Scheduling method and device, processing equipment, storage medium and program product

Abstract

The invention discloses a scheduling method and device, processing equipment, a storage medium and a program product. The method comprises the following steps: receiving first input data of a user; performing topic tracking on the first input data to obtain a topic tracking result; determining a model scheduling demand according to the topic tracking result, and sending the model scheduling demand and encrypted target input data to a cloud side; wherein the model scheduling demand is used for the cloud side to schedule the encrypted target input data to a compression model or a full-amount model for reasoning.

Description

Technical Field

[0001] The present application relates to the field of artificial intelligence technology, and in particular to a scheduling method and apparatus, processing equipment, storage medium, and program product. Background Art

[0002] Artificial Intelligence (AI) technology has achieved rapid development in recent years, especially AI large models based on deep learning have been widely used in various fields. Large language models represented by Generative Pre-Trained (GPT) models have demonstrated strong performance and application value in many fields such as natural language processing.

[0003] Chat-GPT is a typical GPT. Chat-GPT is usually deployed by the model service provider on the cloud side (or cloud) and provided as a remote service. This will cause the end-side (or terminal) user's questions to be obtained by the model service provider when using the model service, so there will be privacy leakage for sensitive content. To this end, two-party secure computing (2PC) can be introduced into the model reasoning process, so that the model service provider cannot know the content of the end-side user's questions and the answers obtained, thereby protecting the user's data privacy. However, when a single full-scale 2PC model performs reasoning services, the reasoning time is significantly extended. Summary of the invention

[0004] Embodiments of the present application provide a scheduling method and apparatus, a processing device, a computer storage medium, and a computer program product.

[0005] The scheduling method provided in the embodiment of the present application is applied to the terminal side, and the method includes:

[0006] receiving first input data from a user;

[0007] Performing topic tracking on the first input data to obtain a topic tracking result;

[0008] The model scheduling requirement is determined according to the topic tracking result, and the model scheduling requirement and the encrypted target input data are sent to the cloud side; wherein the model scheduling requirement is used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

[0009] The scheduling method provided in the embodiment of the present application is applied to the cloud side, and the method includes:

[0010] The model scheduling requirements and encrypted target input data sent by the receiving end;

[0011] According to the model scheduling requirements, the encrypted target input data is scheduled to the compression model or the full model for reasoning to obtain the reasoning result.

[0012] The scheduling device provided in the embodiment of the present application is applied to the terminal side, and the device includes:

[0013] A user interaction module, used for receiving first input data from a user;

[0014] A topic tracking module, used to perform topic tracking on the first input data to obtain a topic tracking result;

[0015] A request module is used to determine the model scheduling requirements based on the topic tracking results, and send the model scheduling requirements and the encrypted target input data to the cloud side; wherein the model scheduling requirements are used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

[0016] The scheduling device provided in the embodiment of the present application is applied to the cloud side, and the device includes:

[0017] The scheduling module is used to receive the model scheduling requirements and encrypted target input data sent by the end side; according to the model scheduling requirements, the encrypted target input data is scheduled to the compression model or the full model for reasoning to obtain the reasoning result.

[0018] The processing device provided in the embodiment of the present application includes: a processor and a memory, the memory is used to store a computer program, and the processor is used to call and run the computer program stored in the memory to execute any one of the above-mentioned scheduling methods.

[0019] The computer-readable storage medium provided in the embodiment of the present application is used to store a computer program, and the computer program enables a computer to execute any one of the above-mentioned scheduling methods.

[0020] The computer program product provided in the embodiments of the present application includes computer program instructions, which enable a computer to execute any one of the above-mentioned scheduling methods.

[0021] The technical solution of the embodiment of the present application determines the corresponding model scheduling requirements according to the number of user interactions on the topic, and schedules the compressed model or the full model for reasoning according to the model scheduling requirements. On the one hand, the compressed model can significantly reduce the time consumption of reasoning, thereby shortening the reasoning time. On the other hand, in view of the situation where the compressed model will cause the reasoning accuracy to decrease, a dynamic model scheduling method is used to schedule the model corresponding to the specific interaction (such as the compressed model or the full model). Through the combination of the above compression model and model scheduling, the disadvantage of the slow execution speed of the single full model reasoning service is improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 It is the overall architecture diagram provided by the embodiment of the present application;

[0023] Figure 2 is a model generation flow chart provided in an embodiment of the present application;

[0024] Figure 3 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 1 ;

[0025] Figure 4 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 2 ;

[0026] Figure 5 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 3 ;

[0027] Figure 6 It is a flowchart of topic tracking provided by an embodiment of the present application;

[0028] Figure 7 It is a flow chart of the service scheduling provided by the embodiment of the present application;

[0029] Figure 8 This is a schematic diagram of the structure of the scheduling device provided in the embodiment of the present application. Figure 1 ;

[0030] Fig. 9 This is a schematic diagram of the structure of the scheduling device provided in the embodiment of the present application. Figure 2 ;

[0031] Fig.10 is a schematic structural diagram of a processing device provided in an embodiment of the present application;

[0032] Fig.11 It is a schematic structural diagram of the chip of an embodiment of the present application. DETAILED DESCRIPTION

[0033] It should be noted that the term "and / or" in this article is only a description of the association relationship of associated objects, indicating that there may be three relationships. For example, A and / or B can represent: A exists alone, A and B exist at the same time, and B exists alone. In addition, the character " / " in this article generally indicates that the objects associated before and after are in an "or" relationship. It should also be understood that the "indication" mentioned in this article can be a direct indication, an indirect indication, or an indication of an association relationship. For example, A indicates B, which can mean that A directly indicates B, for example, B can be obtained through A; it can also mean that A indirectly indicates B, for example, A indicates C, and B can be obtained through C; it can also mean that there is an association relationship between A and B. It should also be understood that the "protocol" mentioned in this article can refer to a standard protocol in the relevant field.

[0034] To facilitate understanding of the technical solutions of the embodiments of the present application, the technical solutions of the present application are described in detail below through specific embodiments. The above related technologies can be combined arbitrarily with the technical solutions of the embodiments of the present application as optional solutions, and they all belong to the protection scope of the embodiments of the present application. The embodiments of the present application include at least part of the following contents.

[0035] It should be noted that the model described in the embodiments of the present application may be, but is not limited to, a 2PC model. For example, the model described in the embodiments of the present application may also be a model of other secure computing protocols. Accordingly, the compression model described in the embodiments of the present application may be, but is not limited to, a compressed 2PC model, and the full model described in the embodiments of the present application may be, but is not limited to, a full 2PC model. It is understood that the "2PC" described in the embodiments of the present application may be replaced by other secure computing protocols.

[0036] Figure 1 This is the overall architecture diagram provided by the embodiment of the present application, such as Figure 1 As shown, the overall architecture provided by the embodiment of the present application consists of a terminal side and a cloud side. The terminal side is deployed with a user interaction module, a 2PC request module, and a topic tracking module; the cloud side is deployed with a 2PC model reasoning service and a service scheduling and orchestration module.

[0037] There are two types of 2PC model inference services deployed on the cloud side, one is the compressed 2PC model inference service, and the other is the full 2PC model inference service. Among them, the number of compressed 2PC model inference services can be multiple, and the number of full 2PC model inference services can be one or more. Generally, the number of compressed 2PC model inference services is greater than the number of full 2PC model inference services.

[0038] The service scheduling and orchestration module deployed on the cloud side will automatically schedule the 2PC requests sent from the client side among multiple 2PC model inference services, that is, forward the 2PC requests to the appropriate 2PC model inference service.

[0039] It should be noted that the 2PC model reasoning service is provided by the 2PC model. In some descriptions in the embodiments of the present application, "2PC model reasoning service" and "2PC model" can be used interchangeably, for example, "scheduling 2PC model reasoning service" and "scheduling 2PC model" can be used interchangeably. For example, "cloud-side deployment 2PC model reasoning service" and "cloud-side deployment 2PC model" can be used interchangeably.

[0040] It should be noted that the cloud side refers to the owner / provider of the 2PC model, and the end side refers to the user of the 2PC model. In specific implementation, the cloud side can be implemented through a server, and the end side can be implemented through a terminal.

[0041] Figure 2 is a flow chart of model generation provided in the embodiment of the present application, such as Figure 2 As shown in the figure, in the model generation process, the original large language model is used to generate a full 2PC model and a compressed 2PC model. The original large language model can be compressed to obtain a model (referred to as a compressed model) through knowledge distillation. Then, the full 2PC model is generated from the original model, and the compressed 2PC model is generated from the compressed model. At this point, the model generation process is completed.

[0042] It should be noted that the original large language model refers to the original model of the large language model based on the Transformer architecture. Examples of large language models based on the Transformer architecture include GPT and BERT.

[0043] It should be noted that the 2PC model described in the embodiments of the present application refers to a specific model optimized for 2PC accelerated reasoning for the Transformer architecture.

[0044] It should be noted that the model described in the embodiments of the present application can be applied to specific scenarios. For example, in a question-and-answer scenario, the corresponding model is a question-and-answer model. For the question-and-answer model, the model input is the user's question, and the model output is the answer to the question. The present application does not limit the specific form of the model input and model output. For example, the model input can be text, voice, picture, etc. The model output can be text, voice, picture, etc.

[0045] Figure 3 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 1 , the scheduling method is applied to the terminal side; Figure 3 As shown, the scheduling method includes the following steps:

[0046] Step 301: Receive first input data from a user.

[0047] Here, the first input data of the user can be understood as the current input data of the user. The present application does not limit the specific form of the first input data, for example, the first input data can be text, voice, picture, etc.

[0048] Exemplarily, in a question-answering scenario, the first input data represents a user's question.

[0049] Step 302: Perform topic tracking on the first input data to obtain a topic tracking result.

[0050] Here, the topic tracking result includes the number of interactions of the user with respect to the first topic, and the first input data belongs to an interaction of the first topic.

[0051] In the embodiment of the present application, topic tracking of the first input data can be achieved through the following process:

[0052] S1: vectorize the first input data to obtain a first vector;

[0053] S2.1: If the first input data belongs to the first interaction of the first topic, the number of interactions of the first topic is set to 1;

[0054] S2.2: If the first input data does not belong to the first interaction of the first topic, searching a target vector in a preset vector set whose similarity with the first vector is less than a first threshold value;

[0055] S2.2.1: If the target vector is found in the vector set, the number of interactions of the first topic is increased by 1;

[0056] S2.2.2: If the target vector is not found in the vector set, the similarity between the first vector and the second vector is calculated;

[0057] S2.2.2.1: If the similarity between the first vector and the second vector is less than a second threshold value, increase the number of interactions of the first topic by 1;

[0058] S2.2.2.2: If the similarity between the first vector and the second vector is greater than or equal to the second threshold value, the number of interactions of the first topic is set to 1.

[0059] The second vector in the above scheme is a vector obtained by vectorizing the second input data, where the second input data is the input data received before the first input data; wherein the similarity between the second vector and all vectors in the vector set is greater than or equal to the first threshold value. That is, the second vector refers to the "vector that is not similar to each vector in the vector set" received last time before the first vector. Here, if the similarity between a vector and each vector in the vector set is greater than or equal to the first threshold value, then the vector can be considered as a "vector that is not similar to each vector in the vector set".

[0060] The similarity in the above solution may be cosine similarity.

[0061] Through the above solution, the number of user interactions on the first topic can be determined. It can be understood that the purpose of performing topic tracking on the first input data is to determine the number of user interactions on the first topic.

[0062] Step 303: Determine the model scheduling requirements based on the topic tracking results, and send the model scheduling requirements and the encrypted target input data to the cloud side; wherein the model scheduling requirements are used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

[0063] In the embodiment of the present application, the model scheduling requirements are determined according to the topic tracking results, which can be achieved through the following process:

[0064] If the number of user interactions on the first topic is less than the first threshold, determining that the model scheduling requirement is a scheduling compression model, and setting a compression model flag representing the model scheduling requirement to true (True);

[0065] If the number of user interactions on the first topic is greater than or equal to the first threshold, the model scheduling requirement is determined to be scheduling the full model, and the compressed model flag representing the model scheduling requirement is set to false (False).

[0066] It should be noted that the model scheduling requirements (i.e., the compressed model flag) sent by the end side to the cloud side are not encrypted, while the target input data sent by the end side to the cloud side is encrypted. For example, the target input data sent by the end side to the cloud side is encrypted via the 2PC protocol. The modules on the cloud side (such as the compressed model and the full model) support reasoning on the encrypted target input data. For example, if the model on the cloud side is a 2PC model (such as a compressed 2PC model and a full 2PC model), the 2PC model supports reasoning on the target input data after 2PC encryption.

[0067] In some implementations, the encrypted target input data may also be referred to as a service call request. In the case where the target input data is encrypted by the 2PC protocol, the service call request may also be referred to as a 2PC request.

[0068] In some implementations, when the number of user interactions on the first topic is less than or greater than a first threshold, the first input data is used as target input data for encryption processing.

[0069] In some implementations, when the number of user interactions on the first topic is equal to a first threshold, the first input data is modified according to all interaction data of the first topic to obtain target input data; and the target input data is encrypted.

[0070] Here, the first threshold can be understood as the service switching threshold (i.e., the threshold for switching from the compressed model service to the full model service), and "the situation where the number of interactions of the first topic is equal to the first threshold" can be understood as "the situation where the scheduling of the service call request is switched from the compressed model to the full model for the first time". In this case, the user's first input data needs to be corrected. Specifically, all interaction data before the current topic (i.e., the first topic) are merged together in order of interaction time, and then merged with the current first input data to form the corrected input data, i.e., the target input data.

[0071] Exemplarily, after receiving the user's first input data, the number of interactions of the current topic can be determined as k through topic tracking, assuming that the service switching threshold is n; if k=n, the user's input data on the terminal side of the previous n-1 interactions (corresponding to k=1, 2, ...n-1) and the output data of the model on the cloud side are merged together in order of interaction time, and then merged with the first input data of the current interaction (corresponding to k=n) to form the corrected input data, i.e., the target input data.

[0072] Further, after step 303, the terminal side receives the inference result for the target input data sent by the cloud side; and displays the inference result to the user.

[0073] The technical solution of the embodiment of the present application proposes an improved solution combining a compression model and service scheduling. On the one hand, the compression model can significantly reduce the time consumption of reasoning, thereby shortening the reasoning time. On the other hand, in view of the situation where the compression model will lead to a decrease in reasoning accuracy, a dynamic model scheduling method is used to schedule the model corresponding to a specific interaction (such as a compression model or a full model). Through the combination of the above compression model and model scheduling, the disadvantage of the slow execution speed of a single full model reasoning service is improved.

[0074] Figure 4 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 2 ,This scheduling method is applied to the cloud side; Figure 4 As shown, the scheduling method includes the following steps:

[0075] Step 401: The model scheduling requirements and encrypted target input data sent by the receiving end side.

[0076] Here, the model scheduling requirements are determined by the client side based on the topic tracking results. Figure 3 Related programs.

[0077] It should be noted that the model scheduling requirements sent from the client to the cloud are not encrypted, while the target input data sent from the client to the cloud is encrypted. Figure 3 Related solutions. For example, the target input data sent from the client to the cloud is encrypted via the 2PC protocol. The cloud-side module (such as the compression model and the full model) supports reasoning on the encrypted target input data. For example, if the cloud-side model is a 2PC model (such as the compression 2PC model and the full 2PC model), the 2PC model supports reasoning on the 2PC-encrypted target input data.

[0078] In some implementations, the encrypted target input data may also be referred to as a service call request. In the case where the target input data is encrypted by the 2PC protocol, the service call request may also be referred to as a 2PC request.

[0079] Step 402: Schedule the encrypted target input data to the compression model or the full model for inference according to the model scheduling requirements to obtain the inference result.

[0080] In some embodiments, the model scheduling requirement is represented by a compressed model flag; if the compressed model flag is set to true, the encrypted target input data is scheduled to the compressed model for reasoning; if the compressed model flag is set to false, the encrypted target input data is scheduled to the full model for reasoning.

[0081] Taking the 2PC protocol as an example, 2PC is a distributed privacy computing solution. During the reasoning process, multiple rounds of communication are required between the two parties executing the 2PC protocol (i.e., the end side and the cloud side) to transmit encrypted data, so that the end side and the cloud side can complete the 2PC reasoning process together.

[0082] Furthermore, the cloud sends the inference results to the client for display.

[0083] Figure 5 This is a schematic diagram of the scheduling method provided in the embodiment of the present application. Figure 3 ,like Figure 5 As shown, the scheduling method includes the following steps:

[0084] Step 501: The terminal receives input data from the user.

[0085] Here, the user's input data corresponds to the "user's first input data" described in the aforementioned solution, and the user's first input data can be understood as the user's current input data. This application does not limit the specific form of the first input data, for example, the first input data can be text, voice, picture, etc.

[0086] Exemplarily, in a question-answering scenario, the first input data represents a user's question.

[0087] Step 502: The terminal side performs topic tracking on the current input data to obtain a topic tracking result.

[0088] Here, the purpose of topic tracking is to determine the number of interactions of users on the same topic (ie, the current topic). The current topic here corresponds to the "first topic" described in the above solution.

[0089] Here, the topic tracking result includes the number of interactions of users on the topic of the day.

[0090] For example, in a question-and-answer scenario, one interaction represents a question-and-answer session, where the "question" represents the question initiated by the client side, and the "answer" represents the "answer" to the question output by the cloud side.

[0091] Step 503: The client determines the scheduling requirements for the server based on the topic tracking results.

[0092] Here, the scheduling requirements for the server are also the “model scheduling requirements” or “service scheduling requirements” described in the aforementioned solution.

[0093] The scheduling requirement is represented by a compression model flag. When the compression model flag is set to true (True), it means that the scheduling requirement is a scheduling compression 2PC model. When the compression model flag is set to false (False), it means that the scheduling requirement is a scheduling full 2PC model.

[0094] Step 504: The terminal side uses the 2PC protocol to encrypt the user's input data.

[0095] Here, the input data to be encrypted may be the input data in the aforementioned step, or the corrected input data.

[0096] Step 505: The client sends the scheduling requirement together with the input data encrypted by the 2PC protocol to the cloud.

[0097] Step 506: The cloud side determines the required service according to the compression model flag; if the compression model flag is set to true (True), execute step 507; if the compression model flag is set to false (False), execute step 508.

[0098] Step 507 : Schedule the input data to the compressed 2PC model and execute step 509 .

[0099] Step 508: Schedule the input data to the full 2PC model and execute step 509.

[0100] Step 509: The cloud side performs the 2PC reasoning process together with the terminal side through the scheduled model.

[0101] Step 510: The cloud side sends the inference result to the terminal side, and the terminal side displays the inference result to the user.

[0102] Above Figure 5 The process is the basic process of scheduling provided by the embodiment of the present application. It can be seen from the previous description that for most user requests (i.e., user input data), they will be routed to the compressed 2PC model service, so the response speed is improved. For a small number of user requests that meet specific conditions, they will be routed to the full 2PC model service, and the full 2PC model service will improve the accuracy of reasoning. Through this method, the problem of reduced user experience caused by the slow speed of a single full 2PC model service when executing 2PC reasoning is improved.

[0103] Figure 6 is a flowchart of topic tracking provided in an embodiment of the present application. The purpose of topic tracking is to determine the number of consecutive interactions of users under the same topic. The process of topic tracking can be combined with the aforementioned Figure 5 In step 502; Figure 6 In the embodiment shown, the user's input data is text data as an example for explanation. Figure 6 As shown, the process includes the following steps:

[0104] Step 601: Obtain text data input by the user.

[0105] Step 602: vectorize the text data to obtain a text vector.

[0106] The text vector obtained here corresponds to the "first vector" described in the aforementioned scheme.

[0107] In some implementations, the text data may be vectorized using techniques such as Doc2Vec.

[0108] Step 603: Determine whether the current text data is the first text data of the current topic; if so, execute step 606; if not, execute step 604.

[0109] Here, “determining whether the current text data is the first text data of the current topic” can also be understood as “determining whether the current text data belongs to the first interaction of the current topic”.

[0110] Step 604: Search the preset vector set to see whether there is an element similar to the current text vector; if not, execute step 605; if so, execute step 607.

[0111] Here, the elements in the preset vector set can be obtained by converting preset text into vectors. For example, the preset text includes "say it again", "say it in more detail", etc.

[0112] In an embodiment of the present application, the cosine similarity between the current text vector and each element in the preset vector set is calculated; if the cosine similarity between the current text vector and an element in the vector set is less than a preset threshold value m1 (i.e., the first threshold value), it is considered that there are elements similar to the current text vector in the vector set; if the cosine similarities between the current text vector and all elements in the vector set are greater than or equal to the preset threshold value m1 (i.e., the first threshold value), it is considered that there are no elements similar to the current text vector in the vector set.

[0113] Step 605: Determine whether the current text vector is similar to the previous text vector; if not, execute step 606; if yes, execute step 607.

[0114] Here, the previous text vector refers to the text vector that is not similar to the elements in the vector set and is received last time before the current text vector. If the cosine similarity between a text vector and each vector in the vector set is greater than or equal to the preset threshold value m1 (i.e., the first threshold value), then the text vector can be considered as a "text vector that is not similar to the elements in the vector set".

[0115] Here, judging whether the current text vector is similar to the previous text vector can be achieved in the following way: calculating the cosine similarity between the current text vector and the previous text vector; if the cosine similarity is less than a preset threshold value m2 (i.e., the second threshold value), the current text vector is considered to be similar to the previous text vector; if the cosine similarity is greater than or equal to the preset threshold value m2 (i.e., the second threshold value), the current text vector is considered to be dissimilar to the previous text vector.

[0116] Step 606 : Set the number of interactions k of the current topic to 1, ie, k=1; and execute step 608 .

[0117] Step 607 : Add 1 to the number of interactions k of the current topic, ie, k=k+1; and execute step 608 .

[0118] Step 608: Output the interaction times k of the current topic.

[0119] Figure 7 is a flow chart of the service scheduling (or model scheduling) provided in the embodiment of the present application. The service scheduling process can be combined with the aforementioned Figure 5 In step 503; Figure 7 As shown, the process includes the following steps:

[0120] Step 701: Determine whether the number of interactions k is less than a preset service switching threshold n; if so, execute step 702; if not, execute step 703.

[0121] Here, n is a preset service switching threshold, which is sent from the cloud side to the terminal side in advance.

[0122] Here, if the topic corresponding to the current interaction is a new topic (corresponding to the case of k = 1) or the number of interactions corresponding to the topic of the current interaction is less than n (corresponding to the case of k < n), the current input data is scheduled to the compressed 2PC model service. If the number of interactions corresponding to the topic of the current interaction (i.e., the consecutive interaction times of the same topic) is greater than or equal to n (corresponding to the case of k ≥ n), the current input data is scheduled to the full 2PC model service.

[0123] Step 702: Set the compressed model flag to true.

[0124] Step 703: Determine whether the number of interactions k is equal to the service switching threshold n; if so, execute Step 704; if not, execute Step 705.

[0125] Step 704: Merge the input data; execute Step 705.

[0126] Here, when the user's input data is first scheduled from the compressed 2PC model service to the full 2PC model service (corresponding to the case of k = n), the user's input data needs to be corrected. Specifically, all the interaction contents before the current topic (i.e., the terminal-side user input data and the cloud-side service output data when k ranges from 1 to n - 1) are merged together in order, and then combined with the current user's input data (i.e., the terminal-side user input data when k = n) to form the corrected user input data.

[0127] Step 705: Set the compressed model flag to false.

[0128] After the terminal side completes the scheduling judgment through the above process, the compressed model flag and the encrypted user input data are sent to the cloud side together. Among them, the user input data is encrypted using the 2PC protocol, and the compressed model flag is not encrypted. After receiving the service request (including the compressed model flag and the encrypted user input data) sent from the terminal side, the cloud side schedules the user input data to the compressed 2PC model service or the full 2PC model service according to the value of the compressed model flag, and ensures that subsequent requests are scheduled to the same service instance when the service type is not switched.

[0129] The technical solution of the embodiment of the present application changes the solution of executing reasoning by a single full 2PC large model reasoning service to a solution that distributes most user requests to the compressed 2PC model service based on service scheduling on the basis of model compression, so as to improve the overall execution speed of the reasoning service. For a small number of specific user requests, they are scheduled to the full 2PC model service to improve the accuracy of reasoning. Therefore, the technical solution of the embodiment of the present application can significantly improve the reasoning execution speed of most user requests while ensuring the overall satisfaction of the user with the reasoning service, and improves the problem of reduced user experience caused by the slow speed of a single full model service when executing 2PC reasoning.

[0130] Figure 8 This is a schematic diagram of the structure of the scheduling device provided in the embodiment of the present application. Figure 1 , applied to the end side, such as Figure 8 As shown, the scheduling device includes:

[0131] The user interaction module 801 is used to receive first input data from a user;

[0132] A topic tracking module 802, configured to perform topic tracking on the first input data to obtain a topic tracking result;

[0133] The request module 803 is used to determine the model scheduling requirements based on the topic tracking results, and send the model scheduling requirements and the encrypted target input data to the cloud side; wherein, the model scheduling requirements are used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

[0134] In some embodiments, the topic tracking result includes the number of interactions of the user on the first topic, and the first input data belongs to an interaction of the first topic; the topic tracking module 802 is used to vectorize the first input data to obtain a first vector; if the first input data belongs to the first interaction of the first topic, the number of interactions of the first topic is set to 1; if the first input data does not belong to the first interaction of the first topic, a target vector whose similarity with the first vector is less than a first threshold value is searched in a preset vector set; if the target vector is found in the vector set, the number of interactions of the first topic is increased by 1; if the target vector is not found in the vector set, the similarity between the first vector and the second vector is calculated; if the similarity between the first vector and the second vector is less than a second threshold value, the number of interactions of the first topic is increased by 1; if the similarity between the first vector and the second vector is greater than or equal to the second threshold value, the number of interactions of the first topic is set to 1.

[0135] In some embodiments, the second vector is a vector obtained by vectorizing second input data, and the second input data is input data received before the first input data; wherein the similarity between the second vector and all vectors in the vector set is greater than or equal to the first threshold value.

[0136] In some embodiments, the request module 803 is used to determine that the model scheduling requirement is a scheduling compression model if the number of interactions of the user on the first topic is less than a first threshold, and set a compression model flag representing the model scheduling requirement to true; if the number of interactions of the user on the first topic is greater than or equal to the first threshold, then determine that the model scheduling requirement is a scheduling full model, and set the compression model flag representing the model scheduling requirement to false.

[0137] In some embodiments, the device further includes: a correction module 804 and an encryption module 805;

[0138] The encryption module 805 is configured to encrypt the first input data as the target input data when the number of interactions of the user on the first topic is less than or greater than the first threshold;

[0139] The correction module 804 is used to correct the first input data according to all interaction data of the first topic to obtain target input data when the number of interactions of the user on the first topic is equal to the first threshold; the encryption module 805 is used to encrypt the target input data.

[0140] In some embodiments, the device further includes: a display module, configured to receive an inference result for the target input data sent by the cloud side; and display the inference result to the user.

[0141] Those skilled in the art should understand that Figure 8 The implementation functions of each unit in the scheduling device shown can be understood by referring to the relevant description of the aforementioned method. Figure 8 The functions of each unit in the scheduling device shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0142] Fig. 9 This is a schematic diagram of the structure of the scheduling device provided in the embodiment of the present application. Figure 2 , applied to the cloud side, such as Fig. 9 As shown, the scheduling device includes:

[0143] The scheduling module 901 is used to receive the model scheduling requirements and encrypted target input data sent by the end side; according to the model scheduling requirements, the encrypted target input data is scheduled to the compression model or the full model for reasoning to obtain the reasoning result.

[0144] In some embodiments, the model scheduling requirement is represented by a compressed model flag;

[0145] In some embodiments, the scheduling module 901 is used to schedule the encrypted target input data to the compression model for reasoning if the compression model flag is set to true; if the compression model flag is set to false, schedule the encrypted target input data to the full model for reasoning.

[0146] In some implementations, the device further includes: a communication module, configured to send the inference result to the end side for display.

[0147] Those skilled in the art should understand that Fig. 9 The implementation functions of each unit in the scheduling device shown can be understood by referring to the relevant description of the aforementioned method. Fig. 9 The functions of each unit in the scheduling device shown can be implemented by a program running on a processor, or by a specific logic circuit.

[0148] Fig.10 1 is a schematic structural diagram of a processing device 1000 provided in an embodiment of the present application. The communication device may be a terminal device or a network device. Fig.10 The processing device 1000 shown includes a processor 1010, and the processor 1010 can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0149] Alternatively, if Fig.10 As shown, the processing device 1000 may further include a memory 1020. The processor 1010 may call and run a computer program from the memory 1020 to implement the method in the embodiment of the present application.

[0150] The memory 1020 may be a separate device independent of the processor 1010 , or may be integrated into the processor 1010 .

[0151] Alternatively, if Fig.10 As shown, the processing device 1000 may further include a transceiver 1030, and the processor 1010 may control the transceiver 1030 to communicate with other devices, specifically, may send information or data to other devices, or receive information or data sent by other devices.

[0152] The transceiver 1030 may include a transmitter and a receiver. The transceiver 1030 may further include an antenna, and the number of antennas may be one or more.

[0153] Optionally, the processing device 1000 may specifically be an end-side device of an embodiment of the present application, and the processing device 1000 may implement corresponding processes implemented by the end-side in various methods of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0154] Optionally, the processing device 1000 may specifically be a cloud-side device of an embodiment of the present application, and the processing device 1000 may implement the corresponding processes implemented by the cloud side in each method of the embodiment of the present application, which will not be described in detail here for the sake of brevity.

[0155] Fig.11 It is a schematic structural diagram of the chip of an embodiment of the present application. Fig.11 The chip 1100 shown includes a processor 1110, which can call and run a computer program from a memory to implement the method in the embodiment of the present application.

[0156] Alternatively, if Fig.11 As shown, the chip 1100 may further include a memory 1120. The processor 1110 may call and run a computer program from the memory 1120 to implement the method in the embodiment of the present application.

[0157] The memory 1120 may be a separate device independent of the processor 1110 , or may be integrated into the processor 1110 .

[0158] Optionally, the chip 1100 may further include an input interface 1130. The processor 1110 may control the input interface 1130 to communicate with other devices or chips, and specifically, may obtain information or data sent by other devices or chips.

[0159] Optionally, the chip 1100 may further include an output interface 1140. The processor 1110 may control the output interface 1140 to communicate with other devices or chips, and specifically, may output information or data to other devices or chips.

[0160] Optionally, the chip can be applied to the terminal side device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the terminal side in each method of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0161] Optionally, the chip can be applied to the cloud-side device in the embodiments of the present application, and the chip can implement the corresponding processes implemented by the cloud side in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0162] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.

[0163] It should be understood that the processor of the embodiment of the present application may be an integrated circuit chip with signal processing capabilities. In the implementation process, each step of the above method embodiment can be completed by the hardware integrated logic circuit or software instructions in the processor. The above processor can be a general processor, a digital signal processor (Digital Signal Processor, DSP), an application-specific integrated circuit (Application Specific Integrated Circuit, ASIC), a field programmable gate array (Field Programmable Gate Array, FPGA) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components. The methods, steps and logic block diagrams disclosed in the embodiments of the present application can be implemented or executed. The general processor can be a microprocessor or the processor can also be any conventional processor. The steps of the method disclosed in the embodiment of the present application can be directly embodied as a hardware decoding processor to perform, or the hardware and software modules in the decoding processor are combined and performed. The software module can be located in a random access memory, a flash memory, a read-only memory, a programmable read-only memory or an electrically erasable programmable memory, a register, and other mature storage media in the art. The storage medium is located in the memory, and the processor reads the information in the memory and completes the steps of the above method in combination with its hardware.

[0164] It can be understood that the memory in the embodiments of the present application can be a volatile memory or a non-volatile memory, or can include both volatile and non-volatile memories. Among them, the non-volatile memory can be a read-only memory (ROM), a programmable read-only memory (PROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), or a flash memory. The volatile memory can be a random access memory (RAM), which is used as an external cache. By way of example and not limitation, many forms of RAM are available, such as static random access memory (SRAM), dynamic random access memory (DRAM), synchronous dynamic random access memory (SDRAM), double data rate synchronous dynamic random access memory (DDR SDRAM), enhanced synchronous dynamic random access memory (ESDRAM), synchronous link dynamic random access memory (SLDRAM), and direct RAM bus random access memory (DR RAM). It should be noted that the memory of the systems and methods described herein is intended to include, but is not limited to, these and any other suitable types of memory.

[0165] It should be understood that the above-mentioned memory is exemplary but not restrictive. For example, the memory in the embodiments of the present application may also be static random access memory (static RAM, SRAM), dynamic random access memory (dynamic RAM, DRAM), synchronous dynamic random access memory (synchronous DRAM, SDRAM), double data rate synchronous dynamic random access memory (double data rate SDRAM, DDR SDRAM), enhanced synchronous dynamic random access memory (enhanced SDRAM, ESDRAM), synchronous link dynamic random access memory (synch link DRAM, SLDRAM) and direct memory bus random access memory (Direct Rambus RAM, DR RAM), etc. That is to say, the memory in the embodiments of the present application is intended to include but not limited to these and any other suitable types of memory.

[0166] An embodiment of the present application also provides a computer-readable storage medium for storing a computer program.

[0167] Optionally, the computer-readable storage medium can be applied to the terminal side device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the terminal side in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0168] Optionally, the computer-readable storage medium can be applied to the cloud-side device in the embodiments of the present application, and the computer program enables the computer to execute the corresponding processes implemented by the cloud side in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0169] An embodiment of the present application also provides a computer program product, including computer program instructions.

[0170] Optionally, the computer program product can be applied to the terminal side device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the terminal side in the various methods of the embodiments of the present application. For the sake of brevity, they are not repeated here.

[0171] Optionally, the computer program product can be applied to the cloud-side device in the embodiments of the present application, and the computer program instructions enable the computer to execute the corresponding processes implemented by the cloud side in the various methods of the embodiments of the present application. For the sake of brevity, they will not be repeated here.

[0172] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, or a combination of computer software and electronic hardware. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0173] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the systems, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0174] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0175] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0176] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0177] If the functions are implemented in the form of software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application can be essentially or partly embodied in the form of a software product that contributes to the prior art. The computer software product is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a magnetic disk or an optical disk.

[0178] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of changes or substitutions within the technical scope disclosed in the present application, which should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A scheduling method, characterized in that: Applied to the terminal side, the method includes: receiving first input data from a user; Performing topic tracking on the first input data to obtain a topic tracking result; The model scheduling requirement is determined according to the topic tracking result, and the model scheduling requirement and the encrypted target input data are sent to the cloud side; wherein the model scheduling requirement is used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

2. The method according to claim 1, characterized in that The topic tracking result includes the number of interactions of the user with respect to the first topic, and the first input data belongs to one interaction of the first topic; The performing topic tracking on the first input data to obtain a topic tracking result includes: Vectorizing the first input data to obtain a first vector; If the first input data belongs to the first interaction of the first topic, the number of interactions of the first topic is set to 1; If the first input data does not belong to the first interaction of the first topic, then search for a target vector whose similarity with the first vector is less than a first threshold value in a preset vector set; if the target vector is found in the vector set, the number of interactions of the first topic is increased by 1; if the target vector is not found in the vector set, the similarity between the first vector and the second vector is calculated; if the similarity between the first vector and the second vector is less than a second threshold value, the number of interactions of the first topic is increased by 1; if the similarity between the first vector and the second vector is greater than or equal to the second threshold value, the number of interactions of the first topic is set to 1.

3. The method according to claim 2, characterized in that The second vector is a vector obtained by vectorizing second input data, and the second input data is input data received before the first input data; wherein the similarity between the second vector and all vectors in the vector set is greater than or equal to the first threshold value.

4. The method according to any one of claims 1 to 3, characterized in that The determining of the model scheduling requirement according to the topic tracking result includes: If the number of interactions of the user on the first topic is less than a first threshold, determining that the model scheduling requirement is a scheduling compression model, and setting a compression model flag representing the model scheduling requirement to true; If the number of interactions of the user on the first topic is greater than or equal to the first threshold, the model scheduling requirement is determined to be a full-volume model, and a compression model flag representing the model scheduling requirement is set to false.

5. The method according to claim 4, characterized in that The method further comprises: When the number of interactions of the user on the first topic is less than or greater than the first threshold, encrypting the first input data as the target input data; When the number of interactions of the user on the first topic is equal to the first threshold, the first input data is modified according to all interaction data of the first topic to obtain target input data; and the target input data is encrypted.

6. The method according to any one of claims 1 to 3, characterized in that The method further comprises: Receiving the inference result for the target input data sent by the cloud side; The inference result is presented to the user.

7. A scheduling method, characterized in that: Applied to the cloud side, the method includes: The model scheduling requirements and encrypted target input data sent by the receiving end; According to the model scheduling requirements, the encrypted target input data is scheduled to the compression model or the full model for reasoning to obtain the reasoning result.

8. The method according to claim 7, characterized in that The model scheduling requirement is represented by a compressed model flag; Scheduling the encrypted target input data to the compression model or the full model for reasoning according to the model scheduling requirement includes: If the compression model flag is set to true, dispatching the encrypted target input data to the compression model for reasoning; If the compression model flag is set to false, the encrypted target input data is scheduled to the full model for inference.

9. The method according to claim 7 or 8, characterized in that: The method further comprises: The inference result is sent to the terminal side for display.

10. A scheduling device, characterized in that: Applied to the terminal side, the device includes: A user interaction module, used for receiving first input data from a user; A topic tracking module, used to track the topic of the first input data to obtain a topic tracking result; A request module is used to determine the model scheduling requirements based on the topic tracking results, and send the model scheduling requirements and the encrypted target input data to the cloud side; wherein the model scheduling requirements are used by the cloud side to schedule the encrypted target input data to the compression model or the full model for inference.

11. A scheduling device, characterized in that: Applied to the cloud side, the device includes: The scheduling module is used to receive the model scheduling requirements and encrypted target input data sent by the end side; according to the model scheduling requirements, the encrypted target input data is scheduled to the compression model or the full model for reasoning to obtain the reasoning result.

12. A processing device, characterized in that: include: A processor and a memory, the memory being used to store a computer program, the processor being used to call and run the computer program stored in the memory to execute the method as claimed in any one of claims 1 to 9.

13. A computer-readable storage medium, characterized in that: Used to store a computer program, wherein the computer program causes a computer to execute the method according to any one of claims 1 to 9.

14. A computer program product, characterized in that The method comprises computer program instructions which cause a computer to execute the method as claimed in any one of claims 1 to 9.

Images