Inference method and device and electronic equipment

Abstract

The reasoning method comprises the following steps: a client sends a problem description to a server; the client receives intermediate data from the server, wherein the intermediate data is obtained by the server at least performing problem description processing; the client processes the intermediate data by using an efficient fine tuning method of parameters on the client and related to the problem description, and returns a processing result to the server; the client receives a reasoning result from the server, wherein the reasoning result is obtained by at least processing the processing result by the server; and the client displays the reasoning result. Therefore, the parameters in the efficient parameter fine tuning method are automatically maintained through the client, and fine tuning is performed on the neural network on the server in a terminal-cloud cooperation manner, so that the neural network on the server can provide personalized services at a relatively low cost.

Description

Technical Field

[0001] This application relates to the technical field of artificial intelligence (AI), and particularly to an inference method, apparatus, and electronic device. Background Art

[0002] Currently, with the dual support of large computing power and big data, the research on pre-trained language models has achieved remarkable results in both model scale and performance, becoming the latest revolutionary breakthrough in the fields of artificial intelligence and deep learning. So far, language models with a scale of hundreds of billions have achieved excellent results in complex artificial intelligence tasks such as article writing, dialogue answering, and automatic programming, and have achieved significant performance improvements compared to small-scale language models in various tasks. Although large-scale language models (i.e., large language models) have strong generality, users still have personalized needs. However, since the computing resources and storage resources required for running large-scale language models are relatively high, large-scale language models are often deployed on cloud servers. At this time, if the large-scale language model on the cloud server is trained for the personalized needs of users, it will consume a large amount of manpower, material resources, and time, resulting in high costs. Therefore, how to enable the large-scale language model deployed on the cloud server to provide personalized services at a lower cost is a technical problem that urgently needs to be solved currently. Summary of the Invention

[0003] This application provides an inference method, apparatus, electronic device, computer storage medium, and computer product, which can enable the large-scale language model deployed on the cloud server to provide personalized services at a lower cost.

[0004] In a first aspect, this application provides an inference method, which is applied to a client and includes: the client sends a problem description to the server; the client receives first intermediate data from the server, and the first intermediate data is obtained by the server processing the problem description using at least a neural network; the client sends a first processing result to the server, and the first processing result is obtained by the client processing the first intermediate data using a first parameter-efficient tuning (PET) method related to the problem description; the client receives an inference result related to the problem description from the server; the client presents the inference result. Exemplarily, the first PET method related to the problem description can be, but is not limited to, selected by the user himself / herself.

[0005] In this way, by deploying the PET method on the client, the client can maintain the parameters in the PET method by itself, and in the inference process, the purpose of fine-tuning the neural network on the server can also be achieved through the client-cloud collaboration method, thereby realizing that the neural network on the server can provide personalized services at a lower cost.

[0006] In a possible implementation, before the client receives the inference result from the server, it further includes: the client receives second intermediate data from the server, where the second intermediate data is obtained by the server using a neural network to process at least the first processing result; the client sends a second processing result to the server, where the second processing result is obtained by the client using a first PET method to process the second intermediate data.

[0007] In a possible implementation, it further includes: the client updates the parameters of the first PET method based on the first processing result. In this way, it is possible to update the parameters of the PET method configured on the client, so that the parameters in the PET method can better meet personalized requirements.

[0008] In a possible implementation, before the client sends the problem description to the server, it further includes: the client converts the problem description into a text vector and adds perturbations to the text vector. Thereby, the security of data transmission is improved, and the leakage of the user's private data is avoided.

[0009] In a possible implementation, before the client sends the problem description to the server, it further includes: the client updates the problem description based on the historical problem description. Thereby, the obtained inference result can conform to the user's preference, and the satisfaction of the inference result is improved.

[0010] In a second aspect, the present application provides an inference method applied to a server, including: the server receives a problem description from a client and an identifier of a first parameter-efficient tuning (PET) method related to the problem description; the server uses a neural network to process at least the problem description, and in the case of obtaining first intermediate data, calls a first interface to send the first intermediate data to the client, where the first interface is determined based on the identifier of the first PET method; the server receives a first processing result from the client, where the first processing result is obtained by the client using the first PET method to process the first intermediate data; the server uses a neural network to process at least the first processing result to obtain an inference result; the server sends the inference result to the client.

[0011] In a possible implementation, the server uses a neural network to process at least the first processing result to obtain an inference result, including: the server uses a neural network to process at least the first processing result, and in the case of obtaining second intermediate data, calls the first interface again to send the second intermediate data to the client; the server receives a second processing result from the client, where the second processing result is obtained by the client using the first PET method to process the second intermediate data; the server uses a neural network to process at least the second processing result to obtain an inference result.

[0012] In a possible implementation, at least one interface is configured in the server, and the at least one interface includes a first interface. One interface is used to call a PET method on the client. Each of the at least one interfaces is associated with a data call point and a data return point in the neural network. The data call point is used to pass the intermediate data of the neural network at the data call point into the interface associated with the data call point, and the data return point is used to pass the return value of the interface associated with the data return point into the neural network, so that the neural network can merge and process the return value with the intermediate data of the neural network at the data return point.

[0013] In a possible implementation, the neural network configured in the server includes: a first network layer and a second network layer. The first network layer includes: a first data call point and a first data return point, and the first data call point and the first data return point are associated with the first interface. The second network layer includes: a second data call point and a second data return point, and the second data call point and the second data return point are associated with the first interface.

[0014] In a third aspect, the present application provides an inference device deployed on the client, including: a communication module and a processing module. The communication module is used to send a problem description to the server. The communication module is also used to receive first intermediate data from the server, and the first intermediate data is obtained by the server using the neural network to process at least the problem description. The processing module is used to process the first intermediate data using a first parameter efficient tuning (PET) method related to the problem description to obtain a first processing result. The communication module is also used to send the first processing result to the server. The communication module is also used to receive an inference result related to the problem description from the server. The processing module is also used to present the inference result.

[0015] In a possible implementation, before the communication module receives the inference result from the server, the communication module is also used to receive second intermediate data from the server, and the second intermediate data is obtained by the server using the neural network to process at least the first processing result; the processing module is also used to process the second intermediate data using the first PET method to obtain a second processing result; the communication module is also used to send the second processing result to the server.

[0016] In a possible implementation, the processing module is also used to: update the parameters of the first PET method based on the first processing result.

[0017] In a possible implementation, before the communication module sends the problem description to the server, the processing module is also used to: convert the problem description into a text vector and add perturbations to the text vector.

[0018] In a possible implementation, before the communication module sends the problem description to the server, the processing module is further configured to: update the problem description based on the historical problem description.

[0019] In a fourth aspect, the present application provides an inference device deployed on the server, including: a communication module and a processing module. The communication module is configured to receive a problem description from a client and an identifier of a first parameter efficient tuning (PET) method related to the problem description. The processing module is configured to process at least the problem description using a neural network. The communication module is further configured to, when the processing module obtains first intermediate data, call a first interface to send the first intermediate data to the client, where the first interface is determined based on the identifier of the first PET method. The communication module is further configured to receive a first processing result from the client, where the first processing result is obtained by the client processing the first intermediate data using the first PET method. The processing module is further configured to process at least the first processing result using a neural network to obtain an inference result. The communication module is further configured to send the inference result to the client.

[0020] In a possible implementation, the processing module is further configured to process at least the first processing result using a neural network, and, when second intermediate data is obtained, call the first interface again and send the second intermediate data to the client through the communication module. The communication module is further configured to receive a second processing result from the client, where the second processing result is obtained by the client processing the second intermediate data using the first PET method. The processing module is further configured to: process at least the second processing result using a neural network to obtain an inference result.

[0021] In a possible implementation, at least one interface is configured in the server, and the at least one interface includes a first interface. One of the at least one interfaces is used to call a PET method on the client. Each of the at least one interfaces is associated with a data call point and a data return point in the neural network. The data call point is used to pass the intermediate data of the neural network at the data call point into the interface associated with the data call point, and the data return point is used to pass the return value of the interface associated with the data return point into the neural network, so that the neural network can perform a merge process on the return value and the intermediate data of the neural network at the data return point.

[0022] In a possible implementation, the neural network configured in the server includes: a first network layer and a second network layer. The first network layer includes: a first data call point and a first data return point, and the first data call point and the first data return point are associated with the first interface. The second network layer includes: a second data call point and a second data return point, and the second data call point and the second data return point are associated with the first interface.

[0023] In a fifth aspect, the present application provides an electronic device, including: one or more processors; a memory for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method described in the first aspect or any possible implementation manner of the first aspect, or implement the method described in the second aspect or any possible implementation manner of the second aspect.

[0024] In a sixth aspect, the present application provides a computer-readable storage medium, including computer program instructions, when the computer program instructions are executed by an electronic device, the electronic device is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect, or execute the method described in the second aspect or any possible implementation manner of the second aspect.

[0025] In a seventh aspect, the present application provides a computer program product containing instructions, when the instructions are run by an electronic device, the electronic device is caused to execute the method described in the first aspect or any possible implementation manner of the first aspect, or execute the method described in the second aspect or any possible implementation manner of the second aspect.

[0026] It can be understood that the beneficial effects of the above second aspect to the seventh aspect can be referred to the relevant descriptions in the first aspect, and will not be elaborated here. Description of the Drawings

[0027] Figure 1 is a schematic diagram of the architecture of a terminal-cloud collaborative inference system provided by an embodiment of the present application;

[0028] Figure 2 is a schematic diagram of the working process of a terminal-cloud collaborative inference system provided by an embodiment of the present application;

[0029] Figure 3 is a schematic flowchart of an inference method provided by an embodiment of the present application;

[0030] Figure 4 is a result schematic diagram of an inference device provided by an embodiment of the present application;

[0031] Figure 5 is a result schematic diagram of another inference device provided by an embodiment of the present application;

[0032] Figure 6 is a schematic diagram of the structure of an electronic device provided by an embodiment of the present application. Detailed Embodiments

[0033] In this text, the term "and / or" describes the relationship between related objects, indicating that there can be three relationships. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. In this text, the symbol " / " indicates that the related objects are in an "or" relationship. For example, A / B means A or B.

[0034] In the specification and claims of this application, terms such as "first" and "second" are used to distinguish different objects, rather than to describe the specific order of the objects. For example, the first response message and the second response message are used to distinguish different response messages, rather than to describe the specific order of the response messages.

[0035] In the embodiments of this application, words such as "exemplary" or "for example" are used to give examples, illustrations, or explanations. Any embodiment or design solution described as "exemplary" or "for example" in the embodiments of this application should not be construed as being more preferred or having more advantages than other embodiments or design solutions. Rather, the use of words such as "exemplary" or "for example" is intended to present relevant concepts in a specific manner.

[0036] In the description of the embodiments of this application, unless otherwise specified, the meaning of "a plurality of" refers to two or more. For example, a plurality of processing units refers to two or more processing units, and a plurality of elements refers to two or more elements, etc.

[0037] Generally, in order to meet the personalized requirements of large language models on cloud servers, the parameter efficient tuning (PET) method can be used. By freezing the parameters of the large language model itself and additionally inserting a small number of PET parameters, the large language model can be fine-tuned. However, the PET method requires the simultaneous use of user data, large language models, and PET parameters. Ordinary users cannot run large language models locally. At the same time, the cloud server does not have user data and it is difficult to simultaneously maintain and schedule the PET parameters of tens of millions of users. This makes it difficult to meet the personalized requirements of large language models through the PET method.

[0038] In view of this, an embodiment of the present application provides an edge-cloud collaborative inference system. By leveraging the characteristics that edge devices have low computing efficiency and small storage capacity, while cloud devices have high computing efficiency, this system adopts a parameter sharing method. It stores the parameters of personalized fine-tuning with fewer parameters (i.e., the parameters of the PET method) on the user's edge device, and stores the computationally intensive and large-parameter pre-trained language model parameters on the cloud device. Through the collaboration of the edge and the cloud in terms of computing and communication, the fine-tuning training and inference of large-scale language models are jointly completed. Thus, large-scale language models on cloud devices can provide personalized services at a relatively low cost.

[0039] Exemplarily, Figure 1 shows a schematic architecture diagram of an edge-cloud collaborative inference system provided by an embodiment of the present application. As Figure 1 shown, the edge-cloud collaborative inference system 100 includes: a client 110 and a server 120. A communication connection can be established between the client 110 and the server 120 through, but not limited to, a wired network or a wireless network.

[0040] The client 110 can be a desktop application, a mobile application, a Web application, or a Web-based application, etc., and it can be deployed on electronic devices such as mobile phones and computers. At least one PET slot is configured in the client 110. One PET slot can be used to carry one PET method. Exemplarily, the PET method can include: a parameter-efficient fine-tuning method based on transform learning (such as: low-rank adaptation (LORA)), a parameter-efficient fine-tuning method based on a plug-in module (such as: adapter), a parameter-efficient fine-tuning method based on a parallel plug-in module (such as: parallel adapter), or a parameter-efficient fine-tuning method based on a bias term (such as: bitfit). In this embodiment, the client 110 is mainly used to receive the problem description input by the user and the selected PET method, and receive the intermediate result of the large language model 121 transmitted by the server 120, and process the intermediate result through a certain PET slot, and return the processed result to the server 120. In addition, the client 110 can also update the parameters of the PET method carried by the PET slot based on the result obtained by processing on the PET slot thereon. In addition, the client 110 can also be used to display the inference result to the user.

[0041] The server 120 may, but is not limited to, be a cloud server. The server 120 is configured with a large language model 121 and at least one PET slot interface. One PET slot interface can be associated with one PET slot. One PET slot interface is used to invoke one PET method in the client 110. In the large language model 121, a data call point and a data return point are configured. One data call point is used to invoke one PET slot interface, so that the intermediate data of the large language model 121 at this data call point is passed into the PET slot interface. One data return point is used to return the return value of one PET slot interface to the large language model 121, so that the return value can be merged with the intermediate data of the large language model 121 at this data return point. In this embodiment, the server 120 is mainly used to determine which PET slot interface to select based on the PET method selected by the user transmitted by the client 110, and to perform inference on the problem description received from the client 110 through the large language model 121. In addition, after selecting the PET slot interface, when the large language model 121 runs to the data call point associated with this PET slot interface, the server 120 can transmit the intermediate data of the large language model 121 passed into this PET slot interface by this data call point to the client 110, and receive the processing result of the intermediate data transmitted by the server 120 through the PET slot on the client 110 that is used to carry the PET method selected by the user, and return the processing result to the large language model 121 through the data return point associated with the PET slot interface in the large language model 121 to continue the inference task. After obtaining the inference result, the server 120 can transmit the inference result to the client 110. In some embodiments, the PET slot interface can be integrated in the large language model 121 or arranged separately from the large language model 121, which can be determined according to the actual situation and is not limited here. In some embodiments, data call points and data return points can be configured in different network layers of the large language model 121, and the data call points and data return points in different network layers can be associated with the same PET slot interface. For example, the large language model 121 may include network layer A and network layer B. Data call point A1 and data return point A2 can be configured in network layer A, and data call point B1 and data return point B2 can be configured in network layer B. Data call point A1 and data return point A2 can be associated with PET slot interface A, and data call point B1 and data return point B2 can be associated with PET slot interface B. At this time, PET slot interface A can be the same as PET slot interface B, or of course different. When PET slot interface A can be the same as PET slot interface B, the large language model 121 can invoke PET slot interface A once when processing data in network layer A and can invoke PET slot interface A again when processing data in network layer B.In some embodiments, the large language model 121 configured in the server 120 can also be replaced by other neural networks, and the replaced solution is still within the protection scope of this application.

[0042] The above is the introduction of the end-cloud collaborative inference system 100 provided by the embodiments of this application. For ease of understanding, the working process of the end-cloud collaborative inference system 100 will be introduced by way of example below.

[0043] Exemplarily, Figure 2 shows a schematic diagram of the working process of the end-cloud collaborative inference system 100 provided by the embodiments of this application. In Figure 2 , there are 4 PET slots configured in the client 110, namely, PET slot 00 carrying PET method 00, PET slot 01 carrying PET method 01, PET slot 02 carrying PET method 02, PET slot 03 carrying PET method 03, and PET slot 04 carrying PET method 04. A large language model 121 is configured in the server 120, and the large language model 121 is of a transformer structure (see details in Figure 2 ). In addition, PET slot interfaces 00, 01, 02, 03, and 04 are configured in the server 120, which are used to call PET methods 00, 01, 02, 03, and 04 respectively. Among them, the PET slot interfaces 00, 01, 02, 03, and 04 are integrated in the large language model 121. It should be understood that the PET slots configured in the client 110 are only examples and do not constitute a limitation to this application; the structure of the large language model 121 in the server 120 is also only an example and does not constitute a limitation to this application; the PET slot interfaces configured in the server 120 are also only examples and do not constitute a limitation to this application.

[0044] Such as Figure 2As shown in the figure, the working process of the edge-cloud collaborative inference system 100 is as follows: In S21, the user can input the problem description to be inferred and select the PET method 04 they expect to use on the client 110. In S22, the client 110 can send the problem description input by the user and the identifier of the PET method 04 to the server 120. In S23, the server 120 can, from the identifier of the PET method 04, know that the PET slot interface 04 needs to be used during the inference process, and can use the large language model 121 to infer the problem description. In S24, during the inference process of the large language model 121, when it executes to the data call point R211 associated with the PET slot interface 04, the server 120 can send the intermediate data β at the data call point R211 to the client 110. At the same time, the server 120 can continue to use the large language model 121 for inference and wait for the return value from the client 110 at the data return point R212 associated with the PET slot interface 04. In S25, after obtaining the intermediate data β, the client 110 can use the PET method 04 carried on its PET slot 04 to process the intermediate data β to obtain the processing result γ. In S26, the client 110 can send the processing result γ to the server 120. In S27, the server 120 can return the processing result γ to the large language model 121 through the data return point R212, and in the large language model 121, the processing result γ and the intermediate data μ at the data return point R212 in the large language model 121 are merged to obtain the merged result θ. Since the data return point R212 is after the last network layer of the large language model 121, the merged result θ can be understood as the inference result. At this time, the server 120 can transmit the inference result θ to the client 110. In S28, after obtaining the inference result θ, the client 110 can present the inference result θ to the user. In S29, the client 110 can update the parameters of the PET method 04 carried on the PET slot 04 based on the processing result γ, so that the training of the PET method 04 can be completed, thereby better providing personalized services.

[0045] Through Figure 2From the working process of the edge-cloud collaborative inference system 100 shown, it can be seen that the system utilizes the characteristics of low computing efficiency and small storage capacity of edge devices and high computing efficiency of cloud devices, and adopts a parameter sharing method. It stores the parameters of personalized fine-tuning with fewer parameters (i.e., the parameters of the PET method) on the user's edge device, while storing the computationally intensive and large-parameter pre-trained language model parameters on the cloud device. Through the collaboration in computing and communication between the edge and the cloud, the fine-tuning training and inference of large-scale language models are jointly completed. In addition, by utilizing the devices of edge users with personalized needs, the storage overhead related to personalized needs on the cloud device is reduced to zero, so that the computing scheduling problem can be transformed into a network scheduling problem. Thus, it is possible to provide personalized services for large-scale language models on cloud devices at a relatively low cost.

[0046] The above is the introduction to the edge-cloud collaborative inference system 100 provided by the embodiment of the present application and its working process. Based on the above content, an inference method provided by the embodiment of the present application will be introduced below.

[0047] Exemplarily, Figure 3 The flowchart of an inference method provided by the embodiment of the present application is shown. As Figure 3 shown, the inference method may include the following steps:

[0048] S301. The client 110 receives the problem description input by the user and the first PET method selected by the user. Among them, the user can input the problem description on the client 110 and select the PET method he expects to use. For example: select LORA, etc. Exemplarily, after the user completes the input and selection and confirms the above operations, the client 110 can receive the input and selection of the user. In some embodiments, the first PET method can also be selected by the client 110 itself, which can be determined according to the actual situation and is not limited here. In addition, the problem description can also be sent to the client 110 by other devices or clients, which is not limited here. Among them, the first PET method can be understood as the PET method related to the problem description.

[0049] S302. The client 110 sends the problem description and the identifier of the first PET method to the server 120. Among them, the client 110 can send the problem description to the server 120 through the network so that the server 120 can reason about the problem description. In addition, the client 110 can also send the identifier of the PET method related to the problem description to the server 120 to inform the server 120 which PET method is used. In some embodiments, in order to reduce the risk of data privacy leakage, the client 110 can perform perturbation processing on the problem description to protect the important privacy information of the user. Exemplarily, the client 110 can first perform vector representation on the problem description through methods such as word embedding to obtain the representation vector of the problem description, thereby realizing the conversion of the problem description into a continuous numerical vector. Then, the client 110 can perturb the problem description by changing the layout of the values in the representation vector of the problem description to obtain the perturbed problem description. Finally, the client 110 can send the problem description with added perturbation to the server 120. In some embodiments, in order to further improve the satisfaction of the inference result, the client 110 can also splice the historical problem description and the current problem description (of course, it can also be done in other ways, which are not limited here), and use the spliced result as the new problem description to complete the update of the problem description; then, the client 110 sends the updated problem description to the server 120. In addition, when it is necessary to add perturbation to the problem description, the client 110 can perturb the spliced result.

[0050] S303. The server 120 processes the problem description using a neural network, and when the first intermediate data is obtained, calls the first interface associated with the first PET method. Among them, after the server 120 obtains the problem description sent by the client 110, it can process the problem description using the neural network deployed on it. In addition, the server 120 can learn from the identifier of the first PET method that the user chooses to use the first PET method. Then, the server 120 can learn from the association relationship between the PET method and the interface on it that the first interface associated with the first PET method needs to be used during the inference process using the neural network. Next, when the server 120 executes to the data call point corresponding to the first interface during the execution of the neural network, it can call the first interface. Among them, the data call point corresponding to the first interface can be a certain position after the first intermediate data is processed in the neural network.

[0051] S304. The server 110 sends the first intermediate data to the client 110. Among them, after the server 110 calls the first interface, it can send the first intermediate data to the client 110 through the first interface. In some embodiments, in order to reduce communication latency, the server 110 can transmit the first intermediate data in a segmented quantization manner. Segmented quantization is a data transmission method that divides the data into multiple smaller segments, performs quantization processing on each segment, and then transmits it. During the transmission process, each segment will be independently compressed and decompressed to effectively transmit and recover the data.

[0052] S305. The client 110 processes the first intermediate data using the first PET method thereon to obtain a first processing result. Among them, after the client 110 obtains the first intermediate data, it can use the first PET method selected by the user to process the first intermediate data to obtain a first processing result. In some embodiments, the client 110 can also calculate at least the first processing result through a loss function to obtain a loss. For example, the client 110 can calculate the first processing result and the local training label through a loss function to obtain a loss. Then, the client 110 can update the parameters in the first PET method with the goal of minimizing the calculated loss. Thus, the update of the parameters in the PET method can be realized on the edge side, so that the parameters in the PET method can better meet personalized needs.

[0053] S306. The client 110 sends the first processing result to the server 120. Exemplarily, the client 110 can also transmit the first processing result in a segmented quantization manner.

[0054] S307. The server 120 combines the first processing result and the second intermediate data obtained by processing the problem description by the neural network to obtain a combined result, and uses the neural network to process the combined result to obtain an inference result related to the problem description. Among them, after the server 120 obtains the first processing result, it can return the first processing result to the neural network through the data return point associated with the first interface in the neural network. Then, the server 120 can combine the first processing result and the second intermediate data obtained between the data return points in the neural network to obtain a combined result. The second intermediate data may be the same as or different from the first intermediate data. When the two are different, the generation time of the second intermediate data is later than the generation time of the first intermediate data. Finally, the server 120 can use the neural network to process the combined result to obtain an inference result.

[0055] S308. The server 120 sends the inference result related to the problem description to the client 110. Among them, after obtaining the inference result, the server 120 can transmit the inference result to the client 110.

[0056] S309. The client 110 presents the inference result. Among them, after obtaining the inference result, the client 110 can present the inference result in ways such as text and voice. Of course, the inference result can also be sent to other devices or clients to present the inference result.

[0057] In this way, by deploying the PET method on the client 110, the client 110 can maintain the parameters in the PET method by itself, and in the inference process, the neural network on the server 120 can also be fine-tuned through the end-cloud collaboration method, so as to achieve the goal of enabling the neural network on the server 120 to provide personalized services at a lower cost.

[0058] It can be understood that in Figure 3 , when the neural network configured in the server 120 contains multiple network layers, and the interfaces associated with the data call points and data return points in different network layers are the same, multiple interactions can also occur between the client 110 and the server 120. In each interaction, the server 120 can process the processing result obtained from the client 110 last time, and when obtaining a certain intermediate data after processing, call the same interface again to transmit the intermediate data to the client 110. In addition, when the server 120 processes the processing result obtained from the client 110 last time, it can merge the processing result with the intermediate data at the data return point associated with the interface it calls, and use the merged result for subsequent processing.

[0059] After obtaining the intermediate data transmitted by the server 120, the client 110 can process the intermediate data using the PET method selected by the user and return the processing result to the server 120. Exemplarily, the client 110 can also update the parameters of the PET method on it using the processing result.

[0060] After obtaining the processing result, the server 120 can continue to process at least the processing result using the neural network to obtain the inference result.

[0061] It can be understood that the magnitudes of the sequence numbers of the steps in the above embodiments do not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation to the implementation process of the embodiments of the present application. In addition, the above-described various embodiments can be combined according to actual situations, and the combined solutions are still within the protection scope of the present application.

[0062] Next, based on the method in the above embodiments, an inference device provided in an embodiment of the present application will be introduced.

[0063] Exemplarily, Figure 4 FIG. shows a schematic structural diagram of an inference device provided in an embodiment of the present application. Among them, the inference device can be deployed in the aforementioned client 110. As Figure 4 shown, the inference device 400 includes: a communication module 401 and a processing module 402. Among them, the communication module 401 is used to send a problem description to the server. The communication module 401 is further used to receive first intermediate data from the server, and the first intermediate data is obtained by the server using a neural network to process at least the problem description. The processing module 402 is used to process the first intermediate data using a first PET method to obtain a first processing result. The communication module 401 is further used to send the first processing result to the server. The communication module 401 is further used to receive an inference result related to the problem description from the server. The processing module 402 is used to present the inference result.

[0064] In some embodiments, before the communication module 401 receives an inference result related to the problem description from the server,

[0065] the communication module 401 is further used to receive second intermediate data from the server, and the second intermediate data is obtained by the server using a neural network to process at least the first processing result;

[0066] the processing module 402 is further used to process the second intermediate data using a first PET method to obtain a second processing result;

[0067] the communication module 401 is further used to send the second processing result to the server.

[0068] In some embodiments, the processing module 402 is further used to: update the parameters of the first PET method based on the first processing result.

[0069] In some embodiments, before the communication module 401 sends the problem description to the server, the processing module 402 is further used to: convert the problem description into a text vector and add perturbations to the text vector.

[0070] In some embodiments, before the communication module 401 sends the problem description to the server, the processing module 402 is further used to: update the problem description based on historical problem descriptions.

[0071] It should be understood that the above device is used to execute the method executed by the client 110 in the above embodiments. For the corresponding program modules in the device, their implementation principles and technical effects are similar to the descriptions in the above method. The working process of the device can refer to the corresponding process in the above method and will not be elaborated here.

[0072] Exemplarily, Figure 5 FIG. shows a schematic structural diagram of another inference device provided by an embodiment of the present application. Among them, the inference device can be deployed in the aforementioned server 120. As Figure 5 shown, the inference device 500 includes: a communication module 501 and a processing module 502. Among them, the communication module 501 is used to receive a problem description from a client and an identifier of a first parameter-efficient fine-tuning (PET) method related to the problem description. The processing module 502 is used to process at least the problem description using a neural network. The communication module 501 is further used to, when first intermediate data is obtained by the processing of the processing module 502, call a first interface to send the first intermediate data to the client, where the first interface is determined based on the identifier of the first PET method. The communication module 501 is further used to receive a first processing result from the client, where the first processing result is obtained by the client processing the first intermediate data using the first PET method. The processing module 502 is further used to process at least the first processing result using a neural network to obtain an inference result. The communication module 501 is further used to send the inference result to the client.

[0073] In some embodiments, the processing module 502 is further used to process at least the first processing result using a neural network, and, when second intermediate data is obtained by the processing, call the first interface again, and send the second intermediate data to the client through the communication module;

[0074] The communication module 501 is further used to receive a second processing result from the client, where the second processing result is obtained by the client processing the second intermediate data using the first PET method;

[0075] The processing module 502 is further used to: process at least the second processing result using a neural network to obtain an inference result.

[0076] In some embodiments, at least one interface is configured in the server, and the at least one interface includes a first interface, and one of the at least one interfaces is used to call a PET method on the client.

[0077] Among them, each of the at least one interfaces is associated with a data call point and a data return point in the neural network. The data call point is used to pass the intermediate data of the neural network at the data call point into the interface associated with the data call point, and the data return point is used to pass the return value of the interface associated with the data return point into the neural network, so that the neural network can merge and process the return value with the intermediate data of the neural network at the data return point.

[0078] In some embodiments, the neural network configured in the server includes a first network layer and a second network layer. Among them, the first network layer includes a first data call point and a first data return point, and the first data call point and the first data return point are associated with a first interface. The second network layer includes a second data call point and a second data return point, and the second data call point and the second data return point are associated with the first interface.

[0079] It should be understood that the above device is used to execute the method executed by the server 120 in the above embodiments. For the corresponding program modules in the device, their implementation principles and technical effects are similar to those described in the above method. The working process of the device can refer to the corresponding process in the above method and will not be elaborated here.

[0080] This application also provides an electronic device 600. As Figure 6 shown, the electronic device 600 includes a bus 602, a processor 604, a memory 606, and a communication interface 608. The processor 604, the memory 606, and the communication interface 608 communicate with each other through the bus 602. The electronic device 600 can be a server (such as a cloud server, etc.) or a terminal device (such as a mobile phone, a computer, etc.). It should be understood that this application does not limit the number of processors and memories in the electronic device 600.

[0081] The bus 602 can be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus, etc. The bus can be divided into an address bus, a data bus, a control bus, etc. For the sake of simplicity of representation, Figure 6 only one line is shown in the figure, but it does not mean that there is only one bus or one type of bus. The bus 604 can include a path for transmitting information between various components of the electronic device 600 (for example, the memory 606, the processor 604, the communication interface 608).

[0082] The processor 604 can include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP), etc.

[0083] The memory 606 may include volatile memory, such as random access memory (RAM). The processor 604 may also include non-volatile memory, such as read-only memory (ROM), flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0084] The executable program code is stored in the memory 606, and the processor 604 executes the executable program code to implement the functions of the acquisition module 401, the communication module 402, the processing module 403, and the display module 404 shown in the foregoing Figure 4 respectively, so as to implement the method executed by the client 110 described in the foregoing embodiments. That is to say, instructions for executing the method executed by the client 110 described in the foregoing embodiments are stored on the memory 606. Alternatively, the processor 604 executes the executable program code to implement the functions of the communication module 501 and the processing module 502 shown in the foregoing Figure 5 respectively, so as to implement the method executed by the server 120 described in the foregoing embodiments. That is to say, instructions for executing the method executed by the server 120 described in the foregoing embodiments are stored on the memory 606.

[0085] Alternatively, executable code is stored in the memory 606, and the processor 604 executes the executable code to implement the functions of the inference device 400 shown in the foregoing Figure 4 respectively, so as to implement the method executed by the client 110 described in the foregoing embodiments. That is to say, instructions for executing the method executed by the client 110 described in the foregoing embodiments are stored on the memory 606. Alternatively, executable code is stored in the memory 606, and the processor 604 executes the executable code to implement the functions of the inference device 500 shown in the foregoing Figure 5 respectively, so as to implement the method executed by the server 120 described in the foregoing embodiments. That is to say, instructions for executing the method executed by the server 120 described in the foregoing embodiments are stored on the memory 606.

[0086] The communication interface 603 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the electronic device 600 and other devices or a communication network.

[0087] It can be understood that in this embodiment Figure 6The schematic structure does not specifically limit the electronic device 600. In some other embodiments, the electronic device 600 may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or have different component arrangements. The components shown in the figure may be implemented in hardware, software, or a combination of software and hardware.

[0088] Based on the method in the above embodiments, an embodiment of the present application provides a computer-readable storage medium, including computer program instructions. When the computer program instructions are executed by an electronic device (such as: Figure 6 the electronic device 600 shown in the figure), the electronic device is caused to execute the method performed by the client 110 or the server 120 in the above embodiments. Exemplarily, the computer-readable storage medium may be any available medium that a computing device can store, or a data storage device such as a data center that contains one or more available media. The available medium may be a magnetic medium (for example, a floppy disk, a hard disk, a magnetic tape), an optical medium (for example, a DVD), or a semiconductor medium (for example, a solid-state drive), etc.

[0089] Based on the method in the above embodiments, an embodiment of the present application provides a computer program product containing instructions. When the instructions are run by an electronic device (such as: Figure 6 the electronic device 600 shown in the figure), the electronic device is caused to execute the method performed by the client 110 or the server 120 in the above embodiments.

[0090] It can be understood that the processor in the embodiments of the present application may be a central processing unit (CPU), or may also be other general-purpose processors, digital signal processors (DSPs), application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), or other programmable logic devices, transistor logic devices, hardware components, or any combination thereof. The general-purpose processor may be a microprocessor or any conventional processor.

[0091] The method steps in the embodiments of this application can be implemented in a hardware manner or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, and the software modules can be stored in a random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), register, hard disk, removable hard disk, CD-ROM, or any other form of storage medium well-known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC.

[0092] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware, or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the processes or functions described in the embodiments of this application are generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable devices. The computer instructions can be stored in a computer-readable storage medium or transmitted through the computer-readable storage medium. The computer instructions can be transmitted from one website, computer, server, or data center to another website, computer, server, or data center in a wired manner (such as coaxial cable, optical fiber, digital subscriber line (DSL)) or wirelessly (such as infrared, wireless, microwave, etc.). The computer-readable storage medium can be any available medium that the computer can access or a data storage device such as a server or data center that includes one or more integrated available media. The available medium can be a magnetic medium (for example, floppy disk, hard disk, magnetic tape), an optical medium (for example, DVD), or a semiconductor medium (for example, solid state disk (SSD)), etc.

[0093] It can be understood that the various numerical numbers involved in the embodiments of this application are only for the convenience of description and are not used to limit the scope of the embodiments of this application.

[0094] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements for some of the technical features; and these modifications or replacements do not cause the essence of the corresponding technical solutions to deviate from the protection scope of the technical solutions of the embodiments of the present application.

Claims

1. A reasoning method, characterized in that, Applied to the client, including: The client sends a problem description to the server; The client receives first intermediate data from the server, where the first intermediate data is obtained by the server processing the problem description at least using a neural network; The client sends a first processing result to the server, where the first processing result is obtained by the client processing the first intermediate data using a first parameter-efficient tuning (PET) method related to the problem description; The client receives an inference result related to the problem description from the server; The client presents the inference result.

2. The method according to claim 1, wherein Before the client receives the inference result related to the problem description from the server, it further includes: The client receives second intermediate data from the server, where the second intermediate data is obtained by the server processing the first processing result at least using the neural network; The client sends a second processing result to the server, where the second processing result is obtained by the client processing the second intermediate data using the first PET method.

3. The method according to claim 1 or 2, characterized in that, It further includes: The client updates the parameters of the first PET method based on the first processing result.

4. The method according to any one of claims 1 to 3, characterized in that Before the client sends the problem description to the server, it further includes: The client converts the problem description into a text vector and adds perturbations to the text vector.

5. The method according to any one of claims 1 to 4, characterized in that, Before the client sends the problem description to the server, it further includes: The client updates the problem description based on historical problem descriptions.

6. A reasoning method, characterized in that, Applied to the server, including: The server receives a problem description from the client and an identifier of a first parameter-efficient tuning (PET) method related to the problem description; The server processes the problem description at least using a neural network, and in the case of obtaining first intermediate data, calls a first interface to send the first intermediate data to the client, where the first interface is determined based on the identifier of the first PET method; The server receives a first processing result from the client, where the first processing result is obtained by the client processing the first intermediate data using the first PET method; The server processes the first processing result at least using the neural network to obtain an inference result; The server sends the inference result to the client.

7. The method according to claim 6, characterized in that, The server processes the first processing result at least using the neural network to obtain an inference result, including: The server processes the first processing result at least using the neural network, and in the case of obtaining second intermediate data, calls the first interface again to send the second intermediate data to the client; The server receives a second processing result from the client, where the second processing result is obtained by the client processing the second intermediate data using the first PET method; The server processes the second processing result at least using the neural network to obtain the inference result.

8. The method according to claim 6 or 7, characterized in that At least one interface is configured in the server, and the at least one interface includes the first interface. One interface among the at least one interface is used to call a PET method on the client; Among them, each interface among the at least one interface is associated with a data call point and a data return point in the neural network. The data call point is used to pass the intermediate data of the neural network at the data call point into the interface associated with the data call point, and the data return point is used to pass the return value of the interface associated with the data return point into the neural network, so that the neural network can perform a merging process on the return value and the intermediate data of the neural network at the data return point.

9. The method according to claim 8, characterized in that, The neural network includes: A first network layer, and the first network layer includes: a first data call point and a first data return point, and the first data call point and the first data return point are associated with the first interface; A second network layer, and the second network layer includes: a second data call point and a second data return point, and the second data call point and the second data return point are associated with the first interface.

10. An inference device, characterized in that, Deployed on the client, including: A communication module for sending a problem description to the server; The communication module is further configured to receive first intermediate data from the server, and the first intermediate data is obtained by the server using the neural network to process at least the problem description; A processing module for processing the first intermediate data using a first parameter efficient tuning (PET) method related to the problem description to obtain a first processing result; The communication module is further configured to send the first processing result to the server; The communication module is further configured to receive an inference result related to the problem description from the server; A display module for presenting the inference result.

11. The device according to claim 10, characterized in that, Before the communication module receives the inference result from the server, The communication module is further configured to receive second intermediate data from the server, and the second intermediate data is obtained by the server using the neural network to process at least the first processing result; The processing module is further configured to process the second intermediate data using the first PET method to obtain a second processing result; The communication module is further configured to send the second processing result to the server.

12. The device according to claim 10 or 11, characterized in that, The processing module is further configured to: Update the parameters of the first PET method based on the first processing result.

13. The device according to any one of claims 10 to 12, characterized in that Before the communication module sends the problem description to the server, the processing module is further configured to: Convert the problem description into a text vector and add perturbations to the text vector.

14. The device according to any one of claims 10-13, characterized in that, Before the communication module sends the problem description to the server, the processing module is further configured to: Update the problem description based on historical problem descriptions.

15. An inference device, characterized in that, Deployed on the server, including: A communication module for receiving a problem description from the client and an identifier of a first parameter efficient tuning (PET) method related to the problem description; A processing module for processing at least the problem description using a neural network; The communication module is further configured to, when the processing module processes and obtains first intermediate data, call a first interface to send the first intermediate data to the client, where the first interface is determined based on the identifier of the first PET method; The communication module is further configured to receive a first processing result from the client, where the first processing result is obtained by the client processing the first intermediate data using the first PET method; The processing module is further configured to process at least the first processing result using the neural network to obtain an inference result; The communication module is further configured to send the inference result to the client.

16. The device according to claim 15, characterized in that The processing module is further configured to process at least the first processing result using the neural network, and, when second intermediate data is processed and obtained, call the first interface again, and send the second intermediate data to the client through the communication module; The communication module is further configured to receive a second processing result from the client, where the second processing result is obtained by the client processing the second intermediate data using the first PET method; The processing module is further configured to: process at least the second processing result using the neural network to obtain the inference result.

17. The device according to claim 15 or 16, characterized in that, At least one interface is configured in the server, and the at least one interface includes the first interface, and one interface in the at least one interface is used to call a PET method on the client; Wherein each interface in the at least one interface is associated with a data call point and a data return point in the neural network, the data call point is used to pass the intermediate data of the neural network at the data call point into the interface associated with the data call point, and the data return point is used to pass the return value of the interface associated with the data return point into the neural network, so that the neural network can perform a merging process on the return value and the intermediate data of the neural network at the data return point.

18. The device according to claim 17, characterized in that, The neural network includes: A first network layer, where the first network layer includes: a first data call point and a first data return point, and the first data call point and the first data return point are associated with the first interface; A second network layer, where the second network layer includes: a second data call point and a second data return point, and the second data call point and the second data return point are associated with the first interface.

19. An electronic device, characterized in that, Includes: One or more processors; A memory for storing one or more programs; When the one or more programs are executed by the one or more processors, the one or more processors are caused to implement the method according to any one of claims 1-5, or execute the method according to any one of claims 6-9.

20. A computer-readable storage medium, characterized in that, Includes computer program instructions, when the instructions are run by an electronic device, the electronic device is caused to execute the method according to any one of claims 1-5, or execute the method according to any one of claims 6-9.

21. A computer program product comprising instructions, characterized in that, When the instruction is run by an electronic device, the electronic device is caused to execute the method according to any one of claims 1 to 5, or execute the method according to any one of claims 6 to 9.

Images