Model reasoning method and device

Abstract

The invention discloses a model reasoning method and device, and relates to the technical field of machine learning. And the client and the server adopt respective deployed models to process different parts of the user data so as to obtain respective output results. And the client obtains an output result of the server and obtains a reasoning result in combination with an output result of the client. Compared with the situation that the server side needs to obtain all the user data in the reasoning process, the server side only obtains part of the user data, and the server side cannot obtain all the content contained in the user data according to the part of the user data, so that guarantee is provided for the safety of the user data. And the client only needs to send a part of the user data to the server, so that bandwidth resources occupied by data transmission between the client and the server and consumed transmission time can be reduced, and the reasoning efficiency is improved.

Description

[0001] This application claims the priority of the Chinese patent application filed with the State Intellectual Property Office on June 8, 2023, with application number 202310680368.6 and invention name “A model interaction method and model management platform based on secure multi-party computing technology”. The entire contents of the application are incorporated by reference in this application. Technical Field

[0002] The present application relates to the field of machine learning technology, and in particular to a method and device for model reasoning. Background Art

[0003] Neural network models can achieve prediction and reasoning of text, images, speech, multi-modality, etc. The processing device divides the neural network model into different parts and deploys each part to the server and client. The server and client use their own models to process user data to obtain output results. The server and client exchange their output results to achieve data prediction and reasoning. In the process of predicting and reasoning data using the above method, the server needs to obtain user data, resulting in the security of user data not being guaranteed. Summary of the invention

[0004] The present application provides a model reasoning method and device for solving the problem of user data insecurity during server and client reasoning.

[0005] In the first aspect, the present application provides a method for model reasoning. The method can be executed by a client deployed on a terminal, and the method includes: the client obtains a first processing result. The first processing result is used to indicate the data obtained by the terminal using the first model to process the user data, and the first model is deployed on the terminal. The client splits the first processing result to obtain the first data and the second data. The client receives part of the model parameters of the second model, and the client processes the first data based on the part of the model parameters to obtain the second processing result. The second model is deployed on the server. The client sends the second data to the server, and receives the third processing result sent by the server. The third processing result is used to indicate the result of the server using the second model to process the second data. The client obtains the inference result based on the second processing result and the third processing result.

[0006] In this application, the client transmits part of the user data (such as the second data) to the server, and obtains the inference result based on the result of the server processing the part of the user data (such as the third processing result) and the result of the client processing another part of the user data (such as the first data) (such as the second processing result). In the case that the server cannot obtain all the contents contained in the user data, the client can still obtain the complete inference result, which effectively improves the security of the user data in the model inference process.

[0007] In another possible implementation manner, the first data is used to indicate: data related to the user's own information.

[0008] Data security can be further ensured by avoiding sending data related to the user's own information or proprietary data to the server and instead processing the data on the terminal side.

[0009] In another possible implementation, before sending the second data to the server, the client selects one or more target encryption algorithms that match the second data from a plurality of preset encryption algorithms. And the client encrypts the second data according to the selected one or more target encryption algorithms to obtain one or more groups of data to be transmitted. Each group of data to be transmitted includes a key corresponding to a target encryption algorithm and data corresponding to the key, and the data corresponding to the key is a part of the second data.

[0010] In another possible implementation, before obtaining the inference result, the client sends the third data to other devices and receives the fourth processing result obtained by other devices processing the third data. The client outputs the inference result according to the second processing result, the third processing result and the fourth processing result.

[0011] In another possible implementation, the client obtains the first processing result, including: the client inputs user data into the first model to obtain the first processing result, wherein the content described by the user data is consistent with the content described by the first processing result.

[0012] In another possible implementation, the second model is a large model.

[0013] In another possible implementation manner, the type of user data includes: at least one of text, image, audio and video, or a combination of several of them.

[0014] In a second aspect, the present application provides another method for model reasoning. The method includes: a client obtains a first processing result. The first processing result is used to indicate the data obtained by the terminal where the client is located using the first model to process the user data, and the first model is deployed at the terminal where the client is located. The client splits the first processing result to obtain first data and second data. The server sends part of the model parameters of the second model to the client. The second model is deployed at the server. The client receives part of the model parameters, and processes the first data based on the part of the model parameters to obtain a second processing result. The client sends the second data to the server, and receives a third processing result sent by the server. The third processing result is used to indicate the result of the server using the second model to process the second data. The client obtains an inference result based on the second processing result and the third processing result.

[0015] In this application, the client and the server process different parts of the user data and obtain their respective output results. The client obtains the output result of the server and obtains the inference result in combination with the output result of the client. Compared with the server needing to obtain all the user data during the inference process, the server in this application only obtains part of the user data. Since the server cannot obtain all the contents contained in the user data based on part of the user data, it provides a guarantee for the security of the user data. The client only needs to send part of the user data to the server, which can reduce the bandwidth resources and transmission time consumed by transmitting data between the client and the server, and improve the efficiency of model reasoning.

[0016] In a third aspect, the present application provides a model reasoning device, which includes modules for executing the method described in the first aspect or any possible design of the first aspect, and / or modules for executing the method described in the second aspect.

[0017] In a fourth aspect, the present application provides a computing device cluster. The computing device cluster includes: at least one computing device. Each computing device includes a processor and a memory. The processor of at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster performs the operation steps of the method in the first aspect or any possible design of the first aspect, or the computing device cluster performs the operation steps of the method in any possible design of the second aspect.

[0018] In a fifth aspect, the present application provides a computer-readable storage medium. The computer-readable storage medium stores: computer program instructions; when the computer program instructions are executed in a computing device cluster, the computing device cluster executes the operation steps of the method described in the first aspect or any possible implementation of the first aspect, or the computing device cluster executes the operation steps of the method described in the second aspect.

[0019] In a sixth aspect, the present application provides a computer program product comprising instructions. When the instructions are executed by a computing device cluster, the computing device cluster executes the operation steps of the method described in the first aspect or any possible implementation of the first aspect, or causes a computer to execute the operation steps of the method described in the second aspect.

[0020] In a seventh aspect, the present application provides a chip system. The chip system includes a processor for implementing the functions of the client in the method of the first aspect above, and / or for implementing the functions of the server in the method of the second aspect above. In a possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system may be composed of a chip, or may include a chip and other discrete devices.

[0021] The beneficial effects of the third to seventh aspects above can be referred to the description of the first aspect or any one of the embodiments of the first aspect, or the description of the second aspect or any one of the embodiments of the second aspect, and will not be repeated here. Based on the implementation methods provided in the above aspects, the present application can also be further combined to provide more implementation methods. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A schematic diagram of a neural network structure;

[0023] Figure 2 A schematic diagram of the structure of a model reasoning system provided in this application;

[0024] Figure 3 A flowchart of the first model reasoning method provided in this application;

[0025] Figure 4 A schematic diagram of obtaining a first processing result provided by the present application;

[0026] Figure 5 A schematic diagram of a process using triple encryption provided for this application;

[0027] Fig. 6A A schematic diagram of a second data encryption process provided by this application;

[0028] Figure 6B A schematic diagram of another encryption process of second data provided by the present application;

[0029] Figure 7 A flowchart of the second model reasoning method provided in this application;

[0030] Figure 8 A flowchart of the third model reasoning method provided in this application;

[0031] Fig. 9 A flowchart of the fourth model reasoning method provided in this application;

[0032] Fig.10 A schematic diagram of the structure of the first model reasoning device provided in this application;

[0033] Fig.11 A schematic diagram of the structure of a computing device provided for this application;

[0034] Fig.12 A schematic diagram of the structure of a computing device cluster provided for this application;

[0035] Fig.13A schematic diagram of a possible connection method of a computing device cluster provided in the present application. DETAILED DESCRIPTION

[0036] The terms used in the implementation method of this application are only used to explain the specific embodiments of this application, and are not intended to limit this application. In order to make the description of the following embodiments clear and concise, a brief introduction to the relevant technologies of the relevant terms is first given.

[0037] (1) Large model

[0038] A large model refers to a deep neural network model with millions or billions of parameters.

[0039] (2) Neural Network

[0040] A neural network can be composed of neurons, and a neuron can be a s The output of this operation unit satisfies the following formula (1).

[0041]

[0042] Where, s = 1, 2, ... n, n is a natural number greater than 1, W s For x s The weight of the neuron is , and b is the bias of the neuron. f is the activation function of the neuron, which is used to introduce nonlinear characteristics into the neural network and convert the input signal in the neuron into the output signal. The output signal of the activation function can be used as the input of the next layer. A neural network is a network formed by connecting multiple single neurons mentioned above. The weight represents the strength of the connection between different neurons and determines the influence of the input on the output.

[0043] Figure 1 is a schematic diagram of a neural network structure. Figure 1 As shown, the neural network 100 includes X processing layers, where X is an integer greater than or equal to 3. The first layer of the neural network 100 is the input layer 110, which is responsible for receiving input signals. The last layer of the neural network 100 is the output layer 130, which is responsible for outputting the processing results of the neural network. The other layers except the first and last layers are intermediate layers 140, and these intermediate layers 140 together constitute the hidden layer 120. Each intermediate layer 140 in the hidden layer 120 can both receive input signals and output signals. The hidden layer 120 is responsible for processing input signals. Each layer represents a logical level of signal processing. Through multiple layers, data signals can be processed by multiple levels of logic.

[0044] Based on a brief introduction to some concepts that may be involved in the present application, the implementation methods of the present application are described below in conjunction with the accompanying drawings.

[0045] Figure 2 A schematic diagram of the structure of a model reasoning system provided in this application, such as Figure 2 As shown, the model reasoning system 200 includes: a terminal 210, a server 220 and a network. The network can realize the function of data transmission between the terminal 210 and the server 220, and the network can include one or more network devices, which can be routers or switches.

[0046] The terminal 210 is used to obtain the data to be inferred, and to cooperate with the server 220 to obtain the inference result based on the data to be inferred. A client may be installed on the terminal 210, and the terminal 210 implements data interaction with the server 220 through the client. The client may be an application with data receiving and processing functions, such as an agent.

[0047] A network model may be deployed on the terminal 210 to cooperate with the server 220 to obtain an inference result. The network model may include, but is not limited to, a convolutional neural network (CNN) model, a deep convolutional neural network (DCNN) model, a Hopfield network (HN) model, a feedforward neural network (FFNN) model, a BP neural network model, a natural language network model (Transformer, BERT), etc.

[0048] The terminal 210 may be, but is not limited to, a user equipment, a mobile station, a mobile terminal, etc. The terminal may be a mobile phone (such as Figure 2 The terminal 211 shown in FIG. 211 ), a tablet computer (such as Figure 2 ), a computer with wireless transceiver function (such as the terminal 212 shown in Figure 2 ), virtual reality (VR) devices (such as the terminal 213 shown in Figure 2 ), augmented reality (AR) devices, industrial control, monitoring devices in smart homes and smart cities (such as Figure 2 Terminal 215 shown in ) and so on.

[0049] The data type of the data to be inferred may be: text, image, audio, video, multimodal, etc. The data to be inferred may come from different scenarios, such as individual users, medical institutions, financial institutions, governments, smart cities, computer synthesis, etc. The data to be inferred may be stored in advance in the terminal 210, or may be generated in real time during the operation of the terminal 210, or may be transmitted by other devices. When the data to be inferred is stored in advance in the terminal 210, the terminal 210 may include a memory. The memory may be a cache, a solid state drive (SSD), a hard disk drive (HDD), a storage class memory (SCM), a memory or other storage medium, exemplarily, a storage particle storing a specific number of bits: a single level cell (SLC), a multi-level cell (MLC), a triple-level cell (TLC) or a quad level cell (QLC), etc.

[0050] The server 220 is used to cooperate with the terminal 210 to obtain the inference result. A network model can be deployed on the server 220 to cooperate with the terminal 210 to obtain the inference result. The network model can be a large model or a general network model, such as a convolutional neural network (CNN) model. The server 220 can be, but is not limited to: a server 221, a data center 222, a computer 223, and a computer cluster 224. The following describes the case where the server 220 is the above devices.

[0051] In a first possible scenario, the service end 220 is a server 221. The server 221 may be arranged on the terminal side or on the cloud side.

[0052] In the second possible scenario, the server 220 is a data center 222. The data center 222 may include one or more physical devices with computing functions, such as servers, mobile phones, tablet computers, etc. When the data center 222 includes multiple physical devices with computing functions, the multiple physical devices may be set in the same physical location or in different physical locations. When multiple physical devices with computing functions are set in different physical locations, a network may be used to implement data interaction between the physical devices. For the relevant description of the network, please refer to the relevant description above, which will not be repeated here.

[0053] In a third possible scenario, the server 220 is a computer 223. The computer 223 may include a memory, a processor, and one or more interfaces.

[0054] The processor included in the computer 223 processes the data transmitted by the terminal 210 to obtain the processing result. The processor may include one or more processor cores. The processor may be a very large-scale integrated circuit. An operating system and other software programs are installed in the processor, so that the processor can access the memory and various peripheral component interconnect (Peripheral Component Interconnect express, PCIe) devices. It is understood that in this embodiment, the core in the processor may be a central processing unit (CPU). The processor may also be other general-purpose processors, digital signal processors (digital signal processing, DSP), application-specific integrated circuits (application-specific integrated circuit, ASIC), field programmable gate arrays (field programmable gate array, FPGA), graphics processing units (graphics processing units, GPU), AI chips, systems on chip (system-on-a-chip, SoC) or other programmable logic devices, discrete gates or transistor logic devices, discrete hardware components, etc. In practical applications, the computer 223 may also include multiple processors.

[0055] The one or more interfaces included in the computer 223 can receive data transmitted by the terminal 210 , and can also transmit processing results obtained by the processor of the computer 223 to the terminal 210 .

[0056] In a fourth possible scenario, the server 220 is a computer cluster 224 .

[0057] Computer cluster 224 refers to a collection of computers connected via a local area network or the Internet (e.g. Figure 2Computers 2241 to 2244 are shown in the figure, and each computer can obtain the content contained therein according to the received data. Exemplarily, the computer cluster 224 may have a rack, and the rack can establish communication for multiple computers contained in the computer cluster 224 through a wired connection, such as a universal serial bus (USB) or a PCIe high-speed bus. The computer cluster 224 is usually used to execute large tasks (also called jobs), where jobs are usually large jobs that require more resources to be processed in parallel. This embodiment does not limit the nature and number of jobs. A job may contain multiple computing tasks generated during the model reasoning process, and these computing tasks can be assigned to multiple computing resources for execution. Each computer in the computer cluster 224 uses the same hardware and the same operating system; different hardware and different operating systems can also be used in the computers in the computer cluster 224 according to business needs.

[0058] Combination of the above Figure 2 The system of model reasoning provided by this application is described below. Figure 3 Describe the model reasoning method provided by this application, Figure 3 A schematic diagram of the flow chart of the first model reasoning method provided in this application, which can be Figure 2 The described client can be executed, or the client and the terminal can be executed in cooperation, or the client, the terminal and the server can be executed in cooperation. The following is an example of the method being executed by the client. The method includes the following steps S310 to S350.

[0059] Step S310: The client obtains a first processing result.

[0060] Figure 4 A schematic diagram of obtaining a first processing result provided by the present application is as follows: Figure 4 As shown, the first processing result is used to indicate data obtained by the terminal where the client is located by processing user data using the first model, and the first model is deployed in the terminal.

[0061] The types of user data may include, but are not limited to: text, image, audio, video, multi-modality, etc. The user data may be as shown in Table 1.

[0062] Table 1

[0063]

[0064] The source of user data may include, but is not limited to, one or more of the following possible scenarios.

[0065] Scenario 1: User data is generated during the operation of the terminal, such as text data generated during the operation of a company.

[0066] Scenario ②: User data is transmitted by other devices, such as other devices transmitting text data related to the company's business operations stored in them to the terminal.

[0067] Scenario ③: User data is synthesized by terminal calculation, such as the terminal using the terminal's computing and storage resources to synthesize text data.

[0068] In a possible scenario, the terminal inputs user data into the first model to obtain a first processing result, wherein the content described by the user data is consistent with the content described by the first processing result.

[0069] Exemplarily, the terminal inputs the user data in Table 1 to the first model, and the first model outputs the first processing result shown in Table 2.

[0070] Table 2

[0071]

[0072] Step S320: The client splits the first processing result to obtain first data and second data.

[0073] In a possible scenario, the client may split the data related to the user's own information in the first processing result into the first data, and the other data into the second data. Exemplarily, the client splits the first processing result x shown in Table 2 into the first data x1 shown in Table 3 and the second data x2 shown in Table 4.

[0074] Table 3

[0075]

[0076] Table 4

[0077]

[0078] The above describes a situation in which the client splits the first processing result into two data (such as first data and second data). In some possible examples, the client can split the first processing result into three or more data, such as the client splits the first processing result to obtain first data, second data and third data.

[0079] Step S330: The client receives partial model parameters of the second model, processes the first data based on the partial model parameters, and obtains a second processing result.

[0080] Among them, the second model is deployed on the server. The second model can be a large model deployed on the server. The large model contains more model parameters than the ordinary model, and the server uses the large model to process the second data to obtain the second processing result. Compared with the second data that can be obtained by using the ordinary model, it contains more content, so that the reasoning result obtained by using the second data reasoning is more in line with the user's expectations. In one possible scenario, before the client receives some model parameters of the second model, the server sends some model parameters of the second model to the client.

[0081] Exemplarily, the server is deployed with Model 2, which includes: model parameter set y1 and model parameter set y2. The server sends the model parameter set y1 of Model 2 to the client. The client receives the model parameter set y1, and uses y1 to process the first data x1 to obtain a second processing result, such as the second processing result is that among the users who purchase the product: women account for a larger proportion, more live in first-tier cities, etc. For example, Model 2 includes 1 million model parameters, Model parameter set 1 includes 10,000 model parameters, Model parameter set 2 includes 990,000 model parameters, and the client receives the 10,000 model parameters contained in Model parameter set 1 sent by the server. The client uses the 10,000 model parameters to obtain the second processing result according to x2.

[0082] In a possible scenario, the first model and the second model are models of the same data type, such as the first model and the second model are models for processing text data. The first model and the second model may come from the same service provider or from different service providers. In the case where the first model and the second model come from different service providers, the different service providers may come from the same manufacturer or from different manufacturers. This application is not limited to this.

[0083] Step S340: The client sends the second data to the server, and receives the third processing result sent by the server.

[0084] The third processing result is used to indicate the result of the server processing the second data using the second model.

[0085] Exemplarily, the server is deployed with model 2, and the server inputs the second data x2 into model 2 deployed on the server. Model 2 processes the second data x2 using model parameter set y2, and outputs a third processing result. For example, the third processing result describes the user's evaluation of the product, for example, the product has more positive evaluations.

[0086] In one possible scenario, before the client sends the second data to the server, the client selects one or more target encryption algorithms that match the second data from a plurality of preset encryption algorithms. The client encrypts the second data according to the selected one or more target encryption algorithms to obtain one or more groups of data to be transmitted, each group of data to be transmitted includes a key corresponding to a target encryption algorithm and data corresponding to the key, and the data corresponding to the key is a part of the second data. The client encrypts the second data using an encryption method that matches the second data to avoid leakage of the second data, which results in insecurity of user data.

[0087] The target encryption algorithm may include, but is not limited to: secret key sharing algorithm, differential privacy algorithm, homomorphic encryption algorithm, obfuscated circuit algorithm, etc. Secret sharing algorithm is a method of distributing, saving, and recovering secret keys (or other secret information). The key manager splits the secret key into a series of interrelated secret information (called subkeys), and then distributes the subkeys to each member of a group; so that after each member of a certain group (authorized set) takes out their subkeys, they can use the established method to recover the secret key, while other groups (unauthorized sets) cannot recover the key, such as triple key pairs. Differential privacy algorithm refers to. Homomorphic encryption (HE) algorithm refers to the original data after homomorphic encryption, and then performs a specific operation on the ciphertext. The plaintext obtained after homomorphic decryption is equivalent to the data result obtained by directly performing the same calculation on the original plaintext data, such as gentry method, BGV method, BFV method, etc. The circuit obfuscation algorithm refers to inserting some random logic gates into the circuit to confuse the circuit structure, making it difficult for attackers to obtain the circuit structure through reverse engineering, thereby achieving the confidentiality of the circuit structure.

[0088] Taking the target encryption algorithm as the triplet key pair in the key sharing algorithm as an example, the process of the client processing the second data to obtain the third processing result and the server processing the first data to obtain the second processing result is explained below. The process includes the following steps 1 to 12. Figure 5 A schematic diagram of a process using triple encryption provided for this application, such as Figure 5 As shown, triple encryption includes steps 1 to 3.

[0089] Step 1. The client generates a key pair triplet.

[0090] For example, the client generates a key pair triple c=a*b.

[0091] Step 2: The client splits the key pair triplet.

[0092] For example, the client splits the key pair triple c=a*b into: c1=a1*b1 and c2=a2*b2, where c=c1+c2.

[0093] Step 3: The client uses the split key pair triplet to encrypt the first data and the second data, and transmits the key for decrypting the second data to the server.

[0094] For example, the client uses c1=a1*b1 to encrypt the model parameter set y1 and the first data x1, and uses c2=a2*b2 to encrypt the model parameter set y2 and the second data x2, to obtain one or more groups of data to be transmitted. The client sends the encrypted second data x2 and c2=a2*b2 to the server. For more information about the triple encryption algorithm, please refer to the relevant description of the general technology, which will not be repeated here.

[0095] Step 4: The client uses a1 to decrypt the encrypted first data x1 to obtain m1, and uses b1 to decrypt the model parameter set y1 to obtain n1, such as m1=x1-a1, n1=y1-b1.

[0096] Step 5: The server uses a2 to decrypt the encrypted second data x2 to obtain m2, and uses b2 to decrypt the model parameter set y2 to obtain n2, such as m2=x2-a2, n2=y2-b2.

[0097] Step 6. The client and server exchange m1, n1 and m2, n2.

[0098] Step 7: The client calculates m01 using m1 and m2, and calculates n01 using n1 and n2, such as m01=m1+m2, n01=n1+n2.

[0099] Step 8. The server side calculates m11 using m1 and m2, and calculates n11 using n1 and n2, such as m11=m1+m2, n11=n1+n2.

[0100] Step 9. The client uses c1, a1, b1, m01 and n01 to calculate r1, such as r1=c1+m01*b1+n01*a1+m01*n01.

[0101] Step 10. The server uses c2, a2, b2, m11 and n11 to calculate r2, such as r2=c2+m11*b1+n11*a1.

[0102] Step 11. The client and server exchange r1 and r2.

[0103] Step 12: The client obtains the second processing result by using r1 and r2, and the server obtains the third processing result by using r1 and r2.

[0104] In some possible situations, the client and the server can use third-party devices to implement the encrypted data transmission process. Fig. 6A As shown, Fig. 6A This is a schematic diagram of a second data encryption process provided by the present application, wherein the third device may be a security device. Figure 6B As shown, Figure 6B Another encryption process diagram of the second data provided by the present application, the third device is an agent. A terminal agent is installed on the terminal, and a server agent is installed on the server. The terminal agent and the server agent are used to implement the function of secure transmission of data between the terminal and the server. The terminal agent and the server agent can be connected through a secure multi-party computation (MPC) coordinator. The process of encrypting the second data in this way includes the following steps ① to ⑩.

[0105] Step 1: The terminal establishes a link with the MPC coordinator through the terminal agent, and the MPC coordinator starts the server agent.

[0106] Step 2: The server agent initializes the secure operating environment.

[0107] Step ③: The server proxy splits the model parameters included in the second model into: model parameter set 1 and model parameter set 2. The server proxy sends the model parameter set 2 to the terminal proxy.

[0108] Step ④: The terminal agent adopts the above Figure 5 The key pair triplet is generated in the described manner, and the generated key pair triplet is split and transmitted to the MPC coordinator.

[0109] Step 5: The MPC coordinator transmits the received key pair triplet to the server agent.

[0110] Step ⑥: The server agent receives the key pair triplet.

[0111] Step 7: The terminal agent splits the user data into multiple data including the first data and the second data, and transmits the second data to the MPC coordinator, and the MPC coordinator transmits the second data to the server agent.

[0112] Step ⑧: The terminal agent obtains a second processing result based on the first data.

[0113] Step 9: The server agent obtains a third processing result according to the second data, and sends the third processing result to the MPC coordinator, and the MPC coordinator transmits the third processing result to the terminal agent.

[0114] Step ⑩: The terminal agent obtains an inference result based on the second processing result and the third processing result.

[0115] In one possible scenario, the terminal and the server may use hardware devices to implement encrypted transmission and processing of data. For example, the terminal and the server establish a dedicated channel to achieve data transmission between the terminal and the server. Another example is that the server installs a processing device for data processing. This application does not limit the encryption method and processing method of data. Users can choose software or hardware to encrypt, transmit or process data according to specific needs.

[0116] Step S350: The client obtains an inference result according to the second processing result and the third processing result.

[0117] Exemplarily, the client adds the second processing result and the third processing result described above to obtain an inference result. Two possible examples of inference results are given below.

[0118] For example, the inference result may be a description of user behavior, such as most users' evaluations of a product are positive.

[0119] For another example, the inference result can also be a description of user needs. For example, different age groups have different demands for products. Older customers (such as those over 25 years old) care more about product quality, cost-effectiveness, taste, etc., while younger customers (such as those 25 years old and below) care more about the appearance of the product. Most of the products purchased are women, and most of the customers live in first-tier cities.

[0120] In one possible scenario, Figure 7 A flow chart of the second model reasoning method provided in this application, such as Figure 7 As shown, before the client obtains the inference result, the client sends the third data to other devices, and receives the fourth processing result obtained by other devices processing the third data. And the client obtains the inference result based on the second processing result, the third processing result and the fourth processing result. The other device can be a terminal or a client, and this application is not limited to this. In this case, the other device receives part of the model parameters of the second model sent by the server, and uses the part of the model parameters to process the third data to obtain the fourth processing result. Exemplarily, model 2 includes: model parameter set y1, model parameter set y2 and model parameter set y3. Other devices receive the model parameter set y3 sent by the server, and use y3 to process the third data to obtain the fourth processing result. For the specific content, please refer to the relevant description above, which will not be repeated here.

[0121] In this application, the client transmits part of the user data (such as the second data) to the server, and obtains the inference result based on the result of the server processing the part of the user data (such as the third processing result) and the result of the client processing another part of the user data (such as the first data) (such as the second processing result). The server cannot obtain all the contents contained in the user data, and the inference result is obtained by the client, which provides a guarantee for the security of the user data.

[0122] The above describes the method of model reasoning by taking the method of executing model reasoning by the client deployed in the terminal as an example. In some possible examples, the method of model reasoning can also have the following two possible examples of execution modes.

[0123] In a first possible example, the model reasoning method may be executed by a terminal. In this case, different from the client execution, steps S310 to S350 are all executed by the terminal.

[0124] In a second possible example, the model reasoning method may also be executed by the terminal and a client deployed on the terminal in cooperation. Figure 8 A flowchart of the third model reasoning method provided in this application is as follows: Figure 8 As shown, the difference from the client execution is that: step S310 is for the terminal to obtain a first processing result, the first processing result is used to indicate the data obtained by the terminal using the first model to process the user data, and the first model is deployed in the terminal. Figures 3 to 8 The data reasoning method provided by this application is described below. Fig. 9 Another model reasoning method provided by the present application is described. Fig. 9 A flowchart of the fourth model reasoning method provided in the present application, which can be executed by the client and the server in cooperation. The difference between this method and the method described above is that before step S330 described above, the method also includes step S330A that the server sends some model parameters of the second model to the client. Among them, the second model is deployed on the server. And before the client receives the third processing result sent by the server in step S340, the method also includes step S340A that the server receives the second data, processes the second data using the second model, obtains the third processing result, and sends the third processing result to the client. For other relevant contents of the method, please refer to the relevant description above, which will not be repeated here.

[0125] In this application, the server receives part of the encrypted user data (such as the second data) and processes the part of the user data using the second model. This reduces the computing resources consumed by the client to calculate the part of the user data, reduces the time consumed by the client to obtain the inference results, and improves efficiency. In addition, part of the user data received by the server is encrypted information, which provides a guarantee that the user data is only obtained by the device allowed by the user.

[0126] Combination of the above Figures 3 to 9 The model reasoning method provided by this application is described below. Fig.10 The device of the first model provided in this application is described. Fig.10 This is a schematic diagram of the structure of the first model reasoning device provided by the present application. The device 1000 includes: a processing module 1010, a transceiver module 1020 and an encryption module 1030. The device 1000 can implement Figure 3 and Figure 4 The functions of the terminal in the method embodiment are described.

[0127] The transceiver module 1020 is used to obtain a first processing result. The processing module 1010 is used to split the first processing result to obtain first data and second data. The transceiver module 1020 is also used to receive some model parameters of the second model. The processing module 1010 is also used to process the first data based on some model parameters to obtain a second processing result. The transceiver module 1020 is also used to send the second data to the server and receive the third processing result sent by the server. The third processing result is used to indicate the result of the server processing the second data using the second model. The processing module 1010 is also used to obtain an inference result based on the second processing result and the third processing result.

[0128] In a possible scenario, the transceiver module 1020 is further used to: send the third data to other devices, and receive the fourth processing result obtained by other devices processing the third data. The processing module 1010 is further used to: output the inference result according to the second processing result, the third processing result and the fourth processing result.

[0129] In a possible scenario, the encryption module 1030 is used to select one or more target encryption algorithms that match the second data from a plurality of preset encryption algorithms. The encryption module 1030 is also used to encrypt the second data according to the selected one or more target encryption algorithms to obtain one or more groups of data to be transmitted. Each group of data to be transmitted includes a key corresponding to a target encryption algorithm and data corresponding to the key, and the data corresponding to the key is a part of the second data.

[0130] In one possible scenario, the second model is a large model.

[0131] In a possible scenario, the type of user data includes: at least one of text, image, audio and video, or a combination of several of them.

[0132] A more detailed description of the processing module 1010, the transceiver module 1020 and the encryption module 1030 can be directly obtained by referring to the relevant description in the method embodiment described above, and will not be repeated here.

[0133] Among them, the processing module 1010, the transceiver module 1020 and the encryption module 1030 can all be implemented by software, or can be implemented by hardware. Exemplarily, the implementation of the processing module 1010 is introduced below by taking the processing module 1010 as an example. Similarly, the implementation of the transceiver module 1020 and the encryption module 1030 can refer to the implementation of the processing module 1010.

[0134] As an example of a software functional unit, the processing module 1010 may include code running on a computing instance. Among them, the computing instance may include at least one of a physical host (computing device), a virtual machine, and a container. Further, the above-mentioned computing instance may be one or more. For example, the processing module 1010 may include code running on multiple hosts / virtual machines / containers. It should be noted that the multiple hosts / virtual machines / containers used to run the code can be distributed in the same region (region) or in different regions. Furthermore, the multiple hosts / virtual machines / containers used to run the code can be distributed in the same availability zone (AZ) or in different AZs, each AZ including a data center or multiple data centers with similar geographical locations. Among them, usually a region can include multiple AZs.

[0135] Similarly, multiple hosts / virtual machines / containers used to run the code can be distributed in the same virtual private cloud (VPC) or in multiple VPCs. Usually, a VPC is set up in a region. For cross-region communication between two VPCs in the same region and between VPCs in different regions, a communication gateway needs to be set up in each VPC to achieve interconnection between VPCs through the communication gateway.

[0136] As an example of a hardware functional unit, the processing module 1010 may include at least one computing device, such as a server, etc. Alternatively, the module A may also be a device implemented using an application-specific integrated circuit (ASIC) or a programmable logic device (PLD). The PLD may be a complex programmable logical device (CPLD), a field-programmable gate array (FPGA), a generic array logic (GAL) or any combination thereof.

[0137] The multiple computing devices included in the processing module 1010 can be distributed in the same region or in different regions. The multiple computing devices included in the processing module 1010 can be distributed in the same AZ or in different AZs. Similarly, the multiple computing devices included in the processing module 1010 can be distributed in the same VPC or in multiple VPCs. The multiple computing devices can be any combination of computing devices such as servers, ASICs, PLDs, CPLDs, FPGAs, and GALs.

[0138] It should be noted that, in other embodiments, the processing module 1010 can be used to execute any step in the model reasoning method, the transceiver module 1020 can be used to execute any step in the model reasoning method, and the encryption module 1030 can be used to execute any step in the model reasoning method. The steps that the processing module 1010, the transceiver module 1020 and the encryption module 1030 are responsible for implementing can be specified as needed. The processing module 1010, the transceiver module 1020 and the encryption module 1030 respectively implement different steps in the model reasoning method to realize the full functions of the model reasoning device.

[0139] The embodiment of the present application also provides a computing device cluster. The computing device cluster includes at least one computing device. The computing device can be a server, such as a central server, an edge server, or a local server in a local data center. In some embodiments, the computing device can also be a terminal device such as a desktop computer, a laptop computer, or a smart phone.

[0140] Fig.11 A schematic diagram of a computing device provided in this application, such as Fig.11As shown, the computing device 1100 includes: a bus 1102, a processor 1104, a memory 1106, and a communication interface 1108. The processor 1104, the memory 1106, and the communication interface 1108 communicate through the bus 1102. The computing device 1100 can be a server or a terminal device. It should be understood that the present application does not limit the number of processors and memories in the computing device 1100.

[0141] The bus 1102 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Fig.11 The bus 1102 may include a path for transmitting information between various components of the computing device 1100 (eg, the memory 1106, the processor 1104, and the communication interface 1108).

[0142] The processor 1104 may include any one or more of a central processing unit (CPU), a graphics processing unit (GPU), a microprocessor (MP), or a digital signal processor (DSP).

[0143] The memory 1106 may include a volatile memory, such as a random access memory (RAM). The processor 1104 may also include a non-volatile memory, such as a read-only memory (ROM), a flash memory, a hard disk drive (HDD), or a solid state drive (SSD).

[0144] The memory 1106 stores executable program codes, and the processor 1104 executes the executable program codes to respectively implement the functions of the pre-processing module 1010, the transceiver module 1020, and the encryption module 1030, thereby implementing the model reasoning method. That is, the memory 1106 stores instructions for executing the model reasoning method.

[0145] The communication interface 1103 uses a transceiver module such as, but not limited to, a network interface card or a transceiver to implement communication between the computing device 1100 and other devices or a communication network.

[0146] Fig.12 A schematic diagram of the structure of a computing device cluster provided in this application, such as Fig.12 As shown, the computing device cluster includes at least one Fig.11 The computing device 1100 described. In the case where the computing device cluster includes two computing devices, and the two computing devices are respectively a server and a terminal with a client deployed thereon, the computing device cluster may constitute Figure 2 The system of model reasoning described, in other words, Figure 2 The system described for model inference is Fig.12 A specific example of a cluster of computing devices is shown.

[0147] The memory 1106 in one or more computing devices in the computing device cluster may store the same instructions for executing the model inference method.

[0148] In some possible implementations, the memory 1106 of one or more computing devices in the computing device cluster may also store partial instructions for executing the method for model reasoning. In other words, a combination of one or more computing devices may jointly execute instructions for executing the method for model reasoning.

[0149] In some other possible implementations, the memories in different computing devices in the computing device cluster may store different instructions, which are respectively used to execute part of the functions of the device for model reasoning. That is, the instructions stored in the memories in different computing devices may implement the functions of one or more modules in the processing module 1010, the transceiver module 1020, and the encryption module 1030.

[0150] In some possible implementations, one or more computing devices in the computing device cluster may be connected via a network, which may be a wide area network or a local area network. Fig.13 A schematic diagram of a possible connection method of a computing device cluster provided in this application, such as Fig.13 As shown, two computing devices 1100A and 1100B are connected via a network. Specifically, the network is connected via a communication interface in each computing device. In this type of possible implementation, the memory in the computing device 1100A stores instructions for executing the functions of the processing module 1010. At the same time, the memory in the computing device 1100B stores instructions for executing the functions of the transceiver module 1020 and the encryption module 1030.

[0151] Fig.13The connection method between the computing device clusters shown can be considered to be that the model inference method provided in this application needs to store a large amount of user data and process the user data to obtain the first processing result, so it is considered to hand over the functions implemented by the transceiver module 1020 and the encryption module 1030 to the computing device 1100B for execution.

[0152] It should be understood that Fig.13 The functions of the computing device 1100A shown in FIG. 1 may also be completed by multiple computing devices. Similarly, the functions of the computing device 1100B may also be completed by multiple computing devices.

[0153] The present application embodiment also provides another computing device cluster. The connection relationship between the computing devices in the computing device cluster can be similar to that of Fig.11 and Fig.13 The connection mode of the computing device cluster is different in that the memory of one or more computing devices in the computing device cluster may store the same instructions for executing the model reasoning method.

[0154] In some possible implementations, the memory of one or more computing devices in the computing device cluster may also store partial instructions for executing the method for model reasoning. In other words, the combination of one or more computing devices can jointly execute the instructions for executing the method for model reasoning.

[0155] The memories in different computing devices in the computing device cluster can store different instructions for executing partial functions of the model reasoning system. That is, the instructions stored in the memories in different computing devices can implement one or more functions deployed in the client and server of the terminal.

[0156] The method steps in this embodiment can be implemented by hardware or by a processor executing software instructions. The software instructions can be composed of corresponding software modules, which can be stored in random access memory (RAM), flash memory, read-only memory (ROM), programmable ROM (PROM), erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), registers, hard disks, mobile hard disks, CD-ROMs, or any other form of storage medium known in the art. An exemplary storage medium is coupled to the processor so that the processor can read information from the storage medium and write information to the storage medium. Of course, the storage medium can also be a component of the processor. The processor and the storage medium can be located in an ASIC. In addition, the ASIC can be located in a computing device. Of course, the processor and the storage medium can also exist as discrete components in a network device or a terminal device.

[0157] The present application also provides a chip system, which includes a processor for implementing the functions of the client and / or server in the above method. In one possible design, the chip system also includes a memory for storing program instructions and / or data. The chip system can be composed of a chip, or can include a chip and other discrete devices.

[0158] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer programs or instructions. When the computer program or instruction is loaded and executed on a computing device, the process or function described in the embodiment of the present application is executed in whole or in part. The computing device may be a general-purpose computer, a special-purpose computer, a computer network, a network device, a user device or other programmable device. The computer program or instruction may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer program or instruction may be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired or wireless means. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server, data center, etc. that integrates one or more available media. The available medium may be a magnetic medium, for example, a floppy disk, a hard disk, a tape; it may also be an optical medium, for example, a digital video disc (DVD); it may also be a semiconductor medium, for example, a solid state drive (SSD).

[0159] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A method for model reasoning, characterized in that: The method comprises: Obtaining a first processing result; wherein the first processing result is used to indicate data obtained by the terminal by processing user data using a first model, and the first model is deployed in the terminal; Splitting the first processing result to obtain first data and second data; receiving partial model parameters of a second model, and processing the first data based on the partial model parameters to obtain a second processing result; wherein the second model is deployed on a server; Sending the second data to a server, and receiving a third processing result sent by the server; wherein the third processing result includes a result of the server processing the second data using the second model; An inference result is obtained according to the second processing result and the third processing result.

2. The method according to claim 1, characterized in that: The first data is used to indicate: data related to the user's own information.

3. The method according to claim 1 or 2, characterized in that: Before sending the second data to the server, the method further includes: Selecting one or more target encryption algorithms that match the second data from a plurality of preset encryption algorithms; The second data is encrypted according to one or more selected target encryption algorithms to obtain one or more groups of data to be transmitted, each group of data to be transmitted includes a key corresponding to a target encryption algorithm and data corresponding to the key, and the data corresponding to the key is part of the second data.

4. The method according to any one of claims 1 to 3, characterized in that Before obtaining the inference result, the method further includes: Sending third data to another device, and receiving a fourth processing result obtained by the other device processing the third data; The obtaining of an inference result according to the second processing result and the third processing result includes: The inference result is outputted according to the second processing result, the third processing result and the fourth processing result.

5. The method according to any one of claims 1 to 4, characterized in that The types of the user data include: at least one of text, image, audio and video, or a combination of several of them.

6. A method for model reasoning, characterized in that: The client obtains a first processing result; wherein the first processing result is used to indicate data obtained by the terminal where the client is located by processing user data using a first model, and the first model is deployed in the terminal; The client splits the first processing result to obtain first data and second data; The server sends part of the model parameters of the second model to the client; wherein the second model is deployed on the server; The client receives the partial model parameters, processes the first data based on the partial model parameters, and obtains a second processing result; The client sends the second data to the server, and receives a third processing result sent by the server; wherein the third processing result is used to indicate the result of the server processing the second data using the second model; The client obtains an inference result according to the second processing result and the third processing result.

7. A device for model reasoning, characterized in that: The device comprises: A transceiver module, used to: obtain a first processing result; wherein the first processing result is used to indicate data obtained by the device by processing user data using a first model, and the first model is deployed in the device; A processing module, used to: split the first processing result to obtain first data and second data; The processing module is further used to: receive partial model parameters of a second model, and process the first data based on the partial model parameters to obtain a second processing result; wherein the second model is deployed on the server; The transceiver module is further used to: send the second data to the server, and receive a third processing result sent by the server; wherein the third processing result includes a result of the server processing the second data using the second model; The processing module is further used to obtain an inference result according to the second processing result and the third processing result.

8. The device according to claim 7, characterized in that The first data is used to indicate: data related to the user's own information.

9. The device according to claim 7 or 8, characterized in that The device also includes: An encryption module, used to: select one or more target encryption algorithms matching the second data from a plurality of preset encryption algorithms; The encryption module is also used to: encrypt the second data according to one or more selected target encryption algorithms to obtain one or more groups of data to be transmitted, each group of data to be transmitted includes a key corresponding to a target encryption algorithm and data corresponding to the key, and the data corresponding to the key is part of the second data.

10. The device according to any one of claims 7 to 9, characterized in that: The transceiver module is further used to: send third data to other devices, and receive a fourth processing result obtained by the other devices processing the third data; The processing module is further used to output the inference result according to the second processing result, the third processing result and the fourth processing result.

11. The device according to any one of claims 8 to 10, characterized in that The types of the user data include: at least one of text, image, audio and video, or a combination of several of them.

12. A device for model reasoning, characterized in that: The device comprises: The transceiver module is used for: the client obtains a first processing result; wherein the first processing result is used to indicate data obtained by the device by processing user data using a first model, and the first model is deployed in the device; A processing module, configured to: split the first processing result by the client to obtain first data and second data; The transceiver module is further used for: the server sends part of the model parameters of the second model to the client; wherein the second model is deployed on the server; The transceiver module is further used for: the client receiving the partial model parameters, processing the first data based on the partial model parameters, and obtaining a second processing result; The transceiver module is further used for: the client sending the second data to the server, and receiving a third processing result sent by the server; wherein the third processing result is used to indicate the result of the server processing the second data using the second model; The processing module is further used for: the client obtaining an inference result according to the second processing result and the third processing result.

13. A computing device cluster, characterized in that: comprising at least one computing device, each computing device comprising a processor and a memory; The processor of the at least one computing device is used to execute instructions stored in the memory of the at least one computing device, so that the computing device cluster executes the method according to any one of claims 1 to 5, or, so that the computing device cluster executes the method according to claim 6.

14. A computer-readable storage medium, characterized in that: The computer readable storage medium stores computer program instructions; When the computer program instructions are executed in a computing device cluster, the computing device cluster is caused to execute the method of any one of claims 1 to 5, or the computing device cluster is caused to execute the method of claim 6.

15. A computer program product comprising instructions, characterized in that When the instructions are executed by a computing device cluster, the computing device cluster executes the method according to any one of claims 1 to 5, or the computing device cluster executes the method according to claim 6.

Images