A code completion method, device and related equipment

Abstract

The present application provides a code completion method, including: obtaining a local data set and initial model parameters sent by an aggregation server; training a local model according to the initial model parameters and the local data set to obtain local model parameters; encrypting the local model parameters and sending them to the aggregation server, so that the aggregation server can aggregate the local model parameters sent by multiple parties to obtain global model parameters; obtaining global model parameters, updating the local model according to the global model parameters and the local model parameters, and obtaining a trained code completion model; obtaining the code input by the user, inputting the code into the code completion model, and obtaining the target code. The embodiment of the present application effectively solves the problem that the data set used in the previous code completion process is a public data set and lacks data privacy protection, and can provide personalized code completion services to speed up the process of software development.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a code completion method, device and related equipment. Background Art

[0002] With the increasing popularity of computer network technology, code completion is one of the most important functions of Integrated Development Environments (IDEs). Code completion can reduce the workload of programmers and speed up the process of software development.

[0003] At present, the main method of code completion is to first obtain a large amount of data from open source code bases (such as open source projects on GitHub) or open source communities as a corpus for deep learning, use code parsers to process the source code, such as converting the source code into a token sequence or an abstract syntax tree, and then select or design a suitable deep neural network to train the data in the corpus. In this process, the data set used is a public data set, which usually lacks data privacy protection, and cannot provide personalized code completion services based on the programming habits of IDEs users. Summary of the invention

[0004] The present application provides a code completion method, apparatus and related equipment, which can effectively solve the problem that the data set used in the previous code completion process is a public data set, lacks data privacy protection and cannot provide personalized code completion services.

[0005] In a first aspect, the present application provides a code completion method, which is applied to a first participant, and the first participant is any participant in a system including an aggregation server and multiple participants, including: obtaining a local data set and an initial model parameter sent by the aggregation server, wherein the local data set includes multiple source codes; training a local model according to the initial model parameters and the local data set to obtain local model parameters; encrypting the local model parameters and sending them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by multiple participants to obtain global model parameters; obtaining the global model parameters, updating the local model according to the global model parameters and the local model parameters to obtain a trained code completion model; obtaining the code input by the user, and inputting the code into the code completion model to obtain the target code.

[0006] In the second aspect, an embodiment of the present application provides a code completion device, including: an acquisition unit, used to acquire a local data set and an initial model parameter sent by an aggregation server, wherein the local data set includes multiple source codes; a local training unit, used to train a local model according to the initial model parameters and the local data set to obtain local model parameters; a sending unit, used to encrypt the local model parameters and send them to the aggregation server, so that the aggregation server can aggregate the local model parameters sent by multiple participants to obtain global model parameters; an updating unit, used to acquire the global model parameters, and update the local model according to the global model parameters and the local model parameters to obtain a trained code completion model; a processing unit, used to acquire the code input by the user, input the code into the code completion model, and obtain the target code.

[0007] In a third aspect, an embodiment of the present application provides a code completion device, including: a processor and a memory, wherein the processor executes the code in the memory to execute the method provided in the first aspect or any one of the implementations of the first aspect.

[0008] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, comprising instructions, which, when executed on a computer, enable the computer to execute the method provided in the first aspect or any one of the implementations of the first aspect.

[0009] By adopting a federated learning architecture, the embodiment of the present application can train an efficient code completion model together while ensuring that the codes of each participant are not made public. The model can provide more personalized code completion services based on the coding habits of each participating user, meet various needs in the development and programming process, and further accelerate the software development process. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] In order to more clearly illustrate the technical solutions in the embodiments of the present application or the background technology, the drawings required for use in the embodiments of the present application or the background technology will be described below.

[0011] Figure 1 It is a federated learning system provided by an embodiment of the present application;

[0012] Figure 2 This is a flow chart of a code completion method provided by an embodiment of the present application;

[0013] Figure 3 It is a code completion schematic diagram provided in an embodiment of the present application;

[0014] Figure 4 is a flow chart of a code completion method for a local model provided in an embodiment of the present application;

[0015] Figure 5It is a schematic diagram of a local model framework provided in an embodiment of the present application;

[0016] Figure 6 It is a structural schematic diagram of a code completion device provided in an embodiment of the present application;

[0017] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0018] The terms used in the embodiments of the present application are only used to explain the specific embodiments of the present application and are not intended to limit the present application.

[0019] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0020] It should be understood that when used in this specification and the appended claims, the terms "include" and "comprises" indicate the presence of described features, integers, steps, operations, elements and / or components, but do not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or combinations thereof.

[0021] It should also be understood that the terms used in this application specification are only for the purpose of describing specific embodiments and are not intended to limit the application. As used in this application specification and the appended claims, unless the context clearly indicates otherwise, the singular forms "a", "an" and "the" are intended to include plural forms.

[0022] It should be further understood that the term “and / or” used in the specification and appended claims refers to any combination and all possible combinations of one or more of the associated listed items, and includes these combinations.

[0023] like Figure 1 As shown, Figure 1 This is a schematic diagram of the system architecture of federated learning. Figure 1 The system shown includes an aggregation server and multiple participants, wherein each participant has its own local data set and local model, each participant has the same local model, and the local data sets of each participant may be partially the same or completely different.

[0024] In an embodiment of the present application, each of the above-mentioned participants first trains a local model according to the local data set and the parameters sent by the aggregation server to obtain local model parameters, and then sends the local model parameters to the aggregation server. The aggregation server aggregates the local model parameters of the above-mentioned multiple participants to obtain global model parameters, and sends the global model parameters to each participant. Each participant receives the global model parameters, updates the local model according to the global model parameters and the local model parameters, obtains a new local model, and then sends the new local model parameters to the aggregation server for aggregation. The aggregation server aggregates the new local model parameters sent by multiple participants to obtain new global model parameters, and sends the new global model parameters to each participant. When the number of interactions between the aggregation server and each participant does not reach the preset number, the steps of the above-mentioned aggregation server interacting with each participant are repeated. When the number of interactions between the aggregation server and each participant reaches the preset number, the training is stopped to obtain a trained code completion model.

[0025] See also Figure 2 , Figure 2 : is a flowchart of a code completion method provided in an embodiment of the present application, the method comprising:

[0026] S101. Obtain a local data set and initial model parameters sent by an aggregation server.

[0027] Specifically, the aggregation server randomly generates the initial model parameters W and then sends the initial model parameters W to n participants. Each participant has a local model M i , n participants each have a model M 1 ,M 2 ,…,M n , each model M i There are m model parameters, and the i-th model has parameters The number of initial model parameters W is consistent with the number of parameters in each local model, that is, it includes m parameters. Each participant has a local data set, and n participants have data sets D 1 ,D 2 ,…,D n The data sets of different participants can be partially the same or completely different, where the local data set includes the source code. For example, the multiple participants are different departments of a company, and each department has the source code corresponding to its own business. Then the data sets of each participant can be partially the same or completely different.

[0028] For example, if there are 4 participants, each of them has M 1 ,M 2 ,M 3 ,M 4 and local dataset D1 ,D 2 ,D 3 ,D 4 , if each model has 3 model parameters, then the first model (i.e. the first participant) M 1 With parameters and dataset D 1 Similarly, the second model (i.e. the second participant) M 2 With parameters and dataset D 2 ; The third model (i.e. the third participant) M 3 With parameters and dataset D 3 ; The fourth model (i.e. the fourth participant) M 4 With parameters and dataset D 4 , the initial model parameters W include 3 parameters.

[0029] S102: Train a local model according to the initial model parameters and the local data set to obtain local model parameters.

[0030] Specifically, for the i-th participant, use the local dataset D i Train your own local model M i , when the convergence condition is met, the local model parameter W is obtained i Among them, the convergence condition is that the number of iterations is equal to the preset number of iterations or the loss function value is less than the preset value.

[0031] S103: Encrypt the local model parameters and send them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by multiple participants to obtain global model parameters.

[0032] Specifically, each participant will use the local model M i The parameters W obtained after training convergence i Perform homomorphic encryption to obtain [W i ], then [W i ] is sent to the aggregation server. The aggregation server aggregates the local model parameters sent by each participant to obtain the global model parameters. The specific implementation process is: for each local model's jth parameter Aggregate the servers and average them to get the global model parameters

[0033]

[0034] It is understandable that for each local model there are m parameters, and after aggregation by the aggregation server, m global model parameters can be obtained. The aggregation server then adds the global model parameters Sent to each participant.

[0035] Homomorphic encryption is a cryptographic technique based on the computational complexity theory of mathematical problems. The data that has been homomorphically encrypted is processed to obtain an output, and the output is decrypted, and the result is the same as the output obtained by processing the unencrypted original data in the same way.

[0036] In the embodiment of the present application, based on the homomorphic encryption method, the parameters of each participant can be effectively protected from being disclosed, and the privacy of each participant's data can be protected. At the same time, the aggregation server does not need to decrypt, reducing the amount of calculation.

[0037] S104: Obtain global model parameters, and update the local model according to the global model parameters and the local model parameters to obtain a trained code completion model.

[0038] Specifically, each participant receives the global model parameters sent by the aggregation server Then, the global model parameters Decrypt to get the global parameters for decryption Each participant obtains a new local model parameter by weighted average of its own local model parameter and the decrypted global model parameter, where the jth local model parameter W of the i-th participant is i,j Updated to:

[0039]

[0040] Where β is a real number ranging from 0 to 1. The local model is updated according to the local model parameters and the global model parameters to obtain a new local model. When the β value is larger, it means that the new local model retains more features of the local model, that is, it provides more personalized code completion; conversely, when the β value is smaller, it means that the new local model retains fewer features of the local model.

[0041] When the number of interactions between the aggregation server and each participant does not reach the preset number, the above S101-S104 are repeated; when the number of interactions between the aggregation server and each participant reaches the preset number, the training is stopped to obtain a trained code completion model. The embodiment of the present application can adjust the value of β to select the features of the local local model to provide more personalized and more accurate code completion according to the actual needs of the participants.

[0042] S105: Obtain the code input by the user, input the code into the code completion model, and obtain the target code.

[0043] Specifically, the code to be predicted is obtained from the user input, and the code completion model automatically prompts and completes the code according to the code input by the user to obtain the target code.

[0044] For example, in a code completion system, the code entered by the user can be as follows: Figure 3 In the code shown, when the user enters the string "co", the system automatically prompts strings such as "config", "contributes", "contributors", "publishConfig", and "categories". When it is detected that the user clicks on one of the strings, for example, when the system detects a touch operation on the string "config", in response to the operation, the system automatically completes the code.

[0045] For example, the code input by the user may be the following code:

[0046]

[0047] Where () represents the missing code. This code is input into the code completion model, and the output target code is:

[0048]

[0049]

[0050] The embodiments of the present application can train an efficient code completion model together while ensuring that the codes of each participant are not made public, provide more accurate and personalized code completion services, meet various needs in the development and programming process, and further accelerate the software development process.

[0051] like Figure 4 As shown above, for the i-th participant, the local dataset D is used i Train your own local model M i , when the convergence condition is met, the local model parameter W is obtained i The specific process is as follows:

[0052] S201. Obtain source code from a local data set, and parse the source code into an abstract syntax tree.

[0053] Specifically, each participant obtains a local dataset D i The source code in the code is created, and an abstract syntax tree (AST) corresponding to each source code is established, and the corresponding source code is represented by AST.

[0054] The abstract syntax tree is an abstract representation of the syntax structure of the source code. The syntax structure of the programming language is represented in a tree form. The abstract syntax tree includes multiple nodes, and each node on the tree represents a structure in the source code. S202: Use the abstract syntax tree as the input of the local model, train the local model, and obtain the target node element.

[0055] Specifically, the local model includes two parallel models: the node value prediction model and the node type prediction model. The node value prediction model is used to encode the abstract syntax tree and predict the value of the target node; the node type prediction model is used to extract the hierarchical structure of the abstract syntax tree and predict the type of the target node. For details, see Figure 5 , Figure 5 A schematic diagram of a local model framework of an embodiment of the present application is shown. When training the local model, the two parallel models are trained in parallel, and the specific steps of the training are:

[0056] Step 1: Perform a depth-first traversal of the abstract syntax tree to obtain a node sequence, and use Path2root to save the hierarchical structure of the abstract syntax tree to obtain the Path2root hierarchical features.

[0057] Specifically, the abstract syntax tree AST is traversed in depth first to obtain a node sequence. At the same time, Path2root is used to save the hierarchical structure of the abstract syntax tree to obtain the Path2root hierarchical features. Among them, Path2root is an encoder, the input is the path from the current node to the root node in the AST, and then the types of all nodes on the path are encoded into a feature vector.

[0058] Depth-first traversal is an algorithm used to traverse or search a tree or graph. The algorithm searches the branches of the tree as deeply as possible. When all the edges of the node have been explored, the search will go back to the starting node of the edge where the node was found. This process continues until all nodes reachable from the source node have been found. If there are still undiscovered nodes, one of them is selected as the source node and the above process is repeated. The whole process is repeated until all nodes have been visited.

[0059] In this embodiment, due to the depth-first traversal, the node sequence in step 1 loses the hierarchical structure of AST, so Path2root is used to preserve the hierarchical structure of AST.

[0060] Step 2: Encode the AST separately, predict the value of the next node and the type of the next node.

[0061] Specifically, the Transformer XL algorithm is used to encode the node sequence to obtain the abstract syntax tree vector, i.e., the AST vector. The Recurrent Neural Network (RNN) algorithm is used to encode the Path2root hierarchical features to obtain the Path2root vector.

[0062] Among them, the Transformer XL algorithm is a language training model and an improvement of the Transformer algorithm.

[0063] For example, assuming S τ =[x τ,1 ,…,x τ,L ] and S τ+1 =[x τ+1,1 ,…,x τ+1,L ] are two consecutive code snippets of length L. Let the τth snippet S τ The nth hidden layer state is Where d represents the dimension of the hidden layer state. τ+1 The nth hidden layer state can be calculated according to the following formula:

[0064]

[0065]

[0066]

[0067] Among them, the function SG(·) means stopping the gradient calculation; Represents the connection of two hidden layer sequences; Represents input containing information about the previous code snippet; represents the n-th layer output of the τ+1th code snippet, that is, the AST vector after Transformer XL encoding; w represents the model parameters.

[0068] The specific steps of the bidirectional RNN algorithm are as follows: Assume that there is in represents the type of the i-th node in the path at the t-th time step. The hidden layer state of the bidirectional RNN can be calculated as follows:

[0069]

[0070]

[0071] Then and Connect them to get the Path2root vector representation P obtained by bidirectional RNN encoding t .

[0072] The embodiment of the present application solves the long dependency problem in the code sequence and the problem of different lengths of the code sequence by using the Transformer XL algorithm; at the same time, due to the depth-first traversal, the node sequence loses the hierarchical structure of the AST, and Path2root is used to preserve the hierarchical structure of the AST, making the model prediction more accurate. Among them, the long dependency problem is that the distance between two highly correlated words is very far.

[0073] Step 3: Output the target element

[0074] The AST vector and the Path2root vector are connected to predict the value of the target node and obtain the value of the target node; the AST vector and the Path2root vector are connected to predict the type of the target node and obtain the type of the target node.

[0075] Specifically, for each prediction model, the output vector from another prediction model is concatenated with the output vector of the prediction model, and then the outputs of the two prediction models are calculated:

[0076]

[0077] Then use the softmax function to generate the probability distribution of the output to select the final code completion result:

[0078]

[0079] in are all trainable parameters. V represents the dictionary size, H represents the dimension of the AST encoding result, and H p Indicates the dimension of the Path2root encoding result. k takes the value of 1 or 2 to represent different prediction models.

[0080] S203. Calculate the loss function value of the entire local model.

[0081] Specifically, for the node value prediction model, the first loss function value Loss is determined according to the value of the target node and the value of the actual node. 1 For the node type prediction model, the type of the target node and the type of the actual node determine the second loss function value Loss 2 . The first loss function value Loss 1 and the second loss function value Loss 2 Perform weighted averaging to obtain the loss function value Loss of the entire local model:

[0082]

[0083] α k The initial value of is set to 0.5. As the model is updated, α k The value of will be updated.

[0084] S204. Update the local model according to the loss function value.

[0085] Update the local model according to the loss function value of the local model, and iterate the entire local model using the stochastic gradient descent method. When the number of iterations is equal to the preset number of iterations or the loss function value is less than the preset value, stop training the local model M. i .

[0086] The embodiment of the present application can extract features of more levels and scales by dividing the local training model into two parallel node value prediction models and node type prediction models, and obtain a more accurate code recommendation service. The node value prediction model preferentially traverses the abstract syntax tree to obtain a node sequence, and uses the Transformer XL algorithm to encode the node sequence to obtain an AST vector, which solves the long dependency problem in the code sequence and the problem of different code sequence lengths. The node type prediction model uses Path2root to save the hierarchical structure of AST, obtains Path2root hierarchical features, and uses the Recurrent Neural Network (RNN) algorithm to encode the Path2root hierarchical features to obtain the Path2root vector, which can capture the structural information of the abstract syntax tree well and provide more accurate code prediction. Finally, the AST vector is connected with the Path2root vector to predict the output of the node value prediction model, and the AST vector is connected with the Path2root vector to predict the output of the node type prediction model, which can more accurately predict the target code.

[0087] The present application also provides a code completion device, which can be used to implement the above-mentioned code completion method embodiments of the present application. Figure 6 , Figure 6 600 is a schematic diagram of a code completion device provided in an embodiment of the present application. The device 600 of this embodiment includes:

[0088] An acquisition unit 601 is used to acquire a local data set and initial model parameters sent by an aggregation server, wherein the local data set includes a plurality of source codes;

[0089] The processing unit 602 is used to train the local model according to the initial model parameters and the local data set to obtain the local model parameters;

[0090] The sending unit 603 is used to encrypt the local model parameters and send them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by multiple participants to obtain the global model parameters;

[0091] An updating unit 604 is used to obtain global model parameters, and update the local model according to the global model parameters and the local model parameters to obtain a trained code completion model;

[0092] The processing unit 602 is further used to obtain the code input by the user, input the code into the code completion model, and obtain the target code.

[0093] In a specific embodiment of the code completion device of the present application, the specific operation of the acquisition unit 601 acquiring the local data set and the initial model parameters sent by the aggregation server can refer to the relevant operations in the above S101, which will not be repeated here; the specific operation of the sending unit 603 encrypting the local model parameters and sending them to the aggregation server can refer to the relevant operations in the above S103, which will not be repeated here; the specific operation of the updating unit 604 obtaining the trained code completion model can refer to the relevant operations in the above S104, which will not be repeated here; the specific operation of the processing unit 602 obtaining the target code can refer to the relevant operations in the above S105, which will not be repeated here.

[0094] In a specific implementation, the processing unit 602 is used to train a local model according to the initial model parameters and the local data set, and the process of obtaining the local model parameters specifically includes:

[0095] Obtain source code in a local data set, and parse the source code into an abstract syntax tree, where the abstract syntax tree includes multiple nodes;

[0096] Input the abstract syntax tree into the local model, train the local model, and obtain the target node element, where the target node element includes the value of the target node and the type of the target node;

[0097] Determine the loss function value based on the target node element and the actual node element;

[0098] Update the local model according to the loss function value;

[0099] When the convergence condition is met, the local model parameters corresponding to the updated local model are obtained. The convergence condition is that the number of iterations is equal to the preset number of iterations or the loss function value is less than the preset value.

[0100] In a specific implementation method, the above-mentioned processing unit 602 obtains the source code in the local data set, and the specific operation of parsing the source code into an abstract syntax tree can refer to the relevant operations in the above-mentioned S201, which will not be repeated here; the specific operation of the above-mentioned processing unit 602 obtaining the target node element can refer to the relevant operations in the above-mentioned S202, which will not be repeated here; the specific operation of the above-mentioned processing unit 602 determining the loss function value can refer to the relevant operations in the above-mentioned S203, which will not be repeated here; the specific operation of the above-mentioned processing unit 602 updating the local model and obtaining the local model parameters corresponding to the updated local model can refer to the relevant operations in the above-mentioned S204, which will not be repeated here.

[0101] In addition, an embodiment of the present application provides an electronic device, which may include the code completion method of any of the above embodiments of the present application. Specifically, the electronic device may be, for example, a terminal device or a server.

[0102] The present application also provides another electronic device, including:

[0103] A processor and a memory, the processor executes the code in the memory, thereby completing the operation of the code completion method of any of the above embodiments of the present application.

[0104] Figure 7 This is a structural block diagram of an electronic device provided in an embodiment of the present application. Figure 7 , which shows a schematic diagram of the structure of an electronic device suitable for implementing a terminal device or a server in an embodiment of the present application. Figure 7 As shown, the electronic device includes: one or more processors 701; a communication interface 702, and a memory 703. The processor 701, the communication interface 702, and the memory 703 are connected via a bus 704. The memory 703 is used to store instructions, and the processor 701 is used to execute the instructions stored in the memory 703. Among them, the processor 701 is configured to call the program instruction execution:

[0105] Obtaining a local data set and initial model parameters sent by an aggregation server, wherein the local data set includes a plurality of source codes;

[0106] Train the local model according to the initial model parameters and the local data set to obtain the local model parameters;

[0107] Encrypting the local model parameters and sending them to the aggregation server, so that the aggregation server can aggregate the local model parameters sent by multiple participants to obtain the global model parameters;

[0108] Get the global model parameters, update the local model according to the global model parameters and the local model parameters, and get the trained code completion model;

[0109] Get the code input by the user, input the code into the code completion model, and get the target code.

[0110] It should be understood that in the embodiment of the present application, the processor 701 may be a central processing unit (CPU), and the processor may also be other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or the processor may also be any conventional processor, etc.

[0111] The communication interface 702 may be a wired interface or a wireless interface for communicating with other modules or devices. The wired interface may be an Ethernet interface, a local interconnect network (LIN), etc., and the wireless interface may be a cellular network interface or a wireless local area network interface, etc.

[0112] The memory 703 may be a non-volatile memory, such as a read-only memory (ROM), a programmable ROM (PROM), an erasable PROM (EPROM), an electrically erasable programmable ROM (EEPROM), or a flash memory. The memory 1003 may also be a volatile memory, which may be a random access memory (RAM) used as an external cache.

[0113] The memory 703 may also be used to store instructions and data, so that the processor 701 can call the instructions stored in the memory 703 to implement the operations performed by the above-mentioned code completion device.

[0114] The bus 704 may be a peripheral component interconnect (PCI) bus or an extended industry standard architecture (EISA) bus. The bus 704 may be divided into an address bus, a data bus, a control bus, etc. For ease of representation, Figure 7Only one thick line is used in the diagram, but this does not mean that there is only one bus or only one type of bus.

[0115] Optionally, an input / output interface 705 may be further included. The input / output interface 705 is connected to an input / output device and is used to receive input information and output operation results.

[0116] In a specific implementation, the processor 701, the communication interface 702, and the implementation methods described in various embodiments of a code completion method and device provided by the embodiments of the present application are described in the embodiments of the present application, and are not described in detail here.

[0117] In another embodiment of the present application, a computer-readable storage medium is provided, which stores a computer program, and the computer program includes program instructions. When the program instructions are executed by a processor, the following are implemented: obtaining a local data set and initial model parameters sent by an aggregation server, wherein the local data set includes multiple source codes; training a local model according to the initial model parameters and the local data set to obtain local model parameters; encrypting the local model parameters and sending them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by multiple participants to obtain global model parameters; obtaining global model parameters, updating the local model according to the global model parameters and the local model parameters to obtain a trained code completion model; obtaining code input by a user, inputting the code into the code completion model to obtain a target code.

[0118] The computer-readable storage medium may be an internal storage unit of the electronic device of any of the foregoing embodiments, such as a hard disk or memory of a terminal. The computer-readable storage medium may also be an external storage device of a terminal, such as a plug-in hard disk, a smart memory card (Smart Media Card, SMC), a secure digital (Secure Digital, SD) card, a flash card (Flash Card), etc. equipped on the terminal. Further, the computer-readable storage medium may also include both an internal storage unit of an electronic device and an external storage device. The computer-readable storage medium is used to store computer programs and other programs and data required by the electronic device. The computer-readable storage medium may also be used to temporarily store data that has been output or is to be output.

[0119] Those of ordinary skill in the art will appreciate that the units and algorithm steps of each example described in conjunction with the embodiments disclosed herein can be implemented in electronic hardware, computer software, or a combination of the two. In order to clearly illustrate the interchangeability of hardware and software, the composition and steps of each example have been generally described in terms of function in the above description. Whether these functions are performed in hardware or software depends on the specific application and design constraints of the technical solution. Professional and technical personnel can use different methods to implement the described functions for each specific application, but such implementation should not be considered to be beyond the scope of this application.

[0120] Technical personnel in the relevant field can clearly understand that, for the convenience and simplicity of description, the specific working processes of the servers, devices and units described above can refer to the corresponding processes in the aforementioned method embodiments, and can also execute the implementation methods of the electronic devices described in the embodiments of the invention, which will not be repeated here.

[0121] In the several embodiments provided in the present application, it should be understood that the disclosed servers, devices and methods can be implemented in other ways. For example, the server embodiments described above are merely schematic. For example, the division of the above-mentioned units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. In addition, the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, or it can be an electrical, mechanical or other form of connection.

[0122] The units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present application.

[0123] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0124] If the above-mentioned integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program code.

[0125] The above is only a specific implementation of the present application, but the protection scope of the present application is not limited thereto. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in the present application, and these modifications or replacements should be included in the protection scope of the present application. Therefore, the protection scope of the present application shall be based on the protection scope of the claims.

Claims

1. A code completion method, characterized in that: Applied to a first participant, the first participant is any participant in a system including an aggregation server and a plurality of participants, including: Acquire a local data set and initial model parameters sent by an aggregation server, wherein the local data set includes a plurality of source codes; Obtain source code in a local data set, and parse the source code into an abstract syntax tree, where the abstract syntax tree includes a plurality of nodes; Inputting the abstract syntax tree into a local model, training the local model, and obtaining a target node element, wherein the target node element includes a value of the target node and a type of the target node; Determine a first loss function value according to the value of the target node and the value of the actual node; Determine a second loss function value according to the type of the target node and the type of the actual node; Taking a weighted average of the first loss function value and the second loss function value, to obtain a loss function value of the local model; Update the local model according to the loss function value; When a convergence condition is met, a local model parameter corresponding to the updated local model is obtained, wherein the convergence condition is that the number of iterations is equal to a preset number of iterations or the loss function value is less than a preset value; Encrypting the local model parameters and sending them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by the multiple participants to obtain global model parameters; Acquire the global model parameters, perform weighted averaging on the local model parameters and the global model parameters to obtain new local model parameters, update the local model according to the new local model parameters, and obtain a trained code completion model; The code input by the user is obtained, and the code is input into the code completion model to obtain the target code.

2. The method according to claim 1, characterized in that The local model includes a parallel node value prediction model and a node type prediction model; the node value prediction model is used to encode the abstract syntax tree and predict the value of the target node, and the node type prediction model is used to extract the hierarchical structure of the abstract syntax tree and predict the type of the target node.

3. The method according to claim 1, characterized in that: The training of the local model to obtain the target node element includes: Performing a depth-first traversal on the abstract syntax tree to obtain a node sequence, using Path2root to save the hierarchical structure of the abstract syntax tree to obtain a Path2root hierarchical feature; The node sequence is encoded by using a Transformer XL algorithm to obtain an abstract syntax tree vector, and the Path2root hierarchical feature is encoded by using a recurrent neural network algorithm to obtain a Path2root vector; The abstract syntax tree vector and the Path2root vector are connected to predict the value of the target node to obtain the value of the target node; the abstract syntax tree vector and the Path2root vector are connected to predict the type of the target node to obtain the type of the target node.

4. The method according to claim 1, characterized in that The step of encrypting the local model parameters and sending the parameters to the aggregation server comprises: The local model parameters are homomorphically encrypted to obtain encrypted local model parameters, and the encrypted local model parameters are sent to the aggregation server.

5. A code completion device, characterized in that: include: An acquisition unit, configured to acquire a local data set and initial model parameters sent by an aggregation server, wherein the local data set includes a plurality of source codes; A processing unit, configured to obtain source code in a local data set, and parse the source code into an abstract syntax tree, wherein the abstract syntax tree includes a plurality of nodes; Inputting the abstract syntax tree into a local model, training the local model, and obtaining a target node element, wherein the target node element includes a value of the target node and a type of the target node; Determine a first loss function value according to the value of the target node and the value of the actual node; Determine a second loss function value according to the type of the target node and the type of the actual node; Taking a weighted average of the first loss function value and the second loss function value, to obtain a loss function value of the local model; Update the local model according to the loss function value; When a convergence condition is met, a local model parameter corresponding to the updated local model is obtained, wherein the convergence condition is that the number of iterations is equal to a preset number of iterations or the loss function value is less than a preset value; A sending unit, configured to encrypt the local model parameters and send them to the aggregation server, so that the aggregation server aggregates the local model parameters sent by multiple participants to obtain global model parameters; An updating unit, configured to obtain the global model parameters, perform weighted averaging of the local model parameters and the global model parameters to obtain new local model parameters, and update the local model according to the new local model parameters to obtain a trained code completion model; The processing unit is further used to obtain a code input by a user, input the code into the code completion model, and obtain a target code.

6. A code completion device, characterized in that: include: A processor and a memory, wherein the processor executes the code in the memory to perform the method according to any one of claims 1 to 4.

7. A computer-readable storage medium, characterized in that: The method comprises instructions, which, when executed on a computer, cause the computer to execute the method according to any one of claims 1 to 4.

Images