A multi-source heterogeneous code conversion method and device for cross-language development

Abstract

The application discloses a multi-source heterogeneous code conversion method for cross-language development, which comprises the following steps: constructing a code sample set by constructing each code sample and a unit code block separated from the code sample into a format of { 'code' x c , 'code type' y c , 'code description' x d}; then using the code sample set to sequentially perform first training of internal features of the model, second training of consistency of code semantics and third training of consistency of mixed codes; taking embedding mapping functions obtained by the model after the first training and the third training as embedding models, obtaining embedding representation vectors after embedding representation mapping by using the two embedding models; in actual use, inputting an initial prompt word into the two embedding models to obtain retrieval enhancement information; and finally inputting the initial prompt word and the retrieval enhancement information into a large language model after splicing to obtain a code conversion result. The application can improve the accuracy of code conversion.

Description

Technical Field

[0001] This invention belongs to the cross-disciplinary field of intelligent control and AI applications, specifically relating to a method and apparatus for cross-language development of multi-source heterogeneous code conversion. Background Technology

[0002] Existing intelligent control algorithms typically require collaborative development by multiple research teams. For example, intelligent transportation involves complex game theory needs and scenarios. The regulation of its multi-node road network often requires consideration of the game theory and collaboration among nodes to achieve intelligent control of traffic lights. Such intelligent game theory and collaborative control algorithms for multi-node road network traffic lights often require collaborative research and development across platforms by multiple teams from universities and research institutes during the design phase. The actual deployment process requires code refactoring based on hardware conditions and development requirements, which involves code conversion and refactoring.

[0003] Existing methods for multi-source heterogeneous code conversion and refactoring primarily achieve code conversion and generation by fine-tuning large language models. For example, Chinese patent application CN119166162A provides a method for constructing a large C-to-Rust code translation model based on intermediate representations, and Chinese patent application CN120295609A provides a software code generation method based on retrieval-augmented generation and large model fine-tuning, mainly achieving code conversion and generation through fine-tuning of large language models. Directly fine-tuning large models requires significant computational power and time, and the deployed resources for the fine-tuned model are also greater. This invention, however, mainly combines retrieval-augmented generation (RAG) methods to design and construct a fine-tuning strategy for the embedded model. Compared to directly fine-tuning large models, the embedded model is smaller and has higher training efficiency. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention proposes a method and apparatus for cross-language development of multi-source heterogeneous code conversion, the specific technical solution of which is as follows:

[0005] A method for converting multi-source heterogeneous code in cross-language development includes:

[0006] S1: Collect multiple code samples and separate the unit code blocks in the code file corresponding to each code sample; then construct {'code'x} for each code sample and unit code block. c , 'code type' y c , 'code description' x d} format to construct a code sample set;

[0007] S2: Input the code sample set into the pre-trained BERT language model, perform the first training of the model, and mine the internal features of the code;

[0008] S3: Input the code sample set into the BERT language model after the first training, and perform a second training on the model to improve the semantic consistency of the code;

[0009] S4: Input the code sample set into the BERT language model after secondary training, and perform tertiary training of the BERT language model for hybrid code consistency.

[0010] S5: The embedding mapping function obtained by the Bert language model after the first training is taken as the first embedding model, denoted as Embed1; the embedding mapping function obtained by the Bert language model after the third training is taken as the second embedding model, denoted as Embed2.

[0011] S6: Use the code files used for training as the knowledge base of the RAG system, and after implementing the embedding representation mapping using Embed1 and Embed2 respectively, obtain the embedding representation vectors as the knowledge vector base;

[0012] S7: In actual use, the intelligent algorithm code that needs to be converted is constructed into initial prompt words according to the prompt word framework {'target language':, 'target task':, 'original code':};

[0013] S8: Input the initial prompt words into Embed1 to obtain the corresponding embedding representations; select the most similar embedding representation vectors from the knowledge vector base through similarity calculation;

[0014] S9: Input the initial prompts and the results obtained in S8 into Embed2 to obtain the corresponding embeddings; select the most similar embedding vectors based on similarity calculation as retrieval enhancement information;

[0015] S10: Concatenate the initial prompt words and search enhancement information to obtain the complete prompt words, input them into the large language model, and obtain the code conversion results.

[0016] Furthermore, in step S2, the loss function for the first training of the model is obtained by summing two parts: the first part is the difference between the embedding representation of the entire code text and the embedding representation vector of each unit code block therein, and the second part is the product of the code type prediction error and a manually adjustable coefficient λ1.

[0017] Furthermore, in step S3, the loss function for the second training of the model is calculated as follows:

[0018] Based on the embedding representations of code files and code descriptions respectively, probability densities centered on each embedding representation are calculated. Then, the KL divergence between the two probability densities is calculated to improve the similarity between the two distributions. This incorporates information from the code descriptions into the embedding representations of the code files, reduces the embedding distance between code file representations with similar code descriptions, increases the embedding distance between code file representations with differences, and improves the semantic consistency of the code.

[0019] Furthermore, in step S4, the loss function for the three training iterations of the model is obtained by summing two parts. The first part is the weighted reconstruction error, which is obtained as follows:

[0020] Calculate the reconstruction error in the embedding representation between each code file and its neighboring samples in different code types, and calculate the dot product between the embedding representation of the code file and its neighboring samples as the weight in the weighted reconstruction error;

[0021] The second part is the product of the code type prediction error and a manually adjustable coefficient λ2.

[0022] Furthermore, the similarity is calculated using cosine similarity.

[0023] Furthermore, in step S4, the method for selecting the embedding representation of the neighborhood samples is as follows:

[0024] First, based on the code types in each code sample, select a set of code samples whose code types are different from those of the current code sample. Then, based on the code descriptions in the code sample set, obtain the corresponding code description embedding representation set. Calculate the Euclidean distance between the code description embedding representation of the current code sample and each code description embedding representation in the set of code description embedding representations. Sort the Euclidean distances from smallest to largest and select the top-ranked code description embedding representations as the embedding representations of the neighboring samples of the current code sample's embedding representation.

[0025] Application of a cross-language multi-source heterogeneous code conversion method in a traffic light control algorithm.

[0026] A cross-language development multi-source heterogeneous code conversion device includes one or more processors for implementing a cross-language development multi-source heterogeneous code conversion method.

[0027] An electronic device, comprising:

[0028] One or more processors;

[0029] A storage device for storing one or more programs, which, when executed by the electronic device, enable the electronic device to implement a method for converting multi-source heterogeneous code across languages.

[0030] A computer-readable storage medium having a program stored thereon that, when executed by a processor, implements a method for converting multi-source heterogeneous code across languages.

[0031] The beneficial effects of this invention are as follows:

[0032] (1) The embedding model obtained by the present invention through a three-level progressive embedding model training strategy combining internal file feature mining, code semantic consistency and hybrid code consistency can help the RAG system to search for relevant auxiliary information more accurately in the database and improve the accuracy of code conversion.

[0033] (2) The basic implementation framework of this invention is based on the large language model of RAG, which can make full use of the generation capability of the large language model. At the same time, using RAG can reduce the illusion problem in the large model generation process. Compared with the existing direct fine-tuning of task-related parameters of the large model, this invention can significantly reduce the cost of model application and improve the efficiency of model application.

[0034] (3) This invention can effectively improve the efficiency of intelligent algorithms from multi-team and cross-platform collaborative development and research to actual deployment and application. Attached Figure Description

[0035] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0036] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0037] Figure 1 The flowchart illustrates the cross-language development method for multi-source heterogeneous code conversion according to the present invention.

[0038] Figure 2 This is a flowchart illustrating the training of the BERT language model and the acquisition of the embedded model in an embodiment of the present invention.

[0039] Figure 3 This is a schematic diagram of a cross-language development multi-source heterogeneous code conversion device according to an embodiment of the present invention. Detailed Implementation

[0040] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0041] The terminology used in this application is for the purpose of describing particular embodiments only and is not intended to be limiting of the application. The singular forms “a,” “the,” and “the” used in this application and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used herein refers to and includes any or all possible combinations of one or more of the associated listed items.

[0042] It should be understood that although the terms first, second, third, etc., may be used in this application to describe various information, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, without departing from the scope of this application, first information may also be referred to as second information, and similarly, second information may also be referred to as first information. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to determination."

[0043] It should also be noted that "multiple" as mentioned in this application refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0044] like Figure 1 and Figure 2 As shown, the method for converting multi-source heterogeneous code for cross-language development in this embodiment includes the following steps:

[0045] Step 1: Collect multiple code samples and separate the individual code blocks in the code file corresponding to each code sample; then construct each code sample and individual code block into {'code' x c , 'code type' y c , 'code description' x d Use the format} to construct a code sample set.

[0046] In this format, the code is the actual code file, the code type indicates the language type of the code file, and the code description contains a description of the functionality implemented by the code file.

[0047] Step 2: Input the code sample set into the pre-trained BERT language model, construct the loss function L1, perform the first training on the pre-trained BERT language model, mine the internal features of the code, and obtain the BERT language model after the first training.

[0048] The expression for the loss function L1 is as follows:

[0049]

[0050] in, This represents the entire code file in the i-th code sample;

[0051] This represents the j-th unit code block in the code file corresponding to the i-th code sample;

[0052] embed() represents an embedded representation of a code file or code module;

[0053] N b Indicates the number of unit code blocks in the code file;

[0054] This represents the global embedding corresponding to the entire code file of the i-th code sample. This represents the sum of the embedding representations of each unit code block in the code file corresponding to the i-th code sample;

[0055] This represents the predicted code type value for the i-th code sample, which is obtained by mapping the embedding representation through a fully connected layer FFN(*). This represents the actual value of the code type for the i-th code sample;

[0056] λ1 represents the adjustable coefficient, and N represents the number of code samples.

[0057] The parameters of the pre-trained BERT language model are initially adjusted by minimizing the loss function L1 using gradient descent.

[0058] Step 3: Input the code sample set into the BERT language model after the first training, construct the loss function L2, and perform a second training on the BERT language model after the first training to improve the semantic consistency of the code. The purpose is to reduce the code file embedding distance between similar code descriptions and increase the code file embedding distance between different code descriptions.

[0059] The expression for the loss function L2 is as follows:

[0060]

[0061]

[0062] Where KL (Kullback-Leibler Divergence) represents the KL divergence;

[0063] The embedding representation of the code description of the i-th sample;

[0064] This represents the global embedding corresponding to the entire code file of the j-th code sample;

[0065] This represents the global embedding corresponding to the entire code file of the k-th code sample;

[0066] The embedding representation of the code description of the j-th code sample;

[0067] The embedding representation of the code description of the k-th code sample;

[0068] In code file embedding representation, given a center point is... In this case, The likelihood of being chosen as a neighbor;

[0069] In the embedded representation described in the code, given the center point is... In this case, The likelihood of being chosen as a neighbor;

[0070] Indicates a coefficient that can be manually adjusted;

[0071] By minimizing the loss function L2, the parameters of the BERT language model after the first training are adjusted.

[0072] Step 4: Input the code sample set into the BERT language model after secondary training to obtain the corresponding embedding representation, construct the loss function L3, and perform tertiary training of the BERT language model for hybrid code consistency, thereby shrinking the embedding representations of different code types but with similar code descriptions.

[0073] The expression for the loss function L3 is as follows:

[0074]

[0075] in, Represents the reconstructed weights, through and The dot product is calculated; T represents the transpose; Let n be the nearest neighbor embedding of the embedding representation of the i-th code sample. The method for selecting the nearest neighbor embedding is as follows: First, based on the code type y in each code sample... c First, select a set of code samples whose code type is different from that of the i-th code sample. Then, extract the corresponding code description from this set and input it into the BERT language model after secondary training to obtain the corresponding embedding representation set. Calculate the Euclidean distance between the i-th code description embedding representation and each code description embedding representation in this embedding representation set, and sort the Euclidean distances from smallest to largest. Select the N embedding representations with the highest Euclidean distances to the i-th code sample. nei λ2 is an embedding representation, which is the nearest neighbor of the embedding representation of the i-th code sample; λ2 is a coefficient that can be manually adjusted.

[0076] Step 5: Use the embedding mapping function obtained from the first training of the BERT language model as the embedding model after feature mining within the code, denoted as Embed1, and store it; use the embedding mapping function obtained from the third training of the BERT language model as the embedding model trained at three levels, denoted as Embed2, and store it.

[0077] Step 6: Store the code files used for training as the knowledge base of the RAG system, and after implementing the embedding representation mapping using two embedding models Embed1 and Embed2 respectively, obtain the embedding representation vectors, store them in the knowledge vector library, and complete the knowledge base construction.

[0078] Step 7: In actual use, construct the initial prompt words for the intelligent algorithm code that needs to be converted according to the prompt word framework {'target language':, 'target task':, 'original code':};

[0079] Step 8: Input the initial prompt words into Embed1 to obtain the corresponding embedding representations. Calculate the cosine similarity based on the embedding representations of Embed1, and select the most similar embedding representation vectors from the knowledge vector base to achieve preliminary screening of the enhanced retrieval information.

[0080] Step 9: Input the initial prompts and the preliminary search enhancement information obtained in Step 8 into Embed2 to obtain the corresponding embedding representations; and select the most similar embedding representation vectors based on cosine similarity, i.e., perform fine screening, as the final search enhancement information.

[0081] Step 10: Construct a complete set of prompts from the initial prompts and the final search enhancement information according to the prompt framework {'Target Language':, 'Target Task':, 'Original Code':, 'Search Enhancement Information'}, and input it into the large language model to obtain the final code conversion result.

[0082] The aforementioned cross-language multi-source heterogeneous code conversion method can be applied to traffic light control algorithms, because traffic light control algorithms involve the regulation of multi-node road networks and the game and coordination among nodes, requiring collaborative development by multiple teams.

[0083] On the other hand, corresponding to the aforementioned method for converting multi-source heterogeneous code in cross-language development, the present invention also provides an embodiment of a device for converting multi-source heterogeneous code in cross-language development.

[0084] See Figure 3 The cross-language development multi-source heterogeneous code conversion device of the present invention includes one or more processors for implementing the cross-language development multi-source heterogeneous code conversion method in the above embodiments.

[0085] The embodiments of the cross-language development multi-source heterogeneous code conversion device of the present invention can be applied to any device with data processing capabilities, such as a computer. The device embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data-processing device reading the corresponding computer program instructions from non-volatile memory into memory for execution. From a hardware perspective, in addition to the processor, memory, network interface, and non-volatile memory, the data-processing device in the embodiments typically includes other hardware depending on its actual functions, which will not be elaborated further.

[0086] The specific implementation process of the functions and roles of each unit in the above device can be found in the implementation process of the corresponding steps in the above method, and will not be repeated here.

[0087] For the device embodiments, since they basically correspond to the method embodiments, the relevant parts can be referred to in the description of the method embodiments. The device embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without creative effort.

[0088] This invention also provides a computer-readable storage medium storing a program thereon, which, when executed by a processor, implements the cross-language development multi-source heterogeneous code conversion method described in the above embodiments.

[0089] The computer-readable storage medium can be an internal storage unit of any data processing device described in any of the foregoing embodiments, such as a hard disk or memory. The computer-readable storage medium can also be an external storage device, such as a plug-in hard disk, smart media card (SMC), SD card, flash card, etc., equipped on the device. Furthermore, the computer-readable storage medium can include both internal storage units of any data processing device and external storage devices. The computer-readable storage medium is used to store the computer program and other programs and data required by the data processing device, and can also be used to temporarily store data that has been output or will be output.

[0090] It will be understood by those skilled in the art that the above descriptions are merely preferred examples of the invention and are not intended to limit the invention. Although the invention has been described in detail with reference to the foregoing examples, those skilled in the art can still modify the technical solutions described in the foregoing examples or make equivalent substitutions for some of the technical features. All modifications and equivalent substitutions made within the spirit and principles of the invention should be included within the scope of protection of the invention.

Claims

1. A method for converting multi-source heterogeneous code in cross-language development, characterized in that, include: S1: Collect multiple code samples and separate the unit code blocks in the code file corresponding to each code sample; Then, each code sample and unit code block is constructed into {'code'x}. c , 'code type' y c , 'code description' x d } format to construct a code sample set; S2: Input the code sample set into the pre-trained BERT language model, perform the first training of the model, and mine the internal features of the code; S3: Input the code sample set into the BERT language model after the first training, and perform a second training on the model to improve the semantic consistency of the code; S4: Input the code sample set into the BERT language model after secondary training, and perform tertiary training of the BERT language model for hybrid code consistency; in step S4, the loss function of the model for the three trainings is obtained by summing two parts. The first part is the weighted reconstruction error, which is obtained as follows: Calculate the reconstruction error in the embedding representation between each code file and its neighboring samples in different code types, and calculate the dot product between the embedding representation of the code file and its neighboring samples as the weight in the weighted reconstruction error; The second part is the product of the code type prediction error and a manually adjustable coefficient λ2; S5: The embedding mapping function obtained by the Bert language model after the first training is taken as the first embedding model, denoted as Embed1; the embedding mapping function obtained by the Bert language model after the third training is taken as the second embedding model, denoted as Embed2. S6: Use the code files used for training as the knowledge base of the RAG system, and after implementing the embedding representation mapping using Embed1 and Embed2 respectively, obtain the embedding representation vectors as the knowledge vector base; S7: In actual use, the intelligent algorithm code that needs to be converted is constructed into initial prompt words according to the prompt word framework {'target language':, 'target task':, 'original code':}; S8: Input the initial prompt words into Embed1 to obtain the corresponding embedding representations; select the most similar embedding representation vectors from the knowledge vector base through similarity calculation; S9: Input the initial prompts and the results obtained in S8 into Embed2 to obtain the corresponding embeddings; select the most similar embedding vectors based on similarity calculation as retrieval enhancement information; S10: Concatenate the initial prompt words and search enhancement information to obtain the complete prompt words, input them into the large language model, and obtain the code conversion results.

2. The method for cross-language development and multi-source heterogeneous code conversion according to claim 1, characterized in that, In step S2, the loss function for the first training of the model is obtained by summing two parts: the first part is the difference between the embedding representation of the entire code text and the embedding representation vector of each unit code block therein, and the second part is the product of the code type prediction error and a manually adjustable coefficient λ1.

3. The method for cross-language development and multi-source heterogeneous code conversion according to claim 1, characterized in that, In step S3, the loss function for the second training of the model is calculated as follows: Based on the embedding representations of code files and code descriptions respectively, probability densities centered on each embedding representation are calculated. Then, the KL divergence between the two probability densities is calculated to improve the similarity between the two distributions. This incorporates information from the code descriptions into the embedding representations of the code files, reduces the embedding distance between code file representations with similar code descriptions, increases the embedding distance between code file representations with differences, and improves the semantic consistency of the code.

4. The method for cross-language development and multi-source heterogeneous code conversion according to claim 1, characterized in that, The similarity is calculated using cosine similarity.

5. The method for cross-language development and multi-source heterogeneous code conversion according to claim 1, characterized in that, In step S4, the method for selecting the embedding representation of the neighborhood sample is as follows: First, based on the code types in each code sample, select a set of code samples whose code types are different from those of the current code sample. Then, based on the code descriptions in the code sample set, obtain the corresponding code description embedding representation set. Calculate the Euclidean distance between the code description embedding representation of the current code sample and each code description embedding representation in the set of code description embedding representations. Sort the Euclidean distances from smallest to largest and select the top-ranked code description embedding representations as the embedding representations of the neighboring samples of the current code sample's embedding representation.

6. The application of a cross-language multi-source heterogeneous code conversion method as described in any one of claims 1 to 5 in a control algorithm for traffic lights.

7. A cross-language development multi-source heterogeneous code conversion device, characterized in that, It includes one or more processors for implementing the cross-language development multi-source heterogeneous code conversion method according to any one of claims 1 to 5.

8. An electronic device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by the electronic device, cause the electronic device to implement the cross-language development multi-source heterogeneous code conversion method according to any one of claims 1 to 5.

9. A computer-readable storage medium, characterized in that, It stores a program that, when executed by a processor, implements the cross-language development method for converting heterogeneous code from multiple sources as described in any one of claims 1 to 5.

Images