Compressed code generation method and device, electronic equipment and readable storage medium

Abstract

The invention provides a compressed code generation method and device, electronic equipment and a readable storage medium. The method comprises the steps that a natural language instruction and context information input by a user are acquired; on the basis of a pre-trained code generation model, under the synergistic effect of an acceleration component and a preset automaton, generating a target compressed code according to the natural language instruction and the context information; the code generation model comprises an encoder and a decoder, and is obtained by performing two-way distillation training on a training sample set formed by samples according to task instruction samples, context samples and original-compression codes; the acceleration component is used for accelerating the reasoning process of the code generation model, and the preset automaton is used for performing grammar constraint verification on phrases gradually generated by the code generation model. According to the method, token redundancy during subsequent model reasoning can be remarkably reduced, the method is not limited by the narrow domain performance of domain specific voice, meanwhile, the problem of grammar errors possibly introduced by code compression is solved, and the consistency of code logic and structure is ensured.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of program code generation, and in particular to a compressed code generation method and device, electronic equipment and a readable storage medium. BACKGROUND

[0002] In recent years, code generation technology has evolved from traditional programming auxiliary tools to an important means of improving the logical reasoning ability of large language models. For example, the "Chain-of-Code" method generates executable code by guiding the model and running the results, replacing natural language reasoning that is susceptible to ambiguity interference, thereby improving reasoning accuracy.

[0003] However, existing large language models still face multiple challenges in code generation and logical reasoning tasks: first, the model tends to generate highly readable but redundant code, significantly increasing token consumption and reducing reasoning efficiency; second, the multi-language training paradigm introduced to pursue multi-language universality brings unnecessary computational burden in actual application scenarios dominated by a single programming language, affecting reasoning accuracy and speed; third, the lack of effective syntax and logic constraint mechanisms leads to frequent syntax errors or logical defects in generated code, weakening its reliability in critical tasks.

[0004] Therefore, there is an urgent need for a lightweight code generation method that balances Turing completeness and logical correctness, which can significantly reduce token overhead while ensuring complete functionality, thereby improving the efficiency and accuracy of large language models in reasoning tasks. SUMMARY

[0005] The present application provides a compressed code generation method and device, electronic equipment and a readable storage medium to overcome the defects of large language models in code generation, such as large token overhead, low reasoning efficiency, and frequent logical errors caused by redundant formats, multi-language interference, and lack of syntax constraints, achieving the unity of efficiency, universality, and correctness, avoiding the redundancy of natural language, not being limited to the narrow domain of domain-specific language, and being more suitable for efficient execution of single tasks than multi-language solutions.

[0006] In one aspect, the present application provides a compressed code generation method, comprising: obtaining a natural language instruction input by a user and context information; generating a target compressed code according to the natural language instruction and the context information based on a pre-trained code generation model under the synergistic action of an acceleration component and a preset automaton; wherein the code generation model comprises an encoder and a decoder, and is obtained by double-path distillation training according to a training sample set composed of task instruction samples, context samples, and original-compressed code pairs; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on word groups generated step by step by the code generation model.

[0007] Further, the code generation model is used to generate a target compressed code according to the natural language instruction and the context information under the synergistic action of an acceleration component and a preset automaton, comprising: encoding the natural language instruction and the context information into a target semantic representation by the encoder, and generating the target compressed code according to the target semantic representation by the decoder; in the decoding process, the inference process of the encoder is accelerated by the acceleration component, and the syntax correctness of word groups generated step by step by the encoder is verified by the preset automaton.

[0008] Further, the acceleration component comprises a plurality of expert sub-networks and a gating network; accordingly, the inference process of the encoder is accelerated by the acceleration component, comprising: inputting the intermediate features of the code generation model in the inference process into the gating network to calculate the selection probabilities of the plurality of expert sub-networks respectively; determining a target expert sub-network for forward inference according to the selection probabilities of the plurality of expert sub-networks respectively; inputting the intermediate features of the code generation model in the inference process into the selected target expert sub-network to obtain a candidate word group probability distribution; and weighting and fusing the candidate word group probability distribution according to the selection probability of the target expert sub-network to obtain a fused result; the fused result is used to output to the code generation model to predict the next word group.

[0009] Further, the preset automaton comprises a finite automaton and a push-down automaton; accordingly, the syntax correctness of word groups generated step by step by the encoder is verified by the preset automaton, comprising: performing regular language recognition on the word groups generated step by step by the encoder by the finite automaton, and / or performing context-free language recognition on the word groups generated step by step by the encoder by the push-down automaton; in the case that an illegal word group or context-free language appears in the generation path, a backtracking and resampling mechanism is triggered.

[0010] Further, when training the code generation model, the code generation model comprises a shared encoder and two decoders, denoted as a first decoder and a second decoder respectively; and the code generation model is trained by double-path distillation, specifically including: inputting the original code sample and the compressed code sample in the original-compressed code pair sample into a pre-trained teacher model respectively to obtain a first target word group probability distribution and a second target word group probability distribution; inputting the task instruction sample and the context sample into the shared encoder to obtain a unified semantic representation; inputting the unified semantic representation into the first decoder and the second decoder respectively to obtain a first predicted word group probability distribution and a second predicted word group probability distribution; calculating a first cross-entropy loss and a first divergence loss according to the first predicted word group probability distribution, the first target word group probability distribution and the original code sample; calculating a second cross-entropy loss and a second divergence loss according to the second predicted word group probability distribution, the second target word group probability distribution and the compressed code sample; iteratively optimizing the first decoder by the first cross-entropy loss and the first divergence loss, and iteratively optimizing the second decoder by the second cross-entropy loss and the second divergence loss; wherein the second decoder trained to convergence is the decoder in the pre-trained code generation model.

[0011] Further, the original code sample is collected and converted into a logically equivalent compressed form to obtain a compressed code sample, specifically including: removing all spaces, line breaks and comments in the original code sample; performing minimum mapping on variables and function names in the original code sample to obtain a compressed code sample corresponding to the original code sample; wherein the original code sample and its corresponding compressed code sample constitute the original-compressed code pair sample.

[0012] Further, it also includes: based on the pre-constructed reversible mapping table from the original identifier to the compressed identifier, replacing the compressed identifier in the target compressed code with the original identifier one by one, and automatically supplementing the target compressed code with indentation, spaces, line breaks and comments to obtain an anti-compressed code; outputting the target compressed code and / or the anti-compressed code to a given external system.

[0013] In a second aspect, the present application also provides a compressed code generation device, comprising: an instruction information acquisition module, configured to acquire a natural language instruction input by a user and context information; a compressed code generation module, configured to generate a target compressed code according to the natural language instruction and the context information based on a pre-trained code generation model, under the cooperation of an acceleration component and a preset automaton; wherein the code generation model comprises an encoder and a decoder, and is obtained by performing double-path distillation training on a training sample set composed of task instruction samples, context samples and original-compressed code pair samples; the acceleration component is configured to accelerate the inference process of the code generation model, and the preset automaton is configured to perform syntax constraint verification on word groups generated by the code generation model step by step.

[0014] In a third aspect, the present application also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor implements the compressed code generation method according to any one of the above aspects when executing the computer program.

[0015] In a fourth aspect, the present application also provides a non-transitory computer readable storage medium, having a computer program stored thereon, wherein the computer program is executable on a processor to implement the compressed code generation method according to any one of the above aspects.

[0016] The compressed code generation method provided by the present application acquires a natural language instruction input by a user and context information, and generates a target compressed code according to the natural language instruction and the context information based on a pre-trained code generation model, under the cooperation of an acceleration component and a preset automaton; wherein the code generation model comprises an encoder and a decoder, and is obtained by performing double-path distillation training on a training sample set composed of task instruction samples, context samples and original-compressed code pair samples; the acceleration component is configured to accelerate the inference process of the code generation model, and the preset automaton is configured to perform syntax constraint verification on word groups generated by the code generation model step by step. This method generates a target compressed code by using a code generation model, accelerates the inference process by using an acceleration component, and performs real-time syntax constraint verification on the compressed code generation process by using a preset automaton, which can significantly reduce token redundancy during subsequent model inference, is not limited to narrow domain of domain-specific voice, and solves the problem of syntax errors that may be introduced by compressed codes, ensures the consistency of code logic and structure, and thus achieves a good balance among inference efficiency, universality and correctness. BRIEF DESCRIPTION OF DRAWINGS

[0017] In order to more clearly illustrate the technical solutions in the present application or the prior art, the following will briefly introduce the drawings required by the embodiments or prior art description. Obviously, the drawings in the following description are some embodiments of the present application, and other drawings can also be obtained by those skilled in the art without creative labor.

[0018] Figure 1 is a flowchart of the compressed code generation method provided by the embodiment of the present application.

[0019] Figure 2 is a structural diagram of the compressed code generation device provided by the embodiment of the present application.

[0020] Figure 3 is a physical structure diagram of the electronic device provided by the embodiment of the present application. DETAILED DESCRIPTION

[0021] In order to make the objects, technical solutions and advantages of the present application clearer, the technical solutions in the present application will be described clearly and completely below in combination with the drawings in the present application. Obviously, the described embodiments are some embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor are within the protection scope of the present application.

[0022] It should be noted that in recent years, code generation has gone beyond the category of auxiliary programming and is gradually used as a means to improve the accuracy of logical reasoning. For example, the "Chain-of-Code" method emphasizes replacing natural language reasoning by generating and executing code, thereby avoiding language ambiguity. However, the problem is that most of the data sets used to train the model are designed for humans, and the goal is "human efficiency", not "model efficiency". The autoregressive model generates long code token by token, which greatly increases the reasoning delay and the error probability in the reasoning process.

[0023] In existing attempts, natural language instructions are the most common way, but the redundancy and ambiguity of natural language are serious, resulting in huge token consumption and often ambiguous expression. Another solution is multi-language mixed training, which improves the applicability of the model across languages, but becomes a burden in actual single-task scenarios, reducing the efficiency of specific tasks. Some studies have proposed a dedicated DSL (Domain-Specific Language), which performs well in a narrow field, but has poor scalability and cannot be applied across tasks. In summary, the current technical route cannot achieve a good balance between efficiency, universality and correctness.

[0024] In view of this, the present application proposes a new compressed code generation method, specifically, Figure 1 A flowchart of the compressed code generation method provided by the embodiment of the present application is shown.

[0025] As Figure 1 shown, the method comprises: S110, obtaining a natural language instruction and context information input by a user; S120, based on a pre-trained code generation model, generating a target compressed code according to the natural language instruction and the context information under the synergistic action of an acceleration component and a preset automaton; wherein the code generation model comprises an encoder and a decoder, and is obtained by performing double-path distillation training on a training sample set composed of task instruction samples, context samples, and original-compressed code pairs; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on the phrase groups generated by the code generation model step by step.

[0026] The following will be described in detail with respect to steps S110-S120 and related steps.

[0027] S110, obtaining a natural language instruction and context information input by a user.

[0028] It is easy to understand that before generating the compressed code, two types of input information need to be obtained, i.e., the natural language instruction input by the user and the optional context information.

[0029] The natural language instruction refers to the task description provided by the user in the form of natural language, which is used to explicitly generate the target. The natural language instruction includes but is not limited to functional requirement description (such as “implement a quick sorting function”), logical reasoning problem (such as “determine whether a number is a prime number”), test case or behavior example (such as input “[1,2,3] should return [3,2,1]”), algorithm step description, etc.

[0030] The optional context information refers to the auxiliary program or environmental information related to the current task, which is used to improve the coherence and adaptability of the generated code, including but not limited to existing partial code segments (such as function skeleton, class definition), declaration and type information of variables or APIs, external dependencies or running environment constraints (such as “only use standard library”, “compatible with Python3.8”), pre-sequence dialogue history or context in multi-round interaction.

[0031] It should be noted that the context information obtained in this step is optional. If there is no context information, the context information in this step can be empty.

[0032] The natural language instruction input by the user and the optional context information jointly constitute the input of the code generation model, and on this basis, step S120 is further performed.

[0033] In S120, the target compressed code is generated according to the natural language instruction and the context information based on the pre-trained code generation model, under the cooperation of the acceleration component and the preset automaton; the code generation model includes an encoder and a decoder, and is obtained by double-path distillation training according to a training sample set composed of a task instruction sample, a context sample and an original-compressed code pair sample; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on the phrase generated by the code generation model step by step.

[0034] Specifically, the embodiment is pre-trained with a code generation model including an encoder and a decoder. The code generation model is constructed based on a large language model architecture or a Transformer structure, which is not specifically limited here.

[0035] In actual generation of the compressed code, the natural language instruction input by the user and the context information obtained in S110 are input into the encoder of the code generation model together, and a unified semantic representation is obtained by encoding, denoted as a target semantic representation; then, the target semantic representation is input into the decoder of the code generation model, and the target compressed code required is obtained by decoding. The target compressed code is a complete or partial program implementation of a target task, which can be directly run by a subsequent execution engine or interpreter. In a logical reasoning scenario, the final answer can be obtained by executing the target compressed code, thereby realizing the goal of “replacing natural language reasoning with code execution”.

[0036] It should be emphasized that in the decoding process of the code generation model, an acceleration component for accelerating the inference process and a preset automaton for performing syntax constraint verification on the generated phrase are introduced.

[0037] In a specific embodiment, the acceleration component uses MoE (Mixture of Experts) to improve the calculation efficiency and throughput performance of the code generation model in the inference process, while maintaining or even enhancing the expression ability of the code generation model. The core idea of the acceleration component is that not all parameters participate in each calculation, but the most relevant “expert subnetwork” is activated dynamically according to the input to realize “conditional calculation”. This will be expanded in detail in the subsequent embodiments.

[0038] The preset automaton includes a finite automaton (FA) and a pushdown automaton (PDA). The finite automaton is a computing model with a finite number of states, which is transferred between states according to input symbols without additional storage space; the pushdown automaton adds a "stack" as auxiliary storage on the basis of the finite automaton. The finite automaton can only identify regular languages, such as languages described by regular expressions; the pushdown automaton can identify context-free languages, which cannot be identified by the finite automaton.

[0039] It should be further noted that the code generation model is obtained by double distillation training of a training sample set composed of task instruction samples, context samples, and original-compressed code pair samples. The original-compressed code pair sample is composed of an original code sample and a compressed code sample, and the compressed code sample is converted from the original code sample. This will be described in detail in the embodiments below.

[0040] In the embodiment, the natural language instruction input by the user and the context information are obtained, and based on the pre-trained code generation model, the target compressed code is generated according to the natural language instruction and the context information under the cooperation of the acceleration component and the preset automaton; wherein the code generation model includes an encoder and a decoder, and is obtained by double distillation training of a training sample set composed of task instruction samples, context samples, and original-compressed code pair samples; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on the phrase generated by the code generation model step by step. The method generates the target compressed code through the code generation model, accelerates the inference process by using the acceleration component, and performs real-time syntax constraint verification on the compressed code generation process by using the preset automaton, which can significantly reduce the token redundancy in the subsequent model inference, is not limited to the narrow domain of domain-specific speech, and at the same time solves the problem of syntax errors that may be introduced by the compressed code, ensures the consistency of the code logic and structure, and achieves a good balance between inference efficiency, universality and correctness.

[0041] On the basis of the above embodiments, further, the training optimization process of the code generation model will be described in detail below.

[0042] The code generation model is trained by two-way distillation, specifically including: inputting the original code sample and the compressed code sample in the original-compressed code pair sample into the pre-trained teacher model respectively to obtain the first target word group probability distribution and the second target word group probability distribution; inputting the task instruction sample and the context sample into the shared encoder to obtain the unified semantic representation; inputting the unified semantic representation into the first decoder and the second decoder respectively to obtain the first predicted word group probability distribution and the second predicted word group probability distribution; calculating the first cross-entropy loss and the first divergence loss according to the first predicted word group probability distribution, the first target word group probability distribution and the original code sample; calculating the second cross-entropy loss and the second divergence loss according to the second predicted word group probability distribution, the second target word group probability distribution and the compressed code sample; iteratively optimizing the first decoder through the first cross-entropy loss and the first divergence loss, and iteratively optimizing the second decoder through the second cross-entropy loss and the second divergence loss; wherein the second decoder trained to convergence is the decoder in the pre-trained code generation model.

[0043] It is easy to understand that in order to train the code generation model, it is necessary to first construct the training sample set required for training. In the embodiment, each training sample in the training sample set includes a task instruction sample, a context sample and an original-compressed code pair sample. The original-compressed code pair sample is composed of an original code sample and a compressed code sample, and the compressed code sample is converted from the original code sample.

[0044] Specifically, the original code sample can be collected from existing public channels (such as open source code repositories, programming competition platforms, Q&A communities, etc.), and the corresponding task instruction sample and context sample can be obtained. If there is no corresponding task instruction sample and context sample, the corresponding task instruction sample and context sample can be constructed by mining auxiliary information, which is not limited here.

[0045] Further, the original code sample is converted into a logically equivalent compressed form to obtain a compressed code sample, specifically including: removing all spaces, line breaks and comments in the original code sample; minimizing the mapping of variables and function names in the original code sample to ensure that the character length is reduced to the maximum extent without destroying the logic, to obtain the compressed code sample corresponding to the original code sample; wherein the original code sample and its corresponding compressed code sample constitute the original-compressed code pair sample.

[0046] For example, the original code sample "function calculateSum(a,b){return a+b;}" will be compressed into the compressed code sample "function f(a,b){return a+b}". This mapping process uses a reversible hash or dictionary mapping mechanism to provide support for subsequent decompression recovery.

[0047] Thus, a training sample set containing task instruction samples, context samples, and original-compressed code pair samples can be constructed.

[0048] Then, the code generation model is trained and optimized by double-path distillation using the training sample set. Specifically, during training, the code generation model adopts a shared encoder + double-branch decoder structure, that is, it includes a shared encoder and two decoders, which are denoted as a first decoder and a second decoder.

[0049] During each round of training, a pre-trained teacher model is introduced, and the teacher model is used to calculate the n-gram probability distribution of the original code sample and the compressed code sample, respectively, denoted as the first target n-gram probability distribution and the second target n-gram probability distribution, which are used as the training target of the code generation model.

[0050] At the same time, the task instruction samples and the context samples corresponding to the original code samples and the compressed code samples are input into the shared encoder together to obtain a unified semantic representation, denoted as a unified semantic representation.

[0051] Next, the unified semantic representation is input into the first decoder to obtain the output first predicted n-gram probability distribution, and the unified semantic representation is input into the second decoder to obtain the output second predicted n-gram probability distribution.

[0052] Subsequently, the cross-entropy loss is calculated according to the difference between the code corresponding to the first predicted n-gram probability distribution and the original code sample, denoted as the first cross-entropy loss, and the cross-entropy loss is calculated according to the difference between the code corresponding to the second predicted n-gram probability distribution and the code compression sample, denoted as the second cross-entropy loss.

[0053] At the same time, the KL divergence loss is calculated according to the difference between the first predicted n-gram probability distribution and the first target n-gram probability distribution, denoted as the first divergence loss, and the KL divergence loss is calculated according to the difference between the second predicted n-gram probability distribution and the second target n-gram probability distribution, denoted as the second divergence loss.

[0054] Finally, the first decoder is optimized by minimizing the first cross-entropy loss and the first divergence loss, and the second decoder is optimized by minimizing the second cross-entropy loss and the second divergence loss, thereby forming double-path distillation learning of the compressed code and the original code.

[0055] After multiple rounds of training and optimization, the trained first decoder and the trained second decoder are obtained. The trained first decoder targets the original code and outputs a human-readable original code token sequence; the trained second decoder targets the compressed code and outputs a compressed code token sequence.

[0056] The embodiment of the present application provides a compressed code generation method, so when actually generating compressed codes, the code generation model uses an encoder and a second decoder. Of course, if there is a demand for original code generation, the code generation model uses an encoder and a first decoder.

[0057] That is to say, through shared encoder and double-branch (first encoder + second encoder) distillation training, the code generation model can simultaneously learn the corresponding relationship between compressed syntax and original syntax, and support flexible switching between the two output modes in the inference stage.

[0058] On the basis of the above embodiment, further, the actual generation process of the compressed code will be described in detail below.

[0059] Based on the pre-trained code generation model, under the synergistic action of the acceleration component and the preset automaton, the target compressed code is generated according to the natural language instruction and the context information, including: the natural language instruction and the context information are encoded into the target semantic representation through the encoder, and the target compressed code is generated according to the target semantic representation through the decoder; in the decoding process, the inference process of the encoder is accelerated by using the acceleration component, and the grammatical correctness of the word group generated by the encoder is verified by using the preset automaton.

[0060] It is easy to understand that the natural language instruction input by the user is input into the encoder of the trained code generation model together with the optional context information to obtain the encoded output target semantic representation. Then the target semantic representation is input into the decoder (i.e. the second decoder in the above) of the trained code generation model, and under the synergistic action of the acceleration component and the preset automaton, the target compressed code is output.

[0061] The synergistic action of the acceleration component and the preset automaton is that the inference process of the encoder is accelerated by using the acceleration component, and the grammatical correctness of the word group generated by the encoder is verified by using the preset automaton.

[0062] Specifically, in a specific embodiment, the acceleration component includes a plurality of expert sub-networks and a gating network; accordingly, accelerating the inference process of the encoder using the acceleration component includes: inputting the intermediate features of the code generation model in the inference process into the gating network, calculating the selection probabilities corresponding to the plurality of expert sub-networks respectively; determining the target expert sub-network used for forward inference according to the selection probabilities corresponding to the plurality of expert sub-networks respectively; inputting the intermediate features of the code generation model in the inference process into the selected target expert sub-network to obtain the candidate word group probability distribution; weighting and fusing the candidate word group probability distribution according to the selection probability corresponding to the target expert sub-network to obtain a fused result; and the fused result is used to output to the code generation model to predict the next word group.

[0063] Specifically, the acceleration component utilizes MoE to improve the computational efficiency and throughput performance of the code generation model in the inference process, while maintaining or even enhancing the expressiveness of the code generation model. In the present embodiment, the acceleration component includes a plurality of expert sub-networks and a gating network, and the actual number of expert sub-networks can be set according to actual needs, for example, 64, which is not specifically limited here.

[0064] Firstly, the gating network calculates the weights / selection probabilities of each expert sub-network according to the intermediate features of the code generation model in the inference process, and selects a few expert sub-networks (target expert sub-networks) according to a preset rule. The intermediate features include the hidden layer representation corresponding to the current generated token sequence and the task context vector. The preset rule can be that the few expert sub-networks with the top preset ranking in the selection probability are selected as the target expert sub-networks, or the few expert sub-networks with the selection probability higher than a preset probability threshold are selected as the target expert sub-networks, which is not specifically limited here.

[0065] The preset ranking and the preset probability threshold can be set according to actual conditions, for example, the preset ranking is 3 and the preset probability threshold is 93%, which is not specifically limited here.

[0066] After the target expert sub-networks are determined, each target expert sub-network performs forward inference on the intermediate features of the code generation model in the inference process to obtain the candidate next token probability distribution or semantic vector, i.e., the candidate word group probability distribution.

[0067] The candidate word group probability distributions are weighted and fused according to the gating weights, and the fused results are output to the code generation model to predict the next token. The gating weights here are the selection probabilities of each expert sub-network output by the gating network.

[0068] The present embodiment significantly reduces the computational load of a single inference by activating only part of the expert sub-networks, while maintaining the generation accuracy, thereby realizing the acceleration of the compressed code generation process.

[0069] In another specific embodiment, the preset automaton includes a finite automaton and a push-down automaton; accordingly, the preset automaton is used to verify the grammatical correctness of the word groups generated by the encoder step by step, including: using the finite automaton to perform regular language recognition on the word groups generated by the encoder step by step, and / or using the push-down automaton to perform context-free language recognition on the word groups generated by the encoder step by step; in the case that an illegal word group or context-free language appears in the generation path, the backtracking and resampling mechanism is triggered.

[0070] Specifically, the preset automaton in the embodiment can support both finite automaton and push-down automaton, the finite automaton is used to constrain local patterns such as keywords and operators, and the push-down automaton is used to constrain global structures such as statement block nesting and parentheses pairing, and the two can be configured as needed according to the syntax complexity of the specific programming language.

[0071] For example, for a scenario that only contains regular syntax constraints, a finite automaton can be used to complete token-level pattern matching, and the finite automaton is used to perform regular language recognition on the phrase generated by the encoder step by step.

[0072] For example, for a scenario that contains context-free grammar features such as parentheses matching and nested structures, a push-down automaton can be used to perform context-free language recognition on the phrase generated by the encoder step by step.

[0073] For example, both are combined to perform real-time syntax checking on the generated target compressed code: the finite automaton is used to perform regular language recognition on the phrase generated by the encoder step by step, and the push-down automaton is used to perform context-free language recognition on the phrase generated by the encoder step by step.

[0074] If it is found that the generated path contains an illegal phrase or does not conform to the context-free grammar, the system will immediately trigger a backtracking and resampling mechanism (return to the last legal state and resample the next phrase to avoid illegal branches) to avoid syntax errors or logical loopholes. In this way, it can be ensured that even under highly compressed representation, the generated target compressed code still strictly follows the syntax specifications of the programming language.

[0075] On the basis of the above embodiment, further, the following will be described in detail with respect to the decompression process.

[0076] Based on the reversibly mapped table from the original identifier to the compressed identifier constructed in advance, the compressed identifiers in the target compressed code are replaced with the original identifiers one by one, and the target compressed code is automatically supplemented with indentation, space, line break and comment to obtain the decompressed code; the target compressed code and / or the decompressed code is output to a given external system.

[0077] It is easy to understand that when constructing the training sample set of the code generation model, the mapping process from the original, redundant human-readable code to the logically equivalent compressed code is involved, and this mapping process can use a reversible hash or dictionary mapping mechanism, thereby establishing a reversible mapping table from the original identifier to the compressed identifier for variable names, function names, etc.

[0078] In the decompression process, the reversible mapping table can be used to replace the short identifiers (compressed identifiers) in the target compressed code with the original human-readable identifiers (original identifiers) one by one, and according to the preset format rule (or the format template learned in the training stage), the appropriate indentation, space, line break is automatically supplemented for the target compressed code without changing the semantics, and the predefined comment template or document comment structure is inserted as necessary, so that the output decompressed code is suitable for efficient execution by machines and meets the inspection and maintenance needs of human developers.

[0079] It is worth mentioning that based on the above-mentioned decompression method, the decompression process is completely based on deterministic mapping relationship and format recovery rule, without introducing additional decompression code generation model. Of course, in other embodiments, a special decompression generation model can be further introduced for more detailed learning recovery of format and annotation.

[0080] After the decompressed code is recovered, the target compressed code and / or the decompressed code can be output to a given external system, and this process can be realized through a specially provided application interface module.

[0081] For example, in industrial control, robots or embodied intelligent agent tasks, the target compressed code is used for automated execution or embodied intelligent agent control. Specifically, the application interface module directly transmits the target compressed code to the embedded execution environment or script execution engine, which quickly parses and executes in a resource-limited or low-latency environment, thereby reducing the token overhead of communication and reasoning and improving online decision speed.

[0082] For example, in a cloud code evaluation platform, an online judging system or a batch logical reasoning service, the target compressed code is used for large-scale batch processing reasoning service. Specifically, the application interface module sends the target compressed code to the backend execution container or sandbox environment to support large-scale concurrent tasks under limited computing power, thereby improving service throughput.

[0083] For example, for human developers, the decompressed code is used for developer interaction and debugging. Specifically, the application interface module outputs the decompressed and well-formatted source code (decompressed code) to the IDE (Integrated Development Environment), web console or code review system, which is convenient for developers to read, modify and debug, and realizes the cooperation of "machine efficient generation - human efficient understanding".

[0084] For example, in an enterprise internal knowledge base or code asset management system, the application interface module archives the decompressed code together with metadata (such as task description, running log, etc.) for subsequent retrieval, reuse and compliance audit.

[0085] The application interface module can flexibly output target compression codes and / or decompression codes according to different scenes, which meets the machine execution efficiency and takes into account the human understanding and management needs.

[0086] The compression code generation method provided by the application forms a complete closed loop from data compression, model training, inference generation, syntax checking, acceleration scheduling to result recovery. Through this design, the application not only solves the problems of excessive redundant tokens and low inference efficiency in the traditional method, but also guarantees the logical correctness of the generated result and the multi-scene adaptation ability, and has strong universality and practical value.

[0087] In particular, in the prior art, although the natural language instruction is intuitive, it has serious defects in token efficiency and logical rigor; multi-language training enhances cross-border capability, but increases model complexity and burden; DSL sacrifices universality and can only be applied in a narrow domain. In contrast, the application realizes the unity of efficiency, universality and correctness through compression codes, avoids the redundancy of natural language, is not limited to the narrow domain of DSL, and is more suitable for efficient execution of single tasks than the multi-language scheme.

[0088] Foreign models such as StarCoder, DeepSeekCoder, Qwen have tried to experiment in compression code generation, but their methods have not formed a complete system, lack syntax automatic machine checking and MOE acceleration mechanism, and still have deficiencies in the consideration of precision and efficiency. The advantage of the application is to propose an end-to-end complete closed loop scheme, which not only realizes systematic design in data preprocessing, generation checking, inference acceleration, result decompression and other links, but also verifies in experiments that while saving 20%-40% tokens, the model accuracy can be kept stable or even improved.

[0089] Corresponding to the compression code generation method described in the above embodiments, the application also provides a compression code generation device. Specifically, Figure 2 The structure diagram of the compression code generation device provided by the embodiment of the application is shown.

[0090] As Figure 2As shown, the device comprises: an instruction information acquisition module 210, configured to acquire a natural language instruction input by a user and context information; and a compressed code generation module 220, configured to generate a target compressed code according to the natural language instruction and the context information based on a pre-trained code generation model, under the cooperative action of an acceleration component and a preset automaton. The code generation model comprises an encoder and a decoder, and is obtained through two-way distillation training based on a training sample set formed by task instruction samples, context samples, and original-compressed code pairs. The acceleration component is configured to accelerate the inference process of the code generation model, and the preset automaton is configured to perform syntax constraint verification on word groups generated by the code generation model step by step.

[0091] In this embodiment, the instruction information acquisition module 210 acquires a natural language instruction input by a user and context information, and the compressed code generation module 220 generates a target compressed code according to the natural language instruction and the context information based on a pre-trained code generation model, under the cooperative action of an acceleration component and a preset automaton. The code generation model comprises an encoder and a decoder, and is obtained through two-way distillation training based on a training sample set formed by task instruction samples, context samples, and original-compressed code pairs. The acceleration component is configured to accelerate the inference process of the code generation model, and the preset automaton is configured to perform syntax constraint verification on word groups generated by the code generation model step by step. The device generates a target compressed code through a code generation model, accelerates the inference process by using an acceleration component, and performs real-time syntax constraint verification on the compressed code generation process by using a preset automaton, which can significantly reduce token redundancy during subsequent model inference, is not limited to narrow-domain specific speech, solves the problem of syntax errors that may be introduced by compressed codes, ensures the consistency of code logic and structure, and thus achieves a good balance among inference efficiency, universality, and correctness.

[0092] It should be noted that the compressed code generation device provided in the embodiments of the present application can be correspondingly referred to the compressed code generation method described in the above embodiments, and will not be described here again.

[0093] Figure 3 An example of an entity structure diagram of an electronic device is shown in FIG. 1. Figure 3As shown, the electronic device can include a processor 310, a communications interface 320, a memory 330, and a communications bus 340, wherein the processor 310, the communications interface 320, and the memory 330 complete mutual communication through the communications bus 340. The processor 310 can invoke a logical instruction in the memory 330 to execute a compressed code generation method, which includes: obtaining a natural language instruction input by a user and context information; based on a pre-trained code generation model, under the synergistic action of an acceleration component and a preset automaton, generating a target compressed code according to the natural language instruction and the context information; wherein the code generation model includes an encoder and a decoder, and is obtained by two-way distillation training according to a training sample set formed by a task instruction sample, a context sample, and an original-compressed code pair sample; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on a word group generated by the code generation model step by step.

[0094] In addition, the logical instruction in the memory 330 described above can be implemented in the form of a software functional unit and sold or used as an independent product, and can be stored in a computer-readable storage medium. Based on such understanding, the technical solutions of the present application essentially or the part that contributes to the prior art or part of the technical solutions can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a plurality of instructions to make a computer device (which can be a personal computer, a server, or a network device, etc.) execute all or part of the steps of the methods described in various embodiments of the present application. The foregoing storage medium includes: a U disk, a mobile hard disk, a read-only memory (ROM, Read-Only Memory), a random access memory (RAM, Random Access Memory), a magnetic disk or an optical disk, and various program code storage media.

[0095] In another aspect, the present application also provides a non-transitory computer-readable storage medium having stored thereon a computer program, which, when executed by a processor, implements the compressed code generation method provided by the above method, and the method comprises: obtaining a natural language instruction input by a user and context information; based on a pre-trained code generation model, generating target compressed code according to the natural language instruction and the context information under the synergistic action of an acceleration component and a preset automaton; wherein the code generation model comprises an encoder and a decoder, and is obtained by performing double-path distillation training on a training sample set constituted by a task instruction sample, a context sample and an original-compressed code pair sample; the acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform syntax constraint verification on the phrase group generated by the code generation model step by step.

[0096] The device embodiments described above are merely illustrative, wherein the units described as separate components can or can not be physically separate, and the components displayed as units can or can not be physical units, i.e., can be located in one place, or can be distributed on multiple network units. Part or all of the modules can be selected to achieve the purpose of the embodiment scheme according to actual needs. Those skilled in the art can understand and implement without creative labor.

[0097] From the above description of the embodiments, those skilled in the art can clearly understand that the embodiments can be realized by means of software and necessary universal hardware platforms, and of course can also be realized by hardware. Based on such understanding, the above technical solutions can be embodied in the form of a software product, which can be stored in a computer readable storage medium, such as ROM / RAM, magnetic disk, optical disk, etc., and includes a plurality of instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0098] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present application, and not to limit them; although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that: it can still modify the technical solutions recorded in the foregoing embodiments, or make equivalent replacement to some technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating compressed code, characterized in that, include: Obtain natural language commands and contextual information input by the user; Based on a pre-trained code generation model, and with the combined action of an acceleration component and a preset automaton, target compressed code is generated according to the natural language instructions and the context information. The code generation model includes an encoder and a decoder, and is trained by dual-path distillation of a training sample set consisting of task instruction samples, context samples, and raw-compressed code. The acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform grammatical constraint verification on the phrases generated step by step by the code generation model.

2. The compressed code generation method according to claim 1, characterized in that, The pre-trained code generation model, through the synergy of acceleration components and a pre-set automaton, generates target compressed code based on the natural language instructions and the context information, including: The encoder encodes the natural language instructions and the context information into a target semantic representation, and the decoder generates the target compressed code based on the target semantic representation. During the decoding process, the acceleration component is used to speed up the inference process of the encoder, and the preset automaton is used to verify the grammatical correctness of the words generated step by step by the encoder.

3. The compressed code generation method according to claim 2, characterized in that, The acceleration component includes multiple expert subnetworks and a gated network; Accordingly, accelerating the inference process of the encoder using the acceleration component includes: The intermediate features of the code generation model during the inference process are input into the gating network, and the selection probabilities corresponding to the multiple expert sub-networks are calculated respectively. Based on the selection probabilities corresponding to the multiple expert subnetworks, the target expert subnetwork for forward inference is determined; The intermediate features of the code generation model during the inference process are input into the selected target expert subnetwork to obtain the probability distribution of candidate word groups; The probability distributions of the candidate word groups are weighted and fused according to the selection probabilities corresponding to the target expert subnetwork to obtain the fused result; the fused result is used to output to the code generation model to predict the next word group.

4. The compressed code generation method according to claim 3, characterized in that, The preset automata include finite automata and pushdown automata; Accordingly, the step of using the preset automaton to perform grammatical correctness verification on the phrases generated step by step by the encoder includes: The finite automaton is used to perform regular language recognition on the word groups generated step by step by the encoder, and / or the pushdown automaton is used to perform context-free language recognition on the word groups generated step by step by the encoder; If an illegal phrase or context-insensitive language is found in the generated path, a backtracking and resampling mechanism is triggered.

5. The compressed code generation method according to claim 1, characterized in that, When training the code generation model, the code generation model includes a shared encoder and two decoders, which are referred to as the first decoder and the second decoder, respectively. Accordingly, the code generation model is trained using a two-path distillation method, specifically including: The original code samples and compressed code samples in the original-compressed code pair are respectively input into the pre-trained teacher model to obtain the probability distribution of the first target word group and the probability distribution of the second target word group; The task instruction sample and the context sample are input into the shared encoder to obtain a unified semantic representation; The unified semantic representation is input into the first decoder and the second decoder respectively to obtain the first predicted word group probability distribution and the second predicted word group probability distribution. Based on the first predicted word group probability distribution, the first target word group probability distribution, and the original code sample, calculate the first cross-entropy loss and the first divergence loss; based on the second predicted word group probability distribution, the second target word group probability distribution, and the compressed code sample, calculate the second cross-entropy loss and the second divergence loss. The first decoder is iteratively optimized using the first cross-entropy loss and the first divergence loss, and the second decoder is iteratively optimized using the second cross-entropy loss and the second divergence loss; wherein the second decoder trained to convergence is the decoder in the pre-trained code generation model.

6. The compressed code generation method according to claim 1, characterized in that, Collect original code samples and convert them into logically equivalent compressed forms to obtain compressed code samples, specifically including: Remove all spaces, line breaks, and comments from the original code sample; Minimize the mapping of variable and function names in the original code sample to obtain the compressed code sample corresponding to the original code sample; The original code sample and its corresponding compressed code sample constitute the original-compressed code pair sample.

7. The compressed code generation method according to any one of claims 1-6, characterized in that, Also includes: Based on a pre-built reversible mapping table from original identifiers to compressed identifiers, the compressed identifiers in the target compressed code are replaced one by one with the original identifiers, and indentation, spaces, newlines and comments are automatically added to the target compressed code to obtain the decompressed code; Output the target compressed code and / or the decompressed code to a given external system.

8. A compressed code generation device, characterized in that, include: The instruction information acquisition module is used to acquire natural language instructions and context information input by the user; The compressed code generation module is used to generate target compressed code based on the natural language instructions and the context information, in conjunction with the acceleration component and the preset automaton, using a pre-trained code generation model. The code generation model includes an encoder and a decoder, and is trained by dual-path distillation of a training sample set consisting of task instruction samples, context samples, and raw-compressed code. The acceleration component is used to accelerate the inference process of the code generation model, and the preset automaton is used to perform grammatical constraint verification on the phrases generated step by step by the code generation model.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and running on the processor, characterized in that, When the processor executes the computer program, it implements the compressed code generation method as described in any one of claims 1 to 7.

10. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the compressed code generation method as described in any one of claims 1 to 7.

Images