Code generation task reply method and device, medium and electronic equipment

Abstract

The embodiment of the invention discloses a code generation task reply method and device, a medium and electronic equipment. A specific embodiment of the method comprises the steps of obtaining a pre-trained code generation task reply model; obtaining a code generation problem submitted by the target user through the target front end; preprocessing the text data and / or audio data corresponding to the code generation problem to obtain preprocessed feature representation information; inputting the preprocessed feature representation information into a pre-trained code generation task reply model to obtain a code generation reply information set; performing visualization processing on the code generation reply information set, and embedding an interaction control to obtain visual interaction page information; and sending the visual interaction page information to a terminal interface for the target user to view and interact. According to the embodiment, code reply can be accurately realized, and the reply content can be visually processed.

Description

Technical Field

[0001] Embodiments of this disclosure relate to the field of computer technology, and more particularly to code generation task response methods, apparatus, electronic devices, and computer-readable media. Background Technology

[0002] With the deep integration of artificial intelligence and software development, code generation technology has become a core capability in scenarios such as intelligent programming assistance and low-code platforms. Existing code generation technologies are typically based on sequence generation techniques using large language models.

[0003] However, when using existing sequence generation techniques based on large language models to generate code, the following technical problems often arise:

[0004] Technical Issue 1: Traditional code generation models are usually trained based on only a single metric (e.g., syntactic correctness), lacking comprehensive consideration of multi-dimensional rule constraints (e.g., formatting rules, naming conventions) and deep semantic relationships (e.g., logical consistency, context adaptation). This leads to problems such as non-standard formatting and semantic deviation from user needs in practical applications.

[0005] Technical Issue 2: Poor user interaction experience, limited result display, and lack of in-depth interactive capabilities.

[0006] The information disclosed in this background section is only intended to enhance the understanding of the background of the inventive concept, and therefore may contain information that does not constitute prior art known to those skilled in the art. Summary of the Invention

[0007] The summary portion of this disclosure is intended to provide a brief overview of the concepts, which will be described in detail in the detailed description portion. This summary portion is not intended to identify key or essential features of the claimed technical solutions, nor is it intended to limit the scope of the claimed technical solutions.

[0008] Some embodiments of this disclosure provide a method, apparatus, electronic device, and computer-readable medium for training a task response model based on dynamic multi-scale rewards, in order to address one or more of the technical problems mentioned in the background section above.

[0009] In a first aspect, some embodiments of this disclosure provide a code generation task response method, comprising: obtaining a pre-trained code generation task response model, wherein the code generation task response model is trained based on a preset model training objective and using reward loss information corresponding to a comprehensive reward set, the comprehensive reward set being generated using a dynamic multi-scale reward model; obtaining target code generation task information submitted by a target user through a target front-end, the target code generation task information including a code generation question; preprocessing the text data and / or audio data corresponding to the code generation question to obtain preprocessed feature representation information; and inputting the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set.

[0010] The code generating the above task response information set is visualized, and interactive controls are embedded to obtain a visual interactive page. The visual interactive page is then sent to the target user's terminal interface for the target user to view and interact with.

[0011] Secondly, some embodiments of this disclosure provide a code generation task response apparatus, comprising: a first acquisition unit configured to acquire a pre-trained code generation task response model, wherein the code generation task response model is trained based on a preset model training objective and reward loss information corresponding to a comprehensive reward set, the comprehensive reward set being generated using a dynamic multi-scale reward model; a second acquisition unit configured to acquire target code generation task information submitted by a target user through a target front-end, the target code generation task information including a code generation question; a preprocessing unit configured to preprocess the text data and / or audio data corresponding to the code generation question to obtain preprocessed feature representation information; an input unit configured to input the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set as a task response information set; a visualization processing unit configured to visualize the task response information set and embed interactive controls to obtain visualized interactive page information; and a sending unit configured to send the visualized interactive page information to the target user's terminal interface for the target user to view and interact with.

[0012] Thirdly, some embodiments of this disclosure provide an electronic device, including: one or more processors; and a storage device having one or more programs stored thereon, such that when the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any implementation of the first aspect.

[0013] Fourthly, some embodiments of this disclosure provide a computer-readable medium having a computer program stored thereon, wherein the program, when executed by a processor, implements the method as described in any implementation of the first aspect.

[0014] The above embodiments of this disclosure have the following beneficial effects: The code generation task response method of some embodiments of this disclosure can accurately achieve code response and visualize the response content. Based on this, the code generation task response method of some embodiments of this disclosure first obtains a pre-trained code generation task response model. This model is trained based on a preset model training objective and reward loss information corresponding to a comprehensive reward set, which is generated using a dynamic multi-scale reward model. Based on the training data and reward mechanism, a basic model capable of stably generating high-quality code is obtained, providing reliable generation capability support for subsequent generation. Then, the target code generation task information submitted by the target user is obtained through the target front-end. This target code generation task information includes the code generation question. Specific user requirements (e.g., text or voice reflecting user requirements) are collected, and the code generation goal (e.g., implementing the addition of two integers using a Python function) is clarified as the core instruction for model input. Then, the text data and / or audio data corresponding to the code generation question are preprocessed to obtain preprocessed feature representation information. The raw input is converted into structured data (e.g., word embedding vectors, speech features) that the model can understand, eliminating noise and extracting key information to improve model processing efficiency and accuracy. Then, the preprocessed feature representations are input into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set. The model generates multiple code generation response information sets based on features (e.g., different implementation paths) to provide diverse and comparable generation results. Next, the code generation response information set is visualized and interactive controls are embedded to obtain a visual interactive page. The code generation response information is formatted (e.g., syntax highlighting), displayed in a structured manner, and interactive buttons such as "copy" and "run" are added to improve user convenience. Finally, the visual interactive page information is sent to the target user's terminal interface for viewing and interaction. Users view the generated code and interactive controls through terminals (e.g., computers and mobile phones), provide feedback on adjustment needs, and complete the closed loop. Attached Figure Description

[0015] The above and other features, advantages, and aspects of the embodiments of this disclosure will become more apparent from the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.

[0016] Figure 1 This is a flowchart of some embodiments of the code-generating task response method according to this disclosure;

[0017] Figure 2 These are schematic diagrams illustrating the structure of some embodiments of the code generation task response apparatus according to this disclosure;

[0018] Figure 3 This is a schematic diagram of the structure of an electronic device suitable for implementing some embodiments of the present disclosure. Detailed Implementation

[0019] Embodiments of this disclosure will now be described in more detail with reference to the accompanying drawings. While some embodiments of this disclosure are shown in the drawings, it should be understood that this disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein. Rather, these embodiments are provided to provide a more thorough and complete understanding of this disclosure. It should be understood that the accompanying drawings and embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of protection of this disclosure.

[0020] It should also be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings. Unless otherwise specified, the embodiments and features described in this disclosure can be combined with each other.

[0021] It should be noted that the concepts of "first" and "second" mentioned in this disclosure are used only to distinguish different devices, modules or units, and are not used to limit the order of functions performed by these devices, modules or units or their interdependencies.

[0022] It should be noted that the terms "a" and "a plurality of" used in this disclosure are illustrative rather than restrictive, and those skilled in the art should understand that, unless otherwise expressly indicated in the context, they should be understood as "one or more".

[0023] The names of messages or information exchanged between multiple devices in the embodiments of this disclosure are for illustrative purposes only and are not intended to limit the scope of such messages or information.

[0024] This disclosure will now be described in detail with reference to the accompanying drawings and embodiments.

[0025] refer to Figure 1The flowchart 100 illustrates some embodiments of a code generation task response method according to the present disclosure. This code generation task response method includes the following steps:

[0026] Step 101: Obtain the pre-trained code generation task response model.

[0027] In some embodiments, the execution entity of the code generation task response method can be hardware or software. When the computing device is hardware, it can be implemented as a distributed cluster composed of multiple servers or terminal devices, or as a single server or a single terminal device. When the computing device is software, it can be installed in the hardware devices listed above. It can be implemented as multiple software or software modules to provide distributed services, or as a single software or software module. No specific limitations are made here.

[0028] In other embodiments, the execution entity can acquire a pre-trained code generation task response model, wherein the code generation task response model is trained using a preset model training objective and reward loss information corresponding to a comprehensive reward set, which is generated using a dynamic multi-scale reward model. The code generation task response model can be a model for responding to code generation tasks. The input to the code generation task response model can be code generation task information, and the output is a set of task response information. The code generation task can be a task that converts natural language requirements into executable code. The task response information can be code information generated in response to the question corresponding to the code generation task. The comprehensive reward set can be a set of comprehensive reward values ​​that measure the comprehensive reward information of the code generation task response. The comprehensive reward value can be a numerical value that measures the quality of the code generation response, a sum of multi-dimensional rewards that integrate rule rewards (e.g., accuracy rewards, format conformity rewards) and learning rewards (e.g., semantic consistency rewards). The reward loss information can be the model optimization loss value calculated based on the comprehensive reward set during the training of the code generation task response model, used to guide parameter updates. The model training objective can be an indicator that guides the optimization of the code generation task response model. For example, the training objective of the above model can be set as the variance reduction rate of the reward variance set corresponding to the comprehensive reward set being less than 5%. The aforementioned dynamic multi-scale reward model can include: a rule-based reward layer, a learning reward layer, and a gated fusion network (dynamic fusion layer). This dynamic multi-scale reward model can be a model that dynamically fuses multi-dimensional rewards (rule-based rewards and learning rewards) through a gated fusion network.

[0029] In practice, the above code generation task response model can be either the Generative Pre-trained Transformer 4 model or the T5 model.

[0030] Step 102: Obtain the target code generation task information submitted by the target user through the target front-end.

[0031] In some embodiments, the aforementioned executing entity can obtain target code generation task information submitted by the target user through the target front-end. The target front-end can be an interface carrier directly operated by the user (e.g., a webpage, the front-end module of an app). The target user can be an entity that submits code generation requirements through the front-end (e.g., a developer, student, or technician). The aforementioned target code generation task information can be a specific code requirement description submitted by the target user, which may include task objectives and language requirements. For example, "Write a function in JavaScript to update the total price of a shopping cart in real time, requiring compatibility with mainstream browsers."

[0032] As an example, firstly, the target front-end receives the user's code generation request through an interactive interface (e.g., input box, voice button). Then, the input is cleaned (e.g., redundancy removal, speech-to-text conversion). Secondly, the cleaned input is formatted and standardized. Finally, the processed code generation request is transmitted to the back-end model processing module.

[0033] Step 103: Preprocess the text data and / or audio data corresponding to the code generation problem to obtain preprocessed feature representation information.

[0034] In some embodiments, the execution entity may preprocess the text data and / or audio data corresponding to the code generation problem to obtain preprocessed feature representation information. The text data may be a description of the code generation problem input by the target user (e.g., "Implement a quicksort function using Python"). The audio data may be a code generation request input by the target user via voice (e.g., "Write me a Java file reading function"). The preprocessed feature representation information may be a high-dimensional vector (e.g., word embedding vector, speech feature vector) that can be processed by the model, and this high-dimensional vector may include the problem semantics, language type, and encoded information requested by the user.

[0035] As an example, if the data for the code generation problem is text data, firstly, a language-specific tokenizer (e.g., jieba for Python, Stanford CoreNLP for Java) is used to split the text data into tokens. Then, a pre-trained language model (e.g., CodeBERT) is used to convert the tokens into low-dimensional vectors (e.g., 768-dimensional). Finally, a bidirectional Transformer encoder is used to extract semantic features, generating context-aware vectors (e.g., 512-dimensional). If the data for the code generation problem is audio data, a speech recognition model (e.g., Whisper) can be used to convert the audio data into text data. Subsequent processing steps are the same as those for text data processing and will not be repeated here.

[0036] Step 104: Input the preprocessed feature representation information into the pre-trained code generation task response model to obtain the code generation response information set, which serves as the task response information set.

[0037] In some embodiments, the execution entity can input the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set. This task response information set can be a collection of code response information generated by the code generation task response model that meets the requirements. For example, if a target user requests "write quicksort in Python," the code generation task response model returns five code generation response messages, each candidate code including: code content, comments, and a time complexity description.

[0038] Step 105: Visualize the task response information set and embed interactive controls to obtain visual interactive page information.

[0039] In some embodiments, the aforementioned executing entity may visualize the task response information set and embed interactive controls to obtain visualized interactive page information. The interactive controls may be functional components (e.g., buttons, links, input boxes) on the page that support user operations. The visualized interactive page information may be page data that displays the code generation results in a visual form (e.g., highlighted code, interactive buttons). For example, the page displays Python code (keyword highlighting), a "Copy Code" button, a "Run Preview" link, and a "Time Complexity" description.

[0040] As an example, first, the code above generates a set of response information, which is then syntax-highlighted (e.g., keywords are marked using Prism.js). Then, interactive controls are embedded (e.g., a copy button, a run preview link). Finally, the code content, additional explanations (e.g., complexity), and control configurations are combined to generate complete page data.

[0041] Step 106: Send the visual interactive page information to the target user's terminal interface for the target user to view and interact with.

[0042] In some embodiments, the aforementioned executing entity may send the aforementioned visual interactive page information to the target user's terminal interface for the target user to view and interact with. The terminal interface may be a display interface directly operated by the target user (e.g., a mobile phone screen or a computer screen) for display and interaction. In practice, firstly, the data related to the visual page (e.g., HTML / CSS / JS code) is pushed to the terminal device. Then, the front-end framework (e.g., React) parses the data and renders the page. Finally, the target user views the code content and additional instructions on the screen and operates interactive controls (e.g., clicking the copy button).

[0043] In some optional implementations, the above code generation task response model is trained through the following steps:

[0044] The first step is to obtain a code generation task information set and an initial code generation task response model. The code generation task information set can be a collection of task information for code generation tasks. For example, this information could be a dataset of user questions such as "Write a file reading function in Java" or "Implement a shopping cart total price calculation using JavaScript."

[0045] The initial code generation task response model described above can be a code generation task response model that has not yet finished training. In practice, the above code generation task response model can be a Generative Pre-trained Transformer 4 model or a T5 model.

[0046] The second step, for the initial code generation task response model mentioned above, can be performed as follows:

[0047] In sub-step one, in response to the initial code generation task response model having fewer recent consecutive training iterations than the target number and / or the variance reduction rate corresponding to the recent consecutive training iterations not meeting the target condition, a code generation task response information set corresponding to the code generation task information set is generated using the initial code generation task response model, based on the task complexity corresponding to each code generation task information set. Here, the recent consecutive training iterations can be the number of uninterrupted consecutive training rounds during the code generation task response model's training. The variance reduction rate corresponding to the recent consecutive training iterations can be the variance reduction rate corresponding to the comprehensive reward set in each training session. The task complexity can be an indicator that quantifies task difficulty using task structure complexity and resource consumption complexity. For example, the task complexity can be 0.5.

[0048] Sub-step two involves using the aforementioned dynamic multi-scale reward model to determine the comprehensive reward set corresponding to the code generation task response information set. This dynamic multi-scale reward model can be a comprehensive reward mechanism combining rule-based rewards and learning rewards, dynamically adjusting reward weights by evaluating the generated code generation task response information across multiple dimensions (e.g., accuracy, format conformity, language consistency, and semantic consistency). There is a one-to-one correspondence between the comprehensive reward set and the code generation task response information set. The comprehensive reward set can be the set of comprehensive reward values ​​from the dynamic multi-scale reward model for the code generation task response information.

[0049] Sub-step three involves updating the initial code generation task response model using the aforementioned comprehensive reward set, resulting in an updated code generation task response model. This updated model can be an improved version of the initial code generation task response model, generated by optimizing the parameters of the initial model using the comprehensive reward set.

[0050] Sub-step four: In response to the target number of most recent consecutive training iterations being accumulated, obtain the reward variance set from the most recent number of training iterations. Here, the reward variance is the variance corresponding to the comprehensive reward set in each training iteration. The target number can be a preset threshold for the total number of consecutive training iterations, used to control the termination condition of the task response model training. The most recent consecutive training iterations can be the number of uninterrupted consecutive training rounds traced back from the current time point.

[0051] Sub-step five: In response to the variance reduction rate corresponding to the aforementioned reward variance set satisfying the target condition, the updated code generation task response model from the last training iteration is determined as the code generation task response model. The aforementioned target condition can be a pre-set variance reduction rate threshold corresponding to the reward variance set. For example, the variance reduction rate threshold could be 5%, and training stops when this threshold is met.

[0052] When training existing code generation models based on large language models, the following technical problems are often encountered: "low model training efficiency and waste of computational resources." The factors leading to these problems are often as follows: During the training process of existing code generation models based on large language models, a fixed number of samples is often used, resulting in oversampling for simple tasks or undersampling for complex tasks. Therefore, the following solution can be adopted:

[0053] In some optional implementations of certain embodiments, the execution entity may, in response to the initial code generation task response model corresponding to a continuous training count less than the target count, generate a code generation task response information set for the code generation task information set based on the task complexity corresponding to each code generation task information set, using the initial code generation task response model. This may include the following steps:

[0054] The first step, for each code generation task training information in the code generation task information set, is to perform the following steps:

[0055] The first sub-step involves determining the task structure complexity corresponding to the task information generated by the above code, using the number of levels, subtasks, targets, and baseline values. In practice, the task structure complexity can be obtained using the following formula:

[0056]

[0057] Among them, t a This refers to the task structure complexity. The number of levels can be a tree-like hierarchy after decomposing the information for each code generation task. For example, a code generation task (e.g., "Write a user login verification function in Python") can be decomposed into three levels: requirement understanding and rule confirmation, module partitioning and logic design, and code implementation and testing verification. The number of subtasks refers to the number of parallel tasks within the same level. For example, in the "requirement understanding and rule confirmation" level, the two subtasks corresponding to this level include: extraction of key requirement elements (e.g., "username and password verification") and confirmation of implicit requirements (e.g., "password needs to be encrypted and stored"). The number of objectives can be the number of independent metrics to be optimized. For example, independent metrics to be optimized may include: accuracy, latency, and security. The baseline value can be a reference value set based on domain experience, used to standardize complexity calculations. For example, the baseline value for a programming task is 10. The task structure complexity can be a comprehensive index value that quantifies the number of levels, subtasks, and objectives, reflecting the overall difficulty of task decomposition, parallel processing, and objective coordination.

[0058] The second sub-step involves using the task's duration, the baseline task's duration, the task's data size, the baseline task's data size, and the percentage of GPU memory to determine the resource consumption complexity corresponding to the task training information generated by the above code.

[0059] In practice, the resource consumption complexity can be obtained using the following formula:

[0060]

[0061] Among them, t bThis refers to the complexity of resource consumption. The task consumption time mentioned above can be the processor computation time consumed to complete the task corresponding to each of the above code generation task information. The benchmark task mentioned above can be a typical task instance completed under the same or similar conditions. Its consumption time, data size, and other parameters serve as a reference standard to evaluate the complexity of the current task's resource consumption. The benchmark task consumption time mentioned above can be the processor computation time consumed by the benchmark task as a reference standard, in "hours". It can be determined through standardized testing (e.g., CPU consumption time measured under fixed data size and fixed algorithm) and is used to measure the relative computational load of the code generation task training information. The task data size mentioned above can refer to the total amount of input data for the code generation task training information, in bytes (B), the number of data samples, and the file size (which must be consistent with the benchmark data size unit). The benchmark task data size mentioned above can be the benchmark task data size as a reference standard, with the unit consistent with the actual data size. Usually, the benchmark task data size and benchmark task consumption time are correlated to form a standardized "data and computation" baseline, which serves as a reference standard for the complexity of the current task's resource consumption. The aforementioned video memory percentage can be the proportion of video memory resources used during task execution to the available video memory.

[0062] The third sub-step involves weightedly combining the task structure complexity and resource consumption complexity to obtain the task complexity. This weighted combination can be a weighted summation. For example, the weights for the task structure complexity and resource consumption complexity can be 0.5.

[0063] The fourth sub-step involves adjusting the number of code generation task response information samples corresponding to each code generation task information in the target task information set, based on the aforementioned task complexity.

[0064] Specifically, the number of samples G of the response information generated by the code above can be determined through the following relationship:

[0065] Response to 0 < t c ≤0.25, defined as a simple task, G=8;

[0066] The response is at 0.25 < t c ≤0.5, defined as a general task, G=16;

[0067] Response at 0.5 < t c ≤0.75 is defined as a difficult task, G=32;

[0068] The response is at 0.75 < t c ≤1, defined as a complex task, G=64.

[0069] The second step involves using the initial code generation task response model described above to generate code generation task information corresponding to each code generation task in the code generation task training set, resulting in a code generation task response information set. The number of task responses corresponding to the code generation task response information is the same as the number of response samples. This code generation task response information set includes a predetermined proportion of code generation task response information representing incorrect responses. These incorrect responses can be syntax errors (e.g., missing parentheses), logical errors (e.g., incorrect loop conditions), or responses that do not meet user requirements (e.g., the user requested a Python function but generated Java code).

[0070] The aforementioned optional steps and related content, as an inventive point of this disclosure, solve the aforementioned technical problem of "low model training efficiency and wasted computing resources." The factors leading to this technical problem are often as follows: traditional methods for training response models for code generation tasks often use a fixed number of samples, resulting in oversampling for simple tasks or undersampling for complex tasks. Oversampling for simple tasks leads to wasted computing resources, while undersampling for complex tasks leads to low model training efficiency. This invention dynamically adjusts the number of samples for the response information of code generation tasks based on task complexity, achieving a match between computing resources and task difficulty, thereby improving model training efficiency and saving computing resources.

[0071] In some optional implementations of certain embodiments, the executing entity may utilize the dynamic multi-scale reward model to determine the comprehensive reward set corresponding to the code generation task response information set, which may include the following steps:

[0072] The first step is to perform a comprehensive reward generation process for each code generation task response in the aforementioned code generation task response set:

[0073] The first sub-step utilizes the rule-based reward layer of a dynamic multi-scale reward model to determine the first rule-based reward information for the code generation task response information. This first rule-based reward information is obtained by weighting accuracy reward, format reward, and language consistency reward. The rule-based reward layer can be a scoring layer based on the accuracy, format reward, and language consistency of the code generation task response information. This rule-based reward layer can include three parallel sub-modules: an accuracy reward module, a format reward module, and a language consistency module. Each parallel sub-module includes: an input layer (code generation task response information), a processing layer, and an output layer (the output layer outputs an accuracy reward (e.g., 1 point), a format reward value (e.g., 0.3 points), and a language consistency reward value (e.g., 0 points)). The processing layer performs feature extraction and scoring calculation on the input code generation task response information, outputting standardized reward values ​​(e.g., accuracy reward, format reward, and language consistency reward). The accuracy reward can assess the degree of matching between the content generated by the task response model and the "true correct answer" or "task objective," reflecting the factual correctness and logical rigor of the output. For example, the user task is "Write a function in Python to determine leap years." If the generated code correctly implements the logic of "divisible by 4 but not by 100, or divisible by 400" (e.g., returning True for input 2000, returning False for input 1900), the accuracy reward is high (e.g., 1 point); if the logic is incorrect (e.g., only checking divisibility by 4), the accuracy reward is low (e.g., 0.2 points). The above format reward can measure whether the content generated by the task response model conforms to predefined structural specifications or style requirements, reflecting the organization and readability of the output. For example, the user may require that "the code must have comments and be indented by 4 spaces." If the generated code has proper indentation (e.g., 4 spaces for each loop / conditional block), clear variable names (e.g., year instead of y), and includes comments explaining the judgment logic (e.g., "#Leap year rule: divisible by 4 but not by 100, or divisible by 400"), the formatting reward is high (e.g., 1 point); if the indentation is messy (e.g., mixed use of spaces and tabs), variable names are arbitrary (e.g., a), and there are no comments, the formatting reward is low (e.g., 0.3 points). The aforementioned language consistency reward can assess the consistency of the generated content in terms of terminology, logical coherence, and stylistic uniformity, reflecting the professional rigor and reliability of the output. The first rule reward information can be a weighted fusion of three sub-rewards: accuracy reward, formatting reward, and language consistency reward, measuring whether each code generation task response conforms to domain norms and providing compliance signals for model optimization. For example, the user explicitly requests "written in Python."If the generated code is Python (e.g., using `def` to define functions and `print` to output results), the language consistency reward is high (e.g., 1 point). If Java code is generated (e.g., using `public class` to define classes), even if the functionality is correct, the language consistency reward is still low (e.g., 0 points).

[0074] The second sub-step involves using the learning reward layer of the aforementioned dynamic multi-scale reward model to execute the first learning reward information generation step:

[0075] Sub-step one involves determining the standard response information set corresponding to the code generation task response information. The learning reward layer can be a scoring layer based on the semantic consistency of the code generation task response information. This learning reward layer may include an encoder module, a reference vector pool module, and a similarity calculation module. The encoder module can be a Transformer encoder. The reference vector pool module can be a storage module for the latent vector set corresponding to the standard response information set. The standard response information set can be high-quality responses annotated by domain experts, or high-quality responses trained on a large-scale, high-quality dataset. The similarity calculation module can be a module in the learning reward layer that determines cosine similarity.

[0076] Sub-step two involves determining the first hidden vector corresponding to the response information of the code generation task. This first hidden vector of the learning reward layer is a high-dimensional vector representation of the response information of the code generation task, obtained by encoding it using a deep model. This deep model can be either a BERT model or a GPT model.

[0077] Sub-step three involves determining the second latent vector corresponding to each standard response in the aforementioned standard response information set, thus obtaining the second latent vector set. Here, the second latent vector is a high-dimensional vector representation obtained by encoding the standard response information using a deep model. This deep model can be either a Transformer model or a GPT model.

[0078] Sub-step four involves generating the mean of the hidden vectors corresponding to the second hidden vector set. In practice, the mean of the second hidden vector can be obtained by averaging each hidden vector in the second hidden vector set dimension by dimension. The first learning reward information can be generated by calculating the cosine similarity between the mean of the first hidden vector and the mean of the second hidden vector. This cosine similarity measures the semantic consistency between the response information of each code generation task and the standard response information. A higher value indicates that the response information of the code generation task better meets the semantic consistency requirements, providing an optimization signal of "semantic rationality" for model optimization.

[0079] Sub-step five involves determining the first learning reward information as the cosine similarity between the first hidden vector and the mean of the first hidden vector. This first learning reward information can be generated by calculating the cosine similarity between the mean of the first and second hidden vectors. The cosine similarity measures the semantic consistency between the response information of each code generation task and the standard response information. A higher value indicates that the response information of the code generation task better meets the semantic consistency requirements, providing an optimization signal of "semantic rationality" for model optimization.

[0080] In practice, learning rewards are obtained through the following formula:

[0081] r leam =sim(h y ,h ref ),

[0082] Among them, h y Generate the first latent vector corresponding to the task response information for the above code. ref r is the mean of the second latent vector set. learn As a learning reward. sin() represents the cosine similarity function.

[0083] The third sub-step involves fusing the first rule reward information and the first learning reward information using the gated fusion network included in the aforementioned dynamic multi-scale reward model to generate a comprehensive reward corresponding to the code generation task response information. This comprehensive reward set can be the set of comprehensive reward values ​​for each task response after the rule reward and learning reward are dynamically weighted through the gated fusion network.

[0084] In practice, the rule-based rewards and learning rewards mentioned above can be combined using a gated fusion network, as shown in the following formula:

[0085] r final =αr rule +(1-α)r learn ,

[0086] Where, r final For reward fusion, α is the weight when fusion rewards. The weight α can be adaptively adjusted through a gating fusion network. For example, in code generation tasks related to mathematical logic, the weight is tilted towards rule-based rewards. rule As a reward for the rules. learn As a reward for learning.

[0087] The above-mentioned optional and related content, as an inventive point of this disclosure, solves the technical problem of "reward model bias" mentioned in the background art. The reasons for reward model bias are often as follows: traditional methods for reinforcing and fine-tuning code task response models (e.g., GRPO, Group Relative Policy Optimization) rely on a single reward model (e.g., based solely on answer correctness) and cannot provide multi-dimensional feedback (e.g., format, semantic consistency, etc.). Solving the above factors can resolve the reward model bias problem.

[0088] However, when using the above scheme to determine the comprehensive reward set corresponding to the code generation task response information set, the following technical problem is often encountered: "Insufficient resource utilization, with high-complexity code generation tasks delayed due to insufficient resources." The factors leading to this technical problem are often as follows: failure to dynamically adjust task resource quotas in real-time based on hardware resource status, resulting in delays for high-complexity tasks due to insufficient resources; and fixed task priorities, which cannot adapt to differences in resource requirements under different scenarios. Therefore, the following scheme can be adopted:

[0089] Optionally, the aforementioned implementing entity can utilize the aforementioned initial dynamic multi-scale reward model to determine the comprehensive reward set corresponding to the code generation task response information set, which may include the following steps:

[0090] The first step is to read the resource status information corresponding to the real-time status data of the current hardware resources from the real-time hardware resource display terminal. This real-time hardware resource display terminal can be a system or interface that monitors and visualizes hardware resource usage in real time (e.g., a server management platform), supporting the dynamic display of the CPU and GPU resource status. The real-time status data can be the specific usage values ​​of the hardware resources at the current moment (e.g., CPU core utilization, GPU memory utilization). The resource status information is a structured description of the real-time status data, for example, "GPU memory utilization: 50% used; CPU core utilization: 85% utilization." In practice, firstly, the real-time usage status data of the current hardware resources can be obtained through the hardware monitoring application programming interface (e.g., nvidia-smi for GPUs, psutil library for CPUs). Then, the real-time usage status data is structured to obtain the resource status information.

[0091] The second step involves constructing a task priority matrix using the task complexity set corresponding to the code generation task information set, the resource status information, and predefined priority tags. The task complexity set can be the set of task complexities for each code generation task in the code generation task information set. The predefined priority tags can be classification labels pre-defined by the system or user to identify task priorities. For example, a core algorithm development task can be defined as a high-priority task, with a priority tag score of 0.9. A regular function development task can be defined as a medium-priority task, with a corresponding priority tag score of 0.6. An auxiliary function development task can be defined as a low-priority task, with a corresponding priority tag score of 0.3.

[0092] As an example, the task priority matrix above can be a 3×3 matrix, where each row represents the task complexity, resource adaptation weight, priority label, and priority score corresponding to the code generation task information set. The first column represents the task complexity corresponding to each code generation task. The second column represents the resource adaptation weight corresponding to the resource status information. For example, resource status information may include GPU memory utilization (e.g., 45%) and CPU core utilization (e.g., 30%). The resource adaptation weight corresponding to the resource status information can be the average of the GPU resource weight (e.g., GPU resource weight = 1 - GPU memory utilization) and the CPU resource weight (e.g., CPU resource weight = 1 - CPU core utilization). The third column represents the score corresponding to the priority label for each code generation task.

[0093] The third step involves generating dynamic resource allocation information for tasks using the aforementioned task priority matrix and resource status information. This dynamic allocation information includes the resource quota for each code generation task in the code generation task information set. Specifically, this dynamic allocation information can be the specific resource information (e.g., GPU memory capacity, CPU thread count) dynamically allocated to each code generation task based on task priority and real-time hardware status. The resource quota can be the specific amount of hardware resources allocated to a single code generation task (e.g., code generation task a has a maximum available GPU memory of 5GB and a CPU core count of 4).

[0094] As an example, firstly, the corresponding priority score set is determined based on the task priority matrix described above. The priority score can be obtained by multiplying the elements of each row in the task priority matrix (task complexity, resource adaptation weight, and the score corresponding to the priority label) sequentially. Then, the priority score set is normalized (e.g., the sum of the priority scores in the set is 1). Next, the GPU unutilized rate and CPU available rate are obtained using the GPU memory utilization and CPU core utilization from the resource status information. Finally, the priority score corresponding to each code generation task is multiplied by the GPU unutilized rate and CPU unutilized rate respectively to obtain the resource quota corresponding to each code generation task. In practice, the resource quota may include the number of available CPU threads and the available GPU memory capacity for each code generation task.

[0095] The fourth step involves retrieving resource quotas corresponding to each code generation task from the various hardware resource devices corresponding to the real-time hardware resource display terminal. This is then used to determine the second rule reward information set and the second learning reward information set corresponding to the code generation task response information set, utilizing the aforementioned dynamic multi-scale reward model. The aforementioned hardware resource devices can be specific hardware devices providing computing, storage, and other resources (e.g., graphics processing units and central processing units). In practice, available resources (e.g., 8GB of remaining video memory on the graphics processor, 4 idle cores on the central processing unit) can be obtained in real-time using hardware monitoring tools (e.g., the nvidia-smi command for graphics processors, the psutil library for central processing units). Combined with the task priority matrix, resource quotas are calculated and allocated (e.g., task b receives 4GB of video memory and 2 threads). The generation of the second rule reward information and the second learning reward information is described in the same way as the generation of the first rule reward information and the first learning reward information, and will not be repeated here.

[0096] The fifth step involves retrieving the resource quotas corresponding to each code generation task from the various hardware resource devices corresponding to the real-time hardware resource display terminal. Then, the second rule reward information set and the second learning reward information set are fused through the gated fusion network included in the dynamic multi-scale reward model to generate a comprehensive reward set corresponding to the code generation task response information set.

[0097] In practice, the rule-based rewards and learning rewards mentioned above can be combined using a gated fusion network, as shown in the following formula:

[0098] r final =αr rule +(1-α)r learn ,

[0099] Where, r finalFor reward fusion, α is the weight when fusion rewards. The weight α can be adaptively adjusted through a gating fusion network. For example, in code generation tasks related to mathematical logic, the weight is tilted towards rule-based rewards. rule As a reward for the rules. learn As a reward for learning.

[0100] The aforementioned optional steps and related content, as an inventive point of this disclosure, solve the aforementioned technical problem of "insufficient resource utilization, and delays in high-complexity code generation tasks due to insufficient resources." Factors leading to this technical problem often include: failure to dynamically adjust task resource quotas based on real-time hardware resource status, resulting in delays in high-complexity tasks due to insufficient resources; and fixed task priorities, which cannot adapt to varying resource requirements in different scenarios. This invention addresses this problem by dynamically allocating resource quotas based on real-time hardware status and task complexity, avoiding resource idleness or overload, and improving hardware resource utilization.

[0101] In some optional implementations, the executing entity can use the comprehensive reward set to update the initial code generation task response model to obtain an updated code generation task response model, which may include the following steps:

[0102] The first step is that the code generation task response information set includes: code generation task response information groups with multiple response types. These multiple response type response information groups can be obtained by grouping the code generation response information of the same code generation task according to different implementation methods (e.g., recursive implementation, iterative implementation, and loop implementation), with each group corresponding to one response type.

[0103] The second step is to determine the intra-group normalized reward value set of the comprehensive reward subset corresponding to the code generation task response information group for each response type, resulting in multiple intra-group normalized reward value sets. These intra-group normalized reward value sets can be sets of values ​​obtained by standardizing the comprehensive reward values ​​of all candidate codes in the code generation task response information group for the same response type to eliminate intra-group distribution bias.

[0104] In practice, the normalized reward value within a group can be obtained using the following formula:

[0105]

[0106] Among them, A intra This is the group-normalized reward value. μ group and σ group These are the mean and standard deviation of the comprehensive reward set corresponding to all code generation task response information in the same code generation task response information group. intra The overall reward value for generating task response information for code within the group.

[0107] The third step is to determine the inter-group normalized reward value set of the comprehensive reward set corresponding to the above code generation task response information set. This inter-group normalized reward value set can be a set of values ​​obtained by globally standardizing the normalized reward values ​​of the above code generation task response information set to eliminate reward bias caused by differences in different response types.

[0108] In practice, the normalized reward value between groups can be obtained using the following formula:

[0109]

[0110] Among them, A inter This represents the normalized reward value between groups. global and σ global These are the mean and standard deviation of the comprehensive reward set corresponding to the response information of all code generation tasks, respectively. inter The overall reward value for generating task response information for inter-group codes.

[0111] The fourth step is to generate the first probability distribution corresponding to the above-mentioned multiple intra-group normalized reward value sets, and to generate the second probability distribution corresponding to the above-mentioned inter-group normalized reward value sets.

[0112] In practice, the first probability distribution and the second probability distribution can be obtained using the following formulas:

[0113]

[0114] Where T is a temperature coefficient used to adjust the sharpness of the distribution, the default value is T = 1. P intra (r j ) represents the first probability distribution. P inter (r j Let be the second probability distribution. J represents the total number of code generation task responses. Jj represents the sequence index of the code generation task response, used to identify its position. rj represents the j-th code generation task response in the set. A intra (r j ) represents r j Normalized dominance value within the group.

[0115] Fifth, determine the reward-loss information based on the relative entropy regularization constraint formula, the first probability distribution, and the second probability distribution mentioned above. This reward-loss information can be a loss value derived from the difference between the first and second probability distributions based on the relative entropy dispersion measure, used to constrain distribution characteristics.

[0116] In practice, reward loss information can be obtained using the following formula:

[0117]

[0118] Among them, the above D KL To reward information on losses. (P) intra (r j ) represents the first probability distribution. P inter (r j ) represents the second probability distribution.

[0119] The sixth step involves using the aforementioned reward and loss information to update the model parameters of the initial code generation task response model via gradient ascent, resulting in the updated code generation task response model.

[0120] In practice, the model parameters of the initial code generation task response model can be updated using the following formula:

[0121]

[0122] Where θ represents the model parameters. The learning rate (e.g., 2e-6), This represents the gradient of the policy. W() is the confidence weighting function for the normalized dominance value. x is the weighted normalized dominance value, σ() is the variance function, and τ is the smoothing term (which can be set to 1e-5). Where L GRPO-new =L GRPO -γ·D KL D KL The reward is the loss information. γ is the coefficient of the hierarchical relative entropy regularization constraint term. L GRPO These are the computational terms using the original GRPO algorithm. This represents the importance sampling ratio. Where π... θ (a|s) represents the probability of choosing action a in state s, determined by the parameter θ of the code generation task response model. θ ( ) represents the policy function of the updated code generation task response model. θ represents the parameters of the updated code generation task response model. θ represents the probability of choosing action a in state s, and is a parameter of the code-generated task response model. old Decide. Generate a strategy function for the task response model in the code before the update. θ oldThe parameters of the code generation task response model before the update are defined. `s` represents the complete representation of the information corresponding to the code generation task, including the information required by the code generation model (e.g., the user-submitted code requirement statement, the code snippet of the current code response, generated variables, and language constraints) and environmental feedback (e.g., the comprehensive reward value corresponding to the generated code response). `a` represents the output or operation chosen by the code generation task response model in a given state `s`. Among them, A inter This represents the normalized reward value between groups. A intra This represents the normalized reward value within the group. The within-group constraint strength β1 = 0.01, and the between-group constraint strength β2 = 0.05. Indicates measurement With π θ The distribution difference of (a|s). min() is the minimum value function. clip() is the clipping function. ∈ is the clipping threshold (usually between 0.1 and 0.2), used to limit the update magnitude of the parameters of the code generation task response model, and β is the original relative entropy divergence penalty coefficient (for example, it can be set to 0.02).

[0123] In some optional implementations, the execution entity may, in response to the initial code generation task response model failing to reach the cumulative target number of consecutive training iterations and / or the variance reduction rate corresponding to the reward variance set failing to meet the target condition, use the updated code generation task response model as the initial code generation task response model and execute the above training steps again. Here, the most recent consecutive training iterations can be the number of consecutive training rounds without interruption during the training of the code generation task response model. The variance reduction rate corresponding to the most recent consecutive training iterations can be the variance reduction rate corresponding to the comprehensive reward set in each training session. The target condition can be a pre-set threshold for the variance reduction rate of the reward variance set (e.g., a reduction rate < 5% for 3 consecutive rounds).

[0124] The above embodiments of this disclosure have the following beneficial effects: the code generation task response method of some embodiments of this disclosure can improve the performance of the code generation task response model, accelerate training efficiency, enhance robustness, and improve multi-task generalization ability. Specifically, the reasons for the poor performance of related code generation task response models are: reward model bias, high variance problem, poor multi-task adaptability, and insufficient noise robustness. Based on this, the code generation task response method of some embodiments of this disclosure firstly obtains a pre-trained code generation task response model, wherein the code generation task response model is trained using the reward loss information corresponding to the comprehensive reward set through a preset model training objective, and the comprehensive reward set is generated using a dynamic multi-scale reward model. Based on the training data and reward mechanism, a basic model that can stably generate high-quality code is obtained, providing reliable generation capability support for subsequent generation. Then, the code generation question submitted by the target user is obtained through the target front-end. The specific user requirements (e.g., text / voice) are collected, and the code generation goal (e.g., using a Python function to add two integers) is clarified as the core instruction for model input. Next, the text and / or audio data corresponding to the code generation problem are preprocessed to obtain preprocessed feature representation information. The original input is converted into structured data that the model can understand (e.g., word embedding vectors, speech features), noise is eliminated and key information is extracted, improving the model's processing efficiency and accuracy. Then, the preprocessed feature representation information is input into the pre-trained code generation task response model to obtain a code generation response information set. The model generates multiple code generation response information sets based on features (e.g., different implementation paths) to provide diverse and comparable generation results. Next, the code generation response information set is visualized and interactive controls are embedded to obtain a visual interactive page. The code generation response information is formatted (e.g., syntax highlighting), structured, and interactive buttons such as "copy" and "run" are added to improve the user's ease of viewing and operation. Finally, the visual interactive page information is sent to the target user's terminal interface for viewing and interaction. Users view the generated code and interactive controls through terminals (e.g., computers and mobile phones), provide feedback on adjustment needs, and complete the closed loop.

[0125] Further reference Figure 2 As an implementation of the methods shown in the above figures, this disclosure provides some embodiments of a code generation task response apparatus, which are similar to... Figure 1 Corresponding to the method embodiments shown, this code generation task response device can be specifically applied to various electronic devices.

[0126] like Figure 2As shown, a code generation task response device 200 includes: a first acquisition unit 201, a second acquisition unit 202, a preprocessing unit 203, an input unit 204, a visualization processing unit 205, and a sending unit 206. The first acquisition unit 201 is configured to acquire a pre-trained code generation task response model, wherein the code generation task response model is trained using a preset model training objective and reward loss information corresponding to a comprehensive reward set, which is generated using a dynamic multi-scale reward model. The second acquisition unit 202 is configured to acquire target code generation task information submitted by a target user through a target front-end. The preprocessing unit 203 is configured to preprocess the text data and / or audio data corresponding to the code generation question to obtain preprocessed feature representation information. The input unit 204 is configured to input the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set. The visualization processing unit 205 is configured to: perform visualization processing on the response information set generated by the above code, and embed interactive controls to obtain visual interactive page information. The sending unit 206 is configured to: send the above visual interactive page information to the terminal interface of the target user for the target user to view and interact with.

[0127] It is understandable that the units described in the code generation task response device 200 and the reference Figure 1 The steps in the described method correspond to each other. Therefore, the operations, features, and beneficial effects described above for the method also apply to the code generation task response device 200 and the units contained therein, and will not be repeated here.

[0128] The following is for reference. Figure 3 It shows a schematic diagram of the structure of an electronic device (e.g., an electronic device) 300 suitable for implementing some embodiments of the present disclosure. Figure 3 The electronic device shown is merely an example and should not be construed as limiting the functionality and scope of the embodiments of this disclosure.

[0129] like Figure 3 As shown, the electronic device 300 may include a processing unit (e.g., a central processing unit, a graphics processing unit, etc.) 301, which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 302 or a program loaded from a storage device 308 into a random access memory (RAM) 303. The RAM 303 also stores various programs and data required for the operation of the electronic device 300. The processing unit 301, ROM 302, and RAM 303 are interconnected via a bus 304. An input / output (I / O) interface 305 is also connected to the bus 304.

[0130] Typically, the following devices can be connected to I / O interface 305: input devices 306 including, for example, touchscreens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 307 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 308 including, for example, magnetic tapes, hard disks, etc.; and communication devices 309. Communication device 309 allows electronic device 300 to communicate wirelessly or wiredly with other devices to exchange data. Although Figure 3 An electronic device 300 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively. Figure 3 Each box shown can represent a device or multiple devices as needed.

[0131] In particular, according to some embodiments of this disclosure, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, some embodiments of this disclosure include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via communication device 309, or installed from storage device 308, or installed from ROM 302. When the computer program is executed by processing device 301, it performs the functions defined in the methods of some embodiments of this disclosure.

[0132] It should be noted that, in some embodiments of this disclosure, the computer-readable medium described above may be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. A computer-readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In some embodiments of this disclosure, a computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In some embodiments of this disclosure, a computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, carrying computer-readable program code. Such propagated data signals may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium can be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, RF (radio frequency), etc., or any suitable combination thereof.

[0133] In some implementations, clients and servers can communicate using any currently known or future-developed network protocol such as HTTP (Hypertext Transfer Protocol) and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (“LANs”), wide area networks (“WANs”), the Internet (e.g., the Internet of Things), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.

[0134] The aforementioned computer-readable medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device. The aforementioned computer-readable medium carries one or more programs, which, when executed by the electronic device, cause the electronic device to: acquire a pre-trained code generation task response model, wherein the code generation task response model is trained using reward loss information corresponding to a comprehensive reward set through a preset model training objective, and the comprehensive reward set is generated using a dynamic multi-scale reward model; acquire target code generation task information submitted by the target user through a target front-end; preprocess the text data and / or audio data corresponding to the code generation question to obtain preprocessed feature representation information; input the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set as the task response information set; perform visualization processing on the code generation response information set and embed interactive controls to obtain visualized interactive page information; and send the visualized interactive page information to the target user's terminal interface for the target user to view and interact with.

[0135] Computer program code for performing operations of some embodiments of this disclosure can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0136] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this disclosure. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0137] The units described in some embodiments of this disclosure can be implemented in software or hardware. The described units can also be housed in a processor; for example, a processor may be described as including a first acquisition unit, a second acquisition unit, a preprocessing unit, an input unit, a visualization processing unit, and a sending unit. The names of these units do not necessarily limit the specific unit; for example, the first acquisition unit may also be described as "acquiring target code generation task information submitted by the target user through the target front-end."

[0138] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Arrays (FPGAs), Application-Specific Integrated Circuits (ASICs), Application Standard Products (ASSPs), System-on-Chip (SoCs), Complex Programmable Logic Devices (CPLDs), and so on.

[0139] The above description is merely a selection of preferred embodiments of this disclosure and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of the invention involved in the embodiments of this disclosure is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-described inventive concept. For example, technical solutions formed by substituting the above-described features with (but not limited to) technical features with similar functions disclosed in the embodiments of this disclosure.

Claims

1. A method for responding to a code generation task, comprising: Obtain a pre-trained code generation task response model, wherein the code generation task response model is trained based on a preset model training objective and reward loss information corresponding to a comprehensive reward set, and the comprehensive reward set is generated using a dynamic multi-scale reward model; The target front-end obtains target code generation task information submitted by the target user, the target code generation task information including code generation questions; The text data and / or audio data corresponding to the code generation problem are preprocessed to obtain preprocessed feature representation information; The preprocessed feature representation information is input into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set. The task response information set is visualized and interactive controls are embedded to obtain a visual interactive page. The visual interactive page information is sent to the target user's terminal interface for the target user to view and interact with.

2. The method according to claim 1, wherein, The code generation task response model is trained through the following steps: Obtain the code generation task information set and the initial code generation task response model; For the initial code generation task response model, the following training steps are performed: In response to the initial code generation task response model having fewer recent consecutive training times than the target number and / or the variance reduction rate corresponding to the recent consecutive training times not meeting the target condition, a code generation task response information set corresponding to the code generation task information set is generated based on the task complexity corresponding to each code generation task information set and using the initial code generation task response model. Using the dynamic multi-scale reward model, determine the comprehensive reward set corresponding to the code generation task response information set; Using the comprehensive reward set, update the initial code generation task response model to obtain the updated code generation task response model; In response to the fact that the target number of consecutive training sessions has been reached, the reward variance set of the most recent number of training sessions is obtained, where the reward variance is the variance of the comprehensive reward set in each training session. In response to the reward variance set corresponding to the variance decrease rate satisfying the target condition, the last trained updated code generation task response model is determined as the code generation task response model.

3. The method according to claim 2, wherein, The step of using the dynamic multi-scale reward model to determine the comprehensive reward set corresponding to the code generation task response information set includes: For each code generation task response in the code generation task response information set, perform the comprehensive reward generation step: Using the rule reward layer of a dynamic multi-scale reward model, the first rule reward information for the code generation task response information is determined, wherein the first rule reward information is obtained by weighting accuracy reward, format reward and language consistency reward. Using the learning reward layer of the dynamic multi-scale reward model, the first learning reward information generation step is performed: Determine the standard response information set corresponding to the response information of the code generation task; Determine the first hidden vector corresponding to the response information of the code generation task; Determine the second latent vector corresponding to each standard response information in the standard response information set to obtain the second latent vector set; Generate the mean of the latent vectors corresponding to the second latent vector set; The cosine similarity between the first hidden vector and the mean of the hidden vectors is determined as the first learning reward information. The first rule reward information and the first learning reward information are fused by the gated fusion network included in the dynamic multi-scale reward model to generate a comprehensive reward corresponding to the code generation task response information.

4. The method according to claim 2, wherein the code generation task response information set includes: Code generation task response information group for multiple response types; as well as The step of updating the initial code generation task response model using the comprehensive reward set to obtain the updated code generation task response model includes: Determine the intra-group normalized reward value set of the comprehensive reward subset corresponding to the code generation task response information group for each response type, and obtain multiple intra-group normalized reward value sets; Determine the inter-group normalized reward value set of the comprehensive reward set corresponding to the code generation task response information set; Generate a first probability distribution corresponding to the multiple intra-group normalized reward value sets, and generate a second probability distribution corresponding to the inter-group normalized reward value sets; The reward loss information is determined based on the relative entropy regularization constraint formula, the first probability distribution, and the second probability distribution; Using the reward loss information, the model parameters of the initial code generation task response model are updated using the gradient ascent method to obtain the updated code generation task response model.

5. The method according to claim 2, further comprising: If the number of consecutive training iterations corresponding to the initial code generation task response model does not reach the cumulative target number and / or if the variance decrease rate corresponding to the reward variance set does not meet the target condition, the updated code generation task response model is used as the initial code generation task response model, and the training steps are executed again.

6. A code generation task response device, comprising: The first acquisition unit is configured to acquire a pre-trained code generation task response model, wherein the code generation task response model is trained based on a preset model training objective and reward loss information corresponding to a comprehensive reward set, and the comprehensive reward set is generated using a dynamic multi-scale reward model. The second acquisition unit is configured to acquire target code generation task information submitted by the target user through the target front end, the target code generation task information including code generation questions; The preprocessing unit is configured to preprocess the text data and / or audio data corresponding to the code generation problem to obtain preprocessed feature representation information; The input unit is configured to input the preprocessed feature representation information into the pre-trained code generation task response model to obtain a code generation response information set, which serves as the task response information set. The visualization processing unit is configured to perform visualization processing on the task response information set and embed interactive controls to obtain visual interactive page information; The sending unit is configured to send the visual interactive page information to the target user's terminal interface for the target user to view and interact with.

7. An electronic device, comprising: One or more processors; Storage device, on which one or more programs are stored, When the one or more programs are executed by the one or more processors, the one or more processors implement the method as described in any one of claims 1-7.

8. A computer-readable medium having a computer program stored thereon, wherein, When the program is executed by the processor, it implements the method as described in any one of claims 1-7.

Images