Code generation method and device, electronic equipment and medium

Abstract

The embodiment of the invention provides a code generation method and device, electronic equipment and a medium. The code generation method specifically comprises the steps that a design image corresponding to a user interface is received; determining a description text corresponding to the design image; fusing the description text with a prompt text for generating a front-end code to obtain a fused prompt text; and inputting the fused prompt text into a large language model, and outputting a target front-end code corresponding to the user interface by the large language model. According to the embodiment of the invention, the efficiency of front-end development can be greatly improved, the labor cost of code writing by developers can be greatly reduced, and the threshold of front-end development can be reduced.

Description

Technical Field

[0001] Embodiments of the present invention relate to the field of Internet technology, and in particular to a code generation method, device, electronic device and medium. Background Art

[0002] With the rapid development of Internet technology, users' standards for web experience and interactivity are increasing, which in turn has led to a significant increase in the demand for front-end development.

[0003] In the existing front-end development process, developers usually need to convert the UI (User Interface) given by designers into corresponding front-end codes.

[0004] In the existing front-end development process, developers often need to spend a lot of time writing code corresponding to the appearance and interaction logic of the user interface, which is not only time-consuming and labor-intensive but also repetitive, greatly affecting the efficiency of front-end development. Summary of the invention

[0005] The purpose of the embodiments of the present invention is to provide a code generation method, device, electronic device, and medium, which can greatly improve the efficiency of front-end development, greatly reduce the manpower cost spent by developers on writing code, and lower the threshold of front-end development.

[0006] The specific technical solutions are as follows:

[0007] In a first aspect of the present invention, a code generation method is provided, the method comprising:

[0008] receiving a design image corresponding to the user interface;

[0009] Determining a description text corresponding to the design image;

[0010] Merging the description text with a prompt text used to generate a front-end code to obtain a merged prompt text;

[0011] The fusion prompt text is input into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

[0012] In a second aspect of the present invention, a code generation device is provided, the device comprising:

[0013] A receiving module, used for receiving a design image corresponding to the user interface;

[0014] An image-to-text module, used to determine the description text corresponding to the design image;

[0015] A prompt fusion module, used for fusing the description text with the prompt text used to generate the front-end code to obtain a fused prompt text;

[0016] The code generation module is used to input the fusion prompt text into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

[0017] Optionally, the image-to-text module is specifically used to input the design image into an image-to-text model, and the image-to-text model outputs an appearance description text corresponding to the design image; the appearance description text is used to describe the appearance information of the design image.

[0018] Optionally, the device further comprises:

[0019] A description text fusion module is used to fuse the appearance description text and the interaction logic description text corresponding to the design image to obtain the description text corresponding to the design image; the interaction logic description text is used to characterize the interaction logic of the user interface, and characterize the response action performed when a trigger operation is received for an interface element.

[0020] Optionally, the appearance description text includes: layout information of interface elements in the user interface, and element attribute information of the interface elements.

[0021] Optionally, the device further comprises:

[0022] A test module, used to test the target front-end code to obtain a test result;

[0023] An error prompt generating module, used for generating an error prompt text when the test result includes error information;

[0024] The first code repair module is used to input the target front-end code and the error prompt text into a large language model, and the large language model outputs a first repair front-end code corresponding to the target front-end code and the error prompt text.

[0025] Optionally, the device further comprises:

[0026] A feedback prompt generation module is used to generate feedback prompt text according to the feedback information submitted by the user for the target front-end code;

[0027] The second code repair module is used to input the target front-end code and the feedback prompt text into a large language model, and the large language model outputs a second repaired front-end code corresponding to the target front-end code and the feedback prompt text.

[0028] In a third aspect of the present invention, there is also provided an electronic device, comprising a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other via the communication bus;

[0029] Memory, used to store computer programs;

[0030] The processor is used to implement the aforementioned method steps when executing the program stored in the memory.

[0031] In a fourth aspect of the implementation of the present invention, a computer-readable storage medium is further provided, wherein the computer-readable storage medium stores instructions, which, when executed on a computer, enable the computer to execute any of the above-described methods.

[0032] The code generation method, device, electronic device and medium provided by the embodiments of the present invention first use the image-to-text technology in the field of computer vision to determine the description text corresponding to the design image; then, the above description text is merged with the prompt text used to generate the front-end code to obtain the fused prompt text; then, the above fused prompt text is input into the large language model, and the above large language model outputs the target front-end code corresponding to the above user interface.

[0033] First, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, the embodiment of the present invention can significantly reduce the time from user interface design to coding, thereby greatly improving the efficiency of front-end development.

[0034] Furthermore, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, the embodiment of the present invention can greatly reduce the manpower cost spent by developers in writing codes. Furthermore, the embodiment of the present invention can reduce the demand for human resources, especially for small and medium-sized enterprises or start-ups, and can efficiently promote projects without hiring a large number of developers.

[0035] In addition, the embodiments of the present invention can lower the threshold for front-end development and enable non-developers such as designers to directly participate in the front-end code generation process, which reduces technical barriers and promotes cross-border cooperation among more people.

[0036] Furthermore, since the target front-end code of the embodiment of the present invention is output by a large language model, the large language model can improve the standardization and consistency of the target front-end code, which makes the maintenance and updating of the target front-end code easier. In other words, the embodiment of the present invention can improve the convenience of maintenance and updating of the target front-end code.

[0037] Furthermore, the embodiments of the present invention can quickly convert the design image of the user interface included in the Internet product into the target front-end code, which is crucial for the testing, feedback collection and iterative development of the Internet product.

[0038] Furthermore, the embodiment of the present invention can realize automatic conversion from the design image corresponding to the user interface to the target front-end code. Since the probability of human error is reduced in the above-mentioned automatic conversion process, the code quality and the quality of Internet products can be improved.

[0039] Furthermore, since the embodiments of the present invention reduce repetitive coding work, developers can focus more time and energy on innovation and function development, thus promoting the innovative development of technology and business.

[0040] In summary, the embodiments of the present invention can push front-end development to a new stage of automation and intelligence, and contribute to the development and progress of the software development industry. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art are briefly introduced below.

[0042] Figure 1 A schematic diagram of the structure of a code generation system according to an embodiment of the present invention;

[0043] Figure 2 A flowchart of a code generation method according to an embodiment of the present invention;

[0044] Figure 3 A schematic diagram of a design image of an embodiment of the present invention;

[0045] Figure 4 An example of fusion prompt text according to an embodiment of the present invention;

[0046] Figure 5 A data flow chart of a code generation method according to an embodiment of the present invention;

[0047] Figure 6 A schematic diagram of the structure of a code generating device according to an embodiment of the present invention;

[0048] Figure 7 The structure block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] The technical solutions in the embodiments of the present invention will be described below in conjunction with the accompanying drawings in the embodiments of the present invention.

[0050] The embodiments of the present invention can be applied to front-end development scenarios in the field of Internet technology. Front-end development is the process of creating a user interface to present to users, and the user interface interaction of Internet products is realized through HTML (Hypertext Markup Language), CSS (Cascading Style Sheets) and derived technologies, frameworks, and solutions.

[0051] In response to the technical problem of low efficiency of front-end development in related technologies, an embodiment of the present invention provides a code generation solution, which specifically includes: receiving a design image corresponding to a user interface; determining a description text corresponding to the above design image; fusing the above description text with a prompt text used to generate a front-end code to obtain a fused prompt text; inputting the above fused prompt text into a large language model, and the above large language model outputs a target front-end code corresponding to the above user interface.

[0052] The embodiment of the present invention first uses the image-to-text technology in the field of computer vision to determine the description text corresponding to the design image; then, the description text is merged with the prompt text used to generate the front-end code to obtain the merged prompt text; then, the merged prompt text is input into the large language model, and the large language model outputs the target front-end code corresponding to the user interface.

[0053] The code generation solution of the embodiment of the present invention combines the image-to-text technology in the field of computer vision with the large language model technology in the field of natural language processing, and can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code.

[0054] First of all, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, thereby reducing the time spent on code writing and code debugging, the embodiment of the present invention can significantly reduce the time from user interface design to coding, thereby greatly improving the efficiency of front-end development.

[0055] Furthermore, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, the embodiment of the present invention can greatly reduce the manpower cost spent by developers in writing codes. Furthermore, the embodiment of the present invention can reduce the demand for human resources, especially for small and medium-sized enterprises or start-ups, and can efficiently promote projects without hiring a large number of developers.

[0056] In addition, the embodiments of the present invention can lower the threshold for front-end development and enable non-developers such as designers to directly participate in the front-end code generation process, which reduces technical barriers and promotes cross-border cooperation among more people.

[0057] Furthermore, since the target front-end code of the embodiment of the present invention is output by a large language model, the large language model can improve the standardization and consistency of the target front-end code, which makes the maintenance and updating of the target front-end code easier. In other words, the embodiment of the present invention can improve the convenience of maintenance and updating of the target front-end code.

[0058] Furthermore, the embodiments of the present invention can quickly convert the design image of the user interface included in the Internet product into the target front-end code, which is crucial for the testing, feedback collection and iterative development of the Internet product.

[0059] Furthermore, the embodiment of the present invention can realize automatic conversion from the design image corresponding to the user interface to the target front-end code. Since the probability of human error is reduced in the above-mentioned automatic conversion process, the code quality and the quality of Internet products can be improved.

[0060] Furthermore, since the embodiments of the present invention reduce repetitive coding work, developers can focus more time and energy on innovation and function development, thus promoting the innovative development of technology and business.

[0061] In summary, the embodiments of the present invention can push front-end development to a new stage of automation and intelligence, and contribute to the development and progress of the software development industry.

[0062] Reference Figure 1 , shows a schematic diagram of the structure of a code generation system according to an embodiment of the present invention, the system may specifically include: a client 101 and a server 102. The client 101 and the server 102 may perform data exchange based on a wireless network or a wired network.

[0063] In actual application, the client 101 can send a design image corresponding to the user interface included in the Internet product to the server 102. It can be understood that an Internet product can include: one or more user interfaces, and one user interface can correspond to one or more design images.

[0064] An Internet product can refer to the functions or services created by a website or application to meet user needs, or it can be an integration of functions and services.

[0065] The server 102 can use the code generation solution of the embodiment of the present invention to convert the design image into the target front-end code corresponding to the user interface. The server 102 can return the target front-end code corresponding to the user interface included in the Internet product to the client 101.

[0066] The client can provide a user interface corresponding to the target front-end code for the user to preview. In addition, user feedback information on the user interface can be collected. Then, the embodiment of the present invention can repair the target front-end code according to the above feedback information to optimize the target front-end code. Examples of the above feedback information may include: a certain interface element is displayed disorderly under the condition of certain device information, or a certain interface element does not respond after being clicked, or the interface freezes, etc.

[0067] The embodiments of the present invention are described below through specific examples.

[0068] Reference Figure 2 , showing a flowchart of the steps of a code generation method according to an embodiment of the present invention, referring to Figure 5 , shows a data flow chart of a code generation method according to an embodiment of the present invention, the method may specifically include the following steps:

[0069] Step 201: receiving a design image corresponding to a user interface;

[0070] Step 202: Determine the description text corresponding to the above-mentioned design image;

[0071] Step 203: merge the description text with the prompt text used to generate the front-end code to obtain a merged prompt text;

[0072] Step 204: input the fusion prompt text into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

[0073] Figure 2 The method embodiment shown can be executed by the server to convert the design image into the target front-end code corresponding to the user interface. Figure 2 The specific execution subject of the illustrated method embodiment is not limited.

[0074] In step 201, the design image corresponding to the user interface may be a visual image provided by a designer for the user interface. The designer may intuitively express the front-end design intention through the design image so that the image-to-text model can understand the front-end design intention.

[0075] The design image may include the appearance information of the user interface. The appearance information may include, but is not limited to, color matching, interface layout, font design, icon design, background image design, advertising image design, illustration design, etc. The interface layout is used to describe the overall structure and layout of the user interface, including the header, navigation bar, content area, sidebar, bottom, etc.

[0076] Reference Figure 3, showing a schematic diagram of a design image of an embodiment of the present invention, wherein the design image specifically includes: areas corresponding to multiple interface elements respectively. The areas of multiple interface elements respectively include: main title area 301, subtitle area 302, first function area 303, second function area 304...nth function 30x, etc., where n can be a positive integer. Among them, the appearance information of the main title area 301 specifically includes: main title text. The appearance information of the subtitle area 302 specifically includes: subtitle text. The appearance information of the first function area 303 specifically includes: first function text and first function icon. The appearance information of the nth function area 30x specifically includes: nth function text and nth function icon.

[0077] It is understood that, in addition to text (such as main title text) or icon (such as first function icon), the appearance information of an interface element may also include: location information in the user interface, background image information, etc. Moreover, the appearance information of text may include: font, word color, etc.

[0078] In step 202, a description text corresponding to the design image may be determined by using a method of generating text from images. The method of generating text from images may automatically analyze the content of the design image according to the design image uploaded by the user, and generate a description text corresponding to the design image.

[0079] In a specific implementation, the process of determining the description text corresponding to the design image specifically includes: inputting the design image into a picture-to-text model, and the picture-to-text model outputs the appearance description text corresponding to the design image. The appearance description text is used to describe the appearance information of the design image.

[0080] In one implementation, the image-to-text model may specifically include: CNNs (Convolutional Neural Networks) and neural networks with self-attention mechanisms. Convolutional neural networks are used to extract image features from design images. The neural networks with self-attention mechanisms are used to learn the relationship information between these image features and generate appearance description text corresponding to the design image based on the relationship information.

[0081] In another implementation, the image-to-text model may include: a convolutional neural network and a pre-trained bidirectional transformer. The convolutional neural network is used to extract image features in the design image. The pre-trained bidirectional transformer is used to extract text features of the text description, and generate appearance description text corresponding to the design image based on the semantic similarity between the image features and the text features. In the training process of the image-to-text model, a loss function can be used to measure the semantic similarity between the image features and the text features, and the parameters of the convolutional neural network and the bidirectional transformer can be updated according to the gradient of the loss function.

[0082] In practical applications, the appearance description text includes: layout information of interface elements in the user interface, and element attribute information of the interface elements. Element attribute information may refer to position information, style information, etc. of the interface elements. Style information may include: font information, background image information, size information, etc.

[0083] In one example, Figure 3 The appearance description text of the user interface shown specifically includes:

[0084] This image represents a Chinese web application user interface that focuses on n functions. The image includes a main content area with multiple interface elements, which may be buttons or tabs, and each interface element has text or icons.

[0085] If you use Vue (a view, a framework for building user interfaces) to write front-end code, this user interface can be divided into the following components:

[0086] Header component: Above the main content area, there is a header navigation that displays the current location within the app.

[0087] Main content area components: The main content area has multiple cards, each representing a different function, which can be independent Vue components dynamically loaded according to user operations. Each card has a title, a short description.

[0088] Each component will be defined in its own .Vue file, including templates, scripts, styles, etc., which conforms to the Vue single-file component architecture.

[0089] In an embodiment of the present invention, the process of determining the description text corresponding to the above-mentioned design image may also include: fusing the appearance description text and the interaction logic description text corresponding to the design image to obtain the description text corresponding to the design image.

[0090] The interaction logic description text can be used to characterize the interaction logic of the user interface. The above interaction logic is used to characterize the response action performed when a trigger operation is received for an interface element. The above trigger operation may include but is not limited to: mouse hover, click, slide, etc. Examples of the above interaction logic may include: when a click operation is received for interface element A1 in user interface A, popping up user interface B, etc.

[0091] The interactive logic description text can be provided by the user. In actual applications, the interactive logic description text corresponding to the interface element can be added to the corresponding position of the interface element in the appearance description text corresponding to the design image, thereby realizing the fusion of the appearance description text and the interactive logic description text, and then obtaining the fused description text.

[0092] For example, if a button is mentioned in the appearance description text, add the interaction logic description text for the button in the description position of the button, such as "upon receiving a click operation on this button, execute [specific response action]".

[0093] The appearance description text provides basic information such as interface structure and element attributes for the generation of front-end code, so that the front-end code can present the appearance of the interface. The interaction logic description text gives the front-end code dynamic interaction capabilities, which not only improves the user experience but also improves the interaction function. After the appearance description text and the interaction logic description text are merged, the front-end code generated based on the fusion result not only has the appearance presentation of the interface, but also covers the response logic of various interactive actions.

[0094] In step 203, the prompt text used to generate the front-end code may include information such as programming language requirements, code style, development library or development framework, etc.

[0095] The embodiment of the present invention fuses the description text with the prompt text used to generate the front-end code to obtain the fused prompt text. The fused prompt text is used to generate the target front-end code that meets the interface requirements corresponding to the description text and the code requirements corresponding to the prompt text.

[0096] The description text and the prompt text used to generate the front-end code are both components of the fusion prompt text.

[0097] The description text describes the content of the interface, including the layout information, element attributes, and interaction logic of the interface, reflecting the appearance and behavior characteristics of the interface. The prompt text clarifies the technical requirements for generating front-end code, covering programming language requirements, code style, and development libraries or frameworks.

[0098] The embodiment of the present application can use natural language to connect the description text and the prompt text to obtain a fused prompt text. Specifically, a template can be provided, which includes: a description text position and a prompt text position; in this way, the description text position can be filled with the description text position, and the prompt text can be filled with the prompt text position. Among them, the embodiment of the present application does not limit the order of the description text position and the prompt text position in the template.

[0099] The description text and the prompt text complement each other in the integrated prompt text, providing comprehensive and accurate guidance for generating front-end code that can be compiled, runned, concise and logically clear.

[0100] Reference Figure 4, shows a schematic diagram of the fusion prompt text of an embodiment of the present invention, wherein the description text and the prompt text used to generate the front-end code can be connected by natural language. Moreover, the description text conforms to the characteristics of natural language and follows the rules of natural language. Similarly, the prompt text used to generate the front-end code conforms to the characteristics of natural language and follows the rules of natural language.

[0101] In step 204, the large language model is an artificial intelligence model designed to understand and generate human language. The large language model can perform a wide range of tasks, such as the text generation task of the embodiment of the present invention, by training on a large amount of text data. The large language model is usually based on a deep learning architecture, such as a transformer structure. The embodiment of the present invention does not limit the specific type of the large language model. For example, the types of large language models may include: GPT (Generative Pre-Trained Transformer) model, GLM (Generative Language Model, General Language Model) and Baichuan large model, etc.

[0102] The embodiment of the present invention utilizes a large language model to generate a target front-end code with a clear structure and satisfactory functions according to the fused prompt text, which simulates the thinking process of human developers to a certain extent.

[0103] The embodiment of the present invention can use training data in advance to train the large language model so that the large language model has the ability to generate code. The training data may include: fusion prompt text samples and code samples; during the training process, the large language model can generate predicted codes based on the fusion prompt text samples; then the loss information can be determined based on the code samples and the predicted codes, and the parameters of the large language model can be updated based on the loss information. When the loss information meets the preset conditions, the large language model converges and can have the ability to generate code.

[0104] The embodiment of the present invention can also verify and adjust the generated target front-end code.

[0105] In one implementation, the generated target front-end code can be verified and adjusted based on the test case. Accordingly, the above method can also include: testing the target front-end code using the test case to obtain a test result; generating an error prompt text when the test result includes error information; inputting the target front-end code and the error prompt text into a large language model, and the large language model outputting a first repair front-end code corresponding to the target front-end code and the error prompt text.

[0106] The process of testing the target front-end code using the test case specifically includes:

[0107] Configure the test environment and run the test cases according to the selected test framework; the test framework will execute each test case in a predetermined order to obtain the test results; or,

[0108] Use the scripts of automated testing tools to automatically execute test cases.

[0109] Among them, the test cases simulate the application scenarios and load conditions of the target front-end code, drive the target front-end code to run under preset conditions such as high load, concurrency, and extreme conditions, thereby testing and recording the performance indicators of the target front-end code.

[0110] During the testing process, the test case executor, such as an automated testing tool or testing framework, interacts with the target front-end code in a variety of ways, such as simulating user behavior and sending network requests. Through these interactive methods, the target front-end code is prompted to execute the corresponding logic.

[0111] The test result may contain at least one of the following error messages:

[0112] Logical error: The code logic does not meet expectations, resulting in the test case's results not meeting expectations. Judgment method: Check the actual results and expected results of the test case output. Usually the test framework will provide a detailed difference report to facilitate analysis of the problem.

[0113] Performance issues: The code runs inefficiently, resulting in long response times or browser freezes. Judgment method: Use performance testing tools to analyze and find code bottlenecks.

[0114] Compatibility issues: The generated code does not behave consistently in certain browsers or environments. Judgment method: Run the same test case on different browsers or devices to see if there are any differences.

[0115] User interface issues: The generated code does not perform correctly in the presentation layer (visual elements, layout, etc.). Judgment method: Perform visual regression testing or manually check the correctness of the user interface.

[0116] The process of obtaining error information in the embodiment of the present invention specifically includes:

[0117] Step A1, execute the test case: wherein the test case drives the target front-end code to run by simulating the application scenario and load conditions of the target front-end code, and determines the test result according to the actual behavior and actual output of the target front-end code.

[0118] Step A2, collect error information: After a test fails, the test framework usually reports the reason for the failure. Record this error information for further analysis.

[0119] Step A3, Analyze the error stack: Check the error stack, which provides the file and line number information where the error occurred.

[0120] Step A4: Reproduce the problem: Under specific conditions, try to reproduce the problem manually to better understand the context of the error.

[0121] Step A5: Use debugging tools to step into the code, observe the program's running status, and check the values ​​of variables at runtime.

[0122] Step A6: Segment the scope of the error: Try to reduce the scope of the target front-end code, or modify the test case to narrow down the possible scope of the problem.

[0123] Step A7: Error classification and recording: Classify the error information and record it in the error tracking system to facilitate subsequent repair and improvement work.

[0124] The embodiment of the present invention can convert the error information into an error prompt text that conforms to the characteristics of natural language, and input the error prompt text into a large language model so that the large language model avoids the error information contained in the error prompt text. In addition, in order to avoid the error information contained in the error prompt text, the large language model can also output the target front-end code and the first repair front-end code corresponding to the error prompt text. The first repair front-end code is the code after the large language model repairs the target front-end code according to the error prompt text.

[0125] The error prompt text may include: error information, conditions under which the error information occurs, and expected behavior. The error prompt text of the embodiment of the present invention may describe the error information and expected behavior (such as avoiding errors) in concise and clear language so that the large language model can understand and give the corresponding first repair front-end code.

[0126] The following is the process of generating the first fix front-end code from a large language model:

[0127] Step B1: In-depth analysis.

[0128] After receiving the target front-end code and error prompt text, the large language model conducts an in-depth analysis of the target front-end code to obtain the code structure.

[0129] Step B2: Error location.

[0130] The large language model determines the error type and location based on the error prompt text and the results of deep analysis. The code at the error location is called the error part.

[0131] The error may be a syntactic error, such as missing key punctuation marks, incorrect variable declaration, etc.; it may also be a logical error, such as inaccurate calculation results, improper conditional judgment, abnormal loop execution, etc.; it may also be an error in external interaction, such as failed network requests, poor connection with back-end data, etc.

[0132] Step B3: Generate a first repair front-end code.

[0133] The large language model uses the programming knowledge and patterns learned from a large amount of training data, combined with the specifications and best practices of front-end development, to modify and adjust the located error parts, and then generate the repaired first-fix front-end code to effectively solve the problem pointed out in the error prompt text.

[0134] In another implementation, the generated target front-end code may be verified and adjusted based on the feedback information.

[0135] Correspondingly, the above method may also include: generating feedback prompt text based on feedback information submitted by the user for the target front-end code; inputting the target front-end code and the feedback prompt text into a large language model, and the large language model outputs a second repair front-end code corresponding to the target front-end code and the feedback prompt text.

[0136] The feedback prompt text of the embodiment of the present invention can use concise and clear language to describe the feedback information and the expected behavior (such as avoiding the behavior corresponding to the feedback information) so that the large language model can understand and give the corresponding second code repair solution. The second code repair solution may include: the target front-end code and the second repair front-end code corresponding to the feedback prompt text.

[0137] The client can provide a user interface corresponding to the target front-end code for the user to preview. In addition, user feedback information on the user interface can be collected. Then, the embodiment of the present invention can repair the target front-end code according to the above feedback information to optimize the target front-end code. Examples of the above feedback information may include: a certain interface element is displayed disorderly under the condition of certain device information, or a certain interface element does not respond after being clicked, or the interface freezes, etc.

[0138] The embodiment of the present invention can also adjust and optimize the target front-end code to improve the code quality. Improving the code quality specifically includes: improving the readability of the code, improving the running efficiency of the code, reducing redundancy, etc.

[0139] The technology used by the embodiment of the present invention to adjust and optimize the target front-end code specifically includes at least one of the following means:

[0140] Technical means B1: code formatting.

[0141] For example, use code formatting tools to improve the consistency of code style in target front-end codes of different user interfaces.

[0142] For example, automatically adjust indentation, spacing, line breaks, and other style issues to improve code readability.

[0143] Technical means B2: code cleanup.

[0144] For example, remove unused variables, functions, classes, and imports to reduce code size and improve clarity. Another example is to simplify the code structure and merge repeated expressions.

[0145] Technical means B3: code structure optimization.

[0146] Maintain a reasonable file and module structure to facilitate code modularization and reuse. Use design patterns and practices to organize code logic.

[0147] Technical means B4, performance optimization. Optimize loops and recursive calls to reduce unnecessary calculations. Delay or asynchronously load resources, such as using the code splitting feature. Use worker threads to handle time-consuming calculation tasks to prevent the main thread from blocking. Worker threads can run in parallel with the main thread without blocking each other.

[0148] In summary, the code generation method of the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, thereby reducing the time spent on code writing and code debugging. Therefore, the embodiment of the present invention can significantly reduce the time from user interface design to coding, thereby greatly improving the efficiency of front-end development.

[0149] Furthermore, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, the embodiment of the present invention can greatly reduce the manpower cost spent by developers in writing codes. Furthermore, the embodiment of the present invention can reduce the demand for human resources, especially for small and medium-sized enterprises or start-ups, and can efficiently promote projects without hiring a large number of developers.

[0150] In addition, the embodiments of the present invention can lower the threshold for front-end development and enable non-developers such as designers to directly participate in the front-end code generation process, which reduces technical barriers and promotes cross-border cooperation among more people.

[0151] Furthermore, since the target front-end code of the embodiment of the present invention is output by a large language model, the large language model can improve the standardization and consistency of the target front-end code, which makes the maintenance and updating of the target front-end code easier. In other words, the embodiment of the present invention can improve the convenience of maintenance and updating of the target front-end code.

[0152] Furthermore, the embodiments of the present invention can quickly convert the design image of the user interface included in the Internet product into the target front-end code, which is crucial for the testing, feedback collection and iterative development of the Internet product.

[0153] Furthermore, the embodiment of the present invention can realize automatic conversion from the design image corresponding to the user interface to the target front-end code. Since the probability of human error is reduced in the above-mentioned automatic conversion process, the code quality and the quality of Internet products can be improved.

[0154] Furthermore, since the embodiments of the present invention reduce repetitive coding work, developers can focus more time and energy on innovation and function development, thus promoting the innovative development of technology and business.

[0155] In summary, the embodiments of the present invention can push front-end development to a new stage of automation and intelligence, and contribute to the development and progress of the software development industry.

[0156] It should be noted that, for the sake of simplicity, the method embodiments are described as a series of action combinations, but those skilled in the art should be aware that the embodiments of the present invention are not limited by the order of the actions described, because according to the embodiments of the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also be aware that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the embodiments of the present invention.

[0157] Reference Figure 6 , shows a schematic diagram of the structure of a code generation device according to an embodiment of the present invention, and the code generation device may specifically include the following modules:

[0158] Receiving module 601, used to receive a design image corresponding to the user interface;

[0159] The image-to-text module 602 is used to determine the description text corresponding to the design image;

[0160] A prompt fusion module 603, used to fuse the description text with the prompt text used to generate the front-end code to obtain a fused prompt text;

[0161] The code generation module 606 is used to input the fusion prompt text into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

[0162] Optionally, the image-to-text module 602 is specifically used to input the design image into an image-to-text model, and the image-to-text model outputs an appearance description text corresponding to the design image; the appearance description text is used to describe the appearance information of the design image.

[0163] Optionally, the device may further include:

[0164] A description text fusion module is used to fuse the appearance description text and the interaction logic description text corresponding to the design image to obtain the description text corresponding to the design image; the interaction logic description text is used to characterize the interaction logic of the user interface, and characterize the response action performed when a trigger operation is received for an interface element.

[0165] Optionally, the appearance description text may include: layout information of interface elements in the user interface, and element attribute information of the interface elements.

[0166] Optionally, the device may further include:

[0167] A test module, used to test the target front-end code to obtain a test result;

[0168] An error prompt generating module, used for generating an error prompt text when the test result may include error information;

[0169] The first code repair module is used to input the target front-end code and the error prompt text into a large language model, and the large language model outputs a first repair front-end code corresponding to the target front-end code and the error prompt text.

[0170] Optionally, the device may further include:

[0171] A feedback prompt generation module is used to generate feedback prompt text according to the feedback information submitted by the user for the target front-end code;

[0172] The second code repair module is used to input the target front-end code and the feedback prompt text into a large language model, and the large language model outputs a second repaired front-end code corresponding to the target front-end code and the feedback prompt text.

[0173] In summary, the code generation device of the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, thereby reducing the time spent on code writing and code debugging. Therefore, the embodiment of the present invention can significantly reduce the time from user interface design to coding, thereby greatly improving the efficiency of front-end development.

[0174] Furthermore, since the embodiment of the present invention can realize the automatic conversion from the design image corresponding to the user interface to the target front-end code, the embodiment of the present invention can greatly reduce the manpower cost spent by developers in writing codes. Furthermore, the embodiment of the present invention can reduce the demand for human resources, especially for small and medium-sized enterprises or start-ups, and can efficiently promote projects without hiring a large number of developers.

[0175] In addition, the embodiments of the present invention can lower the threshold for front-end development and enable non-developers such as designers to directly participate in the front-end code generation process, which reduces technical barriers and promotes cross-border cooperation among more people.

[0176] Furthermore, since the target front-end code of the embodiment of the present invention is output by a large language model, the large language model can improve the standardization and consistency of the target front-end code, which makes the maintenance and updating of the target front-end code easier. In other words, the embodiment of the present invention can improve the convenience of maintenance and updating of the target front-end code.

[0177] Furthermore, the embodiments of the present invention can quickly convert the design image of the user interface included in the Internet product into the target front-end code, which is crucial for the testing, feedback collection and iterative development of the Internet product.

[0178] Furthermore, the embodiment of the present invention can realize automatic conversion from the design image corresponding to the user interface to the target front-end code. Since the probability of human error is reduced in the above-mentioned automatic conversion process, the code quality and the quality of Internet products can be improved.

[0179] Furthermore, since the embodiments of the present invention reduce repetitive coding work, developers can focus more time and energy on innovation and function development, thus promoting the innovative development of technology and business.

[0180] In summary, the embodiments of the present invention can push front-end development to a new stage of automation and intelligence, and contribute to the development and progress of the software development industry.

[0181] An embodiment of the present invention further provides an electronic device that can implement the functions of the aforementioned storage node or scheduling node or ledger node or management node.

[0182] like Figure 7 As shown, the electronic device may include a processor 1001, a communication interface 1002, a memory 1003 and a communication bus 1004, wherein the processor 1001, the communication interface 1002, and the memory 1003 communicate with each other via the communication bus 1004.

[0183] Memory 1003, used for storing computer programs;

[0184] The processor 1001 is used to execute the program stored in the memory 1003, and implements the following steps:

[0185] Storing the storage object;

[0186] Sending the storage object to a coupled second storage node, so that the second storage node stores and / or forwards the storage object;

[0187] After the storage object is successfully stored, the object information of the storage object is sent to the ledger node, so that the ledger node saves the ledger information of the first storage node; the ledger information includes: the object information stored in the corresponding storage node.

[0188] The communication bus mentioned in the above electronic device can be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus can be divided into an address bus, a data bus, a control bus, etc. For ease of representation, only one thick line is used in the figure, but it does not mean that there is only one bus or one type of bus.

[0189] The communication interface is used for communication between the above electronic device and other devices.

[0190] The memory may include a random access memory (RAM) or a non-volatile memory, such as at least one disk memory. Optionally, the memory may also be at least one code generation device located away from the aforementioned processor.

[0191] The above-mentioned processor can be a general-purpose processor, including a central processing unit (CPU), a network processor (NP), etc.; it can also be a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components.

[0192] In another embodiment of the present invention, a computer-readable storage medium is provided. The computer-readable storage medium stores instructions. When the computer-readable storage medium is run on a computer, the computer executes the code generation method described in any one of the above embodiments.

[0193] In another embodiment of the present invention, a computer program product including instructions is provided. When the computer program product is run on a computer, the computer executes the code generation method described in any one of the above embodiments.

[0194] In the above embodiments, it can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented by software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in the embodiment of the present invention is generated in whole or in part. The computer can be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions can be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium. For example, the computer instructions can be transmitted from one website site, computer, server or data center to another website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode. The computer-readable storage medium can be any available medium that can be accessed by a computer or a data storage device such as a server or data center that includes one or more available media integrated. The available medium can be a magnetic medium (e.g., a floppy disk, a hard disk, a tape), an optical medium, or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc. Examples of optical media may include DVD (Digital Video Disc) and the like.

[0195] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the sentence "comprise a ..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0196] Each embodiment in this specification is described in a related manner, and the same or similar parts between the embodiments can be referred to each other, and each embodiment focuses on the differences from other embodiments. In particular, for the system embodiment, since it is basically similar to the method embodiment, the description is relatively simple, and the relevant parts can be referred to the partial description of the method embodiment.

[0197] The above description is only a preferred embodiment of the present invention and is not intended to limit the protection scope of the present invention. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention are included in the protection scope of the present invention.

Claims

1. A code generation method, characterized in that: The method comprises: receiving a design image corresponding to the user interface; Determining a description text corresponding to the design image; Merging the description text with a prompt text used to generate a front-end code to obtain a merged prompt text; The fusion prompt text is input into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

2. The method according to claim 1, characterized in that The determining of the description text corresponding to the design image includes: The design image is input into a picture-to-text model, and the picture-to-text model outputs an appearance description text corresponding to the design image; the appearance description text is used to describe the appearance information of the design image.

3. The method according to claim 2, characterized in that The method further comprises: The appearance description text and the interaction logic description text corresponding to the design image are merged to obtain the description text corresponding to the design image; the interaction logic description text is used to characterize the interaction logic of the user interface, and characterize the response action performed when a trigger operation is received for an interface element.

4. The method according to claim 2, characterized in that: The appearance description text includes: layout information of interface elements in the user interface, and element attribute information of the interface elements.

5. The method according to claim 1, characterized in that The method further comprises: Testing the target front-end code to obtain a test result; In the case where the test result includes error information, generating an error prompt text; The target front-end code and the error prompt text are input into a large language model, and the large language model outputs a first repair front-end code corresponding to the target front-end code and the error prompt text.

6. The method according to claim 1, characterized in that The method further comprises: Generate feedback prompt text based on the feedback information submitted by users for the target front-end code; The target front-end code and the feedback prompt text are input into a large language model, and the large language model outputs a second repair front-end code corresponding to the target front-end code and the feedback prompt text.

7. A code generating device, characterized in that: The device comprises: A receiving module, used for receiving a design image corresponding to the user interface; An image-to-text module, used to determine the description text corresponding to the design image; A prompt fusion module, used for fusing the description text with the prompt text used to generate the front-end code to obtain a fused prompt text; The code generation module is used to input the fusion prompt text into a large language model, and the large language model outputs a target front-end code corresponding to the user interface.

8. The device according to claim 7, characterized in that The image-to-text module is specifically used to input the design image into an image-to-text model, and the image-to-text model outputs the appearance description text corresponding to the design image.

9. An electronic device, characterized in that: It includes a processor, a communication interface, a memory and a communication bus, wherein the processor, the communication interface and the memory communicate with each other through the communication bus; Memory, used to store computer programs; A processor, for implementing the method steps described in any one of claims 1 to 6 when executing a program stored in a memory.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 6 is implemented.

Images