Training method of code generation model and code generation method and device
By training the code generation model, the code corresponding to the remarks information of image elements is automatically generated, which solves the problem of high communication costs between interface design and front-end development, and improves the efficiency and accuracy of code generation.
Patent Information
- Application Number
- CN202410039986.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-01-10
- Publication Date
- 2025-07-11
AI Technical Summary
In the prior art, the communication cost between interface design and front-end development is high, and front-end developers need to manually develop code inefficiently, resulting in inefficient code generation.
By training the code generation model, the image elements marked with remark information and corresponding codes are adjusted to automatically generate the code corresponding to remark information of image elements.
It reduces the communication cost between interface design and front-end developers, improves code generation efficiency, and improves the accuracy and efficiency of code generation.
Smart Images
Figure CN120295612A_ABST
Abstract
Description
Technical Field
[0001] The technical solution of the present disclosure relates to the field of computer technology, and in particular, to a method for training a code generation model, a code generation method, and a device. Background Art
[0002] An image element refers to a basic component of an image. For example, for an image, it may include various image elements such as text and pictures.
[0003] In related technical solutions, when developing a certain interface, the interface designer needs to design the overall layout composition to obtain an image, and then deliver it to the front-end developer. The front-end developer then develops and restores based on this image. This process requires the front-end developer to manually develop the corresponding code, resulting in low efficiency. Summary of the Invention
[0004] In view of this, embodiments of the present disclosure provide a method for training a code generation model, a code generation method, and a device.
[0005] According to a first aspect of the present disclosure, a method for training a code generation model is proposed. The method includes:
[0006] Obtain training data; the training data includes at least one image element labeled with note information and code corresponding to the image element labeled with note information; the image element is used to represent each part of the composition of the image, the note information is used to represent the display requirements for the image element, and the code is used to implement the display requirements;
[0007] Input the note information of the first image element in the training data into the code generation model to be trained, and obtain the predicted code output by the code generation model corresponding to the note information of the first image element;
[0008] Adjust the network parameters of the code generation model according to the difference between the predicted code and the code corresponding to the note information of the first image element in the training data.
[0009] According to a second aspect of the present disclosure, a code generation method is proposed. The method includes:
[0010] Receive an image element labeled with note information; the image element is used to represent each part of the composition of the image, and the note information is used to represent the display requirements for the image element;
[0011] Input the note information of the image element into the code generation model to obtain the code corresponding to the note information of the image element output by the code generation model; wherein, the code generation model is trained by the training method of the code generation model in any embodiment of the present disclosure, and the generated code is used to implement the display requirement.
[0012] Combined with any implementation manner provided by the present disclosure, after obtaining the code corresponding to the note information of the image element output by the code generation model, the method further includes:
[0013] Display the generated code on the visualization interface;
[0014] After receiving the modification information for the code returned based on the visualization interface, update the code based on the modification information.
[0015] Combined with any implementation manner provided by the present disclosure, the receiving the image element marked with note information includes:
[0016] After receiving the target image, perform image recognition on the target image to obtain at least one image element included in the target image; wherein, the image element is used to represent each part that makes up the target image;
[0017] Receive the note information for the at least one image element respectively to obtain at least one image element marked with note information.
[0018] Combined with any implementation manner provided by the present disclosure, the method further includes:
[0019] Display the generated code on the visualization interface;
[0020] After receiving the regeneration instruction for the code returned based on the visualization interface, re - execute the step of performing image recognition on the target image based on the regeneration instruction.
[0021] According to the third aspect of the present disclosure, a training device for a code generation model is proposed, and the device includes:
[0022] An acquisition module, configured to acquire training data; the training data includes at least one image element marked with note information and the code corresponding to the image element marked with note information; the image element is used to represent each part that makes up the image, the note information is used to represent the display requirement for the image element, and the code is used to implement the display requirement;
[0023] A first input module, configured to input the note information of the first image element in the training data into a code generation model to be trained, and obtain a predicted code corresponding to the note information of the first image element output by the code generation model;
[0024] An adjustment module, configured to adjust the network parameters of the code generation model according to the difference between the predicted code and the code corresponding to the note information of the first image element in the training data.
[0025] According to a fourth aspect of the present disclosure, a code generation device is provided, and the device includes:
[0026] A receiving module, configured to receive an image element marked with note information; the image element is used to represent each part of a composed image, and the note information is used to represent the display requirement for the image element;
[0027] A second input module, configured to input the note information of the image element into a code generation model, and obtain a code corresponding to the note information of the image element output by the code generation model; wherein, the code generation model is trained by the training device of the code generation model according to any embodiment of the present disclosure, and the generated code is used to implement the display requirement.
[0028] According to a fifth aspect of the present disclosure, a computer-readable storage medium is provided, and the machine-readable storage medium stores machine-readable instructions, which, when called and executed by a processor, cause the processor to implement the training method and code generation method of the code generation model according to any embodiment of the present disclosure.
[0029] According to a sixth aspect of the present disclosure, an electronic device is provided, including
[0030] A processor;
[0031] A memory for storing executable instructions of the processor;
[0032] Wherein, the processor is configured to execute the training method and code generation method of the code generation model according to any embodiment of the present disclosure.
[0033] The technical solutions provided by the embodiments of the present disclosure may include the following beneficial effects:
[0034] In the training method, code generation method and device of the code generation model provided by the embodiments of the present disclosure, the trained code generation model can be used to automatically generate codes corresponding to the note information of different image elements. Compared with the related technical solution in which front-end developers need to manually develop codes based on the images designed by interface designers, the efficiency can be improved.
[0035] It should be understood that the above general description and the following detailed description are merely exemplary and explanatory, and do not limit the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] The accompanying drawings are incorporated herein and form a part of this specification, showing embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure.
[0037] Figure 1 is a flowchart of a method for training a code generation model according to an exemplary embodiment of the present disclosure;
[0038] Figure 2a is a schematic diagram of the composition of an image according to an exemplary embodiment of the present disclosure;
[0039] Figure 2b is a schematic diagram of training data according to an exemplary embodiment of the present disclosure;
[0040] Figure 3 is a flowchart of a code generation method according to an exemplary embodiment of the present disclosure;
[0041] Figure 4 is a schematic diagram of the process of code generation according to an exemplary embodiment of the present disclosure;
[0042] Figure 5 is a flowchart of another code generation method according to an exemplary embodiment of the present disclosure;
[0043] Figure 6 is a schematic diagram of the structure of a device for training a code generation model according to an exemplary embodiment of the present disclosure;
[0044] Figure 7 is a schematic diagram of the structure of a code generation device according to an exemplary embodiment of the present disclosure;
[0045] Figure 8 is a schematic diagram of the structure of an electronic device according to an exemplary embodiment of the present disclosure. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0046] Exemplary embodiments will be described in detail herein, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of devices and methods consistent with some aspects of the present disclosure as detailed in the appended claims.
[0047] The terms used in this disclosure are for the purpose of describing particular embodiments only and are not intended to limit the disclosure. The singular forms "a", "the", and "said" used in this disclosure and the appended claims are also intended to include the plural forms unless the context clearly dictates otherwise. It should also be understood that the term "and / or" as used herein refers to and encompasses any and all possible combinations of one or more of the associated listed items.
[0048] It should be understood that although the terms first, second, third, etc. may be used in this disclosure to describe various information, such information should not be limited to these terms. These terms are only used to distinguish the same type of information from each other. For example, without departing from the scope of this disclosure, the first information may also be referred to as the second information, and similarly, the second information may also be referred to as the first information. Depending on the context, the word "if" as used herein may be interpreted as "when" or "while" or "in response to determining".
[0049] An image element refers to the basic components of an image. For example, for an image, it may include various image elements such as text, pictures, audio, video, etc.
[0050] In related technical solutions, when developing a certain interface, an interface designer needs to design the overall layout composition of the interface. After obtaining an image, it is delivered to a front-end developer, and the front-end developer develops and restores it based on the various image elements included in the image. During the development and restoration process, for the display requirements of different image elements, communication between the front-end developer and the interface designer is required, resulting in a relatively high communication cost. Moreover, during the development and restoration process, the front-end developer needs to develop corresponding code manually, resulting in low efficiency.
[0051] In view of this, the embodiments of this disclosure provide a training method for a code generation model and a code generation method. In this method, a code generation model can be trained based on the training method of the code generation model, and the trained code generation model can be used to automatically generate the code corresponding to the note information of different image elements, thereby reducing the communication cost between the front-end developer and the interface designer and improving the code generation efficiency.
[0052] Next, in conjunction with the accompanying drawings, a detailed description will be given of the training method of the code generation model for training to obtain the code generation model.
[0053] Figure 1 is a flowchart of a training method for a code generation model shown according to an exemplary embodiment of this disclosure. As Figure 1 shown, the method of this exemplary embodiment may include the following steps:
[0054] In step 101, training data is obtained.
[0055] The training data includes at least one image element labeled with a note, and a code corresponding to the image element labeled with the note. The image element is used to represent each part of the image, the note is used to represent the display requirement for the image element, and the code is used to implement the display requirement.
[0056] As Figure 2a shown, the image is composed of image element 21, image element 22, image element 23, and image element 24.
[0057] The training data may include at least one image element labeled with a note, and a code corresponding to the image element labeled with the note. Running the code can display the image element at an appropriate position in the image.
[0058] As Figure 2b shown, the training data may include image element 21 labeled with a note and code 1 corresponding to the image element 21 labeled with the note. Alternatively, the training data may further include image element 22 labeled with a note and code 2 corresponding to the image element 22 labeled with the note, and image element 23 labeled with a note and code 3 corresponding to the image element 23 labeled with the note. The present disclosure does not limit this.
[0059] In step 102, the note of the first image element in the training data is input into the code generation model to be trained, and the predicted code corresponding to the note of the first image element output by the code generation model is obtained.
[0060] Taking the first image element as the aforementioned image element 21 as an example, the note of image element 21 in the aforementioned training data can be input into the code generation model to be trained, and the predicted code 1 corresponding to the note of image element 21 output by the code generation model is obtained.
[0061] In step 103, according to the difference between the predicted code and the code corresponding to the note of the first image element in the training data, the network parameters of the code generation model are adjusted.
[0062] After obtaining the predicted code 1 corresponding to the note of image element 21 output by the code output model, the network parameters of the code generation model can be adjusted according to the difference between the predicted code 1 and the code 1 corresponding to the note of image element 21 in the training data.
[0063] The above introduced the case where the first image element is the image element 21. In actual applications, the above-mentioned image element 22, image element 23, image element 24, etc. can also be used as the first image element to train the code generation model. And continuously adjust the network parameters of the code generation model until the difference between the predicted code output by the code generation model and the code in the corresponding training data is less than a preset threshold. At this time, the trained code generation model can be obtained.
[0064] Through the training method of the code generation model provided by the embodiments of the present disclosure, a trained code generation model can be obtained, which is convenient for automatically generating codes corresponding to the note information of the image element based on the code generation model subsequently, and improving the code generation efficiency.
[0065] Next, a code generation method for generating corresponding codes using the code generation model trained by the above embodiments will be described in detail.
[0066] Figure 3 is a flowchart of a code generation method shown according to an exemplary embodiment of the present disclosure. This method can be executed by a server. As Figure 3 shown, the method of this exemplary embodiment may include the following steps:
[0067] In step 301, after receiving the target image, perform image recognition on the target image to obtain at least one image element included in the target image.
[0068] Among them, the image element is used to represent each part that makes up the target image.
[0069] When a user needs to develop a certain component in an interface, the image of the component can be uploaded to the server. After receiving the image, the server can use the image as the target image and perform image recognition on the target image to obtain at least one image element included in the target image, including but not limited to pictures, texts, audios, videos, etc.
[0070] Taking the image shown in a in Figure 4 as an example, the server can recognize the image shown in a to obtain the image elements included in the image shown in a, specifically as shown in Figure 4 b in.
[0071] In step 302, respectively receive the note information for the at least one image element to obtain at least one image element marked with note information.
[0072] The note information is used to represent the display requirements for the image element.
[0073] After the server recognizes the image shown in a and obtains the image shown in b, it can display the image shown in b on a visualization interface, and the user can add note information to each of the multiple image elements included in the image shown in b.
[0074] Exemplarily, the user can add note information to the image element as: constant or variable, whether it is loaded on demand or lazily loaded, etc. The present disclosure does not limit the specific content of the added note information, and specifically can be set by relevant staff based on the actual situation.
[0075] After the user adds note information to each of the multiple image elements included in the image shown in b in Figure 4 , the server can receive the note information for different image elements, and then obtain multiple image elements marked with note information, specifically as shown in c in Figure 4 .
[0076] In step 303, input the note information of the image element into the code generation model to obtain the code corresponding to the note information of the image element output by the code generation model.
[0077] Wherein, the code generation model is trained by the training method of the code generation model in any of the foregoing embodiments, and the generated code is used to implement the foregoing display requirements.
[0078] Combined with the foregoing, the note information of the image element can be input into the code generation model respectively to obtain the codes corresponding to the note information of different image elements output by the code generation model, specifically as shown in d in Figure 4 .
[0079] In step 304, display the generated code on the visualization interface.
[0080] After the server generates the code corresponding to the loading logic based on the note information of the image element, it can display the generated code on the visualization interface for the engineer to verify the code.
[0081] In step 305, determine whether to regenerate the code.
[0082] Optionally, a query prompt message can be output on the visualization interface after a preset time period, such as 1 minute, after the generated code is displayed on the visualization interface. The query prompt message is used to query whether the user wants to regenerate the code and provides two options of yes and no for the engineer to choose.
[0083] Optionally, a voice query message may also be output after a preset time period, such as 2 minutes, after the generated code is displayed on the visual interface. The voice query message is used to ask the user whether to regenerate the code. At the same time, a voice collection device may be turned on to collect voice instructions from the engineer.
[0084] If so, execute step 301.
[0085] After receiving the regeneration instruction for the code returned based on the visual interface, it indicates that the engineer believes that there is a large deviation in the currently generated code. At this time, the server may re-execute the step of performing image recognition on the target image based on the foregoing regeneration instruction, that is, regenerate the corresponding code based on the foregoing target image again.
[0086] If not, execute step 306.
[0087] After receiving the instruction for the code returned based on the visual interface not to regenerate, it indicates that the engineer believes that the currently generated code is feasible. At this time, execute step 306.
[0088] In step 306, end.
[0089] So far, the server can successfully generate the code corresponding to the received target image.
[0090] In the code generation method provided by the embodiments of the present disclosure, a trained code generation model can be used to automatically generate the code corresponding to the note information of different image elements. Compared with the related technical solutions in which the front-end developer needs to communicate with the interface designer and then the front-end developer manually develops the code based on the image designed by the interface designer, the communication cost and development cost between upstream and downstream can be reduced, and the code generation efficiency can also be improved.
[0091] Further, in the code generation method provided by the embodiments of the present disclosure, after obtaining the code corresponding to the note information of the image element, the generated code may also be displayed on the visual interface for verification by the engineer. When the engineer believes that there is a problem with the currently generated code, a regeneration instruction may also be issued based on the visual interface, so that the server regenerates the code corresponding to the target image based on the regeneration instruction, thereby improving the accuracy of the generated code.
[0092] In an optional example, as Figure 5 shown, after obtaining the code corresponding to the note information of the image element output by the foregoing code generation model, the following steps may further be included:
[0093] In step 501, the generated code is displayed on a visualization interface.
[0094] In step 502, after receiving modification information for the code returned based on the visualization interface, the code is updated based on the modification information.
[0095] After the generated code is displayed on the visualization interface, an engineer can modify the displayed code on this visualization interface. After the server receives the modification information obtained by the engineer for modifying the foregoing code, the code can be updated based on this modification information.
[0096] In the code generation method provided by the embodiments of the present disclosure, after obtaining the code corresponding to the note information of the image element, the generated code can also be displayed on a visualization interface for the engineer to check. When the engineer believes that there is a problem with the currently generated code, the generated code can be modified on this visualization interface. After the server receives the modification information of the engineer for the code, the code can be automatically updated based on this modification information, thereby improving the accuracy of the generated code.
[0097] For the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present disclosure is not limited by the described action sequence, because according to the present disclosure, certain steps can be performed in other sequences or simultaneously.
[0098] Secondly, those skilled in the art should also know that the embodiments described in the specification are all optional embodiments, and the actions and modules involved are not necessarily essential to the present disclosure.
[0099] Corresponding to the foregoing method embodiments for implementing application functions, the present disclosure also provides embodiments of an apparatus for implementing application functions and a corresponding terminal.
[0100] Figure 6 It is a schematic structural diagram of a training apparatus for a code generation model shown by the present disclosure according to an exemplary embodiment, as Figure 6 shown, the training apparatus for the code generation model may include:
[0101] An acquisition module 61, configured to acquire training data; the training data includes at least one image element labeled with note information and a code corresponding to the image element labeled with note information; the image element is used to represent each part of the composition of the image, the note information is used to represent the display requirements for the image element, and the code is used to implement the display requirements.
[0102] The first input module 62 is configured to input the note information of the first image element in the training data into the code generation model to be trained, and obtain the predicted code corresponding to the note information of the first image element output by the code generation model.
[0103] The adjustment module 63 is configured to adjust the network parameters of the code generation model according to the difference between the predicted code and the code corresponding to the note information of the first image element in the training data.
[0104] Figure 7 It is a schematic structural diagram of a code generation device shown by the present disclosure according to an exemplary embodiment. As Figure 7 shown, the code generation device may include:
[0105] The receiving module 71 is configured to receive the image elements marked with note information; the image elements are used to represent each part of the composed image, and the note information is used to represent the display requirements for the image elements.
[0106] The second input module 72 is configured to input the note information of the image elements into the code generation model, and obtain the code corresponding to the note information of the image elements output by the code generation model; wherein, the code generation model is trained by the training device of the code generation model according to any embodiment of the present disclosure, and the generated code is used to implement the display requirements.
[0107] Optionally, on the basis of the Figure 7 shown modules, the code generation device further includes:
[0108] The display module is configured to display the generated code on the visualization interface.
[0109] The update module is configured to update the code based on the modification information after receiving the modification information for the code returned based on the visualization interface.
[0110] Optionally, when the receiving module 71 is configured to receive the image elements marked with note information, it includes:
[0111] After receiving the target image, perform image recognition on the target image to obtain at least one image element included in the target image; wherein, the image elements are used to represent each part of the composed target image.
[0112] Receive the note information for the at least one image element respectively, and obtain at least one of the image elements marked with note information.
[0113] Optionally, on the basis of the Figure 7 shown modules, the code generation device may further include:
[0114] A display module for displaying the generated code on a visualization interface.
[0115] A processing module for, after receiving a regeneration instruction for the code returned based on the visualization interface, re-executing the step of performing image recognition on the target image based on the regeneration instruction.
[0116] For the apparatus embodiment, since it basically corresponds to the method embodiment, the relevant parts can be referred to the partial description of the method embodiment. The apparatus embodiments described above are only illustrative. The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place, or may be distributed to multiple network units. Some or all of the modules can be selected according to actual needs to achieve the purpose of the present disclosure solution. A person of ordinary skill in the art can understand and implement it without creative work.
[0117] Figure 8 It is a schematic structural diagram of an electronic device 800 shown according to an exemplary embodiment. For example, the electronic device 800 can be a server.
[0118] Referring to Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802, a memory 804, a power component 806, a multimedia component 808, an audio component 810, an input / output (I / O) interface 812, a sensor component 814, and a communication component 816.
[0119] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, telephone calls, data communication, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.
[0120] The memory 804 is configured to store various types of data to support the operation of the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, and the like. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, a magnetic disk, or an optical disk.
[0121] The power supply component 806 provides power to various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power for the electronic device 800.
[0122] The multimedia component 808 includes a screen that provides an output interface between the above-mentioned electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen can be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, swipes, and gestures on the touch panel. The above-mentioned touch sensors can not only sense the boundaries of touch or swipe actions but also detect the duration and pressure associated with the above-mentioned touch or swipe operations. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the electronic device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each of the front camera and the rear camera can be a fixed optical lens system or have a focal length and optical zoom capabilities.
[0123] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC) that is configured to receive external audio signals when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode. The received audio signals can be further stored in the memory 804 or transmitted via the communication component 816. In some embodiments, the audio component 810 further includes a speaker for outputting audio signals.
[0124] The I / O interface 812 provides an interface between the processing component 802 and a peripheral interface module, and the above-mentioned peripheral interface module can be a keyboard, a click wheel, buttons, etc. These buttons may include, but are not limited to: a home button, a volume button, a start button, and a lock button.
[0125] The sensor assembly 814 includes one or more sensors for providing status assessment of various aspects for the electronic device 800. For example, the sensor assembly 814 can detect the on / off state of the electronic device 800, the relative positioning of components, such as the display and keypad of the electronic device 800 as described above. The sensor assembly 814 can also detect a change in the position of the electronic device 800 or a component of the electronic device 800, the presence or absence of user contact with the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 can include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 can also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 can also include an acceleration sensor, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.
[0126] The communication component 816 is configured to facilitate communication between the electronic device 800 and other devices in a wired or wireless manner. The electronic device 800 can access a wireless network based on communication standards, such as WiFi, 4G or 5G, 4G LTE, 5G NR, or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 further includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology, and other technologies.
[0127] In an exemplary embodiment, the electronic device 800 can be implemented by one or more application specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components for performing the above methods.
[0128] In an exemplary embodiment, a non-transitory computer-readable storage medium is also provided, such as a memory 804 including instructions, which when executed by a processor 820 of the electronic device 800, enables the electronic device 800 to execute the training method and code generation method of the code generation model according to any embodiment of the present disclosure.
[0129] The non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.
[0130] Those skilled in the art will readily conceive of other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. The present disclosure is intended to cover any variations, uses, or adaptations of the present disclosure that follow the general principles of the present disclosure and include known common general knowledge or conventional technical means in the technical field not disclosed by the present disclosure. The specification and examples are only to be considered as exemplary, and the true scope and spirit of the present disclosure are pointed out by the following claims.
[0131] It should be understood that the present disclosure is not limited to the exact structures described above and shown in the drawings, and various modifications and changes can be made without departing from its scope. The scope of the present disclosure is only limited by the appended claims.
Claims
1. A training method for a code generation model, characterized in that, The method includes: Obtaining training data; the training data includes at least one image element labeled with note information and a code corresponding to the image element labeled with note information; the image element is used to represent each part of the composed image, the note information is used to represent the display requirement for the image element, and the code is used to implement the display requirement; Inputting the note information of the first image element in the training data into the code generation model to be trained, and obtaining a predicted code corresponding to the note information of the first image element output by the code generation model; Adjusting the network parameters of the code generation model according to the difference between the predicted code and the code corresponding to the note information of the first image element in the training data.
2. A code generation method, characterized in that, The method includes: Receiving an image element labeled with note information; the image element is used to represent each part of the composed image, and the note information is used to represent the display requirement for the image element; Inputting the note information of the image element into the code generation model, and obtaining a code corresponding to the note information of the image element output by the code generation model; wherein, the code generation model is trained by the method described in claim 1, and the generated code is used to implement the display requirement.
3. The method according to claim 2, characterized in that, After obtaining the code corresponding to the note information of the image element output by the code generation model, the method further includes: Displaying the generated code on a visualization interface; After receiving modification information for the code returned based on the visualization interface, updating the code based on the modification information.
4. The method according to claim 2, wherein The receiving an image element labeled with note information includes: After receiving a target image, performing image recognition on the target image to obtain at least one image element included in the target image; wherein, the image element is used to represent each part of the target image; Receiving note information for the at least one image element respectively to obtain at least one image element labeled with note information.
5. The method according to claim 4, wherein The method further includes: Displaying the generated code on a visualization interface; After receiving a regeneration instruction for the code returned based on the visualization interface, re-performing the step of performing image recognition on the target image based on the regeneration instruction.
6. A training device for a code generation model, characterized in that, The apparatus includes: An obtaining module, configured to obtain training data; the training data includes at least one image element labeled with note information and a code corresponding to the image element labeled with note information; the image element is used to represent each part of the composed image, the note information is used to represent the display requirement for the image element, and the code is used to implement the display requirement; A first input module, configured to input the note information of the first image element in the training data into the code generation model to be trained, and obtain a predicted code corresponding to the note information of the first image element output by the code generation model; An adjustment module, configured to adjust network parameters of the code generation model according to a difference between the prediction code and a code corresponding to the note information of the first image element in the training data.
7. A code generation device, characterized in that, The apparatus includes: A receiving module, configured to receive an image element marked with note information; the image element is used to represent each part of a composition of an image, and the note information is used to represent a display requirement for the image element; A second input module, configured to input the note information of the image element into a code generation model to obtain a code corresponding to the note information of the image element output by the code generation model; wherein, the code generation model is trained by the apparatus according to claim 6, and the generated code is used to implement the display requirement.
8. The device according to claim 7, characterized in that, The apparatus further includes: A display module, configured to display the generated code on a visualization interface; An update module, configured to update the code based on modification information returned based on the visualization interface after receiving the modification information for the code.
9. A computer-readable storage medium, on which a computer program is stored, and when the program is executed by a processor, the steps of the method according to claim 1, or any one of claims 2-5 are implemented.
10. An electronic device, including: A processor; A memory for storing processor-executable instructions; Wherein, the processor is configured to execute the steps of the method according to claim 1, or any one of claims 2-5.