Code generation method and device and computer readable storage medium

Abstract

The invention relates to a code generation method and device and a computer readable storage medium, and relates to the technical field of computers. The code generation method comprises the following steps: acquiring multimedia content and text information input by a user; determining a running environment of a to-be-generated code according to the text information; according to the running environment and the multimedia content, a code corresponding to the multimedia content is generated, the code corresponding to the multimedia content is used for generating an interaction object, and at least part of features of the interaction object are the same as those of the multimedia content. According to the invention, the generation efficiency of the code of the interaction object is improved.

Description

Technical Field

[0001] The present disclosure relates to the field of computer technologies, and particularly to a code generation method and apparatus, and a computer-readable storage medium. Background Art

[0002] In today's digital age, the functions of the Internet have been continuously extended and penetrated into all aspects of people's lives and work. Users' requirements for interactive experiences have been continuously climbing, prompting the explosive growth of front-end development demands.

[0003] In the development process of an interactive object, first, a designer designs a design draft of the interactive object, and then a developer converts the design draft provided by the designer into corresponding code, thereby generating a product that can interact with users. Summary of the Invention

[0004] According to some embodiments of the present disclosure, there is provided a code generation method, including: obtaining multimedia content and text information input by a user; determining a running environment for the code to be generated according to the text information; and generating code corresponding to the multimedia content according to the running environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interactive object, and at least some features of the interactive object are the same as those of the multimedia content.

[0005] According to some other embodiments of the present disclosure, there is provided a code generation apparatus, including: an obtaining module configured to obtain multimedia content and text information input by a user; a determining module configured to determine a running environment for the code to be generated according to the text information; and a generating module configured to generate code corresponding to the multimedia content according to the running environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interactive object, and at least some features of the interactive object are the same as those of the multimedia content.

[0006] According to some other embodiments of the present disclosure, there is provided a code generation apparatus, including: a memory; and a processor coupled to the memory, where the processor is configured to execute the code generation method according to any embodiments of the present disclosure based on instructions stored in the memory.

[0007] According to some other embodiments of the present disclosure, there is provided a computer-readable storage medium having computer program instructions stored thereon, and when the instructions are executed by a processor, the code generation method according to any embodiments of the present disclosure is implemented.

[0008] According to some other embodiments of the present disclosure, there is provided a computer program product including computer program instructions, and when the computer program instructions are executed by a processor, the code generation method according to any embodiments of the present disclosure is implemented.

[0009] Other features, aspects, and advantages of the present disclosure will become apparent from the following detailed description of the exemplary embodiments of the present disclosure with reference to the accompanying drawings. Description of the Drawings

[0010] Embodiments of the present disclosure will now be described with reference to the accompanying drawings. It should be understood that the drawings in the following description only relate to some embodiments of the present disclosure and do not constitute a limitation on the present disclosure. In the drawings:

[0011] Figure 1 A flowchart showing a code generation method according to some embodiments of the present disclosure is shown;

[0012] Figure 2 A flowchart showing the process of generating code according to some embodiments of the present disclosure is shown;

[0013] Figures 3 - 4 A code generation interface according to some embodiments of the present disclosure is shown;

[0014] Figure 5 A block diagram showing a code generation device according to some embodiments of the present disclosure is shown;

[0015] Figure 6 A block diagram showing a code generation device according to some other embodiments of the present disclosure is shown;

[0016] Figure 7 A block diagram showing an electronic device according to some other embodiments of the present disclosure is shown.

[0017] It should be understood that, for the sake of convenience of description, the sizes of the various parts shown in the drawings are not necessarily drawn according to actual proportional relationships. The same or similar reference numerals are used in the various drawings to represent the same or similar components. Therefore, once an item is defined in one drawing, it may not be further discussed in subsequent drawings. Detailed Embodiments

[0018] The technical solutions in the embodiments of the present disclosure will be clearly and completely described below with reference to the accompanying drawings in the embodiments of the present disclosure. It should be understood that the present disclosure can be implemented in various forms and should not be construed as limited to the embodiments set forth herein.

[0019] It should be understood that the various steps described in the method embodiments of the present disclosure can be executed in different orders and / or in parallel. In addition, the method embodiments may include additional steps and / or omit the steps shown. The scope of the present disclosure is not limited in this regard. Unless otherwise specifically stated, the relative arrangements of the components and steps set forth in these embodiments should be construed as merely exemplary and do not limit the scope of the present disclosure.

[0020] As used in this disclosure, the term "comprising" and its variants refer to an open-ended term that includes at least the subsequent element / feature, but does not exclude other elements / features, that is, "including but not limited to". The term "based on" means "at least partially based on".

[0021] It should be noted that the concepts such as "first", "second", etc. mentioned in this disclosure are not intended to imply that the objects so described must be in a given order in terms of time, space, ranking, or any other way.

[0022] It should be noted that the modification of "one" and "multiple" mentioned in this disclosure is illustrative rather than restrictive. Those skilled in the art should understand that, unless clearly indicated otherwise in the context, it should be understood as "one or more".

[0023] The embodiments of the present disclosure will be described in detail below with reference to the accompanying drawings, but the present disclosure is not limited to these specific embodiments. These specific embodiments can be combined with each other, and the same or similar concepts or processes may not be repeated in some embodiments. In addition, in one or more embodiments, specific features, structures, or characteristics can be combined in any suitable manner that will be clear to those of ordinary skill in the art from this disclosure.

[0024] It should be understood that the present disclosure also does not limit how to obtain the multimedia content to be applied / processed. In some embodiments of the present disclosure, it can be obtained from a storage device, such as an internal memory or an external storage device.

[0025] Figure 1 A flowchart showing a code generation method according to some embodiments of the present disclosure is shown.

[0026] As Figure 1 shown, the code generation method includes: step S1, obtaining multimedia content and text information input by a user; step S2, determining the operating environment of the code to be generated according to the text information; step S3, generating code corresponding to the multimedia content according to the operating environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interaction object, and at least some features of the interaction object are the same as those of the multimedia content.

[0027] The forms of multimedia content may include videos, images, animations, etc. The text information includes, for example, the user's instructions for generating interactive objects, the user's intention to generate code, the description of the interactive objects, the conditions that the interactive objects should meet, etc. The text information is, for example, "Refer to this video and generate the code for a small game", "Refer to this picture and generate the code for a web page", etc. The user can input text through methods such as keyboard input, voice input, handwriting input, photo scanning input, etc., and the present disclosure does not limit this. The interactive objects are, for example, content generated through programming, which are the visual products of programming and the user's design goals, and can interact with the user. The interactive objects include, for example, web pages, charts, games, interactive interfaces, etc. The multimedia content input by the user includes, for example, demonstration examples, sketches, design drawings, screenshots, etc. of the interactive objects. The operating environment for the code to be generated includes, for example, browsers, local operating environments, mobile terminal operating environments, etc.

[0028] Taking the interactive object as a web page as an example, the user first inputs the design drawing of the web page to be generated and the text information about generating the web page. The model determines the user's needs based on the text information, and thus determines the programming language, code environment, technology, architecture, etc. Then, the model generates the code of the web page according to the operating environment and the design drawing. If the code is run in the operating environment, a web page similar to or the same as the design drawing can be generated.

[0029] According to the present disclosure, based on the text information input by the user, the operating environment for the code to be generated is determined, and based on the operating environment and the multimedia content input by the user, the code corresponding to the multimedia content is generated. In this way, the user can obtain the required code by inputting text information and multimedia content, improving the generation efficiency of the code of the interactive object.

[0030] The code generation method of this embodiment can be executed on the client side or partially on the server side.

[0031] Figure 2 A flowchart showing the process of generating code according to some embodiments of the present disclosure is shown.

[0032] The following combines Figure 2 , and introduces the process of generating code according to some embodiments of the present disclosure.

[0033] As Figure 2 shown, step S3 includes: step S31, determining one or more visual elements according to the multimedia content; step S32, generating the code of each visual element of the one or more visual elements according to the operating environment; step S33, generating the code corresponding to the multimedia content according to the code of each visual element.

[0034] For example, the model first parses out the visual elements included in the multimedia content. The visual elements may include, for example, buttons, links, dropdown menus, etc. Then, based on the multimedia content, the model analyzes the characteristics of each visual element and determines its appearance attributes according to the characteristics, such as position coordinates, size, color, material, transparency, visibility, etc. Based on these attributes, code describing the appearance and state of the element in the corresponding operating environment is generated.

[0035] The model can also parse the nested relationships between visual elements to determine which are parent elements and which are child elements. According to the nested relationships of the elements, the child elements are added to the corresponding parent elements in the code. Moreover, the model can also determine the interaction functions of the visual elements based on the multimedia content and add code to implement the interaction functions to the visual elements.

[0036] In some embodiments, generating code corresponding to the multimedia content according to the operating environment and the multimedia content includes: extracting the visual features of the multimedia content and the text features of the text information through a multimodal model; fusing the visual features and the text features to obtain fused features; generating code corresponding to the multimedia content according to the operating environment and the fused features.

[0037] The multimodal model is, for example, a model capable of processing, understanding, and fusing visual information and text information. The multimodal model extracts the visual features of the multimedia content and the text features of the text information, and fuses the visual features and the text features. Then, based on the fused features and the operating environment, the model performs a code generation task to obtain code for generating interactive objects.

[0038] According to the operating environment, the model can identify the parts that need to be focused on from the already fused multimodal features. For example, if the operating environment requires concise code syntax, the model pays more attention to the main features of the objects in the image and the core keywords in the text, assigns higher weights to these parts, and reduces the attention to minor detail features. Moreover, if the operating environment requires concise code syntax, the generated code adopts a simpler task logic.

[0039] In some embodiments, the multimedia content includes the animation of the interactive interface. Generating code corresponding to the multimedia content according to the operating environment and the multimedia content includes: determining the controls and control information in the interactive interface according to the animation of the interactive interface, where the control information includes at least one of the definition, attributes, position, style, and interaction events of the control; generating code corresponding to the multimedia content according to the control, the control information, and the operating environment, where the code corresponding to the multimedia content is used to generate the interactive interface.

[0040] For example, the interaction object includes an interaction interface, and the interaction interface includes multiple controls. The types of controls are, for example, buttons, text boxes, links, etc. The attributes of the controls include, for example, the identification of the controls, whether the controls are available, whether they are visible, etc. The model determines the position information of the controls in the interaction interface according to the positions of the controls displayed in the multimedia content. The styles of the controls include, for example, shape, color, font, size, etc. The interaction functions include, for example, click, hover, input validation, etc. Combining with the operating environment, the model can convert the control information into corresponding rules in the code.

[0041] In some embodiments, determining the controls and control information in the interaction interface according to the animation of the interaction interface includes: determining the key frames in the animation according to the animation and text information of the interaction interface; determining the controls and control information in the interaction interface according to the key frames.

[0042] The key frames include, for example, frames where operation changes occur, initial frames, etc. According to the frames where operation changes occur, the state changes caused by system or user behaviors can be captured. The initial frame is the starting point of the dynamic process, which carries the original state information of the system or interface and provides a basic reference for subsequent dynamic changes. According to the key frames, the model can, for example, analyze the layout, style, and attributes of the controls, the visual changes and / or functional responses of the controls after the user performs operations such as clicking, inputting, and dragging, etc. For example, according to the key frames, the model can determine the color gradient when a button in the interaction interface is pressed, the position update of an element after a drag operation, etc.

[0043] In some embodiments, determining the key frames in the animation according to the animation and text information of the interaction interface includes: determining the keywords according to the text information; determining the similarity between the vectors of each animation frame of the multiple animation frames of the animation and the vectors of the keywords; determining the animation frames whose similarity exceeds the threshold as the key frames.

[0044] For example, the model can determine the keywords related to the important functions and important controls of the interaction interface according to the text information. The model converts these keywords into vectors, and identifies the key frames according to the keywords related to the key functions, so as to find the key frames that can reflect the key functions. For example, if the text information includes that "the interface should include a confirmation function", then the keyword is "confirmation", and the model filters the animation frames before and after the "confirmation" control is triggered from the animation as the key frames.

[0045] In some embodiments, determining the controls and control information in the interaction interface according to the key frames includes: determining the control information according to the changes of the controls in adjacent key frames.

[0046] The model not only analyzes the morphological features of each control in a single frame, but also vertically compares and analyzes the associations between different key frames. By analyzing the visual changes, position migrations, and the triggering and response order of multiple controls across multiple frames, the model determines the collaboration mechanism between controls and more accurately identifies the controls and control information.

[0047] For example, in two adjacent key frames, a button may change from its initial normal state to a highlighted state when the mouse hovers over it. The model compares the same button in adjacent key frames, analyzes the state changes of the button and the corresponding appearance features of these state changes, such as changes in color and transparency. Based on these changes, the function, interaction logic, etc. of the control are determined.

[0048] In some embodiments, the multimedia content includes one or more first statistical charts. Codes corresponding to the multimedia content are generated according to the operating environment and the multimedia content, including: obtaining a data set; generating codes corresponding to the one or more first statistical charts according to the data set, the operating environment, and the one or more first statistical charts, where the codes are used to generate one or more second statistical charts, and at least one of the one or more second statistical charts has the same style as at least one of the one or more first statistical charts, and the style includes at least one of a title style, an axis style, a legend style, colors, fonts, element layout, graph types, and graph styles.

[0049] The first statistical charts include, for example, samples such as bar charts, line charts, and pie charts, but the data represented by the first statistical charts is not exactly the same as the data set that the user needs to analyze.

[0050] The model analyzes the data set input by the user and analyzes the types and styles, etc. of the first statistical charts to generate codes. The codes can be used to generate second statistical charts that are similar in appearance to the first statistical charts but analyze different data. Moreover, the user can interact with the second statistical charts through operations such as clicking, hovering, and zooming, thereby obtaining more data information and more intuitively understanding the data analysis results.

[0051] In some embodiments, generating codes corresponding to the one or more first statistical charts according to the data set, the operating environment, and the one or more first statistical charts includes: in response to the multimedia content including multiple first statistical charts, determining the graph type and variables of each of the multiple first statistical charts; determining the variables in the data set that match each of the first statistical charts according to the graph type and variables of each of the first statistical charts; generating codes corresponding to each of the first statistical charts according to the variables that match each of the first statistical charts and the operating environment, where the codes corresponding to each of the first statistical charts are used to generate second statistical charts with the same style for each of the first statistical charts.

[0052] For example, if the first statistical chart input by the user includes a bar chart and a pie chart, the model analyzes the visual information of the first statistical chart and determines that the second statistical chart should also include a bar chart and a pie chart, thereby generating code for generating a bar chart and code for generating a pie chart. The model extracts the text in the first statistical chart, determines the variables involved in the bar chart based on the text, selects target variables with the same or similar meanings in the dataset according to the variables involved in the bar chart, and generates a bar chart for analyzing the target variables. Similarly, the model determines the variables involved in the pie chart, selects target variables with the same or similar meanings in the dataset according to the variables involved in the pie chart, and generates a pie chart for analyzing the target variables.

[0053] In some embodiments, according to the operating environment and multimedia content, code corresponding to the multimedia content is generated, including: in response to code corresponding to one or more first statistical charts being used to generate multiple second statistical charts, generating interaction code between the multiple second statistical charts, where the interaction code is used to display the correlation relationship between the statistical data of the multiple second statistical charts.

[0054] For example, the model not only generates code for generating multiple second statistical charts, but also generates code for secondary editing of the multiple second statistical charts. The code for secondary editing of the multiple second statistical charts includes, for example, code for analyzing, constructing, and displaying the correlation relationship between the multiple second statistical charts. Displaying the correlation relationship between the statistical data of the multiple second statistical charts includes, for example, when the user selects a certain variable in the second statistical chart A, the data related to the variable will be highlighted, and at the same time, it can trigger the highlighting of the data related to the variable in the second statistical chart B. When the user performs a filtering operation on one second statistical chart, the other second statistical charts will filter the data according to the filtering result. For example, in a visualization interface including a bar chart and a line chart, the bar chart shows the number of employees in each unit each year, and the line chart shows the revenue change of each unit from 2010 to 2020. When the user selects the number of employees of a certain unit in a certain year in the bar chart, the line chart highlights the annual revenue of that unit.

[0055] Figures 3 - 4 Shows a code generation interface according to some embodiments of the present disclosure.

[0056] As Figure 3 shown, the user can input the design drawing 403 of the interaction object and the text information 402 in the input box 301. When the intelligent assistant starts generating code, it can enter the interface as Figure 4 shown. The user can also click the "Programming" button in each function bar or recommendation bar in the interface as Figure 3 shown and enter the interface as Figure 4 shown.

[0057] In Figure 4The conversation display area on the left side of the shown interface can display the design drawing 403 and text information 402 input by the user. The user can also supplement the input of text information through the dialog box 401. In the code display area on the right side of the interface, the generated code is displayed. And after the code is generated, a preview of the interaction object that can be obtained by running the code can also be displayed in the interface.

[0058] Figure 5 The block diagram of a code generation device according to some embodiments of the present disclosure is shown.

[0059] As Figure 5 shown, the code generation device 5 includes: an acquisition module 51 configured to acquire the multimedia content and text information input by the user; a determination module 52 configured to determine the operating environment of the code to be generated according to the text information; a generation module 53 configured to generate the code corresponding to the multimedia content according to the operating environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interaction object, and at least part of the features of the interaction object are the same as those of the multimedia content.

[0060] The acquisition module 51 of the code generation device 5 can be used to execute Figure 1 step S1, the determination module can be used to execute Figure 1 step S2, and the generation module 53 can be used to execute Figure 1 step S3.

[0061] According to the above device of the present disclosure, the operating environment of the code to be generated is determined according to the text information input by the user, and the code corresponding to the multimedia content is generated according to the operating environment and the multimedia content input by the user. In this way, the user can obtain the required code by inputting text information and multimedia content, improving the generation efficiency of the code of the interaction object.

[0062] In some embodiments, the generation module 53 is further configured to: extract the visual features of the multimedia content and the text features of the text information through a multimodal model; fuse the visual features and the text features to obtain fused features; generate the code corresponding to the multimedia content according to the operating environment and the fused features.

[0063] In some embodiments, the generation module 53 is further configured to: determine the controls and control information in the interaction interface according to the animation of the interaction interface, where the control information includes at least one of the definition, attributes, position, style, and interaction events of the control; generate the code corresponding to the multimedia content according to the control, the control information, and the operating environment, where the code corresponding to the multimedia content is used to generate an interaction interface.

[0064] In some embodiments, the generation module 53 is further configured to: determine key frames in the animation according to the animation and text information of the interaction interface; determine the controls and control information in the interaction interface according to the key frames.

[0065] In some embodiments, the generation module 53 is further configured to: determine keywords according to the text information; determine the similarity between the vector of each animation frame of the multiple animation frames of the animation and the vector of the keywords; determine the animation frames with similarity exceeding the threshold as key frames.

[0066] In some embodiments, the generation module 53 is further configured to: determine the control information according to the changes of the controls in adjacent key frames.

[0067] In some embodiments, the multimedia content includes one or more first statistical charts, and the generation module 53 is further configured to: obtain a data set; generate code corresponding to the one or more first statistical charts according to the data set, the operating environment, and the one or more first statistical charts, wherein the code is used to generate one or more second statistical charts, and at least one of the one or more second statistical charts has the same style as at least one of the one or more first statistical charts, and the style includes at least one of a title style, an axis style, a legend style, a color, a font, an element layout, a graph type, and a graph style.

[0068] In some embodiments, the generation module 53 is further configured to: in response to the multimedia content including multiple first statistical charts, determine the graph type and variables of each of the multiple first statistical charts; determine the variables in the data set that match each of the first statistical charts according to the graph type and variables of each of the first statistical charts; generate code corresponding to each of the first statistical charts according to the variables that match each of the first statistical charts and the operating environment, wherein the code corresponding to each of the first statistical charts is used to generate a second statistical chart with the same style for each of the first statistical charts.

[0069] In some embodiments, the generation module 53 is further configured to: in response to the code corresponding to the one or more first statistical charts being used to generate multiple second statistical charts, generate interaction code between the multiple second statistical charts, wherein the interaction code is used to display the association relationship between the statistical data of the multiple second statistical charts.

[0070] Figure 6 The block diagram of a code generation device according to some embodiments of the present disclosure is shown.

[0071] As Figure 6As shown, the code generation device 6 includes: at least one memory 61; and at least one processor 62 coupled to the at least one memory 61, the at least one processor 62 being configured to execute the code generation method as described in any of the foregoing embodiments based on instructions stored in the at least one memory 61.

[0072] According to the above device of the present disclosure, the operating environment for the code to be generated is determined based on the text information input by the user, and the code corresponding to the multimedia content is generated according to the operating environment and the multimedia content input by the user. In this way, the user can obtain the required code by inputting text information and multimedia content, improving the generation efficiency of the code of the interaction object.

[0073] The memory 61 is used to store one or more computer-readable instructions. The memory 61 may include any combination of various forms of computer-readable storage media, such as volatile memory and / or non-volatile memory, including but not limited to random access memory (RAM), dynamic random access memory (DRAM), static random access memory (SRAM), read-only memory (ROM), flash memory. The memory 61 may store, for example, an operating system, application programs, a boot loader (BootLoader), a database, and other programs, and may also store various application programs and various data, etc.

[0074] The processor 62 is used to run the computer-readable instructions to implement the method described in any of the foregoing embodiments. For the specific implementation of each step of the method, reference may be made to the above embodiments, and the repeated parts will not be elaborated here.

[0075] The processor 62 may be configured to execute Figure 1 each step in. The processor 62 may be embodied as various processing devices, such as a central processing unit (CPU), a network processor (NP), etc.; it may also be a digital signal processor (DSP), an application-specific integrated circuit (ASIC), a field programmable gate array (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components. The central processing unit (CPU) may be of the X86 or ARM architecture, etc.

[0076] The processor 62 and the memory 61 may communicate with each other directly or indirectly. For example, the processor 62 and the memory 61 may communicate through a network. The network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network. The processor 62 and the memory 61 may also communicate with each other through a system bus, and the present disclosure does not limit this.

[0077] It should be noted that Figure 6The components of the code generating device 6 shown are merely exemplary and non-limiting. The code generating device 6 may further include other components according to actual application requirements. The processor 62 may control other components in the code generating device 6 to perform desired functions.

[0078] The code generating device 6 may be implemented by software, firmware and / or hardware, and may be integrated into a device installed with relevant application programs.

[0079] Figure 7 A block diagram of an electronic device according to some other embodiments of the present disclosure is shown.

[0080] Figure 7 The electronic device 7 shown may be a computer system with a dedicated hardware structure, which can execute corresponding functions when a relevant application program is installed.

[0081] Electronic devices include, but are not limited to, mobile terminals such as smartphones, laptops, personal digital assistants (PDAs), tablet personal computers (Tablet PCs), PMPs (portable multimedia players), vehicle-mounted terminals (such as vehicle-mounted navigation terminals), wearable devices, etc., as well as fixed terminals such as digital televisions and desktop computers, etc.

[0082] like Figure 7 As shown, the central processing unit (CPU) 71 executes various processes according to the program stored in the read-only memory (ROM) 72 or the program loaded from the storage part 78 to the random access memory (RAM) 73. In the RAM 73, data required when the CPU 71 executes various processes is stored as needed. The central processing unit is merely an example, and it can also be other types of processors, such as the various processors described above. The ROM 72, RAM 73 and the storage part 78 can be various forms of computer-readable storage media. It should be noted that although Figure 7 ROM 72, RAM 73 and storage portion 78 are shown separately in FIG, but one or more of them may be combined or located in the same or different memory or storage modules.

[0083] The CPU 71, the ROM 72, and the RAM 73 are connected to one another via a bus 74. To the bus 74, an input / output interface 75 is also connected.

[0084] The following components are connected to the input / output interface 75: an input section 76, such as a touch screen, a touch pad, a keyboard, a mouse, an image sensor, a microphone, an accelerometer, a gyroscope, etc.; an output section 77, including a display, such as a cathode ray tube (CRT), a liquid crystal display (LCD), a speaker, a vibrator, etc.; a storage section 78, including a hard disk, a magnetic tape, etc.; and a communication section 79, including a network interface card such as a LAN card, a modem, etc. The communication section 79 allows communication processing to be performed via a network such as the Internet. It is easily understandable that although Figure 7 the parts shown in Figure 7 communicate with each other through the bus 74 in the electronic device 7, they may also communicate through a network or other means, where the network may include a wireless network, a wired network, and / or any combination of a wireless network and a wired network.

[0085] As needed, a drive 710 is also connected to the input / output interface 75. A removable medium 711, such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc., is installed on the drive 710 as needed, so that a computer program read therefrom is installed in the storage section 78 as needed.

[0086] In the case where the above series of processes are implemented by software, a program constituting the software can be installed from a network such as the Internet or a storage medium such as the removable medium 711.

[0087] According to an embodiment of the present disclosure, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, some embodiments of the present disclosure include a computer program product, which, when running on a computer, causes the computer to implement the method described in any one of the foregoing embodiments. The computer program product includes computer instructions carried on a computer-readable medium, including program codes for executing the method shown in the flowchart. In such an embodiment, the computer instructions can be downloaded and installed from the network through the communication section 79, or installed from the storage section 78, or installed from the ROM 72. When the computer program is executed by the CPU 71, the method of the embodiment of the present disclosure is executed.

[0088] It should be noted that, in the context of the present disclosure, a computer-readable medium can be a tangible medium that can contain or store a program for use by or in connection with an instruction execution system, apparatus, or device.

[0089] A computer-readable medium can be a computer-readable storage medium, or a computer-readable signal medium, or any combination of the above two.

[0090] Computer-readable storage media include, but are not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or components, or any combination of the above. More specific examples of computer-readable storage media may include, but are not limited to, electrical connections with one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fibers, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the above. In the present disclosure, a computer-readable storage media may be any tangible medium that contains or stores a program that can be used by or in conjunction with an instruction execution system, device, or component. Computer instructions are stored on the computer-readable storage media, and when executed by a processor, implement the method described in any of the foregoing embodiments.

[0091] According to the above-mentioned device of the present disclosure, based on the text information input by the user, the operating environment for the code to be generated is determined, and based on the operating environment and the multimedia content input by the user, code corresponding to the multimedia content is generated. In this way, the user can obtain the required code by inputting text information and multimedia content, improving the generation efficiency of the code for the interaction object.

[0092] A computer-readable signal medium may include a data signal propagated in a baseband or as part of a carrier wave, which carries computer-readable program code. Such a propagated data signal may take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, device, or component. The program code contained on the computer-readable medium may be transmitted using any appropriate medium, including but not limited to: wires, optical cables, RF (radio frequency), etc., or any suitable combination of the above.

[0093] The above-mentioned computer-readable medium may be included in the above-mentioned electronic device; or it may exist separately and not be assembled into the electronic device.

[0094] In some embodiments, a computer program is also provided, including: instructions that, when executed by a processor, cause the processor to execute the method described in any of the foregoing embodiments. For example, the instructions may be embodied as computer program code.

[0095] In embodiments of the present disclosure, computer program code for performing the operations of the present disclosure may be written in one or more programming languages or combinations thereof. The foregoing programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, C++, and also include conventional procedural programming languages such as the "C" language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on the remote computer or server. In the case of a remote computer, the remote computer may be connected to the user's computer through any type of network (including a local area network (LAN) or a wide area network (WAN)), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0096] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present disclosure. In this regard, each block in the flowchart or block diagram may represent a module, a program segment, or a part of code that contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions noted in the blocks may occur in a different order than noted in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram and / or flowchart, and combinations of blocks in the block diagram and / or flowchart, may be implemented by a dedicated hardware-based system for performing the specified functions or operations, or may be implemented by a combination of dedicated hardware and computer instructions.

[0097] The functions described above may be performed, at least in part, by one or more hardware logic components. For example, by way of non-limitation, exemplary hardware logic components that may be used include: field programmable gate arrays (FPGA), application specific integrated circuits (ASIC), application specific standard products (ASSP), system on a chip (SOC), complex programmable logic devices (CPLD), and the like.

[0098] Although some specific embodiments of the present disclosure have been described in detail by way of examples, those skilled in the art should understand that the above examples are for illustrative purposes only and are not intended to limit the scope of the present disclosure. Those skilled in the art should understand that the above embodiments may be modified without departing from the scope and spirit of the present disclosure. The scope of the present disclosure is defined by the appended claims.

Claims

1. A code generation method, comprising: Obtaining multimedia content and text information input by a user; Determining a running environment for the code to be generated according to the text information; Generating code corresponding to the multimedia content according to the running environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interaction object, and at least part of the features of the interaction object are the same as those of the multimedia content.

2. The code generation method according to claim 1, wherein, Generating code corresponding to the multimedia content according to the running environment and the multimedia content, comprising: Determining one or more visual elements according to the multimedia content; Generating code for each of the one or more visual elements according to the running environment; Generating code corresponding to the multimedia content according to the code for each visual element.

3. The code generation method according to claim 1, wherein, Generating code corresponding to the multimedia content according to the running environment and the multimedia content, comprising: Extracting visual features of the multimedia content and text features of the text information through a multimodal model; Fusing the visual features and the text features to obtain fused features; Generating code corresponding to the multimedia content according to the running environment and the fused features.

4. The code generation method according to claim 1, wherein, The multimedia content includes an animation of an interaction interface. Generating code corresponding to the multimedia content according to the running environment and the multimedia content, comprising: Determining controls and control information in the interaction interface according to the animation of the interaction interface, wherein the control information includes at least one of a definition, attributes, position, style, and interaction event of the control; Generating the code corresponding to the multimedia content according to the control, the control information, and the running environment, wherein the code corresponding to the multimedia content is used to generate the interaction interface.

5. The code generation method according to claim 4, wherein, Determining controls and control information in the interaction interface according to the animation of the interaction interface, comprising: Determining key frames in the animation according to the animation of the interaction interface and the text information; Determining controls and control information in the interaction interface according to the key frames.

6. The code generation method according to claim 5, wherein, Determining key frames in the animation according to the animation of the interaction interface and the text information, comprising: Determining keywords according to the text information; Determining the similarity between the vector of each animation frame of the animation and the vector of the keyword; Determining the animation frames with a similarity exceeding a threshold as the key frames.

7. The code generation method according to claim 5, wherein, Determining controls and control information in the interaction interface according to the key frames, comprising: Determining the control information according to the change of the control in adjacent key frames.

8. The code generation method according to claim 1, wherein, The multimedia content includes one or more first statistical charts. Generating code corresponding to the multimedia content according to the running environment and the multimedia content, comprising: Obtaining a data set; Generate the code corresponding to the one or more first statistical charts according to the data set, the operating environment, and the one or more first statistical charts, wherein the code is used to generate one or more second statistical charts, and the one or more second statistical charts are in the same style as at least one of the one or more first statistical charts, and the style includes at least one of a title style, an axis style, a legend style, colors, fonts, element layout, graph type, and graph style.

9. The code generation method according to claim 8, wherein, Generating the code corresponding to the one or more first statistical charts according to the data set, the operating environment, and the one or more first statistical charts includes: In response to the multimedia content including a plurality of first statistical charts, determine the graph type and variables of each of the plurality of first statistical charts; According to the graph type and variables of each of the first statistical charts, determine the variables in the data set that match each of the first statistical charts; Generate the code corresponding to each of the first statistical charts according to the variables matching each of the first statistical charts and the operating environment, wherein the code corresponding to each of the first statistical charts is used to generate a second statistical chart in the same style for each of the first statistical charts.

10. The code generation method according to claim 8, wherein, The generating the code corresponding to the multimedia content according to the operating environment and the multimedia content includes: In response to the code corresponding to one or more first statistical charts being used to generate a plurality of second statistical charts, generate interaction code between the plurality of second statistical charts, wherein the interaction code is used to display the correlation between the statistical data of the plurality of second statistical charts.

11. A code generation device, comprising: An acquisition module configured to acquire multimedia content and text information input by a user; A determination module configured to determine the operating environment for which code is to be generated according to the text information; A generation module configured to generate code corresponding to the multimedia content according to the operating environment and the multimedia content, wherein the code corresponding to the multimedia content is used to generate an interaction object, and at least some features of the interaction object are the same as those of the multimedia content.

12. A code generation device, comprising: A memory; And A processor coupled to the memory, the processor being configured to execute the code generation method according to any one of claims 1 to 10 based on instructions stored in the memory.

13. A computer-readable storage medium having computer program instructions stored thereon, which when executed by a processor, implement the code generation method according to any one of claims 1 to 10.

14. A computer program product comprising computer program instructions, which when executed by a processor, implement the code generation method according to any one of claims 1 to 10.

Images