Automatic control generation method and system, equipment and storage medium

Abstract

The invention discloses an automatic control generation method and system, equipment and a storage medium, and belongs to the technical field of computers.The automatic control generation method includes the steps that requirements of a user for controls are collected and analyzed, and the requirements of the user for the controls are converted into semantic vectors capable of being understood by a computer through a natural language processing technology; the missing data and the abnormal data are processed by using a Pandas library, and the collected demand data are cleaned, labeled and formatted; training a deep learning model by using the labeled formatted data set, so that the deep learning model can accurately understand user requirements and generate corresponding codes and styles; and inputting the user demand data into the deep learning model, and generating a source code and a style file of the control according to the output of the deep learning model. According to the method, the source code and the style of the control can be intelligently generated according to the demand description input by the user, automatic conversion from the user demand to the code and the style is realized, and the development efficiency of the control is greatly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer technology, and in particular to a control automatic generation method and an automatic generation system, device and storage medium. Background Art

[0002] A control is a visual component that is embedded in an application's window, dialog box, or other interface elements to provide users with a way to interact with the program. It also helps the program display information, receive user input, and perform corresponding functions.

[0003] The visual interface of the development platform can be optimized through the custom control development interface of the development platform. With the rapid development of software development, the demand for custom controls is growing. The traditional custom control development method requires developers to have a deep programming foundation, and the development process is cumbersome and time-consuming. In recent years, although some low-code platforms and automation tools have emerged, they usually only provide limited customization capabilities and cannot meet the complex and changing control requirements.

[0004] In view of this, it is necessary to provide a new technical solution to solve the above problems. Summary of the invention

[0005] To solve the above technical problems, the present application provides a method and system for automatically generating custom controls, which can intelligently generate the source code and style of the control according to the requirement description input by the user, thereby realizing the automatic conversion from user requirements to code and style.

[0006] A control automatic generation method, comprising: Collect and analyze user requirements for controls, and convert them into semantic vectors that can be understood by computers through natural language processing technology; Use the Pandas library to process missing and abnormal data, and clean, annotate, and format the collected demand data; Use annotated formatted data sets to train deep learning models so that they can accurately understand user needs and generate corresponding code and styles; Input user demand data into the deep learning model, and generate the source code and style files of the control based on the output of the deep learning model to complete the automatic generation of the control.

[0007] Preferably, the collecting and parsing of user requirements for controls, and converting the user requirements for controls into a semantic vector understandable by a computer through natural language processing technology, includes: Build an interface for users to input their requirements for custom controls; Users input their needs into the constructed interface in the form of natural language; Adopt NLP word embedding technology to convert the natural language requirements input by users into semantic vectors that can be understood by computers; Among them, the user's requirements for custom controls include control style, control function, control input properties and output properties.

[0008] Preferably, the NLP word embedding technology is used to convert the natural language requirements input by the user into semantic vectors that can be understood by the computer, including: Filter keywords in the natural language text input by the user, and take each filtered keyword as a requirement; For each requirement, create a vector space containing several requirement elements; Convert the requirement element corresponding to each requirement to one or more Boolean values.

[0009] Preferably, the cleaning, labeling and formatting of the collected demand data includes: Process and clean user input to remove redundant information; Label the corresponding functions and attributes for the requirements entered by the user; Expand the labeled data; Collect multiple requirement texts and corresponding annotations with associated controls, and build training sets and validation sets.

[0010] Preferably, the expansion of the annotated data includes: synonym replacement, sentence conversion, adding specific examples, combination generation, adding negative sentences and context changes.

[0011] Preferably, the step of inputting user demand data into a deep learning model and generating a source code and a style file of a control according to an output of the deep learning model, thereby completing the automatic generation of the control, includes: Convert user requirements into vector inputs; Select predefined control templates according to your needs; The model output contains the required code snippet of the control's structure, properties, and style; Generate the complete code using the decoder in the neural network; Use static code analysis tools to perform syntax checks and use legal code as source code; Output the legal code to the corresponding library file and integrate it into the corresponding development platform.

[0012] According to another aspect of the present application, there is also provided a control automatic generation system for executing the control automatic generation method, comprising: a collection module, a data preprocessing module, a deep learning model, a code generation module and a style generation module; The collection module is used for user input control requirements; the data preprocessing module is used to clean, annotate and format the control requirements input by the user; the deep learning model is used to extract and convert features; the code and style generation module is used to generate source code and style files for the control.

[0013] Preferably, it also includes an evaluation feedback module; the evaluation feedback module is used to perform automatic testing and user evaluation on the generated code and style.

[0014] According to another aspect of the present application, a computing device is also provided, including: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the control automatic generation method as described above is executed.

[0015] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed on a computer, the computer executes the automatic control generation method.

[0016] Compared with the prior art, this application has at least the following beneficial effects: 1. The present invention can intelligently generate the source code and style of the control according to the demand description input by the user, realizes the automatic conversion from user demand to code and style, and greatly improves the efficiency of control development.

[0017] 2. The present invention can generate personalized custom controls according to different needs of users to meet diverse development scenarios.

[0018] 3. The present invention can use deep learning technology to accurately understand user needs and generate codes and styles that meet the requirements. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Hereinafter, some specific embodiments of the present invention will be described in detail in an exemplary and non-limiting manner with reference to the accompanying drawings. The same reference numerals in the accompanying drawings indicate the same or similar components or parts. It should be understood by those skilled in the art that these drawings are not necessarily drawn to scale. In the accompanying drawings: Figure 1 It is a schematic diagram of the overall process of the method for automatically generating controls of the present invention; Figure 2 A schematic diagram of an input collection interface in an embodiment of the present invention; Figure 3 The figure is a flow chart of converting a natural language requirement into a semantic vector in an embodiment of the present invention. DETAILED DESCRIPTION

[0020] In order to make the purpose, technical solution and advantages of the present application clearer, the technical solution of the present application will be clearly and completely described below in combination with the specific embodiments of the present application and the corresponding drawings. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of the present application.

[0021] like Figure 1 As shown, a method for automatically generating a control comprises the following steps: Step S1, collecting and analyzing user requirements for controls, and converting user requirements for controls into semantic vectors that can be understood by computers through natural language processing technology; Step S2: Use the Pandas library to process missing data and abnormal data, and clean, annotate and format the collected demand data; Step S3: Use the labeled formatted data set to train the deep learning model so that it can accurately understand user needs and generate corresponding code and style; Step S4: Input the user demand data into the deep learning model, and generate the source code and style file of the control according to the output of the deep learning model, so as to complete the automatic generation of the control.

[0022] In this embodiment, the automatic generation method of the dashboard control is taken as an example to describe the automatic generation method of the control. The automatic generation method of the control includes: Requirements collection: Through natural language processing technology, collect and analyze users' requirements for the functions, properties, and styles of custom controls.

[0023] Input collection: Build an interface that allows users to input their requirements for custom controls. For example, users can describe the required input properties, output properties, and style requirements. Figure 2 shown.

[0024] Semantic analysis: Use NLP word embedding technology to convert users' natural language needs into semantic vectors that can be understood by computers.

[0025] The text description of the user-entered requirement is "1. Dashboard 2. The overall style is dark 3. Supports changeable size 4. Supports changeable color; 5. Supports input of current value; 6. Supports input value range; 7. Has scale lines; 8. Supports setting scale line precision; 9. Supports display unit;" Such a description is converted into a structured or vectorized representation.

[0026] Filter keywords in the text, each keyword is a requirement, and create a vector space for each requirement, which contains the unique elements of the requirement. Convert each requirement into one or more Boolean values. For example, "support changeable size" can be converted into a separate Boolean feature. Figure 3 shown.

[0027] Data preprocessing: Use Pandas to process missing values, detect outliers, check duplicate values, encode labels, etc., and clean, annotate, and format the collected required data for use in deep learning model training.

[0028] Data cleaning: Process and clean user input to remove redundant information.

[0029] Annotate data: Annotate relevant functions and attributes for the requirements entered by the user.

[0030] The above text is structurally annotated to obtain: { "type": "Dashboard", "style": "Dark", "size_change": true, "color_change": true, "input_current_value": true, "input_value_range": true, "has_scale": true, "scale_precision": true, "show_unit": true } Data enhancement: Data enhancement is to expand the annotated data, such as generating synonymous sentences, changing word order, etc., to enrich the training data. The current text demand data expansion is shown in Table 1.

[0031] Table 1 Text requirement data extension example table

[0032] Dataset construction: Collect the requirement texts and corresponding annotations of multiple related controls to build training sets and validation sets to ensure the diversity and adequacy of the dataset.

[0033] Deep learning model training: Use labeled data sets to train deep learning models so that they can accurately understand user needs and generate corresponding code and styles.

[0034] At present, the Transformer model is planned to be used. This type of model performs well in natural language understanding and generation and is more suitable for processing user demand descriptions. Compared with the Transformer model, the RNN model and the LSTM model may have weaker ability to understand and generate complex codes; the CNN model is mainly used for image processing and is not very suitable for processing text data and generating code.

[0035] The following is an understanding and training of the model to realize user needs, and finally generate control code for a detailed description of the model structure and components: The model includes: input module, Transformer encoder, demand understanding module and Transformer decoder. The model structure is as follows: Input module, used to implement user demand input and preprocessing.

[0036] User requirement input: The user enters the requirement description for the control through the graphical interface or command line.

[0037] Preprocessing: Preprocess the requirement description entered by the user, including word segmentation, removal of stop words, and standardization, to ensure the consistency and cleanliness of the input data.

[0038] Transformer encoder, which is used to implement embedding layers, positional encoding, multi-head self-attention mechanism, feedforward neural network, residual connection and layer normalization.

[0039] Embedding layer: Converts the input text into a vector representation. Each word is mapped to a vector of fixed dimension.

[0040] Positional encoding: Adds position information to help the model understand the order of words.

[0041] Multi-head self-attention mechanism: Capture different relationships in the input sequence through multiple attention heads to enhance the model's understanding ability.

[0042] Feedforward neural network: Perform nonlinear transformation on the vector at each position to further extract features.

[0043] Residual connections: Residual connections are added after each sub-layer to help the gradient flow and avoid the gradient vanishing problem.

[0044] Layer Normalization: Normalize the output of each sub-layer to stabilize model training.

[0045] The demand understanding module is used to implement demand analysis, demand type classification, and demand attribute extraction.

[0046] Demand analysis: The feature vector generated by the encoder is used to analyze the key information of user needs, such as control type, style attributes, behavior, etc.

[0047] Requirement type classification: Use classifiers to classify requirement types and determine the specific control types (such as buttons, input boxes, drop-down menus, etc.) required by users.

[0048] Required attribute extraction: Extract specific attributes of controls, such as background color, font color, border style, etc.

[0049] The Transformer decoder is used to implement embedding layers, position encoding, multi-head attention mechanism, encoder-decoder attention mechanism, feedforward neural network, residual connection, and layer normalization.

[0050] Embedding layer: Converts the initial sequence of control codes and styles into a vector representation.

[0051] Positional encoding: Adds position information to help the model understand the order of code sequences.

[0052] Multi-head attention mechanism: Capture the relationship between encoder output and decoder input through multiple attention heads.

[0053] Encoder-decoder attention mechanism: combines the output of the encoder and the input of the decoder to generate new feature representations.

[0054] Feedforward neural network: Perform nonlinear transformation on the vector at each position to further extract features.

[0055] Residual Connection: A residual connection is added after each sub-layer.

[0056] Layer Normalization: Normalize the output of each sub-layer.

[0057] The output module is used for code generation, style generation, and output formatting.

[0058] Code generation: Generate Python code for the control based on the feature vector generated by the decoder.

[0059] Style generation: Generates the corresponding CSS style sheet to define the visual appearance of the control.

[0060] Output formatting: Format the generated code and style to ensure that the output code is neat and easy to read.

[0061] An example of this embodiment is as follows: User input: “I needed a dashboard with a blue background and white text, showing real-time temperature and humidity data, and the date and time in the lower right corner.

[0062] Preprocessing: Word segmentation: The input text is segmented into "I", "need", "one", "with", "blue", "background", "and", "white", "text", "of", "dashboard", "display", "real-time", "of", "temperature", "and", "humidity", "data", "and", "in", "lower right corner", "display", "date", "and", and "time".

[0063] Remove stop words: Remove stop words such as "I", "a", "and", "of", "and", and "in".

[0064] Normalize: Normalize the remaining words to lowercase and uniform format.

[0065] Requirements analysis: Control type: Dashboard.

[0066] Style attributes: background color (blue), text color (white).

[0067] Display content: Real-time temperature and humidity data, with date and time displayed in the lower right corner.

[0068] The code is generated as follows: import tkinter as tk from tkinter import ttk from datetime import datetime import time class DashboardApp: def __init__(self, root): self.root = root self.root.title("Real-time Dashboard") self.root.configure(bg="blue") # Create a display area self.data_frame = tk.Frame(root, bg="blue") self.data_frame.pack(pady=20) self.temperature_label = tk.Label(self.data_frame, text="Temperature: 25°C", fg="white", bg="blue", font=("Arial", 16)) self.temperature_label.pack() self.humidity_label = tk.Label(self.data_frame, text="Humidity: 60%", fg="white", bg="blue", font=("Arial", 16)) self.humidity_label.pack() self.date_time_frame = tk.Frame(root, bg="blue") self.date_time_frame.pack(side=tk.BOTTOM, anchor=tk.SE, padx=20, pady=20) self.date_label = tk.Label(self.date_time_frame, fg="white", bg="blue", font=("Arial", 12)) self.date_label.pack() self.time_label = tk.Label(self.date_time_frame, fg="white", bg="blue", font=("Arial", 12)) self.time_label.pack() # Start the time update thread self.update_time() def update_time(self): current_time = datetime.now().strftime("%Y-%m-%d %H:%M:%S") self.date_label.config(text=current_time.split()[0]) self.time_label.config(text=current_time.split()[1]) self.root.after(1000, self.update_time) if __name__ == "__main__": root = tk.Tk() app = DashboardApp(root) root.mainloop() Style Generation body, html { height: 100%; margin: 0; padding: 0; } .dashboard { background-color: blue; color: white; height: 100%; display: flex; flex-direction: column; align-items: center; justify-content: center; font-family: Arial, sans-serif; } .data { margin-top: 20px; } .data div { font-size: 1.6rem; margin-bottom: 10px; } .date-time { position: absolute; bottom: 20px; right: 20px; font-size: 1.2rem; } Select model: Select a deep learning model (Transformer) for demand understanding and training.

[0069] Training data: Use labeled data sets to train the model so that the model can generate control codes from the requirements document description.

[0070] Code and style generation: Generate source code and style files of controls based on the output of the deep learning model.

[0071] Input vector: Convert user requirements into vector input through the model.

[0072] Template selection: Select a predefined control template based on your needs.

[0073] Generate code: The model outputs the required code snippet, including the structure, properties, and style of the control. Use the decoder part in the neural network to generate the complete code. The above requirements use the decoder example: class Dashboard: def __init__(self, size, color, current_value, value_range, scale_precision, unit): self.size = size # variable size self.color = color # Variable color self.current_value = current_value # Current value self.value_range = value_range # value range self.scale_precision = scale_precision # scale line precision self.unit = unit # Display unit # Instantiate the dashboard dashboard = Dashboard(size='variable size', color='dark', current_value=0,value_range=(0, 100), scale_precision=1, unit='unit') Code processing: Use static code analysis tools to perform syntax checks to ensure the legality of the generated code.

[0074] Generate library files: Because the control ultimately serves a specific business, it may be necessary to generate a library to be integrated into the corresponding development platform, such as a DLL file.

[0075] Furthermore, it also includes evaluation and feedback: testing and user evaluation of the generated code and style, and continuously optimizing the deep learning model based on the feedback results.

[0076] Provide a user feedback mechanism to allow users to evaluate the generated controls and collect feedback information. Iterate and improve the training data, model, and generation logic based on user feedback to improve the accuracy and practicality of the model.

[0077] Specifically, testing includes automatic testing, unit testing and performance testing.

[0078] Automatic testing: Perform automated testing on the generated code to ensure that it meets user requirements and is error-free.

[0079] Unit testing: Write unit test cases to test the execution effect of the generated controls under different parameters. For example: verify whether the size change is effective; verify whether the color change can be displayed normally; verify whether the input value and range meet the expectations.

[0080] Visual testing: Perform visual verification in the graphical interface to check whether the appearance of the dashboard controls meets the requirements. For example, you can use the UI testing framework (PyQt) to perform interface verification.

[0081] Performance testing: Evaluate the performance of dashboard controls under high load or extreme conditions.

[0082] Based on the same inventive concept, a control automatic generation system is also provided for executing the control automatic generation method, including: a collection module, a data preprocessing module, a deep learning model, a code generation module and a style generation module.

[0083] Among them, the collection module is used for user input control requirements; the data preprocessing module is used to clean, annotate and format the control requirements input by the user; the deep learning model is used to extract and convert features; the code and style generation module is used to generate the source code and style file of the control.

[0084] Preferably, a control automatic generation system further includes an evaluation feedback module; the evaluation feedback module is used to perform automatic testing and user evaluation on the generated code and style.

[0085] According to another aspect of the present application, a computing device is also provided, including: a processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the control automatic generation method as described above is executed.

[0086] According to another aspect of the present application, a computer-readable storage medium is provided, on which computer instructions are stored. When the instructions are executed on a computer, the computer executes the automatic control generation method.

[0087] For ease of description, spatially relative terms such as "above", "above", "on the upper surface of", "above", etc. may be used here to describe the spatial positional relationship between a device or feature and other devices or features as shown in the figure. It should be understood that spatially relative terms are intended to include different orientations of the device in use or operation in addition to the orientation described in the figure. For example, if the device in the accompanying drawings is inverted, the device described as "above other devices or structures" or "above other devices or structures" will be positioned as "below other devices or structures" or "below other devices or structures". Thus, the exemplary term "above" can include both "above" and "below". The device can also be positioned in other different ways (rotated 90 degrees or in other orientations), and the spatially relative descriptions used here are interpreted accordingly.

[0088] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application. As used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, it indicates the presence of features, steps, operations, devices, components and / or combinations thereof.

[0089] It should be noted that the terms "first", "second", etc. in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the numbers used in this way can be interchanged where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein.

[0090] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. For those skilled in the art, the present invention may have various modifications and variations. Any modification, equivalent replacement, improvement, etc. made within the spirit and principle of the present invention shall be included in the protection scope of the present invention.

Claims

1. A method for automatically generating a control, characterized in that: include: Collect and analyze user requirements for controls, and convert them into semantic vectors that can be understood by computers through natural language processing technology; Use the Pandas library to process missing and abnormal data, and clean, annotate, and format the collected demand data; Use annotated formatted data sets to train deep learning models so that they can accurately understand user needs and generate corresponding code and styles; Input user demand data into the deep learning model, and generate the source code and style files of the control based on the output of the deep learning model to complete the automatic generation of the control.

2. The method for automatically generating controls according to claim 1, characterized in that: The collecting and parsing of user requirements for controls, and converting the user requirements for controls into a semantic vector understandable by a computer through natural language processing technology, includes: Build an interface for users to input their requirements for custom controls; Users input their needs into the constructed interface in the form of natural language; Adopt NLP word embedding technology to convert the natural language requirements input by users into semantic vectors that can be understood by computers; Among them, the user's requirements for custom controls include control style, control function, control input properties and output properties.

3. The method for automatically generating controls according to claim 2, characterized in that: The NLP word embedding technology is used to convert the natural language requirements input by the user into semantic vectors that can be understood by the computer, including: Filter keywords in the natural language text input by the user, and take each filtered keyword as a requirement; For each requirement, create a vector space containing several requirement elements; Convert the requirement element corresponding to each requirement to one or more Boolean values.

4. The method for automatically generating controls according to claim 3, characterized in that: The collected demand data is cleaned, annotated and formatted, including: Process and clean user input to remove redundant information; Label the corresponding functions and attributes for the requirements entered by the user; Expand the labeled data; Collect multiple requirement texts and corresponding annotations with associated controls, and build training sets and validation sets.

5. The method for automatically generating controls according to claim 4, characterized in that: The expansion of the annotated data includes: synonym replacement, sentence conversion, adding specific examples, combination generation, adding negative sentences and context changes.

6. The method for automatically generating controls according to claim 5, characterized in that: The user demand data is input into the deep learning model, and the source code and style file of the control are generated according to the output of the deep learning model, so as to complete the automatic generation of the control, including: Convert user requirements into vector inputs; Select predefined control templates according to your needs; The model output contains the required code snippet of the control's structure, properties, and style; Generate the complete code using the decoder in the neural network; Use static code analysis tools to perform syntax checks and use legal code as source code; Output the legal code to the corresponding library file and integrate it into the corresponding development platform.

7. A control automatic generation system, characterized in that: Used to execute the control automatic generation method according to any one of claims 1 to 6, comprising: a collection module, a data preprocessing module, a deep learning model, a code generation module and a style generation module; The collection module is used for user input control requirements; the data preprocessing module is used to clean, annotate and format the control requirements input by the user; the deep learning model is used to extract and convert features; the code and style generation module is used to generate source code and style files for the control.

8. The automatic control generation system according to claim 7, characterized in that: It also includes an evaluation feedback module; the evaluation feedback module is used to perform automatic testing and user evaluation on the generated code and style.

9. A computing device, characterized in that include: A processor and a memory storing a computer program, wherein when the computer program is executed by the processor, the control automatic generation method according to any one of claims 1 to 6 is executed.

10. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores computer instructions, and when the instructions are executed on a computer, the computer executes the control automatic generation method according to any one of claims 1 to 6.

Images