Method, device, equipment and storage medium for automatically generating design interface

Abstract

The automatic generation method, device, equipment and storage medium of the design interface provided by the present invention obtains a real-scene image of a target smart device; then, the real-scene image is preprocessed; and then the preprocessed real-scene image and user requirements are input into a pre-trained UI generation model to obtain a button UI design interface including the target smart device. The present invention improves the efficiency and quality of interface design, reduces human omissions and errors, and realizes the generation of design interfaces more quickly, intelligently and efficiently in the design interface process through image scene recognition technology and a method for automatically generating component interfaces.

Description

Technical Field

[0001] The present invention relates to the field of UI design technology, and in particular to a method, device, equipment and storage medium for automatically generating a design interface. Background Art

[0002] In the process of designing the interface, programmers need to manually write front-end code to layout and draw components according to the requirements and described scenarios. This process involves technologies such as HTML, CSS, and JavaScript, including but not limited to selecting the appropriate layout mode, setting the position and size of elements, defining styles and interactive behaviors, etc. However, this manual code writing method has a series of shortcomings:

[0003] First, this process is very time-consuming. Programmers need to carefully think about and calculate the position and size of each component, as well as various layout-related properties. Second, the original process has high technical requirements for programmers. They need to be familiar with front-end development-related technologies and tools, and have good coding skills and experience to write reliable and efficient front-end code.

[0004] In summary, the problems existing in the prior art need to be solved urgently. Summary of the invention

[0005] The present invention provides a method, device, equipment and storage medium for automatically generating a design interface, so as to solve the defects in the prior art and realize the automatic generation of the design interface.

[0006] The present invention provides a method, device, equipment and storage medium for automatically generating a design interface, including:

[0007] Acquire the real-life image of the target smart device;

[0008] Performing image preprocessing on the real scene image;

[0009] Input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface;

[0010] Wherein, the UI design interface includes buttons of the target smart device.

[0011] According to a method for automatically generating a design interface provided by the present invention, the step of performing image preprocessing on the real scene image specifically includes:

[0012] Denoising the real scene image;

[0013] Perform format conversion on the denoised real scene image;

[0014] Enhance the contrast and brightness of the converted real-life images.

[0015] According to a method for automatically generating a design interface provided by the present invention, the training steps of the UI generation model are as follows:

[0016] Acquire a training data set, wherein the training data set includes user demand data and a synthetic image generated by a generative adversarial network;

[0017] normalizing the user demand data and the synthesized image;

[0018] The standardized synthetic images are divided into training set images and test set images;

[0019] Constructing the loss function and prediction task of the UI generation model;

[0020] Training the UI generation model according to the training set images, the loss function and the prediction task;

[0021] Evaluate whether the trained UI generation model meets the requirements based on the test set images. If not, continue training; if yes, obtain a trained UI generation model.

[0022] According to a method for automatically generating a design interface provided by the present invention, after the step of training the UI generation model according to the training set image, the loss function and the prediction task, the method specifically includes:

[0023] When the predetermined number of training times is not reached or the loss function does not converge, the value of the loss function is calculated according to the output of the UI generation model;

[0024] Calculating the gradient of the parameters of the UI generation model according to the loss function;

[0025] The optimizer updates the parameters of the UI generation model according to the gradient.

[0026] According to a method for automatically generating a design interface provided by the present invention, the step of evaluating whether the trained UI generation model meets the requirements based on the test set images specifically includes:

[0027] Calculate true positive examples, true negative examples, false negative examples, and false positive examples based on the test set images and the output of the UI generation model;

[0028] Calculate the accuracy, precision and recall of the UI generation model according to the true positive examples, the true negative examples, the false negative examples and the false positive examples;

[0029] When the accuracy, the precision and the recall rate all reach corresponding preset thresholds, the UI generation model meets the requirements.

[0030] According to a method for automatically generating a design interface provided by the present invention, after the step of obtaining the UI design interface, the step further includes:

[0031] Generate a connection identifier for a button of the target smart device;

[0032] The connection identifier is used to generate a corresponding control command according to the connection identifier when a button trigger signal of the target smart device is received, and send the command to the central control server so that the central control server can control the target smart device.

[0033] According to a method for automatically generating a design interface provided by the present invention, the UI generation model is a convolutional neural network.

[0034] The present invention also provides an automatic generation device for a design interface, comprising:

[0035] An image acquisition module, used to acquire a real-scene image of a target smart device;

[0036] An image processing module, used for performing image preprocessing on the real scene image;

[0037] The interface design module is used to input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface;

[0038] Wherein, the UI design interface includes buttons of the target smart device.

[0039] The present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the method for automatically generating a design interface as described above is implemented.

[0040] The present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the method for automatically generating a design interface as described in any one of the above is implemented.

[0041] The present invention also provides a computer program product, comprising a computer program, wherein when the computer program is executed by a processor, the method for automatically generating a design interface as described above is implemented.

[0042] The automatic generation method, device, equipment and storage medium of the design interface provided by the present invention obtains a real-scene image of a target smart device; then, the real-scene image is preprocessed; and then the preprocessed real-scene image and user requirements are input into a pre-trained UI generation model to obtain a button UI design interface including the target smart device. The present invention improves the efficiency and quality of interface design, reduces human omissions and errors, and realizes the generation of design interfaces more quickly, intelligently and efficiently in the design interface process through image scene recognition technology and a method for automatically generating component interfaces. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] In order to more clearly illustrate the technical solutions in the present invention or the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0044] Figure 1 It is a flow chart of the method for automatically generating a design interface provided by the present invention;

[0045] Figure 2 It is a structural schematic diagram of the automatic generation device of the design interface provided by the present invention;

[0046] Figure 3 It is a structural schematic diagram of the electronic device provided by the present invention. DETAILED DESCRIPTION

[0047] In order to make the purpose, technical solution and advantages of the present invention clearer, the technical solution of the present invention will be clearly and completely described below in conjunction with the drawings of the present invention. Obviously, the described embodiments are part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0048] In the interface design process, programmers need to manually write front-end code to layout and draw components according to requirements and description scenarios. This process has high technical requirements for programmers. For some non-professional designers or beginners, this technical requirement may exceed their capabilities, resulting in interface design effects that are not as expected or requiring additional technical support. In addition, manual code writing not only wastes valuable time, but is also prone to tedious errors and debugging processes. Especially when the interface becomes complex or requires multiple modifications, the time cost increases further.

[0049] In order to solve the problems in the prior art, the present invention proposes an automatic generation method of a design interface to achieve automatic generation of a design interface. The automatic generation method of the design interface is described below. Figure 1 As shown, including but not limited to the following steps:

[0050] Step 110: Acquire a real-scene image of the target smart device.

[0051] In step 110, it is necessary to obtain a real-scene image of the target smart device. It is understandable that the UI generation model in the present invention is mainly for images containing devices that need to be intelligently controlled, including lights, curtains, air conditioners, cameras, industrial equipment, etc. In this embodiment, there is no restriction on the method of obtaining the real-scene image. You can use a camera, a smart phone, or a surveillance camera to shoot the real-scene image, or you can access the real-time video stream provided by a network camera or a monitoring system, and then capture the required image, and you can also download the required real-scene image in the image library.

[0052] Step 120: perform image preprocessing on the real scene image.

[0053] In step 120, the real scene image needs to be preprocessed to standardize the image. There are many ways to preprocess the real scene image: first, use filtering technology to remove noise, then adjust the contrast and brightness of the image to make it clearer; uniformly adjust the image size, crop the area of ​​interest, and rotate or flip the image as needed; convert the image to grayscale or color, perform histogram equalization to enhance the contrast, and use the edge detection algorithm to extract edge information; binarize the image to facilitate target segmentation, fill or remove blank areas to reduce interference, and finally perform color space conversion according to task requirements. These preprocessing steps help improve image quality and provide a more reliable data basis for subsequent UI generation.

[0054] Step 130: input the preprocessed real-scene image and user requirements into a pre-trained UI generation model to obtain a UI design interface; wherein the UI design interface includes buttons of the target smart device.

[0055] In step 120, the preprocessing of the real-scene image has been completed and has been integrated into the input data format required by the model. The preprocessed real-scene image and user requirements are input into the selected UI generation model, and the generation process is started. Based on the input real-scene image and user requirements, the model will generate a UI design interface containing the target smart device button, which includes the layout, style, size and other aspects of the device button. As needed, the generated UI design interface is optimized and adjusted to ensure that it meets user needs and has a good user experience. The generated UI design interface is evaluated and iterated based on feedback to improve the design quality and meet user expectations. The finalized and confirmed UI design interface can be used for practical applications, such as developing applications or websites, and integrating target smart device buttons to achieve user control needs.

[0056] The automatic generation method of the design interface provided by the present invention obtains a real-scene image of a target smart device; then, the real-scene image is preprocessed; and then, the preprocessed real-scene image and user needs are input into a pre-trained UI generation model to obtain a button UI design interface including the target smart device.

[0057] The present invention improves the efficiency and quality of interface design, reduces human omissions and errors, and realizes the generation of design interfaces more quickly, intelligently and efficiently in the design interface process through image scene recognition technology and automatic generation of component interfaces. The method proposed by the present invention does not require programmers to write code from 0 to 1. The AI-trained model will give a design interface that almost meets the requirements. Programmers can directly apply it or fine-tune it according to the suggestions of designers, which greatly reduces the time cost. Designers do not need to draw design drawings according to user requirements and then write code for programmers. Programmers do not need to write from 0.

[0058] It is understandable that users can pass any picture to the UI generation model. Users can pass in pictures without smart devices, or pictures with people, animals and other elements. The UI generation model will automatically identify the picture and generate it. If it is an invalid picture (the UI generation model determines that there is no smart device), no interface will be generated and the invalid picture will be directly filtered out.

[0059] As a further optional embodiment, the step of performing image preprocessing on the real scene image specifically includes:

[0060] Denoising the real scene image;

[0061] Perform format conversion on the denoised real scene image;

[0062] Enhance the contrast and brightness of the converted real-life images.

[0063] In this embodiment, it is necessary to perform image preprocessing on the real scene image. Specifically, a filtering technique (such as Gaussian filtering or median filtering) is used to denoise the real scene image. This will help to eliminate noise in the image and improve image quality. After denoising, the real scene image is converted into a format acceptable to the model. For example, if the model requires the input to be a specific type of image (such as JPEG, PNG, etc.), the denoised image needs to be converted to that format. The contrast and brightness of the real scene image after format conversion are enhanced. This can be achieved by histogram equalization or adjusting the exposure and contrast of the image. Enhancing the contrast and brightness of the image can make the image clearer and improve the effect of subsequent processing steps.

[0064] As a further optional embodiment, the training steps of the UI generation model are as follows:

[0065] Acquire a training data set, wherein the training data set includes user demand data and a synthetic image generated by a generative adversarial network;

[0066] normalizing the user demand data and the synthesized image;

[0067] The standardized synthetic images are divided into training set images and test set images;

[0068] Constructing the loss function and prediction task of the UI generation model;

[0069] Training the UI generation model according to the training set images, the loss function and the prediction task;

[0070] Evaluate whether the trained UI generation model meets the requirements based on the test set images. If not, continue training; if yes, obtain a trained UI generation model.

[0071] In this embodiment, it is necessary to collect user demand data and real-life images, which can be existing images or synthetic images generated by a generative adversarial network (GAN). Subsequently, these user demand data and synthetic images are standardized to ensure that they have similar scales and distributions. Next, the standardized synthetic images are divided into training sets and test sets to evaluate the performance of the model during the training process. In order to guide the training process of the model, it is necessary to design a loss function suitable for the UI generation task, such as a pixel-level loss function or an adversarial loss function. Define the prediction task that the generation model needs to perform, that is, generate an image that matches the user's needs. The UI generation model is trained using the training set images, loss functions, and prediction tasks. Use the test set images to evaluate whether the trained UI generation model meets the requirements. This can be evaluated by calculating the similarity between the images generated by the model and the user's needs or by user surveys. If the model does not meet the requirements, the model architecture, loss function, or training parameters can be adjusted and training continues.

[0072] Specifically, data normalization usually requires vectorizing the input data for model processing. This means converting the user demand data and the synthetic image into vector representations in numerical form. The user demand data may contain text descriptions, labels, or other forms of information. First, convert the text description into a numerical representation. You can use techniques such as bag of words, TF-IDF, Word2Vec, BERT, etc. to convert the text into a dense vector. If the user demand contains labels or classification information, it can be encoded as a one-hot encoding or represented using an embedded vector. The synthetic image is usually represented as a pixel matrix, which needs to be converted into a numerical vector. This can be achieved by flattening the pixel values ​​into a one-dimensional array or using a convolutional neural network (CNN) to extract the feature vector of the image. The extracted feature vector can be the activation value of a layer in the CNN model or the vector representation output by the fully connected layer. The vector representation of the user demand data and the vector representation of the synthetic image are combined into an input vector. In this way, each training sample is represented by a complete input vector, which contains the user demand and the corresponding synthetic image information. Next, the input data is standardized or normalized to ensure that the scale and distribution of the data are similar, which is conducive to model training and convergence. Finally, the vectorized input data is used to train the UI generation model to generate a UI design interface that meets user needs.

[0073] In some optional embodiments, the prediction task includes the following:

[0074] 1. Image to code translation: Given an image, the model is required to generate the code associated with the image. This task can help the model learn the correspondence between the image and the code and generate a code representation suitable for the input image.

[0075] 2. Image description generation: Given an image, the model is required to generate a descriptive text instead of directly generating code. Then, natural language processing techniques can be used to convert the generated text into code representation. This task can help the model learn the conversion from image to natural language description, and then convert the natural language description into code.

[0076] 3. Image Part Annotation: Given an image and a region or object of interest, the model is required to generate a code associated with the region or object. This task can help the model learn to associate specific regions in the image with code representations, enabling it to generate codes associated with a given region or object.

[0077] 4. Image-conditional code generation: Given an image and a code snippet, the model is required to generate complete code that can implement the function shown in the image. This task can help the model learn to generate code that meets the expected function based on the image content and the given code snippet.

[0078] 5. Code completion: Given a partial code and a corresponding image, the model is required to generate complete code that can complete the given code snippet. This task can help the model learn to infer the missing code from the code snippet and generate complete code that is legal and has the expected function.

[0079] As an optional embodiment, the generator network can be divided into multiple branches, each branch is responsible for generating image details of different scales or different levels. The lower resolution branches usually focus on the overall structure and large features, while the higher resolution branches focus on more subtle details and textures. The existing image generation step does not have multiple branches. Through this change, it is possible to generate images at different resolutions, and then gradually enlarge and merge these images to obtain a more detailed and realistic synthetic real-life image. At the same time, this embodiment can also input different styles and scenes as conditional information into the generator, allowing the generator to generate corresponding types of real-life images according to different conditional information. The discriminator needs to determine whether the generated image conforms to the given style and scene. Through this mechanism, a synthetic real-life image with a specific style and scene can be generated.

[0080] As a further optional embodiment, after the step of training the UI generation model according to the training set images, the loss function and the prediction task, the method further includes:

[0081] When the predetermined number of training times is not reached or the loss function does not converge, the value of the loss function is calculated according to the output of the UI generation model;

[0082] Calculating the gradient of the parameters of the UI generation model according to the loss function;

[0083] The optimizer updates the parameters of the UI generation model according to the gradient.

[0084] In this embodiment, during the training process, each round of iteration shall perform the following steps: a. When the model has not reached the predetermined number of training times or the loss function has not converged, continue training. b. Use the training set image as input to generate the predicted output of the UI design interface through the UI generation model. c. Calculate the value of the loss function based on the generated UI design interface and the real target UI design interface. The loss function measures the difference between the generated UI design interface and the real target. d. Calculate the gradient of the UI generation model parameters based on the value of the loss function. e. Use an optimizer (such as stochastic gradient descent, Adam, etc.) to update the parameters of the UI generation model according to the gradient to minimize the loss function.

[0085] When the number of training times reaches a predetermined value or the loss function converges, training stops. Convergence of the loss function usually means that the change in the loss function is small enough and no longer significantly affects the performance of the model. Depending on the actual training situation and model performance, it may be necessary to adjust the definition of the loss function, the parameters of the optimizer, or the architecture of the model to further improve the performance and convergence speed of the model.

[0086] As a further optional embodiment, the step of evaluating whether the trained UI generation model meets the requirements according to the test set images specifically includes:

[0087] Calculate true positive examples, true negative examples, false negative examples, and false positive examples based on the test set images and the output of the UI generation model;

[0088] Calculate the accuracy, precision and recall of the UI generation model according to the true positive examples, the true negative examples, the false negative examples and the false positive examples;

[0089] When the accuracy, the precision and the recall rate all reach corresponding preset thresholds, the UI generation model meets the requirements.

[0090] Specifically, we need to calculate true positives, true negatives, false negatives, and false positives:

[0091] True Positives (TP): The number of samples where the model correctly predicts that the real-life image can correctly generate the code.

[0092] True Negatives (TN): The number of samples where the model correctly predicts that the real-life image cannot correctly generate the code.

[0093] False Negatives (FN): The number of samples where the model incorrectly predicts that the real-life image cannot correctly generate the code.

[0094] False Positives (FP): The number of samples where the model incorrectly predicts that the real-life image can correctly generate the code.

[0095] Then, you can calculate indicators such as accuracy, precision, and recall by following the steps below:

[0096] Accuracy: It indicates the proportion of correct predictions made by the model in all test samples. The calculation formula is (TP+TN) / (TP+TN+FP+FN).

[0097] Precision: It indicates the proportion of samples that the model can actually generate correct codes for among the samples that it predicts can correctly generate codes for real-life images. The calculation formula is TP / (TP+FP).

[0098] Recall rate: It indicates the proportion of samples that the model can correctly predict to generate the correct code from the real scene pictures. The calculation formula is TP / (TP+FN).

[0099] Next, the accuracy, precision, and recall are evaluated according to the preset thresholds. Usually, the preset thresholds are set according to specific tasks and requirements. If the accuracy, precision, and recall all reach the preset thresholds, the UI generation model is considered to meet the requirements and can be further applied in actual scenarios.

[0100] As a further optional embodiment, after the step of obtaining the UI design interface, the step further includes:

[0101] Generate a connection identifier for a button of the target smart device;

[0102] The connection identifier is used to generate a corresponding control command according to the connection identifier when a button trigger signal of the target smart device is received, and send the command to the central control server so that the central control server can control the target smart device.

[0103] In this embodiment, the generated UI design interface will include a button for the target smart device, and the corresponding connection identifier will be generated when the UI component is generated. This can be a unique identifier of the device, a combination of device type and location, or any other identifier that can accurately identify the device and associate it with it. Associate the connection identifier with the trigger behavior of each button. When the user clicks or touches the button, the connection identifier will be used to identify the specific smart device that the user wants to control. Then, according to the button trigger behavior and the connection identifier, the corresponding control command is generated. The user's intention is translated into appropriate control instructions, such as turning on, off, adjusting brightness, etc. The generated control command is sent to the central control server. The central control server is a system that centrally manages smart devices. It is responsible for receiving control commands from the UI design interface and passing them to the corresponding smart devices. After receiving the control command, the central control server will parse the connection identifier and identify the target smart device. Then, it sends the corresponding control command to the target smart device to perform the operation required by the user, such as turning on and off the light, adjusting the temperature, etc.

[0104] As a further optional embodiment, the UI generation model is a convolutional neural network.

[0105] It is understandable that the selection of model architecture is related to task requirements and datasets. Different model architectures may perform better on different tasks and datasets. Therefore, in practice, it is necessary to compare and evaluate multiple model architectures to determine the best model architecture. In addition to convolutional neural networks, there are other models that are suitable, such as the pre-trained model ResNet.

[0106] The automatic generation device of the design interface provided by the present invention is described below. Figure 2 As shown, the automatic generation device of the design interface described below and the automatic generation method of the design interface described above can refer to each other.

[0107] An automatic generation device for a design interface, comprising:

[0108] An image acquisition module 210 is used to acquire a real-scene image of a target smart device;

[0109] An image processing module 220, used for performing image preprocessing on the real scene image;

[0110] The interface design module 230 is used to input the pre-processed real scene image and user requirements into the pre-trained UI generation model to obtain a UI design interface;

[0111] Wherein, the UI design interface includes buttons of the target smart device.

[0112] Figure 3An example of a physical structure diagram of an electronic device is shown in FIG. Figure 3 As shown, the electronic device may include: a processor 310, a communication interface 320, a memory 330 and a communication bus 340, wherein the processor 310, the communication interface 320 and the memory 330 communicate with each other through the communication bus 340. The processor 310 may call the logic instructions in the memory 330 to execute the automatic generation method of the design interface, and the method includes:

[0113] Acquire the real-life image of the target smart device;

[0114] Performing image preprocessing on the real scene image;

[0115] Input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface;

[0116] Wherein, the UI design interface includes buttons of the target smart device.

[0117] In addition, the logic instructions in the above-mentioned memory 330 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art or the part of the technical solution, can be embodied in the form of a software product, which is stored in a storage medium and includes several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as a U disk, a mobile hard disk, a read-only memory (ROM, Read-On l yMemory), a random access memory (RAM, Random Access Memory), a disk or an optical disk.

[0118] On the other hand, the present invention further provides a computer program product, the computer program product comprising a computer program, the computer program can be stored in a non-transitory computer-readable storage medium, when the computer program is executed by a processor, the computer can execute the automatic generation method of the design interface provided by the above methods, the method comprising:

[0119] Acquire the real-life image of the target smart device;

[0120] Performing image preprocessing on the real scene image;

[0121] Input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface;

[0122] Wherein, the UI design interface includes buttons of the target smart device.

[0123] In another aspect, the present invention further provides a non-transitory computer-readable storage medium having a computer program stored thereon, which is implemented when the computer program is executed by a processor to perform the automatic generation method of the design interface provided by the above methods, the method comprising:

[0124] Acquire the real-life image of the target smart device;

[0125] Performing image preprocessing on the real scene image;

[0126] Input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface;

[0127] Wherein, the UI design interface includes buttons of the target smart device.

[0128] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0129] Through the description of the above implementation methods, those skilled in the art can clearly understand that each implementation method can be implemented by means of software plus a necessary general hardware platform, and of course, it can also be implemented by hardware. Based on this understanding, the above technical solution is essentially or the part that contributes to the prior art can be embodied in the form of a software product, and the computer software product can be stored in a computer-readable storage medium, such as ROM / RAM, a disk, an optical disk, etc., including a number of instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to execute the methods described in each embodiment or some parts of the embodiments.

[0130] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit it. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A method for automatically generating a design interface, characterized in that: include: Acquire the real-life image of the target smart device; Performing image preprocessing on the real scene image; Input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface; The UI design interface includes a button of the target smart device, and the button is used to trigger the generation of a corresponding control command and send it to the central control server, so that the central control server can control the target smart device; The training steps of the UI generation model are as follows: Acquire a training data set, wherein the training data set includes user demand data and a synthetic image generated by a generative adversarial network; normalizing the user demand data and the synthesized image; Specifically, the user demand data and the synthetic image are converted into vector representations in numerical form, where the user demand data includes text descriptions, labels or classification information; the vector representations of the user demand data and the vector representations of the synthetic image are combined into an input vector, where each training sample is represented by a complete input vector, which includes user demand and corresponding synthetic image information; the input data is standardized or normalized, and the vectorized input data is used to train the UI generation model; The standardized synthetic images are divided into training set images and test set images; Constructing the loss function and prediction task of the UI generation model; Training the UI generation model according to the training set images, the loss function and the prediction task; Evaluate whether the trained UI generation model meets the requirements based on the test set images. If not, continue training. If yes, obtain a trained UI generation model. The prediction task is to generate an image that matches user needs.

2. The method for automatically generating a design interface according to claim 1, characterized in that: The step of performing image preprocessing on the real scene image specifically includes: Denoising the real scene image; Perform format conversion on the denoised real scene image; Enhance the contrast and brightness of the converted real-life images.

3. The method for automatically generating a design interface according to claim 2, characterized in that: After the step of training the UI generation model according to the training set images, the loss function and the prediction task, the method further includes: When the predetermined number of training times is not reached or the loss function does not converge, the value of the loss function is calculated according to the output of the UI generation model; Calculating the gradient of the parameters of the UI generation model according to the loss function; The optimizer updates the parameters of the UI generation model according to the gradient.

4. The method for automatically generating a design interface according to claim 3, characterized in that: The step of evaluating whether the trained UI generation model meets the requirements based on the test set images specifically includes: Calculate true positive examples, true negative examples, false negative examples, and false positive examples based on the test set images and the output of the UI generation model; Calculate the accuracy, precision and recall of the UI generation model according to the true positive examples, the true negative examples, the false negative examples and the false positive examples; When the accuracy, the precision and the recall rate all reach corresponding preset thresholds, the UI generation model meets the requirements.

5. The method for automatically generating a design interface according to claim 1, characterized in that: After the step of obtaining the UI design interface, the steps further include: Generate a connection identifier for a button of the target smart device; The connection identifier is used to generate a corresponding control command according to the connection identifier when a button trigger signal of the target smart device is received, and send the command to the central control server so that the central control server can control the target smart device.

6. The method for automatically generating a design interface according to claim 1, characterized in that: The UI generation model is a convolutional neural network.

7. An automatic generation device for a design interface, characterized in that: include: An image acquisition module, used to acquire a real-scene image of a target smart device; An image processing module, used for performing image preprocessing on the real scene image; The interface design module is used to input the pre-processed real-scene images and user requirements into the pre-trained UI generation model to obtain the UI design interface; The UI design interface includes a button of the target smart device, and the button is used to trigger the generation of a corresponding control command and send it to the central control server, so that the central control server can control the target smart device; The training steps of the UI generation model are as follows: Acquire a training data set, wherein the training data set includes user demand data and a synthetic image generated by a generative adversarial network; normalizing the user demand data and the synthesized image; Specifically, the user demand data and the synthetic image are converted into vector representations in numerical form, where the user demand data includes text descriptions, labels or classification information; the vector representations of the user demand data and the vector representations of the synthetic image are combined into an input vector, where each training sample is represented by a complete input vector, which includes user demand and corresponding synthetic image information; the input data is standardized or normalized, and the vectorized input data is used to train the UI generation model; The standardized synthetic images are divided into training set images and test set images; Constructing the loss function and prediction task of the UI generation model; Training the UI generation model according to the training set images, the loss function and the prediction task; Evaluate whether the trained UI generation model meets the requirements based on the test set images. If not, continue training. If yes, obtain a trained UI generation model. The prediction task is to generate an image that matches user needs.

8. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the method for automatically generating a design interface as described in any one of claims 1 to 6 is implemented.

9. A non-transitory computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the method for automatically generating a design interface as claimed in any one of claims 1 to 6 is implemented.

Images