Interface code generation method, device, equipment and storage medium

Abstract

The present application provides an interface code generation method, device, equipment and storage medium. The method quickly generates the interface code of a GUI image based on a GUI image, saves a lot of repetitive work for front-end developers, reduces the complexity of GUI development, and improves development efficiency. Moreover, in the process of generating the interface code of the GUI image, the embodiment of the present application uses a first preset model to identify the GUI image, wherein the first preset model extracts features of the extracted features again after extracting features of the GUI image, so that the feature information of the extracted GUI image is richer, and the image recognition rate is improved. Therefore, the subsequent embodiment of the present application generates the interface code of the above-mentioned GUI image to be processed based on the features of the GUI image extracted by the first preset model, and the accuracy of the interface code of the generated GUI image is improved.

Description

Technical Field

[0001] The present application relates to computer technology of financial technology (Fintech), and in particular to an interface code generation method, device, equipment and storage medium. Background Art

[0002] With the development of computer technology, more and more technologies are applied in the financial field. The traditional financial industry is gradually transforming into financial technology. The code generation technology of graphical user interface (GUI) is no exception. However, due to the security and real-time requirements of the financial industry, higher requirements are also placed on the code generation technology of GUI.

[0003] In the related art, in the process of engineering project development in the financial industry, the development of the GUI part usually requires the participation of system analysts, artists or GUI designers, and front-end developers. First, the system analyst analyzes the requirements and generates design documents; the artist or GUI designer creates low-fidelity or high-fidelity GUI diagrams based on the design documents. Finally, the front-end developer develops the corresponding code based on the designed GUI diagram.

[0004] As the engineering and projects progress, the products will usually undergo continuous version updates and iterations. Even in the development of the same version, there will be multiple fine-tuning changes. Therefore, front-end developers need to constantly develop corresponding codes based on different GUI diagrams, which makes the development process complicated and inefficient. Summary of the invention

[0005] In order to solve the problems existing in the prior art, the present application provides an interface code generation method, device, equipment and storage medium to solve the problem of how to quickly obtain the interface code of a GUI diagram during the GUI development process to reduce the complexity of GUI development and improve development efficiency.

[0006] In a first aspect, an embodiment of the present application provides a method for generating an interface code, the method comprising:

[0007] Acquire a GUI image to be processed, and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain a first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed;

[0008] Acquire a correspondence between a pre-stored image feature of a GUI image and a code feature of a domain specific language (DSL) code, and determine a code feature of the DSL code corresponding to the second image feature according to the correspondence;

[0009] According to the code feature of the DSL code corresponding to the second image feature, the interface code of the GUI image to be processed is obtained.

[0010] In a possible implementation, the first preset model further includes a noise reduction layer, which removes noise points in the first image features, and the feature extraction layer extracts features from the first image features after the noise points are removed to obtain the second image features.

[0011] In a possible implementation, the feature extraction layer is a hash layer, and the hash layer calculates a hash value of the first image feature, and performs feature extraction on the first image feature according to the hash value to obtain the second image feature.

[0012] In a possible implementation manner, before determining, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature, the method further includes:

[0013] Determining a loss value of the feature extraction layer according to the first image feature and the second image feature;

[0014] According to the loss value of the feature extraction layer, the feature extraction layer is optimized so that the loss value of the optimized feature extraction layer is equal to a preset minimum loss value, and the optimized feature extraction layer performs feature extraction on the first image feature to obtain the second image feature.

[0015] In a possible implementation, the obtaining of the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code includes:

[0016] Inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is trained by reference image features, reference code features, and the corresponding relationship between the reference image features and the reference code features;

[0017] The corresponding relationship is obtained according to the output of the second preset model.

[0018] In a possible implementation manner, before inputting the image features of the preset GUI image and the code features of the preset DSL code into the second preset model, the method further includes:

[0019] inputting the preset DSL code into a third preset model, wherein the third preset model is obtained by training a reference DSL code and code features of the reference DSL code;

[0020] According to the output of the third preset model, a code feature of the preset DSL code is obtained.

[0021] In a possible implementation, obtaining the interface code of the to-be-processed GUI image according to the code feature of the DSL code corresponding to the second image feature includes:

[0022] determining the DSL code corresponding to the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature;

[0023] According to the DSL code corresponding to the GUI image to be processed, the interface code of the GUI image to be processed is obtained.

[0024] In a possible implementation, determining the DSL code corresponding to the to-be-processed GUI image according to the code feature of the DSL code corresponding to the second image feature includes:

[0025] inputting the code feature of the DSL code corresponding to the second image feature into a fourth preset model, wherein the fourth preset model is obtained by training the code feature of the reference DSL code and the reference DSL code;

[0026] According to the output of the fourth preset model, the DSL code corresponding to the GUI diagram to be processed is obtained.

[0027] In a possible implementation, obtaining the GUI image to be processed includes:

[0028] Performing image preprocessing on at least one GUI image of the target GUI, wherein the image preprocessing includes one or more of adjusting image size and image normalization;

[0029] According to the image preprocessing result, the GUI image to be processed is obtained.

[0030] In a possible implementation manner, before performing image preprocessing on at least one GUI image of the target GUI, the method further includes:

[0031] Receive the at least one GUI image sent by a preset image acquisition device.

[0032] In a possible implementation manner, before performing image preprocessing on at least one GUI image of the target GUI, the method further includes:

[0033] Receive the at least one GUI image sent by the device where the target GUI is located.

[0034] In a second aspect, an embodiment of the present application provides an interface code generation device, the device comprising:

[0035] An input module, used for acquiring a GUI image to be processed, and inputting the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain a first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed;

[0036] An acquisition module, used for acquiring the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code;

[0037] a determination module, configured to determine, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature;

[0038] The obtaining module is used to obtain the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature.

[0039] In a possible implementation, the first preset model further includes a noise reduction layer, which removes noise points in the first image features, and the feature extraction layer extracts features from the first image features after the noise points are removed to obtain the second image features.

[0040] In a possible implementation, the feature extraction layer is a hash layer, and the hash layer calculates a hash value of the first image feature, and performs feature extraction on the first image feature according to the hash value to obtain the second image feature.

[0041] In a possible implementation manner, the input module is further configured to determine a loss value of the feature extraction layer according to the first image feature and the second image feature before the determination module determines, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature;

[0042] According to the loss value of the feature extraction layer, the feature extraction layer is optimized so that the loss value of the optimized feature extraction layer is equal to a preset minimum loss value, and the optimized feature extraction layer performs feature extraction on the first image feature to obtain the second image feature.

[0043] In a possible implementation, the acquisition module is specifically used to:

[0044] Inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is trained by reference image features, reference code features, and the corresponding relationship between the reference image features and the reference code features;

[0045] The corresponding relationship is obtained according to the output of the second preset model.

[0046] In a possible implementation, the acquisition module is further used to:

[0047] inputting the preset DSL code into a third preset model, wherein the third preset model is obtained by training a reference DSL code and code features of the reference DSL code;

[0048] According to the output of the third preset model, a code feature of the preset DSL code is obtained.

[0049] In a possible implementation, the obtaining module is specifically used to:

[0050] determining the DSL code corresponding to the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature;

[0051] According to the DSL code corresponding to the GUI image to be processed, the interface code of the GUI image to be processed is obtained.

[0052] In a possible implementation, the obtaining module is specifically used to:

[0053] inputting the code feature of the DSL code corresponding to the second image feature into a fourth preset model, wherein the fourth preset model is obtained by training the code feature of the reference DSL code and the reference DSL code;

[0054] According to the output of the fourth preset model, the DSL code corresponding to the GUI diagram to be processed is obtained.

[0055] In a possible implementation, the input module is further used to:

[0056] Performing image preprocessing on at least one GUI image of the target GUI, wherein the image preprocessing includes one or more of adjusting image size and image normalization;

[0057] According to the image preprocessing result, the GUI image to be processed is obtained.

[0058] In a possible implementation, the input module is further used to:

[0059] Receive the at least one GUI image sent by a preset image acquisition device.

[0060] In a possible implementation, the input module is further used to:

[0061] Receive the at least one GUI image sent by the device where the target GUI is located.

[0062] In a third aspect, an embodiment of the present application provides an interface code generating device, including:

[0063] processor;

[0064] Memory; and

[0065] Computer programs;

[0066] The computer program is stored in the memory and is configured to be executed by the processor, wherein the computer program includes instructions for executing the method as described in the first aspect.

[0067] In a fourth aspect, an embodiment of the present application provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and the computer program enables a server to execute the method described in the first aspect.

[0068] In a fifth aspect, an embodiment of the present application provides a computer program product, comprising computer instructions, wherein the computer instructions are executed by a processor according to the method described in the first aspect.

[0069] The interface code generation method, device, equipment and storage medium provided in the embodiment of the present application first input the GUI image to be processed into a first preset model, the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the above-mentioned GUI image to be processed to obtain a first image feature, the feature extraction layer performs feature extraction on the above-mentioned first image feature to obtain a second image feature, and then, the method determines the code feature of the DSL code corresponding to the above-mentioned second image feature according to the correspondence between the image feature of the pre-stored GUI image and the code feature of the DSL code, and then, according to the code feature, obtains the interface code of the above-mentioned GUI image to be processed. That is, the embodiment of the present application quickly generates the interface code of the GUI image based on a GUI image, saves a lot of repetitive work of front-end developers, reduces the complexity of GUI development, and improves development efficiency. Moreover, after the first preset model in the embodiment of the present application performs feature extraction on the GUI image, the extracted features are extracted again, so that the feature information of the extracted GUI image is richer, the information of the obtained GUI image is more, and image recognition is performed based on the obtained information, which improves the image recognition rate, thereby improving the accuracy of the interface code of the subsequently generated GUI image. BRIEF DESCRIPTION OF THE DRAWINGS

[0070] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0071] Figure 1 A schematic diagram of a model training process provided in an embodiment of the present application;

[0072] Figure 2 Based on the embodiments of the present application Figure 1 Schematic diagram of the trained model generating user interface code;

[0073] Figure 3 A schematic diagram of an interface code generation system architecture provided in an embodiment of the present application;

[0074] Figure 4 A flowchart of a method for generating interface code provided in an embodiment of the present application;

[0075] Figure 5 A schematic diagram of a first preset model provided in an embodiment of the present application;

[0076] Figure 6 A flowchart of another method for generating interface code provided in an embodiment of the present application;

[0077] Figure 7 A schematic diagram of a code generation model provided in an embodiment of the present application;

[0078] Figure 8 A schematic diagram of the structure of an interface code generation device provided in an embodiment of the present application;

[0079] Fig. 9 A possible structural diagram of the interface code generation device of the present application is shown. DETAILED DESCRIPTION

[0080] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0081] The terms "first", "second", "third", and "fourth", etc. (if any) in the specification and claims of the present application and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchangeable where appropriate, so that the embodiments of the present application described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "including" and "having" and any of their variations are intended to cover non-exclusive inclusions, for example, a process, method, system, product, or device that includes a series of steps or units is not necessarily limited to those steps or units that are clearly listed, but may include other steps or units that are not clearly listed or inherent to these processes, methods, products, or devices.

[0082] In the related art, in the process of engineering project development in the financial industry, the development of the GUI part usually requires the participation of system analysts, artists or GUI designers, and front-end developers. First, the system analyst analyzes the requirements and generates design documents; the artist or GUI designer creates low-fidelity or high-fidelity GUI diagrams based on the design documents. Finally, the front-end developer develops the corresponding code based on the designed GUI diagram.

[0083] As the engineering and projects progress, the products will usually undergo continuous version updates and iterations. Even in the development of the same version, there will be multiple fine-tuning changes. Therefore, front-end developers need to continuously develop corresponding codes based on different GUI diagrams. The development process is complicated and inefficient. Therefore, how to quickly obtain the interface code of the GUI diagram during the GUI development process to reduce the complexity of GUI development and improve development efficiency has become an urgent problem to be solved.

[0084] In order to solve the above problems, existing research provides a method for generating user interface code. For example, during the training process, Figure 1 As shown, the client or server inputs a GUI image I into the Convolutional Neural Networks (CNN) model, and the CNN model recognizes the GUI image I and encodes it into a vector representation p. The client or server inputs a DSL code into the Long Short-Term Memory (LSTM) model, and the LSTM model encodes the DSL code into an intermediate representation qt. The client or server inputs p and qt into the LSTM* model, and then inputs the output of LSTM* into the softmax function module to obtain the relationship between p and qt. In the sampling stage, as Figure 2As shown, the client or server inputs the empty code text into the LSTM model, and then inputs the GUI diagram to be processed into the CNN. Finally, the output of the CNN and the output of the LSTM are input into the LSTM*, and then the output of the LSTM* is input into the softmax function module, so that the DSL code of the above GUI diagram to be processed can be obtained.

[0085] However, the above CNN model recognizes the image by reducing the dimension of the GUI image I, that is, the CNN model reduces the dimension of the GUI image I into computer-processable information, and recognizes the image based on the computer-processable information. In the process of dimensionality reduction, a lot of information of the GUI image I is discarded by the CNN model, which makes the recognition rate of the GUI image I by the CNN model low, and further, the accuracy of the vector p obtained by the above CNN model to recognize the above GUI image I is low, which reduces the accuracy of the DSL code of the subsequently generated GUI image.

[0086] Therefore, the embodiment of the present application proposes a method for generating interface code, which quickly generates the interface code of a GUI image based on a GUI image, saves a lot of repetitive work for front-end developers, reduces the complexity of GUI development, and improves development efficiency. In addition, in the process of generating the interface code of the GUI image, the embodiment of the present application uses a first preset model to identify the GUI image, wherein the first preset model extracts features from the extracted features again after extracting features from the GUI image, so that the feature information of the extracted GUI image is richer, and compared with the existing CNN model that reduces the dimension of the GUI image to identify the image, the embodiment of the present application extracts features from the GUI image twice, obtains more information about the GUI image, and performs image recognition based on the obtained information, thereby improving the image recognition rate. Therefore, the subsequent embodiment of the present application generates the interface code of the above-mentioned GUI image to be processed based on the features of the GUI image extracted by the first preset model, thereby improving the accuracy of the interface code of the generated GUI image.

[0087] The interface code generation method provided in the embodiment of the present application can be applied to the GUI development of various devices, such as the GUI development of mobile devices, the GUI development of personal computer (PC) devices, the GUI development of gaming devices, etc. The specific method can be determined based on actual conditions, and the embodiment of the present application does not impose any special restrictions on this.

[0088] Here, taking the GUI development of a mobile terminal device as an example, the GUI diagram of the mobile terminal device can be acquired through a preset image acquisition device, such as a camera. Exemplarily, taking the preset image acquisition device as the above-mentioned camera as an example, the number, position and type of the above-mentioned camera and other information can be uniformly specified by the code generation device. Among them, the code generation device is used to generate the interface code of the GUI diagram, and manage the devices used in the process of generating the interface code of the GUI diagram, such as the above-mentioned camera. For example, the code generation device can check whether the installation of the above-mentioned camera meets the requirements according to the relevant regulations of the camera, such as checking whether the installation of the above-mentioned camera meets the requirements through the installation image of the above-mentioned camera uploaded by the mobile terminal device. If it meets the requirements, the code generation device can establish a connection with the above-mentioned camera. During the GUI development process of the mobile terminal device, the code generation device can start the above-mentioned camera to acquire the GUI diagram of the mobile terminal device.

[0089] In addition to the above, the GUI image can also be obtained by the mobile device itself, for example, the mobile device obtains the GUI image by taking a screenshot. Exemplarily, the code generation device can send an image acquisition request to the mobile device during the GUI development process of the mobile device. The mobile device takes a screenshot according to the request to obtain the GUI image.

[0090] Here, after the camera or mobile device acquires the GUI image, it can be sent to the code generation device. Furthermore, the code generation device can input the GUI image into the first preset model to quickly generate the interface code of the GUI image, saving a lot of repetitive work for front-end developers, reducing the complexity of GUI development, and improving development efficiency. Moreover, the first preset model includes a convolution layer and a feature extraction layer. The convolution layer performs feature extraction on the GUI image to obtain a first image feature. The feature extraction layer performs feature extraction on the first image feature to obtain a second image feature, so that the feature information of the extracted GUI image is richer, the image recognition rate is improved, and thus, the accuracy of the interface code of the GUI image generated based on the features of the GUI image is improved.

[0091] Optionally, Figure 3 A schematic diagram of an interface code generation system architecture provided in an embodiment of the present application. Figure 3 In the example, the GUI development of a mobile terminal device is taken as an example. The above architecture includes a code generation device 11 and a preset image acquisition device 12.

[0092] It is understandable that the structure illustrated in the embodiment of the present application does not constitute a specific limitation on the interface code generation architecture. In other feasible implementations of the present application, the above architecture may include more or fewer components than shown in the figure, or combine certain components, or split certain components, or arrange the components differently, which can be determined according to the actual application scenario and is not limited here. Figure 3 The components shown may be implemented in hardware, software, or a combination of software and hardware.

[0093] In the specific implementation process, the code generation device 11 may first determine whether to control the preset image acquisition device 12 to start. Exemplarily, the code generation device 11 may control the preset image acquisition device 12 to start and collect the GUI image of the mobile terminal device after determining that the GUI development of the mobile terminal device has started. Alternatively, the code generation device 11 may determine whether to control the preset image acquisition device 12 to start according to the needs of the front-end developer. Exemplarily, if the front-end developer requires to turn on the preset image acquisition device 12, the code generation device 11 controls the preset image acquisition device 12 to start and collect the GUI image of the mobile terminal device.

[0094] After the preset image acquisition device 12 is started and the GUI image of the mobile device is acquired, the acquired image can be sent to the code generation device 11. After receiving the above GUI image, the code generation device 11 inputs the above GUI image into the first preset model, performs feature extraction on the above GUI image, obtains the first image feature, and performs feature extraction on the above first image feature to obtain the second image feature. Thus, the code generation device 11 quickly generates the interface code of the above GUI image based on the features of the GUI image extracted by the first preset model, saving a lot of repetitive work for front-end developers and improving development efficiency. Moreover, after the first preset model performs feature extraction on the GUI image, the extracted features are extracted again, so that the feature information of the extracted GUI image is richer, the information of the obtained GUI image is more, and image recognition is performed based on the obtained information, thereby improving the image recognition rate. Thus, the code generation device 11 generates the interface code of the above GUI image to be processed based on the features of the GUI image extracted by the first preset model, thereby improving the accuracy of the interface code of the generated GUI image.

[0095] In addition, the system architecture and business scenarios described in the embodiments of the present application are intended to more clearly illustrate the technical solutions of the embodiments of the present application, and do not constitute a limitation on the technical solutions provided in the embodiments of the present application. Ordinary technicians in this field can know that with the evolution of the system architecture and the emergence of new business scenarios, the technical solutions provided in the embodiments of the present application are also applicable to similar technical problems.

[0096] The technical solution of the present application is described below by taking several embodiments as examples, and the same or similar concepts or processes may not be repeated in some embodiments.

[0097] Figure 4 The present invention provides a flow chart of an interface code generation method provided in an embodiment of the present invention. The present invention provides an interface code generation method, which can be applied to the processing of interface code generation of a GUI diagram, and the method can be executed by any device that executes the interface code generation method, which can be implemented by software and / or hardware. Figure 4 As shown, in Figure 3 Based on the system architecture shown, the interface code generation method provided in the embodiment of the present application includes the following steps:

[0098] S401: Obtain a GUI image to be processed, and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain a first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed.

[0099] The GUI diagram to be processed may be determined according to actual conditions. For example, taking the GUI development of the mobile terminal device as an example, the GUI diagram to be processed may be the GUI diagram of the mobile terminal device.

[0100] In the embodiment of the present application, taking the execution subject as the above-mentioned code generation device as an example, the above-mentioned GUI image to be processed can be the GUI image of the above-mentioned mobile terminal device sent by the above-mentioned preset image acquisition device directly received by the above-mentioned code generation device, or the GUI image of the above-mentioned mobile terminal device sent by the device (such as the above-mentioned mobile terminal device) where the received target GUI (such as the GUI of the above-mentioned mobile terminal device) is located.

[0101] Here, in order to improve the recognition rate of the subsequent first preset model recognition picture, the above-mentioned code generation device can perform image preprocessing on the received GUI image. Exemplarily, the above-mentioned preset image acquisition device can send at least one GUI image of the above-mentioned target GUI to the above-mentioned code generation device. Similarly, the device where the above-mentioned target GUI is located can also send at least one GUI image of the above-mentioned target GUI to the above-mentioned code generation device. After receiving the above-mentioned at least one GUI image, the above-mentioned code generation device can perform image preprocessing on the above-mentioned at least one GUI image, such as adjusting the image size and image normalization, etc., so as to obtain the above-mentioned GUI image to be processed according to the image preprocessing result. Exemplarily, the above-mentioned code generation device can adjust the size of the above-mentioned at least one GUI image to a preset size, and then normalize the image, etc., and finally use the processed image as the above-mentioned GUI image to be processed. Among them, the above-mentioned preset size can be determined according to the actual situation, and the embodiment of the present application does not impose any special restrictions on this.

[0102] In the embodiment of the present application, when the convolution layer extracts features from the GUI image to be processed, the feature extraction can be performed according to the weight of each dimension of the feature of the GUI image to be processed, wherein the weight of each dimension of the feature represents the importance of the feature of the dimension, that is, the weight of the feature with high importance is large, and the weight of the feature with low importance is small. Here, the importance of each dimension of the feature can be determined according to the actual situation.

[0103] Exemplarily, the convolution layer may first sort each dimension of the GUI image to be processed according to the weight of each dimension of the GUI image to be processed, for example, sorting from large weight to small weight, and then extracting features with larger weights from each dimension of the GUI image to be processed according to the sorting result, for example, extracting features with weights greater than preset weights, thereby obtaining the first image feature of the GUI image to be processed. The preset weights may be set according to actual conditions, and the present application embodiment does not impose any particular restrictions on this.

[0104] In this way, the embodiment of the present application first obtains the features with larger weights of the above-mentioned GUI image to be processed through the first preset model, that is, extracts the features with higher importance, and obtains more accurate features of the GUI image. As a result, the extracted features are subsequently subjected to feature extraction again, so that the feature information of the extracted GUI image is richer, and the information of the obtained GUI image is more. Image recognition is performed based on the obtained information, thereby improving the image recognition rate.

[0105] Here, after the first preset model extracts features from the GUI image, it extracts features from the extracted features again, so that the feature information of the extracted GUI image is richer, the image recognition rate is improved, and thus, the accuracy of the interface code of the subsequently generated GUI image is improved. In order to further improve the accuracy of the first preset model in recognizing images, the first preset model may also include a noise reduction layer, which removes noise points from the first image features and improves the clarity of the image. Then, the feature extraction layer extracts features from the first image features after the noise points are removed to obtain the second image features, further improving the accuracy of the interface code of the subsequently generated GUI image.

[0106] Exemplarily, the feature extraction layer may be a hash layer, which calculates the hash value of the first image feature, and extracts the first image feature according to the hash value to obtain the second image feature. The hash layer calculates the hash value of the first image feature, and then marks the first image feature based on the hash value, thereby extracting the first image feature according to the mark to obtain richer feature information of the GUI image, thereby improving the image recognition rate of the first preset model.

[0107] Here, the above hash layer can use the formula: i (x; w) = tanh(w T [X i (f n-2 (x); f n-1 (x))]) Calculate the hash value of the first image feature. Where i = 1, 2, ..., k, h i (x; w) represents the hash value of the i-th bit, tanh is the hyperbolic tangent function, w represents the weight of the hash layer, f n-2 (x), f n-1 (x) represent the first image features mentioned above, X i (f n-2 (x); f n-1 (x)) represents the feature representation of the i-th image segment after segmentation.

[0108] For example, a schematic diagram of a first preset model provided in an embodiment of the present application is as follows: Figure 5As shown. The first preset model includes a convolution layer, a denoising layer and a feature extraction layer, and the feature extraction layer is a hash layer. The convolution layer performs feature extraction on the GUI image to be processed to obtain the first image feature of the GUI image to be processed. The denoising layer removes the noise points in the first image feature. The hash layer calculates the hash value of the first image feature after the noise points are removed, and then marks the first image feature after the noise points are removed based on the hash value, thereby performing feature extraction on the first image feature after the noise points are removed according to the mark to obtain the second image feature of the GUI image to be processed, that is, obtaining more accurate and richer feature information of the GUI image, thereby improving the image recognition rate of the first preset model, and further improving the accuracy of the interface code of the GUI image generated based on the features of the GUI image extracted by the first preset model.

[0109] Here, in addition to the above-mentioned convolutional layer, denoising layer, and hash layer, the above-mentioned first preset model can also use a hierarchical attention mechanism. Based on the hierarchical attention mechanism, the feature information of the above-mentioned GUI image is extracted, thereby making the interface code of the GUI image subsequently generated based on the extracted features of the above-mentioned GUI image more accurate.

[0110] In addition, the first preset model can be independently set from the code generation device. Before the code generation device obtains the GUI image to be processed and inputs the GUI image to be processed into the first preset model, the first preset model can be trained first, so that the code generation device can directly use the trained first preset model to extract features from the GUI image to be processed, thereby shortening the GUI development time and improving the development efficiency.

[0111] S402: Obtain a correspondence between the image features of the pre-stored GUI image and the code features of the DSL code, and determine the code feature of the DSL code corresponding to the second image feature according to the correspondence.

[0112] Here, before determining the code feature of the DSL code corresponding to the second image feature according to the corresponding relationship, the code generating device may also determine the loss value of the feature extraction layer according to the first image feature and the second image feature, and then optimize the feature extraction layer according to the loss value of the feature extraction layer so that the loss value of the optimized feature extraction layer is equal to the preset minimum loss value. The optimized feature extraction layer extracts features from the first image feature to obtain the second image feature.

[0113] In an embodiment of the present application, the above-mentioned code generation device optimizes the above-mentioned feature extraction layer according to the input and output of the above-mentioned feature extraction layer, so that the features extracted by the optimized feature extraction layer are more accurate, thereby improving the accuracy of the interface code of the GUI image subsequently generated based on the features extracted by the optimized feature extraction layer.

[0114] Exemplarily, the above code generation device can be based on the formula: Determine the loss value L of the feature extraction layer. Where W = [w1, w2, ..., wc] is the weight of the classifier in the feature extraction layer, y i =[y1,y2,…,yn] represents the first image feature, bi represents the second image feature, s represents the continuous value encoding without thresholding, which is a constant value and can be determined through a large number of experiments. Furthermore, the code generation device optimizes the feature extraction layer according to the loss value L of the feature extraction layer, so that the loss value of the optimized feature extraction layer is equal to the preset minimum loss value. Here, the preset minimum loss value can be determined according to the actual situation, and the embodiment of the present application does not impose any special restrictions on this.

[0115] S403: Obtaining the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature.

[0116] Exemplarily, the code generating device may first determine the DSL code corresponding to the GUI image to be processed based on the code features of the DSL code corresponding to the second image features, and then obtain the interface code of the GUI image to be processed based on the DSL code corresponding to the GUI image to be processed.

[0117] The code generation device may input the code feature of the DSL code corresponding to the second image feature into a fourth preset model, where the fourth preset model is obtained by training the code feature of the reference DSL code and the reference DSL code. The fourth preset model may be specifically determined according to actual conditions, such as an LSTM model, which is not particularly limited in the embodiment of the present application. Furthermore, the code generation device obtains the DSL code corresponding to the GUI image to be processed according to the output of the fourth preset model.

[0118] Here, the interface code of the GUI image to be processed can be determined according to actual needs. The code generation device can compile the DSL code corresponding to the GUI image to be processed, and convert the DSL code corresponding to the GUI image to be processed into the interface code of the GUI image to be processed.

[0119] In an embodiment of the present application, the code generation device first inputs the GUI image to be processed into a first preset model, the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain a first image feature, the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature, then, the code generation device determines the code feature of the DSL code corresponding to the second image feature according to the correspondence between the image feature of the pre-stored GUI image and the code feature of the DSL code, and then, according to the code feature, obtains the interface code of the GUI image to be processed, that is, the embodiment of the present application quickly generates the interface code of the GUI image based on a GUI image, saves a lot of repetitive work of front-end developers, reduces the complexity of GUI development, and improves development efficiency. Moreover, in the embodiment of the present application, after the first preset model performs feature extraction on the GUI image, the extracted features are extracted again, so that the feature information of the extracted GUI image is richer, the information of the obtained GUI image is more, and image recognition is performed based on the obtained information, which improves the image recognition rate, thereby improving the accuracy of the interface code of the subsequently generated GUI image.

[0120] Here, before the code generating device inputs the code features of the DSL code corresponding to the second image features into the fourth preset model, it is necessary to train the fourth preset model, so that the code features of the DSL code corresponding to the second image features can be subsequently input into the trained fourth preset model, and the DSL code corresponding to the GUI image to be processed can be obtained according to the output of the fourth preset model. In the training process, the code generating device can input the code features of the reference DSL code into the fourth preset model, and then determine the output accuracy according to the DSL code output by the fourth preset model and the reference DSL code. If the output accuracy is lower than the preset accuracy threshold, the code generating device can adjust the fourth preset model according to the output accuracy to improve the output accuracy, use the adjusted fourth preset model as the new fourth preset model, and re-execute the step of inputting the code features of the reference DSL code into the fourth preset model.

[0121] The reference DSL code may be a DSL code of a known GUI diagram, such as a DSL code of an old version of a GUI diagram of the mobile terminal device. The code features of the reference DSL code may be obtained by extracting features from the reference DSL code by a code generation device. Exemplarily, the code generation device may input the reference DSL code into an LSTM model, encode the reference DSL code into an intermediate representation qt through the LSTM model, and then obtain the code features of the reference DSL code according to the qt.

[0122] In addition, in the embodiment of the present application, when obtaining the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code, the code generation device also considers inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is obtained by training the reference image features, the reference code features, and the correspondence between the reference image features and the reference code features. The second preset model can be specifically determined according to actual conditions, such as an LSTM model, and the embodiment of the present application does not impose any special restrictions on this. Furthermore, the code generation device obtains the correspondence between the image features of the GUI image and the code features of the DSL code based on the output of the second preset model. Figure 6 This is a flow chart of another method for generating interface code proposed in the embodiment of the present application. Figure 6 As shown, the method includes:

[0123] S601: Obtain a GUI image to be processed, and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain a first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed.

[0124] Among them, the implementation method of step S601 is the same as that of the above-mentioned step S401, which will not be repeated here.

[0125] S602: Inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is trained by reference image features, reference code features, and the correspondence between the reference image features and the reference code features.

[0126] Here, before inputting the image features of the preset GUI image and the code features of the preset DSL code into the second preset model, the code generating device may also obtain the code features of the preset DSL code.

[0127] Exemplarily, the code generation device may input the preset DSL code into a third preset model, wherein the third preset model is obtained by training a reference DSL code and the code features of the reference DSL code. The third preset model may be specifically determined according to actual conditions, such as an LSTM model, which is not particularly limited in the embodiment of the present application. Furthermore, the code generation device obtains the code features of the preset DSL code according to the output of the third preset model.

[0128] The third preset model may be an LSTM model, which is trained by using a reference DSL code and code features of the reference DSL code. The code generation device inputs the preset DSL code into the LSTM model, and obtains the code features of the preset DSL code according to the output of the LSTM model.

[0129] S603: According to the output of the second preset model, the correspondence between the image feature of the GUI image and the code feature of the DSL code is obtained, and according to the correspondence, the code feature of the DSL code corresponding to the second image feature is determined.

[0130] Here, after obtaining the code features of the preset DSL code, the code generation device may input the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, which may also be an LSTM model. The LSTM model is trained by reference image features, reference code features, and the correspondence between the reference image features and the reference code features. The code generation device obtains the correspondence between the image features of the GUI image and the code features of the DSL code based on the output of the LSTM model, thereby making the obtained correspondence more accurate and improving the accuracy of the interface code of the GUI image subsequently generated based on the correspondence.

[0131] For example, Figure 7A schematic diagram of a code generation model is shown, and the code generation model includes the first preset model, the second preset model and the third preset model. After obtaining the GUI image to be processed, the code generation device can input the GUI image to be processed into the first preset model. The first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain the first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain the second image feature of the GUI image to be processed. The code generation device can input the image feature of the preset GUI image and the code feature of the preset DSL code into the second preset model, wherein the second preset model is trained by reference image features, reference code features, and the corresponding relationship between the reference image features and the reference code features. Furthermore, the code generation device obtains the corresponding relationship between the image feature of the GUI image and the code feature of the DSL code according to the output of the second preset model, and determines the code feature of the DSL code corresponding to the second image feature according to the corresponding relationship, and obtains the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature. The code generating device may obtain the code features of the preset DSL code by inputting the preset DSL code into the third preset model. Here, the third preset model is trained by a reference DSL code and the code features of the reference DSL code.

[0132] S604: Obtaining the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature.

[0133] Among them, the implementation method of step S604 is the same as that of the above-mentioned step S403, which will not be repeated here.

[0134] The embodiment of the present application obtains the correspondence between the image features of the above-mentioned GUI image and the code features of the DSL code through the second preset model, thereby making the above-mentioned correspondence more accurate and improving the accuracy of the interface code of the GUI image subsequently generated based on the above-mentioned correspondence. Moreover, the embodiment of the present application quickly generates the interface code of the GUI image based on a GUI image, saving a lot of repetitive work for front-end developers, reducing the complexity of GUI development, and improving development efficiency. In addition, after the first preset model in the embodiment of the present application performs feature extraction on the GUI image, the extracted features are extracted again, so that the feature information of the extracted GUI image is richer, the information of the obtained GUI image is more, and image recognition is performed based on the obtained information, which improves the image recognition rate, thereby improving the accuracy of the interface code of the GUI image subsequently generated based on the features of the above-mentioned GUI image.

[0135] Here, before the code generation device inputs the image features of the preset GUI image and the code features of the preset DSL code into the second preset model, it is necessary to train the second preset model so that the image features of the preset GUI image and the code features of the preset DSL code can be subsequently input into the trained second preset model to obtain the correspondence between the image features of the GUI image output by the second preset model and the code features of the DSL code. During the training process, the code generation device can input the reference image features and the reference code features into the second preset model, and then determine the output accuracy based on the relationship between the two output by the second preset model and the correspondence between the reference image features and the reference code features. If the output accuracy is lower than the preset accuracy threshold, the code generation device can adjust the second preset model based on the output accuracy to improve the output accuracy, use the adjusted second preset model as the new second preset model, and re-execute the step of inputting the reference image features and the reference code features into the second preset model.

[0136] Among them, the above-mentioned reference image features and reference code features can be image features of a known GUI image and code features of the DSL code of the GUI image, such as image features of the old version GUI image of the above-mentioned mobile device and code features of the DSL code of the old version GUI image.

[0137] The code features of the DSL code of the old version of the GUI diagram can be obtained by extracting features from the DSL code of the old version of the GUI diagram by the code generation device. Exemplarily, the code generation device can input the DSL code of the old version of the GUI diagram into an LSTM model, and encode the reference DSL code into an intermediate representation qt through the LSTM model, and then obtain the code features of the DSL code of the old version of the GUI diagram according to the qt.

[0138] Similarly, before the code generation device inputs the preset DSL code into the third preset model, it needs to train the third preset model. Here, the training process of the third preset model can refer to the training process of the second preset model, which will not be repeated here.

[0139] Corresponding to the interface code generation method of the above embodiment, Figure 8 This is a schematic diagram of the structure of the interface code generation device provided in the embodiment of the present application. For the convenience of explanation, only the parts related to the embodiment of the present application are shown. Figure 8A schematic diagram of the structure of an interface code generating device provided in an embodiment of the present application, the interface code generating device 80 includes: an input module 801, an acquisition module 802, a determination module 803 and an acquisition module 804. The interface code generating device here can be the above-mentioned code generating device itself, or a chip or integrated circuit that implements the function of the code generating device. It should be noted here that the division of the input module, the acquisition module, the determination module and the acquisition module is only a division of logical functions, and the two can be physically integrated or independent.

[0140] Among them, the input module 801 is used to obtain the GUI image to be processed and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed to obtain the first image feature of the GUI image to be processed, and the feature extraction layer performs feature extraction on the first image feature to obtain the second image feature of the GUI image to be processed.

[0141] The acquisition module 802 is used to acquire the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code.

[0142] The determination module 803 is configured to determine, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature.

[0143] The obtaining module 804 is used to obtain the interface code of the to-be-processed GUI image according to the code feature of the DSL code corresponding to the second image feature.

[0144] In a possible design, the first preset model also includes a noise reduction layer, which removes noise points in the first image features, and the feature extraction layer extracts features from the first image features after the noise points are removed to obtain the second image features.

[0145] In a possible implementation, the feature extraction layer is a hash layer, and the hash layer calculates a hash value of the first image feature, and performs feature extraction on the first image feature according to the hash value to obtain the second image feature.

[0146] In a possible implementation, the input module 801 is further configured to determine a loss value of the feature extraction layer according to the first image feature and the second image feature before the determination module 803 determines, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature;

[0147] According to the loss value of the feature extraction layer, the feature extraction layer is optimized so that the loss value of the optimized feature extraction layer is equal to a preset minimum loss value, and the optimized feature extraction layer performs feature extraction on the first image feature to obtain the second image feature.

[0148] In a possible implementation, the acquisition module 802 is specifically configured to:

[0149] Inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is trained by reference image features, reference code features, and the corresponding relationship between the reference image features and the reference code features;

[0150] The corresponding relationship is obtained according to the output of the second preset model.

[0151] In a possible implementation, the acquisition module 802 is further configured to:

[0152] inputting the preset DSL code into a third preset model, wherein the third preset model is obtained by training a reference DSL code and code features of the reference DSL code;

[0153] According to the output of the third preset model, a code feature of the preset DSL code is obtained.

[0154] In a possible implementation, the obtaining module 804 is specifically configured to:

[0155] determining the DSL code corresponding to the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature;

[0156] According to the DSL code corresponding to the GUI image to be processed, the interface code of the GUI image to be processed is obtained.

[0157] In a possible implementation, the obtaining module 804 is specifically configured to:

[0158] inputting the code feature of the DSL code corresponding to the second image feature into a fourth preset model, wherein the fourth preset model is obtained by training the code feature of the reference DSL code and the reference DSL code;

[0159] According to the output of the fourth preset model, the DSL code corresponding to the GUI diagram to be processed is obtained.

[0160] In a possible implementation, the input module 801 is further configured to:

[0161] Performing image preprocessing on at least one GUI image of the target GUI, wherein the image preprocessing includes one or more of adjusting image size and image normalization;

[0162] According to the image preprocessing result, the GUI image to be processed is obtained.

[0163] In a possible implementation, the input module 801 is further configured to:

[0164] Receive the at least one GUI image sent by a preset image acquisition device.

[0165] In a possible implementation, the input module 801 is further configured to:

[0166] Receive the at least one GUI image sent by the device where the target GUI is located.

[0167] The device provided in the embodiment of the present application can be used to execute the technical solution of the above-mentioned method embodiment. Its implementation principle and technical effect are similar, and the embodiment of the present application will not be repeated here.

[0168] Optionally, Fig. 9 A possible basic hardware architecture of the interface code generating device described in the present application is schematically provided.

[0169] See also Fig. 9 The interface code generating device 900 includes at least one processor 901 and a communication interface 903. Optionally, it may also include a memory 902 and a bus 904.

[0170] The interface code generating device 900 may be the above-mentioned processing device, and the present application does not impose any particular limitation on this. In the interface code generating device 900, the number of processors 901 may be one or more. Figure 8 Only one of the processors 901 is illustrated. Optionally, the processor 901 may be a central processing unit (CPU), a graphics processing unit (GPU) or a digital signal processing (DSP). If the interface code generating device 900 has a plurality of processors 901, the types of the plurality of processors 901 may be different or the same. Optionally, the plurality of processors 901 of the interface code generating device 900 may also be integrated into a multi-core processor.

[0171] The memory 902 stores computer instructions and data; the memory 902 may store computer instructions and data required to implement the above-mentioned interface code generation method provided in the present application, for example, the memory 902 stores instructions for implementing the steps of the above-mentioned interface code generation method. The memory 902 may be any one or any combination of the following storage media: non-volatile memory (e.g., read-only memory (ROM)), solid state disk (SSD), hard disk (HDD), optical disk), volatile memory.

[0172] The communication interface 903 can provide information input / output for the at least one processor, and can also include any one or any combination of the following devices: a network interface (eg, an Ethernet interface), a wireless network card, and other devices with network access functions.

[0173] Optionally, the communication interface 903 may also be used for the interface code generating device 900 to perform data communications with other computing devices or terminals.

[0174] Further optional, Fig. 9 A thick line represents bus 904. Bus 904 can connect processor 901 with memory 902 and communication interface 903. Thus, processor 901 can access memory 902 through bus 904, and can also use communication interface 903 to exchange data with other computing devices or terminals.

[0175] In the present application, the interface code generating device 900 executes the computer instructions in the memory 902, so that the interface code generating device 900 implements the above-mentioned interface code generating method provided in the present application, or enables the interface code generating device 900 to deploy the above-mentioned interface code generating device.

[0176] From the perspective of logical function division, for example, Fig. 9 As shown, the memory 902 may include an input module 801, an acquisition module 802, a determination module 803, and an acquisition module 804. The inclusion here only refers to the fact that when the instructions stored in the memory are executed, the functions of the input module, the acquisition module, the determination module, and the acquisition module can be realized respectively, and is not limited to the physical structure.

[0177] In addition, the above-mentioned interface code generation device can also be used as described above Fig. 9 In addition to being implemented through software, it can also be implemented through hardware as a hardware module or as a circuit unit.

[0178] The present application provides a computer-readable storage medium, which stores a computer program, and the computer program enables a server to execute the above-mentioned interface code generation method provided by the present application.

[0179] The present application provides a computer program product, including computer instructions, wherein the computer instructions are executed by a processor to implement the above-mentioned interface code generation method.

[0180] The present application provides a chip, comprising at least one processor and a communication interface, wherein the communication interface provides information input and / or output for the at least one processor. Further, the chip may also include at least one memory, wherein the memory is used to store computer instructions. The at least one processor is used to call and run the computer instructions to execute the above-mentioned interface code generation method provided by the present application.

[0181] In the several embodiments provided in the present application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interfaces, indirect coupling or communication connection of devices or units, which can be electrical, mechanical or other forms.

[0182] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0183] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of hardware plus software functional units.

Claims

1. A method for generating interface code, characterized in that: include: Acquire a GUI image to be processed, and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed, and obtains a first image feature of the GUI image to be processed, specifically: perform feature extraction according to the weight of each dimensional feature of the GUI image to be processed, wherein the weight of each dimensional feature represents the importance of the dimensional feature, that is, the feature with high importance has a large weight, and the feature with small importance has a large weight; sort each dimensional feature according to the weight of each dimensional feature, and the importance of each dimensional feature is determined according to actual conditions; extract features with weights greater than preset weights from each dimensional feature according to the sorting results, and obtain the first image feature, and the preset weight is set according to actual conditions; the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed; Acquire a correspondence between a pre-stored image feature of a GUI image and a code feature of a domain specific language DSL code, and determine, based on the correspondence, a code feature of the DSL code corresponding to the second image feature; Obtaining the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature; Before determining the code feature of the DSL code corresponding to the second image feature according to the corresponding relationship, the method further includes: Determining a loss value of the feature extraction layer according to the first image feature and the second image feature; According to the loss value of the feature extraction layer, the feature extraction layer is optimized so that the loss value of the optimized feature extraction layer is equal to a preset minimum loss value, and the optimized feature extraction layer extracts features of the first image features to obtain the second image features.

2. The method according to claim 1, characterized in that The first preset model also includes a noise reduction layer, which removes noise points in the first image features. The feature extraction layer extracts features from the first image features after the noise points are removed to obtain the second image features.

3. The method according to claim 1, characterized in that The feature extraction layer is a hash layer, which calculates a hash value of the first image feature, and performs feature extraction on the first image feature according to the hash value to obtain the second image feature.

4. The method according to any one of claims 1 to 3, characterized in that The obtaining of the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code includes: Inputting the image features of the preset GUI image and the code features of the preset DSL code into a second preset model, wherein the second preset model is trained by reference image features, reference code features, and the corresponding relationship between the reference image features and the reference code features; The corresponding relationship is obtained according to the output of the second preset model.

5. The method according to claim 4, characterized in that Before inputting the image features of the preset GUI image and the code features of the preset DSL code into the second preset model, the method further includes: inputting the preset DSL code into a third preset model, wherein the third preset model is obtained by training a reference DSL code and code features of the reference DSL code; According to the output of the third preset model, a code feature of the preset DSL code is obtained.

6. The method according to any one of claims 1 to 3, characterized in that The obtaining, according to the code feature of the DSL code corresponding to the second image feature, the interface code of the GUI image to be processed comprises: determining the DSL code corresponding to the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature; According to the DSL code corresponding to the GUI image to be processed, the interface code of the GUI image to be processed is obtained.

7. The method according to claim 6, characterized in that The determining, according to the code feature of the DSL code corresponding to the second image feature, the DSL code corresponding to the to-be-processed GUI image comprises: inputting the code feature of the DSL code corresponding to the second image feature into a fourth preset model, wherein the fourth preset model is obtained by training the code feature of the reference DSL code and the reference DSL code; According to the output of the fourth preset model, the DSL code corresponding to the GUI diagram to be processed is obtained.

8. The method according to any one of claims 1 to 3, characterized in that The obtaining of the GUI image to be processed comprises: Performing image preprocessing on at least one GUI image of the target GUI, wherein the image preprocessing includes one or more of adjusting image size and image normalization; According to the image preprocessing result, the GUI image to be processed is obtained.

9. An interface code generating device, characterized in that: include: An input module is used to obtain a GUI image to be processed, and input the GUI image to be processed into a first preset model, wherein the first preset model includes a convolution layer and a feature extraction layer, the convolution layer performs feature extraction on the GUI image to be processed, and obtains a first image feature of the GUI image to be processed, specifically: feature extraction is performed according to the weight of each dimensional feature of the GUI image to be processed, wherein the weight of each dimensional feature represents the importance of the dimensional feature, that is, the feature with high importance has a large weight, and the feature with small importance has a large weight; each dimensional feature is sorted according to the weight of each dimensional feature, and the importance of each dimensional feature is determined according to actual conditions; according to the sorting result, features with weights greater than a preset weight are extracted from each dimensional feature to obtain the first image feature, and the preset weight is set according to actual conditions; the feature extraction layer performs feature extraction on the first image feature to obtain a second image feature of the GUI image to be processed; An acquisition module, used for acquiring the correspondence between the image features of the pre-stored GUI image and the code features of the DSL code; a determination module, configured to determine, according to the corresponding relationship, a code feature of the DSL code corresponding to the second image feature; An obtaining module, configured to obtain the interface code of the GUI image to be processed according to the code feature of the DSL code corresponding to the second image feature; Before determining the code feature of the DSL code corresponding to the second image feature according to the corresponding relationship, the device is further specifically configured to: Determining a loss value of the feature extraction layer according to the first image feature and the second image feature; According to the loss value of the feature extraction layer, the feature extraction layer is optimized so that the loss value of the optimized feature extraction layer is equal to a preset minimum loss value, and the optimized feature extraction layer performs feature extraction on the first image feature to obtain the second image feature.

10. An interface code generating device, characterized in that: include: processor; Memory; as well as Computer programs; The computer program is stored in the memory and is configured to be executed by the processor, wherein the computer program includes instructions for executing the method according to any one of claims 1 to 8.

11. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores a computer program, and the computer program enables the server to execute the method according to any one of claims 1 to 8.

12. A computer program product, characterized in that The method comprises computer instructions, wherein the computer instructions are executed by a processor to perform the method according to any one of claims 1 to 8.

Images