Interface layout method, apparatus, device, and computer program product
By constructing an interface layout topology diagram and using the trained model to predict constraint relationships, the basic elements of the interface are automatically laid out, solving the problem of high cost and low efficiency of manual design for 5G new call interfaces, and realizing efficient and personalized interface design.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- CHINA MOBILE GROUP ZHEJIANG
- Filing Date
- 2021-08-05
- Publication Date
- 2026-08-04
AI Technical Summary
The current design of new 5G call interfaces relies on manual design, resulting in high costs and low efficiency, making it difficult to meet the personalized needs of various industries.
By obtaining the sequence of basic interface elements, a topology graph of the interface layout is constructed, and the training interface layout generation model is used to predict constraint relationships and automatically lay out the basic interface elements.
It improves interface layout efficiency, reduces the cost and time of manual design, and adapts to the personalized needs of different industries.
Smart Images
Figure CN115705183B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of data processing technology, and in particular to an interface layout method, apparatus, device, and computer program product. Background Technology
[0002] With the rapid development of 5G (5th Generation Mobile Communication Technology), the application of 5G calling is becoming increasingly widespread, enabling users to make video calls based on phone numbers. However, currently, the interface for 5G calling relies on manual design by industry clients or operators. Since the interface design requirements of various industries are highly personalized, manual design is costly, time-consuming, labor-intensive, and inefficient. Summary of the Invention
[0003] The main objective of this invention is to provide an interface layout method, apparatus, device, and computer program product, which aims to improve interface layout efficiency.
[0004] To achieve the above objectives, the present invention provides an interface layout method, which includes the following steps:
[0005] Obtain the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes.
[0006] Based on the aforementioned basic interface element nodes, construct the interface layout topology diagram of the interface to be laid out.
[0007] Based on the trained interface layout generation model and the interface layout topology graph, constraint relationship prediction is performed on the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction result.
[0008] Optionally, the step of obtaining the sequence of basic interface elements of the interface to be laid out includes:
[0009] Obtain the interface layout requirements information for the interface to be laid out;
[0010] Based on the trained basic element generation model and the interface layout requirement information, the basic elements of the interface to be laid out are predicted to obtain the interface basic element sequence.
[0011] Optionally, the basic element generation model includes an encoder and a decoder. The step of predicting the basic elements of the interface to be laid out based on the trained basic element generation model and the interface layout requirement information to obtain the sequence of basic elements includes:
[0012] Based on the encoder, feature extraction is performed on the interface layout requirement information to obtain a context vector;
[0013] Based on the context vector and the decoder, the basic elements of the interface to be laid out are predicted, and a sequence of basic elements of the interface is obtained.
[0014] Optionally, the basic element generation model further includes an embedding layer, and the step of extracting features from the interface layout requirement information based on the encoder to obtain a context vector includes:
[0015] The interface layout requirement information is serialized into text to obtain indexed interface layout requirement information.
[0016] Based on the embedding layer, the indexed interface layout requirement information is vector-mapped to obtain the interface layout requirement vector.
[0017] Based on the encoder, feature extraction is performed on the interface layout requirement vector to obtain the context vector.
[0018] Optionally, the interface layout generation model includes several relation graph convolutional layers, and the step of predicting constraint relationships in the interface layout topology graph based on the trained interface layout generation model and the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph includes:
[0019] Based on the aforementioned several relational graph convolutional layers, feature extraction is performed on the interface layout topology graph to obtain topological latent features;
[0020] Based on the aforementioned topological latent features, a tensor factorization operation is performed to predict the scoring results of the pairwise node constraint relationships among the aforementioned interface basic element nodes.
[0021] Based on the scoring results, the constraint relationship prediction results of the interface layout topology diagram are determined.
[0022] Optionally, the interface layout generation model further includes several fully connected layers and several discard layers, with each of the relation graph convolutional layers, each of the fully connected layers, and each of the discard layers arranged sequentially. The step of extracting features from the interface layout topology graph based on the several relation graph convolutional layers to obtain the topological latent features includes:
[0023] Based on the aforementioned relational graph convolutional layers, fully connected layers, and discard layers, feature extraction is performed on the interface layout topology graph to obtain potential topological features.
[0024] Optionally, before the step of predicting the constraint relationships of the interface layout topology based on the trained interface layout generation model and the interface layout topology, and obtaining the constraint relationship prediction result of the interface layout topology, the method further includes:
[0025] Obtain training data for the interface layout topology graph, and label the interface layout topology graph training data for constraint relationships to obtain constraint relationship data;
[0026] Obtain the model to be trained, and select training sample data from the interface layout topology training data and the constraint relationship data;
[0027] Based on the training sample data, the model to be trained is iteratively trained to obtain the interface layout generation model.
[0028] Furthermore, to achieve the above objectives, the present invention also provides an interface layout device, the interface layout device comprising:
[0029] The acquisition module is used to acquire the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes.
[0030] The construction module is used to construct the interface layout topology diagram of the interface to be laid out based on the aforementioned basic interface element nodes.
[0031] The prediction module is used to predict the constraint relationships of the interface layout topology based on the trained interface layout generation model and the interface layout topology, and obtain the constraint relationship prediction results of the interface layout topology, so as to perform interface layout based on the constraint relationship prediction results.
[0032] In addition, to achieve the above objectives, the present invention also provides an interface layout device, the interface layout device comprising: a memory, a processor, and an interface layout program stored in the memory and executable on the processor, wherein the interface layout program, when executed by the processor, implements the steps of the interface layout method as described above.
[0033] In addition, to achieve the above objectives, the present invention also provides a computer-readable storage medium storing an interface layout program, which, when executed by a processor, implements the steps of the interface layout method as described above.
[0034] In addition, to achieve the above objectives, the present invention also provides a computer program product, the computer program product comprising a computer program, which, when executed by a processor, implements the steps of the interface layout method as described above.
[0035] This invention provides an interface layout method, apparatus, device, and computer program product. The method involves obtaining a sequence of basic interface elements for an interface to be laid out, the sequence including several basic interface element nodes; constructing an interface layout topology graph based on these basic interface element nodes; and predicting constraints on the interface layout topology graph based on a trained interface layout generation model and the topology graph, obtaining the constraint prediction results for interface layout. This approach only requires obtaining the basic interface element nodes needed for the interface to be laid out, constructing the interface layout topology graph based on these nodes, and then automatically predicting the constraints of the topology graph based on the trained interface layout generation model. This allows for layout of all basic interface element nodes of the interface to be laid out based on these constraints, thereby improving the efficiency of interface layout. Attached Figure Description
[0036] Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention;
[0037] Figure 2 This is a flowchart illustrating the first embodiment of the interface layout method of the present invention;
[0038] Figure 3 This is a flowchart illustrating the second embodiment of the interface layout method of the present invention;
[0039] Figure 4 This is a schematic diagram of the basic element generation model involved in an embodiment of the present invention;
[0040] Figure 5 This is a schematic diagram of the interface layout generation model involved in an embodiment of the present invention.
[0041] The realization of the objective, functional features and advantages of the present invention will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation
[0042] It should be understood that the specific embodiments described herein are merely illustrative of the invention and are not intended to limit the invention.
[0043] The main solution of this invention is as follows: First, obtain the sequence of basic interface elements for the interface to be laid out, including several basic interface element nodes. Second, construct an interface layout topology graph based on these basic interface element nodes. Third, predict the constraint relationships of the interface layout topology graph based on the trained interface layout generation model and the topology graph, obtaining the constraint relationship prediction results for interface layout. Through this method, only the basic interface element nodes required for the interface to be laid out need to be obtained. Then, an interface layout topology graph is constructed based on these basic interface element nodes. Afterward, based on the built and trained interface layout generation model, the constraint relationships of the interface layout topology graph are automatically predicted, allowing for the layout of all basic interface element nodes of the interface to be laid out based on these constraint relationships, thereby improving the efficiency of interface layout.
[0044] Technical terms involved in the embodiments of this invention:
[0045] With the advent of the 5G era, operators can upgrade user calling experiences based on VoNR / VoLTE high-definition video calling services. This leverages advantages not found in OTT applications, such as the ability to avoid app installations, number-based services, and guaranteed deterministic network performance, to boost data traffic revenue, cultivate user habits, capture users, and lay the groundwork for the future. The GSMA, in its RCS (Regulatory Services) framework, proposed Enriched Calling, introducing enhanced capabilities into calls and defining three main scenarios: Pre-Call, In-Call, and Post-Call. Before a call, the caller can send the call subject, importance, images, and location information to the callee. During a call, both parties can send each other any files, send IM messages, share locations, maps, doodles, and real-time video. If the caller cannot answer, the caller can send a text or voice message. Operators can upgrade VoNR / VoLTE high-definition video calling services based on these three scenarios. 3GPP has also developed the IMS Data Channel standard, which, based on VoLTE / VoNR high-definition audio and video calls and combined with WebRTC technology, provides an extended data channel (IMS Data Channel) to synchronize voice and video calls with the extended data channel. This enables screen sharing, AR overlay, and even a fully immersive experience with synchronized auditory, visual, tactile, and kinematic sensations during high-definition video calls. The IMS Data Channel, based on UDP, provides a high-real-time single-stream or multi-stream data interaction channel, allowing simultaneous desktop sharing, whiteboard sharing, and file transfer during calls. By introducing the real-time interactive channel (IMS Data Channel), it adds industry-specific interactive applications to high-definition real-time multimedia calls for vertical industries. End-to-end QoS assurance goes beyond just voice and video QoS policies and enforcement; it also requires customized QoS policy control for different data service types and supports negotiation of media QoS attributes based on multiple scenarios. The IMS Data Channel does not concern itself with the content or format of the transmitted channel; it only requires the communicating parties to agree on the communication format. Diverse application content can be transmitted through the IMS Data Channel using a common Webpage + JavaScript approach.
[0046] The embodiments of the present invention take into account that in existing related solutions, the interface of 5G new calls needs to be designed manually by industry customers or operators. Since the interface design requirements of various industries are quite personalized, manual design costs are increased, time and effort are wasted, and efficiency is low.
[0047] Reference Figure 1 , Figure 1 This is a schematic diagram of the terminal structure of the hardware operating environment involved in the embodiments of the present invention.
[0048] In this embodiment of the invention, the terminal is an interface layout device, which can be a PC (personal computer), microcomputer, laptop computer, server, or other terminal device with processing capabilities.
[0049] like Figure 1 As shown, the terminal may include: a processor 1001, such as a CPU (Central Processing Unit), a communication bus 1002, a user interface 1003, a network interface 1004, and a memory 1005. The communication bus 1002 is used to enable communication between these components. The user interface 1003 may include a display screen and an input unit such as a keyboard; optionally, the user interface 1003 may also include a standard wired interface or a wireless interface. The network interface 1004 may optionally include a standard wired interface or a wireless interface (such as a Wi-Fi interface). The memory 1005 may be high-speed RAM or non-volatile memory, such as a disk drive. Optionally, the memory 1005 may also be a storage device independent of the aforementioned processor 1001.
[0050] Those skilled in the art will understand that Figure 1 The terminal structure shown does not constitute a limitation on the terminal and may include more or fewer components than shown, or combine certain components, or have different component arrangements.
[0051] like Figure 1 As shown, the memory 1005, which serves as a computer storage medium, may include an operating system, a network communication module, a user interface module, and an interface layout program.
[0052] exist Figure 1 In the terminal shown, the processor 1001 can be used to call the interface layout program stored in the memory 1005 and perform the following operations:
[0053] Obtain the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes.
[0054] Based on the aforementioned basic interface element nodes, construct the interface layout topology diagram of the interface to be laid out.
[0055] Based on the trained interface layout generation model and the interface layout topology graph, constraint relationship prediction is performed on the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction result.
[0056] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0057] Obtain the interface layout requirements information for the interface to be laid out;
[0058] Based on the trained basic element generation model and the interface layout requirement information, the basic elements of the interface to be laid out are predicted to obtain the interface basic element sequence.
[0059] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0060] Based on the encoder, feature extraction is performed on the interface layout requirement information to obtain a context vector;
[0061] Based on the context vector and the decoder, the basic elements of the interface to be laid out are predicted, and a sequence of basic elements of the interface is obtained.
[0062] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0063] The interface layout requirement information is serialized into text to obtain indexed interface layout requirement information.
[0064] Based on the embedding layer, the indexed interface layout requirement information is vector-mapped to obtain the interface layout requirement vector.
[0065] Based on the encoder, feature extraction is performed on the interface layout requirement vector to obtain the context vector.
[0066] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0067] Based on the aforementioned several relational graph convolutional layers, feature extraction is performed on the interface layout topology graph to obtain topological latent features;
[0068] Based on the aforementioned topological latent features, a tensor factorization operation is performed to predict the scoring results of the pairwise node constraint relationships among the aforementioned interface basic element nodes.
[0069] Based on the scoring results, the constraint relationship prediction results of the interface layout topology diagram are determined.
[0070] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0071] Based on the aforementioned relational graph convolutional layers, fully connected layers, and discard layers, feature extraction is performed on the interface layout topology graph to obtain potential topological features.
[0072] Furthermore, the processor 1001 can be used to call the interface layout program stored in the memory 1005, and also perform the following operations:
[0073] Obtain training data for the interface layout topology graph, and label the interface layout topology graph training data for constraint relationships to obtain constraint relationship data;
[0074] Obtain the model to be trained, and select training sample data from the interface layout topology training data and the constraint relationship data;
[0075] Based on the training sample data, the model to be trained is iteratively trained to obtain the interface layout generation model.
[0076] Based on the above hardware structure, various embodiments of the interface layout method of the present invention are proposed.
[0077] This invention provides an interface layout method.
[0078] Reference Figure 2 , Figure 2 This is a flowchart illustrating the first embodiment of the interface layout method of the present invention.
[0079] In this embodiment, the interface layout method includes:
[0080] Step S10: Obtain the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements of the interface.
[0081] In this embodiment, the interface layout method can be applied to an interface layout device, which can be an IMS data channel and can be managed by the operator; it can also be applied to an interface layout system composed of the IMS data channel and other related devices, which is subordinate to the above-mentioned... Figure 1 The interface layout of the device.
[0082] In one embodiment, the step of obtaining the sequence of basic interface elements of the interface to be laid out includes:
[0083] The system obtains interface creation request information from industry customers via the IMS data channel. This interface creation request information includes an interface layout requirement description. Based on the interface layout requirement description information, the sequence of basic interface elements to be laid out is determined. Specifically, the industry customer initiates a request to the IMS data channel to create a new 5G call main interface, and the request message carries the industry customer's interface layout requirement description.
[0084] In this embodiment, a sequence of basic interface elements for the interface to be laid out is obtained. This sequence includes several basic interface element nodes. The basic interface element nodes in the sequence are the basic element nodes required for the layout of the interface, such as buttons, titles, tables, images, and videos.
[0085] In this embodiment, the method for obtaining the sequence of basic elements of the interface to be laid out can be:
[0086] Method 1: Receive a new interface request from an industry client. Based on the interface layout requirement description information carried in the request, and then, based on this description and a trained basic element generation model, predict the basic elements of the interface to be laid out by the industry client, thus obtaining a sequence of basic interface elements. The specific execution process can be found in the second embodiment below, and will not be elaborated further here.
[0087] Method 2: Receive the interface creation request sent by the industry client, and based on the interface layout requirement description information carried in the interface creation request, retrieve the interface basic element sequence from the memory based on the interface layout requirement description information. The interface basic element sequence is a set of interface basic elements that are pre-set according to the interface layout requirement description information.
[0088] Step S20: Based on the aforementioned basic interface element nodes, construct the interface layout topology diagram of the interface to be laid out.
[0089] In this embodiment, an interface layout topology graph of the interface to be laid out is constructed based on several basic interface element nodes. Specifically, the interface layout topology graph is constructed by using several determined basic interface element nodes as nodes, the content of the elements as the features of the nodes, and the constraints between the elements as the edges between the nodes.
[0090] It should be noted that the constraint relationships between any two nodes in this layout topology graph need to be reconstructed so that subsequent layout can be carried out based on the reconstructed layout topology graph.
[0091] Step S30: Based on the trained interface layout generation model and the interface layout topology graph, perform constraint relationship prediction on the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction result.
[0092] In this embodiment, based on the trained interface layout generation model and interface layout topology graph, constraint relationship prediction is performed on the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction result, that is, to map the distance between the basic elements of the interface through the constraint relationship to realize the generation of the interface topology to be laid out.
[0093] Specifically, based on the feature extractor in the trained interface layout generation model, the topological feature information of the interface layout in the interface layout topology graph is extracted. Then, based on the topological feature information of the interface layout and the classifier in the interface layout generation model, the constraint relationship of the interface layout topology graph is predicted to obtain the constraint relationship prediction result, and the constraint relationship prediction result is used as the basis for prediction.
[0094] The specific execution process of the classifier involves obtaining a classification probability vector and then determining the constraint relationship prediction result corresponding to the largest classification probability value in the classification probability vector.
[0095] In another embodiment, the interface layout generation model is a relational graph convolutional network model, and the feature extractor includes several relational graph convolutional layers. Step S30 above includes:
[0096] Based on the aforementioned relational graph convolutional layers, feature extraction is performed on the interface layout topology graph to obtain topological latent features; based on the topological latent features, tensor factorization is performed to predict the scoring results of pairwise node constraint relationships among the aforementioned interface basic element nodes; based on the scoring results, the constraint relationship prediction results of the interface layout topology graph are determined. The specific execution process is described in the third embodiment below.
[0097] It should be noted that the distance between basic interface elements can be mapped by the constraint relationships in the constraint relationship prediction results, and then the layout of basic interface elements can be based on the distance between them.
[0098] This invention provides an interface layout method. The method involves obtaining a sequence of basic interface elements for the interface to be laid out, including several basic interface element nodes. Based on these basic interface element nodes, an interface layout topology graph is constructed. Then, based on a trained interface layout generation model and the interface layout topology graph, constraint relationships are predicted on the topology graph to obtain the predicted constraint relationship results, which are then used for interface layout. This approach only requires obtaining the basic interface element nodes needed for the interface to be laid out, constructing the interface layout topology graph based on these nodes, and then automatically predicting the constraint relationships of the topology graph using the trained interface layout generation model. This allows for the layout of all basic interface element nodes of the interface to be laid out based on these constraints, thereby improving the efficiency of interface layout.
[0099] Furthermore, based on the first embodiment described above, a second embodiment of the interface layout method of the present invention is proposed.
[0100] Reference Figure 3 , Figure 3 This is a flowchart illustrating the second embodiment of the interface layout method of the present invention.
[0101] In this embodiment, step S10 includes:
[0102] Step S11: Obtain the interface layout requirements information of the interface to be laid out;
[0103] In this embodiment, the interface layout requirements information of the interface to be laid out is obtained. Specifically, the interface creation request information of the industry customer is obtained, which includes interface layout requirement description information. For example, an industry customer initiates a request to create a new 5G call main interface to the IMS data channel. The request message carries the industry customer's interface layout requirement description. Then, the interface layout device obtains the industry customer's interface creation request information through the IMS data channel, which includes the interface layout requirement description information.
[0104] Step S12: Based on the trained basic element generation model and the interface layout requirement information, predict the basic elements of the interface to be laid out to obtain the interface basic element sequence.
[0105] In this embodiment, based on the trained basic element generation model and interface layout requirement information, the basic interface elements to be included in the interface to be laid out are predicted to obtain a sequence of basic interface elements. The basic element generation model is a deep learning-based model used to generate the sequence of basic interface elements required for the interface to be laid out.
[0106] Specifically, based on the feature extractor in the trained basic element generation model, the interface layout requirement feature information is extracted from the interface layout requirement information. Then, based on the interface layout requirement feature information and the classifier in the basic element generation model, the basic elements of the interface to be laid out are predicted to obtain the interface basic element sequence.
[0107] The specific execution process of the classifier involves obtaining a classification probability vector and then determining the sequence of basic interface elements corresponding to the largest classification probability value in the classification probability vector.
[0108] In one embodiment, the basic element generation model is an encoder-decoder neural network model, which includes an encoder and a decoder. Step S12 includes:
[0109] Step A121: Based on the encoder, perform feature extraction on the interface layout requirement information to obtain a context vector;
[0110] Step A122: Based on the context vector and the decoder, predict the basic elements of the interface to be laid out, and obtain the sequence of basic elements of the interface.
[0111] In this embodiment, based on the encoder, features are extracted from the interface layout requirement information to obtain a context vector; then, based on the context vector and the decoder, the basic elements of the interface to be laid out are predicted to obtain a sequence of basic elements of the interface.
[0112] It should be noted that the encoder can be composed of a recurrent neural network, which can be an LSTM (long short-term memory) neural network, or it can be composed of a deep convolutional neural network, etc. Correspondingly, the decoder can be composed of a recurrent neural network, which can be an LSTM (long short-term memory) neural network, or it can be composed of a deep convolutional neural network, etc.
[0113] In some embodiments, the encoder includes three LSTM neural network layers, and step A121 above includes:
[0114] Based on the three LSTM neural network layers of the encoder, feature extraction is performed on the interface layout requirement information to obtain a context vector.
[0115] In some embodiments, the decoder includes three LSTM neural network layers, and step A122 above includes:
[0116] Based on the context vector and the three LSTM neural network layers of the decoder, the basic elements of the interface to be laid out are predicted, and a sequence of basic elements of the interface is obtained.
[0117] For ease of understanding, a specific embodiment will be used for illustration below. (Refer to...) Figure 4 , Figure 4 This is a schematic diagram of the basic element generation model involved in this embodiment of the invention. The first layer is the input layer: it inputs the indexed industry customer layout content description; the second layer is the embedding layer: it uses word embedding to convert each word into a vector, and the output is set to convert the words into 128-dimensional spatial vectors. The function of this layer is to perform vector mapping on the input words, converting the index of each word into a 128-dimensional fixed-shape vector; the third and fourth layers are LSTM encoding layers: containing 128 LSTM neurons, with the activation function set to "relu"; the fifth and sixth layers are LSTM decoding layers: containing 128 LSTM neurons, with the activation function set to "relu"; the seventh layer is a fully connected (Dense) layer (output layer): the activation function is set to "softmax", and the softmax output result is fed into a multi-class cross-entropy loss function to obtain the interface basic element sequence.
[0118] Furthermore, the interface layout requirements information is preprocessed, and then the preprocessed interface layout requirements information is input into the basic element generation model.
[0119] Data preprocessing can include text serialization and vector mapping. Specifically, the interface layout requirement information is cleaned and serialized to obtain indexed interface layout requirement information. Then, a word embedding layer is used to perform vector mapping on the indexed interface layout requirement information to obtain the interface layout requirement vector. Specifically, all punctuation marks in the interface layout requirement information are removed. If the text is Chinese, it is segmented into words; if the text is English, the letters are converted to lowercase. Each word is also tokenized, so that each segment of text is converted into an index number, and zeros are padded for sequences that do not reach the maximum text length. For example, the longest length of the interface layout requirement information is taken as its index sequence length, and its dictionary size is taken as the data dimension. Then, the word embedding layer is used to convert each word into a vector. Specifically, the input data dimension of the word embedding layer is the dictionary size of the interface layout requirement information, and the output is set to convert the words into 128-dimensional spatial vectors. In other words, the function of the word embedding layer is to perform vector mapping on the input words, converting the index of each word into a 128-dimensional fixed-shape vector.
[0120] In one embodiment, the basic element generation model further includes an embedding layer, and step A121 above includes:
[0121] Step A1211: Based on the embedding layer, perform vector mapping on the indexed interface layout requirement information to obtain the interface layout requirement vector;
[0122] Step A1212: Based on the encoder, feature extraction is performed on the interface layout requirement vector to obtain the context vector.
[0123] In this embodiment, the indexed interface layout requirement information is vector-mapped based on the embedding layer to obtain an interface layout requirement vector; based on the encoder, features are extracted from the interface layout requirement vector to obtain a context vector. Specifically, the indexed interface layout requirement information is vector-mapped based on the word embedding layer to obtain an interface layout requirement vector. The word embedding layer converts each word into a vector. Specifically, the input data dimension of the word embedding layer is the size of the dictionary of interface layout requirement information, and the output is set to convert the words into 128-dimensional spatial vectors. That is, the function of the word embedding layer is to perform vector mapping on the input words, converting the index of each word into a 128-dimensional fixed-shape vector.
[0124] To train the basic element generation model, prior to step S12 above, this interface layout method also includes:
[0125] Obtain training data for interface layout requirements, and label the interface layout requirements for basic interface elements to obtain basic interface element data; obtain the model to be trained, and select training sample data from the interface layout requirement training data and the basic interface element data; based on the training sample data, iteratively train the model to be trained to obtain an interface layout topology generator.
[0126] Specifically, the representation values of each interface layout requirement information in the training data of interface layout requirement information are extracted, and then based on each representation value of interface layout requirement information, the corresponding sequence of basic interface layout elements is matched for the training data of interface layout requirement information, thereby obtaining the sequence data of basic interface layout elements.
[0127] The training sample data includes at least one training sample, and the training sample includes interface layout requirement information from the interface layout requirement information training data.
[0128] In one embodiment, historical interface layout requests are obtained, and then the interface layout requirement information carried in the interface layout requests is used as training data for all interface layout requirement information. In other embodiments, the training data can be set manually or through other means.
[0129] Furthermore, the interface layout requirements information is preprocessed, and then the preprocessed interface layout requirements information is input into the basic element generation model.
[0130] Data preprocessing can include text serialization and vector mapping. Specifically, the interface layout requirement information is cleaned and serialized to obtain indexed interface layout requirement information. Then, a word embedding layer is used to perform vector mapping on the indexed interface layout requirement information to obtain the interface layout requirement vector. Specifically, all punctuation marks in the interface layout requirement information are removed. If the text is Chinese, it is segmented into words; if the text is English, the letters are converted to lowercase. Each word is also tokenized, so that each segment of text is converted into an index number, and zeros are padded for sequences that do not reach the maximum text length. For example, the longest length of the interface layout requirement information is taken as its index sequence length, and its dictionary size is taken as the data dimension. Then, the word embedding layer is used to convert each word into a vector. Specifically, the input data dimension of the word embedding layer is the dictionary size of the interface layout requirement information, and the output is set to convert the words into 128-dimensional spatial vectors. In other words, the function of the word embedding layer is to perform vector mapping on the input words, converting the index of each word into a 128-dimensional fixed-shape vector.
[0131] Furthermore, the training sample data is divided into a training set and a test set. For example, 90% of the training sample data is allocated to the training set, and 10% is allocated to the test set. The training set is used to train the model, and the test set is used to test the model.
[0132] Specifically, training samples are selected from the training sample data, and the interface layout requirement information corresponding to the training samples is input into the training model. The model prediction is performed to obtain the model output label. Then, the difference between the model output label and the permission category of the interface layout topology map corresponding to the training sample is calculated to obtain the model loss. Based on the model loss, the training model is updated until the number of iterations of the training model reaches the preset number of iterations.
[0133] It should be noted that the number of training epochs is set to 1000, the batch size is set to 100, categorical cross-entropy is chosen as the loss function and objective function (loss = 'categorical_crossentropy'), and the Adam optimizer is selected to improve the learning speed of traditional gradient descent (optimizer = 'adam'). The neural network uses gradient descent to find the optimal weight values that minimize the objective function, and the neural network learns these weight values autonomously through training. The weights of the model are derived after convergence.
[0134] In this embodiment, based on the basic element generation model that has been built and trained, the basic elements of the interface to be laid out are automatically predicted to obtain the sequence of basic elements of the interface, thereby further improving the efficiency of interface layout.
[0135] Furthermore, based on the first embodiment described above, a third embodiment of the interface layout method of the present invention is proposed.
[0136] In this embodiment, the interface layout generation model includes several relational graph convolutional layers, and step S30 includes:
[0137] Step A31: Based on the aforementioned relational graph convolutional layers, feature extraction is performed on the interface layout topology graph to obtain potential topological features;
[0138] Step A32: Perform tensor factorization based on the topological latent features to predict the score results of the pairwise node constraint relationships among the several interface basic element nodes.
[0139] Step A33: Based on the scoring results, determine the constraint relationship prediction results of the interface layout topology diagram.
[0140] In this embodiment, feature extraction is performed on the interface layout topology graph based on the several relation graph convolutional layers to obtain topological latent features; tensor factorization is performed on the topological latent features to predict the scoring results of the pairwise node constraint relationships among the several interface basic element nodes; and the constraint relationship prediction results of the interface layout topology graph are determined based on the scoring results.
[0141] Furthermore, the interface layout generation model also includes several fully connected layers and several discard layers, with each of the relation graph convolutional layers, each of the fully connected layers, and each of the discard layers arranged sequentially. Step A31 includes:
[0142] Step A311: Based on the several relation graph convolutional layers, the several fully connected layers, and the several discard layers, feature extraction is performed on the interface layout topology graph to obtain topological latent features.
[0143] For ease of understanding, a specific embodiment will be used for illustration below. (Refer to...) Figure 5 , Figure 5 This is a schematic diagram of the interface layout generation model involved in an embodiment of the present invention. The first layer is a relational graph convolutional layer (R-GCN): the number of convolutional kernels is 128 (i.e., the output dimension), and the activation function is set to "relu"; the second layer is a fully connected layer (Dense): the number of neurons is 128, and the activation function is set to "relu"; the third layer is a dropout layer: the dropout probability is set to 0.2, and input neurons are randomly disconnected with a certain probability (20%) each time parameters are updated during training. The dropout layer is used to prevent overfitting; the fourth layer is a relational graph convolutional layer (R-GCN): the number of convolutional kernels is 64, and the activation function is set to "lambda"; the fifth layer is a fully connected layer (Dense): the number of neurons is 64, and the activation function is set to "relu"; the sixth layer is a dropout layer: the dropout probability is set to 0.2; the seventh layer is a tensor factorization layer (DistMult). Factorization: Using the latent space vector representations zi and zj of nodes i and j output by the encoder, the decoder predicts candidate edges (vi, r, vj) through factorization, with the activation function set to "softmax". The decoder scores the possible edges (vi, r, vj) using the function g(vi, r, vj) to determine the probability that these edges belong to set E. This score g(vi, r, vj) represents the probability that the basic element nodes vi and vj are associated through relation r. Using the latent space vector representations zi and zj of nodes i and j output by the encoder, the decoder predicts candidate edges (vi, r, vj) through DistMult factorization as the scoring function:
[0144]
[0145] Here, Rr is a d*d diagonal matrix that represents the importance of each dimension in zi to the relationship r. Finally, softmax(g(vi,r,vj)) represents the constraint relationship between basic element node i and basic element node j, with a value between 0 and 1. The larger the value, the stronger the constraint relationship and the closer the layout distance.
[0146] In this embodiment, the interface layout generation model, which is built and trained based on the relational graph network, automatically predicts the constraint relationships of the interface layout topology graph, so as to lay out all the basic interface element nodes of the interface to be laid out based on the constraint relationships, thereby further improving the efficiency of interface layout.
[0147] Furthermore, based on the first embodiment described above, a fourth embodiment of the interface layout method of the present invention is proposed.
[0148] In this embodiment, prior to step S30, the interface layout method further includes:
[0149] Step A40: Obtain interface layout topology training data, and label the interface layout topology training data with constraints to obtain constraint data.
[0150] In this embodiment, interface layout topology graph training data is obtained, and the interface layout topology graph training data is labeled with constraints to obtain constraint relationship data. Specifically, the representation values of each interface layout topology graph set in the interface layout topology graph training data and the representation values of each message content in the delivery message training data are extracted. Then, based on the representation values of each interface layout topology graph set and each message content, corresponding constraint relationship prediction results are matched for the interface layout topology graph training data and the delivery message training data to obtain constraint relationship prediction result data.
[0151] In one embodiment, an interface layout topology map set corresponding to customers in various industries is obtained, and then the interface layout topology map set is used as training data for the interface layout topology map. In other embodiments, the interface layout topology map training data can be set manually or through other means. The interface layout topology map training data includes at least one interface layout topology map set.
[0152] In one embodiment, historical interface layout requests are obtained, and then the interface layout requirement information carried in the interface layout requests is used as training data for all interface layout requirement information. In other embodiments, the training data can be set manually or through other means.
[0153] Furthermore, after the step of obtaining the interface layout topology training data and the message delivery training data, the interface layout method further includes:
[0154] The interface layout topology training data is processed by integer serialization to obtain indexed interface layout topology training data. The delivery message training data is cleaned and serialized to obtain indexed delivery message training data. The indexed message training data is vectorized through a word embedding layer to obtain vectorized delivery message training data. This data is then used to label the indexed interface layout topology training data and the vectorized delivery message training data for constraint relationship prediction results, thus obtaining constraint relationship prediction result data.
[0155] The training data for the delivery messages undergoes text cleaning and serialization. Specifically, all punctuation marks are removed. If the text is Chinese, it is segmented into words; if the text is English, all letters are converted to lowercase. Each word is also tokenized, transforming each text segment into an index number. Sequences shorter than the maximum text length are padded with zeros. For example, the longest length of the delivery message training data is used as its index sequence length, and the dictionary size is used as the data dimension for vector mapping.
[0156] Step A50: Obtain the model to be trained, and select the number of training samples from the interface layout topology training data and the constraint relationship data;
[0157] In this embodiment, a model to be trained is obtained, and a number of training samples are selected from the interface layout topology graph training data and the constraint relationship data. The training sample data includes at least one training sample, and each training sample includes an interface layout topology graph set from the interface layout topology graph training data.
[0158] Furthermore, the training sample data is divided into a training set and a test set. For example, 90% of the training sample data is allocated to the training set, and 10% is allocated to the test set. The training set is used to train the model, and the test set is used to test the model.
[0159] Step A60: Based on the training sample data, iteratively train the model to be trained to obtain the interface layout generation model.
[0160] In this embodiment, the model to be trained is iteratively trained based on the training sample data to obtain the interface layout generation model. Specifically, training samples are selected from the training sample data, and the interface layout topology atlas corresponding to the training samples is input into the model to be trained. Model prediction is performed to obtain the model output label. Then, the difference between the model output label and the constraint relationship prediction result corresponding to the training sample is calculated to obtain the model loss. Based on the model loss, the model to be trained is updated until the number of iterations of the model to be trained reaches the preset number of iterations.
[0161] It should be noted that the error between the predicted and actual constraints can be calculated, and the training objective is to minimize this error. The objective function chosen is the 'categorical_crossentropy' multi-class log loss function. The number of training epochs is set to 2000, and the Adam optimizer is selected to improve the learning speed of traditional gradient descent (optimizer='adam'). The neural network uses gradient descent to find the optimal weight values that minimize the objective function; the neural network learns these weight values autonomously through training. Training is performed on a training set to minimize the objective function, and the model is evaluated and validated using a test set after each training epoch. Once the model converges, its weights are derived.
[0162] In this embodiment, based on the interface layout generation model that has been built and trained, the constraint relationship of the interface layout topology is automatically predicted so that all basic interface element nodes of the interface to be laid out can be laid out based on the constraint relationship, thereby improving the efficiency of interface layout.
[0163] The present invention also provides an interface layout device.
[0164] In this embodiment, the interface layout device includes:
[0165] The acquisition module is used to acquire the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes.
[0166] The construction module is used to construct the interface layout topology diagram of the interface to be laid out based on the aforementioned basic interface element nodes.
[0167] The prediction module is used to predict the constraint relationships of the interface layout topology based on the trained interface layout generation model and the interface layout topology, and obtain the constraint relationship prediction results of the interface layout topology, so as to perform interface layout based on the constraint relationship prediction results.
[0168] Furthermore, the acquisition module includes:
[0169] The information acquisition unit is used to acquire the interface layout requirements information of the interface to be laid out;
[0170] The element prediction unit is used to predict the basic elements of the interface to be laid out based on the trained basic element generation model and the interface layout requirement information, so as to obtain the interface basic element sequence.
[0171] Furthermore, the basic element generation model includes an encoder and a decoder, and the element prediction unit includes:
[0172] The feature extraction subunit is used to extract features from the interface layout requirement information based on the encoder to obtain a context vector;
[0173] The element prediction subunit is used to predict the basic elements of the interface to be laid out based on the context vector and the decoder, so as to obtain the sequence of basic elements of the interface.
[0174] Furthermore, the basic element generation model also includes an embedding layer, and the feature extraction subunit is also used to serialize the interface layout requirement information into text to obtain indexed interface layout requirement information; to perform vector mapping on the indexed interface layout requirement information based on the embedding layer to obtain an interface layout requirement vector; and to extract features from the interface layout requirement vector based on the encoder to obtain a context vector.
[0175] Furthermore, the interface layout generation model includes several relational graph convolutional layers, and the prediction module includes:
[0176] The feature extraction unit is used to extract features from the interface layout topology graph based on the plurality of relation graph convolutional layers to obtain topological latent features;
[0177] Tensor decomposition unit is used to perform tensor factor decomposition operation based on the topological latent features and predict the scoring results of the pairwise node constraint relationship among the several interface basic element nodes.
[0178] The result determination unit is used to determine the constraint relationship prediction result of the interface layout topology diagram based on the scoring result.
[0179] Furthermore, the interface layout generation model also includes several fully connected layers and several discard layers, with each of the relation graph convolutional layers, each of the fully connected layers, and each of the discard layers arranged sequentially. The feature extraction unit is also used to extract features from the interface layout topology graph based on the several relation graph convolutional layers, the several fully connected layers, and the several discard layers to obtain topological latent features.
[0180] Furthermore, the interface layout device also includes:
[0181] The data acquisition module is used to acquire interface layout topology training data and to label the interface layout topology training data for constraint relationships to obtain constraint relationship data.
[0182] The sample selection module is used to obtain the model to be trained and select training sample data from the interface layout topology training data and the constraint relationship data.
[0183] The model training module is used to iteratively train the model to be trained based on the training sample data to obtain the interface layout generation model.
[0184] The functions of each module in the above-mentioned interface layout device correspond to the steps in the above-mentioned interface layout method embodiment, and their functions and implementation processes will not be described in detail here.
[0185] The present invention also provides a computer-readable storage medium storing an interface layout program, which, when executed by a processor, implements the steps of the interface layout method as described in any of the above embodiments.
[0186] The specific embodiments of the computer-readable storage medium of the present invention are basically the same as the embodiments of the interface layout method described above, and will not be repeated here.
[0187] The present invention also provides a computer program product, the computer program product comprising a computer program, wherein when the computer program product is executed by a processor, it implements the steps of the interface layout method as described in any of the above embodiments.
[0188] The specific embodiments of the computer program product of the present invention are basically the same as the embodiments of the interface layout method described above, and will not be repeated here.
[0189] It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or system that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or system. Unless otherwise specified, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or system that includes that element.
[0190] The sequence numbers of the above embodiments of the present invention are for descriptive purposes only and do not represent the superiority or inferiority of the embodiments.
[0191] Through the above description of the embodiments, those skilled in the art can clearly understand that the methods of the above embodiments can be implemented by means of software plus necessary general-purpose hardware platforms. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above, and includes several instructions to cause a terminal device (which may be a mobile phone, computer, server, or network device, etc.) to execute the methods described in the various embodiments of the present invention.
[0192] The above are merely preferred embodiments of the present invention and do not limit the scope of the patent. Any equivalent structural or procedural transformations made based on the description and drawings of the present invention, or direct or indirect applications in other related technical fields, are similarly included within the scope of patent protection of the present invention.
Claims
1. A method for interface layout, characterized in that, The interface layout method includes the following steps: Obtain the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes. Based on the aforementioned basic interface element nodes, construct the interface layout topology diagram of the interface to be laid out. Based on the trained interface layout generation model and the interface layout topology graph, constraint relationship prediction is performed on the interface layout topology graph to obtain the constraint relationship prediction result of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction result; wherein, the interface layout generation model is a relational graph convolutional network model; the constraint relationship in the constraint relationship prediction result is used to map the distance between interface basic elements, and then the interface basic elements are laid out based on the distance between interface basic elements.
2. The interface layout method as described in claim 1, characterized in that, The steps for obtaining the sequence of basic elements of the interface to be laid out include: Obtain the interface layout requirements information for the interface to be laid out; Based on the trained basic element generation model and the interface layout requirement information, the basic elements of the interface to be laid out are predicted to obtain the interface basic element sequence.
3. The interface layout method as described in claim 2, characterized in that, The basic element generation model includes an encoder and a decoder. The step of predicting the basic elements of the interface to be laid out based on the trained basic element generation model and the interface layout requirement information to obtain the sequence of basic elements includes: Based on the encoder, feature extraction is performed on the interface layout requirement information to obtain a context vector; Based on the context vector and the decoder, the basic elements of the interface to be laid out are predicted, and a sequence of basic elements of the interface is obtained.
4. The interface layout method as described in claim 3, characterized in that, The basic element generation model further includes an embedding layer. The step of extracting features from the interface layout requirement information based on the encoder to obtain a context vector includes: The interface layout requirement information is serialized into text to obtain indexed interface layout requirement information. Based on the embedding layer, the indexed interface layout requirement information is vector-mapped to obtain the interface layout requirement vector. Based on the encoder, feature extraction is performed on the interface layout requirement vector to obtain the context vector.
5. The interface layout method as described in claim 1, characterized in that, The interface layout generation model includes several relation graph convolutional layers. The step of predicting the constraint relationships of the interface layout topology graph based on the trained interface layout generation model and the interface layout topology graph, and obtaining the constraint relationship prediction results of the interface layout topology graph, includes: Based on the aforementioned several relational graph convolutional layers, feature extraction is performed on the interface layout topology graph to obtain topological latent features; Based on the aforementioned topological latent features, a tensor factorization operation is performed to predict the scoring results of the pairwise node constraint relationships among the aforementioned interface basic element nodes. Based on the scoring results, the constraint relationship prediction results of the interface layout topology diagram are determined.
6. The interface layout method as described in claim 5, characterized in that, The interface layout generation model further includes several fully connected layers and several discard layers, with each of the relation graph convolutional layers, each of the fully connected layers, and each of the discard layers arranged sequentially. The step of extracting features from the interface layout topology graph based on the several relation graph convolutional layers to obtain the topological latent features includes: Based on the aforementioned relational graph convolutional layers, fully connected layers, and discard layers, feature extraction is performed on the interface layout topology graph to obtain potential topological features.
7. The interface layout method as described in claim 1, characterized in that, Before the step of predicting the constraint relationships of the interface layout topology based on the trained interface layout generation model and the interface layout topology, and obtaining the constraint relationship prediction result of the interface layout topology, the method further includes: Obtain training data for the interface layout topology graph, and label the interface layout topology graph training data for constraint relationships to obtain constraint relationship data; Obtain the model to be trained, and select training sample data from the interface layout topology training data and the constraint relationship data; Based on the training sample data, the model to be trained is iteratively trained to obtain the interface layout generation model.
8. An interface layout device, characterized in that, The interface layout device includes: The acquisition module is used to acquire the sequence of basic elements of the interface to be laid out, wherein the sequence of basic elements includes several basic elements nodes. The construction module is used to construct the interface layout topology diagram of the interface to be laid out based on the aforementioned basic interface element nodes. The prediction module is used to predict the constraint relationships of the interface layout topology graph based on the trained interface layout generation model and the interface layout topology graph, and obtain the constraint relationship prediction results of the interface layout topology graph, so as to perform interface layout based on the constraint relationship prediction results; wherein, the interface layout generation model is a relational graph convolutional network model; the constraint relationships in the constraint relationship prediction results are used to map the distances between the basic elements of the interface, and then the basic elements of the interface are laid out based on the distances between the basic elements of the interface.
9. An interface layout device, characterized in that, The interface layout device includes: a memory, a processor, and an interface layout program stored in the memory and executable on the processor, wherein the interface layout program, when executed by the processor, implements the steps of the interface layout method as described in any one of claims 1 to 7.
10. A computer program product, characterized in that, The computer program product includes a computer program that, when executed by a processor, implements the steps of the interface layout method as described in any one of claims 1 to 7.