UI design scheme generation method and device

Abstract

The invention discloses a UI design scheme generation method and device, and the method comprises the steps: generating a user experience UE process based on a business demand document through a large language model LLM; generating a UI (User Interface) design scheme based on the UE process through AIGC (Artificial Intelligence Generation Content); creating a virtual user environment based on a UI design scheme through digital twinning, simulating an interactive behavior between a user and the UI, and collecting behavior pattern data of the user in the process of simulating the interactive behavior; performing emotion analysis based on the behavior mode data to obtain an emotion state of the user; and updating the UI design scheme based on the emotional state. According to the method, from demand interpretation to design generation to user interaction simulation and emotion analysis, full-process automation of UE design / UI design is realized.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to a method and device for generating a UI design scheme. Background Art

[0002] In today's rapidly developing digital age, the design of user experience (UE) and user interface (UI) is crucial to the success of software products. However, the traditional UE / UI design process usually relies on manual requirements analysis, design sketches, testing and optimization, which is time-consuming and labor-intensive. Summary of the invention

[0003] The embodiments of the present application provide a method and device for generating a UI design solution, which realizes the full process automation of UE design / UI design, from demand interpretation to design generation, and then to user interaction simulation and emotion analysis.

[0004] In a first aspect, a method for generating a UI design scheme is provided, the method comprising:

[0005] Generate user experience UE process based on business requirement document through large language model LLM;

[0006] Generate a user interface UI design scheme based on the UE process through artificial intelligence content generation AIGC;

[0007] Creating a virtual user environment based on the UI design solution through digital twins, simulating the interaction between the user and the UI, and collecting the user's behavior pattern data in the process of simulating the interaction;

[0008] Performing emotion analysis based on the behavior pattern data to obtain the emotional state of the user;

[0009] The UI design is updated based on the emotional state.

[0010] In some implementations, generating a user experience UE process based on a business requirement document using a large language model LLM includes:

[0011] Through LLM, user requirements, functional points and design indicators are extracted from the content of the business requirements document through the self-attention mechanism;

[0012] Based on the user requirements, the functional points and the design indicators, a UE flow chart and test cases are generated.

[0013] In some embodiments, the LLM is pre-trained by performing a masked language model task and a next sentence prediction task based on text data;

[0014] In the process of performing the masked language model task, some words in the input text data of the LLM are randomly masked, and the LLM predicts the original content of the masked words;

[0015] In the process of performing the next sentence prediction task, the LLM determines whether two sentences appear adjacent to each other in the original text.

[0016] In some implementations, the generating of the user interface UI design scheme based on the UE process by artificial intelligence content generation AIGC includes:

[0017] Generate a user interface UI design scheme based on the UE process through a generative adversarial network GAN, wherein the GAN includes a generator and a discriminator, wherein the generator is used to generate the UI design scheme, and the discriminator is used to discriminate between the generated UI design scheme and the real UI design scheme; or,

[0018] Encode the UE process into a latent space through a variational autoencoder VAE, and decode from the latent space to generate a new layout or style; or,

[0019] In the process of performing the conditional generation task, a UI design scheme is generated based on a predetermined color scheme or design style through the model.

[0020] In some implementations, the generating of the user interface UI design scheme based on the UE process by artificial intelligence content generation AIGC includes:

[0021] Based on the requirements of the UE process, set the generation parameters of AIGC, the generation parameters including color scheme, font type and layout style;

[0022] Generate an initial UI design scheme based on the generation parameters and the design knowledge learned from the training data, wherein the initial UI design scheme includes layout, elements and interaction methods;

[0023] Presenting an initial UI design solution to a user, and receiving evaluation indicators input by the user for the initial UI design solution, wherein the evaluation indicators include aesthetics, usability, and compliance;

[0024] The generation parameters of the AIGC are adjusted based on the evaluation index, and the UI design scheme is regenerated based on the AIGC with the adjusted generation parameters, until the evaluation index corresponding to the generated UI design scheme meets the user's expectations.

[0025] In some embodiments, the creating a virtual user environment based on the UI design solution through a digital twin to simulate the interaction between the user and the UI includes:

[0026] The state data of the physical entity is collected in real time by sensors installed on the physical entity, wherein the state data includes temperature, pressure and speed;

[0027] Transmitting the collected state data to a cloud or a local server via a network, and synchronizing the state data with a digital twin model;

[0028] creating a digital replica of the physical entity in the virtual user environment based on the state data, the digital replica including geometric structure, material properties, and functional characteristics;

[0029] The digital twin model simulates the interaction process between the user and the physical entity based on the digital copy of the physical entity.

[0030] In some implementations, performing emotion analysis based on the behavior pattern data to obtain the user's emotional state includes:

[0031] Extracting target behavior pattern data that has an impact on sentiment analysis from the behavior pattern data, the target behavior pattern data including operation frequency, response time, task completion rate, and number of errors;

[0032] Encoding the target behavior pattern data to obtain behavior characteristics;

[0033] The behavior features are input into a random forest model, emotion classification results are output for the behavior features through each decision tree in the random forest model, and the emotional state of the user is determined based on the emotion classification results of each decision tree.

[0034] In some implementations, performing emotion analysis based on the behavior pattern data to obtain the user's emotional state includes:

[0035] converting the behavior pattern data into a time series;

[0036] Extracting emotional features from the time series through an LSTM layer in a long short-term memory network LSTM model, and obtaining temporal dependencies between operations from the time series through a gating mechanism of the LSTM model;

[0037] Based on the emotional characteristics and the time dependency, the emotional state of the user at a target time point is determined.

[0038] In a second aspect, a UI design scheme generating device is provided, the device comprising:

[0039] Large Language Model (LLM) unit, used to generate user experience (UE) flow based on business requirement documents through LLM;

[0040] An artificial intelligence generated content AIGC unit, used to generate a user interface UI design scheme based on the UE process through AIGC;

[0041] A digital twin unit, used to create a virtual user environment based on the UI design solution through a digital twin, simulate the interaction between the user and the UI, and collect the user's behavior pattern data in the process of simulating the interaction behavior;

[0042] An emotion analysis unit, configured to perform emotion analysis based on the behavior pattern data to obtain the emotional state of the user;

[0043] An updating unit is used to update the UI design scheme based on the emotional state.

[0044] In some embodiments, the LLM unit is used to extract user requirements, function points and design indicators from the content of the business requirement document through the LLM based on the self-attention mechanism; and generate a UE flowchart and test cases based on the user requirements, the function points and the design indicators.

[0045] In some embodiments, the LLM is pre-trained by performing a masked language model task and a next sentence prediction task based on text data; in the process of performing the masked language model task, some words in the input text data of the LLM are randomly masked, and the LLM predicts the original content of the masked words; in the process of performing the next sentence prediction task, the LLM determines whether two sentences appear adjacent to each other in the original text.

[0046] In some embodiments, the AIGC unit is used to generate a user interface UI design scheme based on the UE process through a generative adversarial network GAN, the GAN includes a generator and a discriminator, the generator is used to generate a UI design scheme, and the discriminator is used to discriminate between the generated UI design scheme and the real UI design scheme; or, encode the UE process into a latent space through a variational autoencoder VAE, and decode from the latent space to generate a new layout or style; or, in the process of performing a conditional generation task, generate a UI design scheme based on a predetermined color scheme or design style through a model.

[0047] In some embodiments, the AIGC unit is used to set the generation parameters of the AIGC based on the requirements of the UE process, and the generation parameters include a color scheme, a font type, and a layout style; generate an initial UI design scheme based on the generation parameters and the design knowledge learned from the training data, and the initial UI design scheme includes a layout, elements, and an interaction method; present the initial UI design scheme to the user, and receive evaluation indicators input by the user for the initial UI design scheme, and the evaluation indicators include aesthetics, usability, and compliance; adjust the generation parameters of the AIGC based on the evaluation indicators, and regenerate the UI design scheme based on the AIGC with adjusted generation parameters until the evaluation indicators corresponding to the generated UI design scheme meet the user's expectations.

[0048] In some embodiments, the digital twin unit is used to collect status data of the physical entity in real time through sensors installed on the physical entity, and the status data includes temperature, pressure and speed; transmit the collected status data to the cloud or local server through the network, and synchronize the status data with the digital twin model; create a digital copy of the physical entity in the virtual user environment based on the status data, and the digital copy includes geometric structure, material properties and functional characteristics; simulate the interaction process between the user and the physical entity based on the digital copy of the physical entity through the digital twin model.

[0049] In some embodiments, the emotion analysis unit is used to extract target behavior pattern data that has an impact on emotion analysis from the behavior pattern data, wherein the target behavior pattern data includes operation frequency, response time, task completion rate, and number of errors; encode the target behavior pattern data to obtain behavior characteristics; input the behavior characteristics into a random forest model, output emotion classification results for the behavior characteristics through each decision tree in the random forest model, and determine the user's emotional state based on the emotion classification results of each decision tree.

[0050] In some embodiments, the emotion analysis unit is used to convert the behavior pattern data into a time series; extract emotion features from the time series through the LSTM layer in a long short-term memory network LSTM model, and obtain the time dependency between operations from the time series through the gating mechanism of the LSTM model; and determine the emotional state of the user at a target time point based on the emotion features and the time dependency.

[0051] The embodiment of the present application proposes an automated UE / UI design solution generation method that integrates a large language model (LLM), artificial intelligence generated content (AIGC), digital twin technology, and sentiment analysis. The business requirement document is automatically interpreted by LLM to generate the UE process; the UI design solution is automatically generated according to the UE process using AIGC technology; the real user is simulated to perform interactive operations through digital twin technology, and the user's behavior pattern data is collected; finally, the user's emotional state is analyzed using emotional computing technology, and the UI design solution is updated based on the user's emotional state. The embodiment of the present application realizes the automation of the entire process of UE / UI design, from requirement interpretation to design generation, and then to user interaction simulation and sentiment analysis. The automated process significantly reduces manual participation, shortens the design cycle, and improves efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0052] Figure 1 This is a flowchart of a UE generation process through LLM provided by an embodiment of the present application;

[0053] Figure 2 It is a flow chart of a method for generating a UI design solution based on AIGC provided in an embodiment of the present application;

[0054] Figure 3 It is a flow chart of a method for simulating interaction through digital twins provided in an embodiment of the present application;

[0055] Figure 4 is a flow chart of a method for determining a user's emotional state through a random forest model provided in an embodiment of the present application;

[0056] Figure 5 This is a flow chart of a method for determining a user's emotional state through an LSTM model provided in an embodiment of the present application;

[0057] Figure 6 It is a collaborative flow chart of LLM, AIGC, digital twin and affective computing provided in an embodiment of the present application;

[0058] Figure 7 It is a data processing flow chart provided in an embodiment of the present application. DETAILED DESCRIPTION

[0059] In order to make the objectives, technical solutions and advantages of the present application clearer, the implementation methods of the present application will be further described in detail below with reference to the accompanying drawings.

[0060] The following is an explanation of some terminology concepts involved in the embodiments of the present application.

[0061] In today's rapidly developing digital age, the design of user experience (UE) and user interface (UI) is crucial to the success of software products. However, the traditional UE / UI design process usually relies on manual demand analysis, design sketches, testing and optimization. This process is not only time-consuming and labor-intensive, but also easily affected by subjective bias, resulting in a deviation between the design results and the actual needs of users. In addition, with the diversification of user groups and the continuous evolution of needs, how to quickly respond to market changes and provide UE / UI designs that meet users' emotional and psychological needs has become a major challenge facing the design field.

[0062] At present, although there are some automated testing tools and prototyping software on the market, they often focus on the implementation of a single function and lack a deep understanding of user emotions and behaviors. In addition, these tools have limited capabilities in handling complex interactions and sentiment analysis, and it is difficult to fully capture and simulate the user experience of real users. Therefore, existing technologies still have significant deficiencies in improving design efficiency, accuracy, and user satisfaction.

[0063] In view of this, the embodiment of the present application proposes an automated UE / UI design and testing solution that integrates a large language model (LLM), artificial intelligence generated content (AIGC), digital twin technology, and emotional computing. The solution automatically interprets business requirements through LLM and generates UE processes; uses AIGC technology to automatically generate UI design solutions based on UE processes; uses digital twin technology to simulate real users to perform interactive operations and collect user behavior pattern data; finally, uses emotional computing technology to analyze the user's emotional state, identifies potential problems in the design based on the user's emotional state, and updates and optimizes the generated UI design solutions based on this.

[0064] Compared with the prior art, the effects achieved by the embodiments of the present application include the following aspects.

[0065] First, full automation has been achieved. From demand interpretation to design generation, to user interaction simulation and sentiment analysis, the entire process of UE / UI design has been automated. The automated process significantly reduces manual involvement, shortens the design cycle, and improves efficiency.

[0066] Second, deep learning and sentiment analysis. By combining LLM and sentiment computing technology, not only the accuracy of the design is improved, but also the user's emotions can be deeply understood, thereby optimizing the design to improve user satisfaction, and reducing the cost of design and development by reducing manual testing and iteration.

[0067] Third, real-time feedback and iterative optimization: Digital twin technology enables UI design solutions to be tested and optimized in real time in a virtual environment, quickly responding to market changes and user needs. Through accurate user sentiment analysis, UE / UI that better meets user psychological needs can be designed to enhance user experience.

[0068] Fourth, highly personalized. AIGC technology can generate personalized UI design solutions based on user feedback to meet the needs of different user groups. Real-time interaction simulation and sentiment analysis can help discover and solve problems in the design in advance and improve product quality.

[0069] The following is an explanation of some terminology concepts involved in the embodiments of the present application.

[0070] Large Language Model (LLM): LLM is an artificial intelligence model based on deep learning, which is specially used to process and generate natural language text. LLM learns the patterns and structures of language from large-scale text data through unsupervised learning, and can perform complex tasks such as text analysis, sentiment analysis, language translation and speech recognition.

[0071] Artificial Intelligence Generated Content (AIGC): AIGC refers to the process of automatically creating content such as text, images, audio and video using artificial intelligence technology, especially algorithms in fields such as natural language processing and computer vision. AIGC technology can generate new and original content based on given inputs or parameters, and can be used in a variety of application scenarios, such as content creation, design prototype generation, etc.

[0072] Digital twin technology is a technology that creates a virtual copy of a physical entity and uses sensor data and model analysis to simulate the behavior and performance of the entity in the real world. Digital twins can be used in many fields such as product design, production process optimization, equipment maintenance prediction, etc., and improve the accuracy and efficiency of decision-making through real-time data and simulation analysis.

[0073] Affective computing is a branch of artificial intelligence that involves developing systems and devices that can recognize, interpret, process and simulate human emotions. Affective computing technology understands and responds to the user's emotional state by analyzing speech, facial expressions, text and other physiological or behavioral signals.

[0074] User Experience (UE) refers to the overall feelings and evaluations formed by users during the use of products, systems or services. UE design focuses on improving user satisfaction and efficiency, and enables users to use products more conveniently and happily by optimizing interaction processes, interface design and functional layout.

[0075] User Interface (UI) is the medium for interaction between users and computer systems or software, including layout, visual elements, buttons, icons and other design elements. The goal of UI design is to create an intuitive, easy-to-use and beautiful interface to improve the user's interactive experience.

[0076] In the field of user experience (UE) and user interface (UI) design, the solutions commonly used in the industry mainly include the following aspects:

[0077] First, manual design and testing: Designers manually create UE / UI design prototypes based on requirements and experience, and collect feedback and perform iterative optimization through user testing.

[0078] Second, automated design tools: Use a variety of design software and platforms to provide templates and component libraries to speed up the design process.

[0079] Third, user feedback collection: Collect user feedback through questionnaires, user interviews, focus groups, etc. to guide design improvements.

[0080] Fourth, data analysis: Use data analysis tools, heat map analysis, etc. to quantify user behavior and preferences.

[0081] Although the above methods have improved design efficiency and quality to a certain extent, they still have the following problems and limitations:

[0082] First, it is highly subjective: the manual design process is easily influenced by the designer’s personal experience and preferences, and may not fully meet user needs.

[0083] Second, limited efficiency: Even with the assistance of automated tools, the design and testing process is still time-consuming, especially for complex interactions and diverse user groups.

[0084] Third, insufficient data utilization: Traditional user feedback collection and data analysis methods often lack depth and find it difficult to accurately capture user emotions and behavioral motivations.

[0085] Fourth, the iteration cycle is long: the entire iteration cycle from design to testing to optimization is long, making it difficult to quickly respond to market changes and user needs.

[0086] In view of this, the solution proposed in the embodiments of the present application aims to solve the above-mentioned problems and limitations by integrating multiple technologies.

[0087] First, use the large language model (LLM) to automatically interpret business needs: Through the powerful natural language processing capabilities of LLM, reduce dependence on manual interpretation and improve the accuracy and efficiency of demand analysis.

[0088] Second, use artificial intelligence generated content (AIGC) technology to automatically generate UI design solutions: According to the UE process interpreted by LLM, AIGC can quickly generate diverse UI design solutions and accelerate the design iteration process.

[0089] Third, use digital twin technology to simulate user interactions: create a virtual user environment, simulate the operations and interaction behaviors between real users and UI, and provide more comprehensive and in-depth test data.

[0090] Fourth, combine emotional computing to conduct user sentiment analysis: By analyzing the user’s behavior and reactions in the digital twin environment, identify problem points in the design that may cause negative emotions and make optimization suggestions.

[0091] The embodiment of the present application includes the following steps.

[0092] Step 1: Requirements interpretation: Analyze the business requirements document through LLM to extract key information and design indicators.

[0093] Step 2: UE process generation: Automatically generate UE flow charts and test cases based on the key information and design indicators extracted by LLM.

[0094] Step 3: UI design generation: Use AIGC technology to automatically generate multiple UI design solutions based on the UE process.

[0095] Step 4: User interaction simulation: Create a virtual user environment through digital twin technology, simulate the interaction behavior between real users and UI, and collect user behavior pattern data in the process of simulating interaction behavior.

[0096] Step 5: Emotional analysis and optimization: Use sentiment analysis technology to analyze the user's emotional state, identify and solve potential problems in the design based on the user's emotional state, and make optimization suggestions.

[0097] Step 6: Iterative optimization: According to the feedback from the sentiment analysis, the UE / UI design solution is iteratively optimized until the emotional state of the user's feedback on the UE / UI design solution meets the sentiment index.

[0098] The technical solution provided in the embodiments of the present application can effectively overcome the limitations of existing solutions and provide a more efficient, accurate and user-centric design process, thereby significantly improving the overall quality and user satisfaction of UE / UI design solutions.

[0099] The embodiments of the present application can closely combine multiple links such as demand interpretation, design generation, user interaction simulation, emotion analysis and iterative optimization, and integrate technologies such as automated testing tools, AI-based design assistance tools, virtual users and simulation testing, emotion recognition technology and digital twin platforms to form a closed-loop automated UE / UI design and testing process, which can understand user needs more comprehensively and deeply, and provide more accurate and personalized design optimization suggestions.

[0100] The following is an analysis and explanation of some technical problems related to this application.

[0101] As for automated testing tools, since they mainly focus on functional testing and do not involve in-depth analysis of user experience, these tools cannot fully capture user emotions and behavioral motivations, resulting in inaccurate design optimization. However, the embodiments of the present application can more deeply understand user needs and emotional reactions by integrating large language models and emotional computing technology, thereby providing more accurate design optimization suggestions.

[0102] For AI-based design assistance tools, because existing AI design assistance tools usually only provide design suggestions, but do not involve the collection and analysis of user interaction and emotional feedback. Therefore, these tools cannot ensure that the design solution can meet the actual emotional needs of users, and there may be deficiencies in user experience. However, the embodiment of the present application simulates real user interaction through digital twin technology, and combines emotional computing for emotional analysis to ensure that the design solution can better meet the emotional needs of users.

[0103] For virtual users and simulation tests, because virtual users and simulation test platforms mainly rely on data recording or simulation of real users, they lack personalized and automated design iteration processes. Therefore, these platforms have limitations in quickly responding to design changes and optimizations, and may not be able to adapt to changes in market and user needs in a timely manner. However, the embodiments of the present application can quickly generate and iterate UI design solutions through automated UE process generation and AIGC technology, thereby achieving rapid response and adaptation to market changes.

[0104] Regarding emotion recognition technology, since emotion recognition technology is usually independent of the design and testing process and is not directly integrated with UE / UI design tools, these technologies are insufficient in converting emotion analysis results into specific design improvement measures. The embodiment of the present application closely combines emotion computing technology with the UE / UI design process to ensure that the results of emotion analysis can directly guide the iteration and optimization of the design.

[0105] With regard to digital twin platforms, because existing digital twin platforms are mainly used in the fields of industry and urban planning, insufficient consideration is given to user experience and interface design. Therefore, these platforms may not be sophisticated and comprehensive enough in simulating user interactions and collecting user experience data. The embodiments of this application combine digital twin technology and emotional computing to optimize UE / UI design and provide more sophisticated and comprehensive user interaction simulation and emotional analysis.

[0106] The embodiment of the present application aims to realize an automated process from business requirement interpretation to user experience (UE) and user interface (UI) design through an integrated framework. The framework integrates multiple technical components such as large language model (LLM), artificial intelligence generated content (AIGC), digital twin technology and emotional computing to form a closed-loop design and testing system. The overall structure of the embodiment of the present application includes five main links: demand analysis, design generation, user interaction simulation, emotion analysis and iterative optimization. The embodiment of the present application interprets business requirements through LLM, generates design solutions using AIGC, simulates user interactions through digital twins, uses emotional computing for emotion analysis, and performs design iterations based on the analysis results.

[0107] The following is an explanation of the functions and execution method flows of some components involved in the embodiments of the present application.

[0108] (1) Large Language Model (LLM)

[0109] The core working mechanism of the Large Language Model (LLM) is natural language processing technology based on deep learning. These models usually adopt the Transformer architecture, which processes the input text data through the Self-Attention Mechanism. The Self-Attention Mechanism enables the model to capture the dependencies between different parts of the text sequence, and effectively interact and integrate information no matter how far apart these parts are.

[0110] In an unsupervised learning environment, LLM is pre-trained with a large amount of text data. The text data used for pre-training includes various types of texts such as books, articles, and conversation records. In this way, LLM learns the common patterns and structures of language, including grammatical rules, vocabulary usage, and contextual relationships.

[0111] The implementation of LLM involves large-scale neural network models that contain billions or even trillions of parameters. These parameters are constantly adjusted during the training process to minimize the error in predicting the next word or phrase. In the pre-training stage, LLM is mainly used to perform two tasks: masked language model and next sentence prediction.

[0112] In the process of performing the masked language model task, some words in the LLM input text will be randomly masked, and LLM needs to predict the original content of these masked words. This helps LLM learn the contextual information and language structure of the words. In the process of performing the next sentence prediction task, LLM needs to determine whether two sentences appear adjacent to each other in the original text, which helps the model understand the relationship between sentences.

[0113] In the embodiment of the present application, the role of LLM is to serve as the starting point of the entire automated UE / UI design process. For example, please refer to the attached Figure 1 , attached Figure 1 The process of generating a UE process through LLM is shown, and the process of generating a UE process through LLM includes the following steps.

[0114] Step 1: Requirement document analysis: First, LLM receives the business requirement document as input, which contains product description, user stories, market analysis, etc.

[0115] Step 2: Extract key information: LLM analyzes the content of the business requirement document through the self-attention mechanism and extracts user requirements, functional points and design indicators from the content of the business requirement document.

[0116] Step 3, UE process generation: Based on the extracted user requirements, function points and design indicators, LLM generates preliminary UE flow charts and test cases. This step involves the prediction of user behavior, planning of task processes and arrangement of interaction steps.

[0117] Step 4: Output result integration: The UE flowchart and test cases output by LLM are used as input of AIGC to provide a basis for the next step of UI design solution generation.

[0118] In this way, LLM not only improves the accuracy and efficiency of demand interpretation, but also provides a reliable basis for the generation and testing of subsequent UI design solutions. In addition, the UE process generated by LLM can also be fine-tuned and optimized according to the specific needs of the project to ensure that the final design results can better meet the actual needs and emotional expectations of users.

[0119] (2) Artificial Intelligence Generated Content (AIGC)

[0120] The core of AIGC technology is to use machine learning models, especially deep learning models, to automatically create new text, images, audio and other content. AIGC technology usually relies on a large number of data sets for training. Through these data sets, the model learns how to generate output content that meets the requirements based on specific inputs and parameters.

[0121] In the field of UI design, AIGC technology can use existing design patterns, styles, and best practices of user interaction to train models. These models can understand the constituent elements of design, such as layout, color, fonts, and icons, and how they are combined to create a user interface that is both beautiful and practical.

[0122] When AIGC technology is used to implement UI design, the following implementation methods are used.

[0123] Implementation method 1: Generative Adversarial Network (GAN): GAN consists of two parts, the generator and the discriminator. The generator is responsible for creating new UI design solutions, while the discriminator tries to distinguish between the generated UI design solutions and the real UI design solutions. Through this adversarial process, the generator continuously improves the quality of the content of the UI design solutions it generates.

[0124] Implementation method 2: Variational Autoencoder (VAE): VAE is a generative model that encodes input data into a latent space and then decodes it from this latent space to generate new data. In UI design, VAE can be used to generate new layouts or styles.

[0125] Implementation method 3: Conditional generation: When performing conditional generation tasks, the model generates UI design solutions based on specific conditions or attributes. For example, a UI design solution can be generated based on a color scheme or design style specified by the user.

[0126] In the application process of the embodiment of the present application, AIGC plays a key role in converting the UE process generated by LLM into a specific UI design solution. Figure 2 , attached Figure 2 A flow chart of a method for generating a UI design scheme based on AIGC is shown. The specific process of generating a UI design scheme based on AIGC includes the following steps.

[0127] Step 1: Receive UE process: AIGC receives UE flow charts and test cases from LLM. These documents include user interaction steps, functional requirements, and design indicators.

[0128] Step 2: Parameter setting: According to the requirements of the UE process, set the generation parameters of AIGC, including the specified color scheme, font type, and layout style.

[0129] Step 3: UI design scheme generation: AIGC automatically generates a series of initial UI design schemes based on the set generation parameters and the design knowledge learned from the training data. The initial UI design schemes can include different layouts, elements, and interaction methods.

[0130] Step 4: Scheme evaluation: The generated initial UI design scheme is then submitted to real users (such as designers or stakeholders) for evaluation. After the initial UI design scheme is presented to the user, the evaluation indicators input by the user for the initial UI design scheme are received. The evaluation indicators may include aesthetics, usability, and compliance.

[0131] Step 5, iterative optimization: According to the evaluation indicators fed back by users for the initial UI design solution, AIGC can further adjust the generation parameters and regenerate the UI design solution based on the AIGC with adjusted generation parameters until the evaluation indicators corresponding to the generated UI design solution meet the user's expectations.

[0132] Through AIGC technology, the embodiments of the present application can greatly improve the efficiency and innovation of UI design, while reducing the reliance on manual UI design solutions. This method can not only speed up the design process, but also create a variety of UI design solutions, providing end users with a richer and more personalized experience.

[0133] (3) Digital Twin Technology

[0134] Digital twin technology is a technology that achieves real-time data synchronization and behavior simulation by creating a virtual copy of a physical entity. This technology uses sensors to collect real-time data from physical entities and transmits this data to their digital copies, thereby replicating and simulating the state and behavior of physical entities in a virtual environment. Digital twins can capture many aspects of physical entities, including appearance, performance, operating environment, etc., as well as various conditions and influencing factors that change over time.

[0135] Please refer to the attached Figure 3 , attached Figure 3 A flow chart of the method performed by digital twins is shown, and the implementation of digital twin technology includes the following steps.

[0136] Step 1: Data collection: The status data of the physical entity is collected in real time through sensors installed on the physical entity. The status data of the physical entity includes temperature, pressure, speed, etc.

[0137] Step 2: Data synchronization: The collected status data of the physical entity is transmitted to the cloud or local server through the network, and the status data of the physical entity is synchronized with the digital twin model.

[0138] Step 3: Model building: Create an accurate digital copy of the physical entity in the virtual user environment based on the state data of the physical entity. The digital copy includes geometric structure, material properties, functional characteristics, etc.

[0139] Step 4: Simulation analysis: Use simulation algorithms, such as finite element analysis (FEA) and computational fluid dynamics (CFD), to simulate and analyze the digital twin and predict its performance under different conditions.

[0140] Step 5: Interaction simulation: Use the digital twin model to simulate the interaction process between the user and the physical entity based on the digital copy of the physical entity, such as operation methods, usage scenarios, etc., to verify the feasibility of the design and user experience.

[0141] In the application process of the embodiment of the present application, the digital twin technology has the following functions.

[0142] First, simulate user interaction: Use the digital twin model to simulate the interactive operations between real users and the UI, such as clicking, sliding, and inputting.

[0143] Second, data collection: During the simulated interaction process, collect user behavior pattern data, which includes operation sequence, reaction time, error rate, etc.

[0144] Third, behavioral analysis: analyze user behavior pattern data, identify user behavior patterns and potential problem points based on user behavior pattern data, and provide data support for sentiment analysis.

[0145] Fourth, the basis for sentiment analysis: input the collected data into the sentiment computing model to analyze the user's emotional response, such as satisfaction, frustration, etc.

[0146] Fifth, design optimization feedback: Based on the results of sentiment analysis, provide optimization suggestions for UI / UE design, such as improving layout, adjusting functions, etc.

[0147] Through digital twin technology, the embodiments of the present application can simulate the real-world user experience in a virtual environment, provide a scientific basis for design iteration, and ensure that the final product can meet the user's actual needs and emotional expectations.

[0148] (4) Affective computing

[0149] In the embodiments of the present application, the algorithms, models and working principles involved in emotional computing are to analyze and infer the user's emotional state from the user's behavior pattern data. There are many algorithms and models for implementing emotional computing. The following is an example of an algorithm and model that can implement emotional computing.

[0150] A. Statistical analysis: Use descriptive statistics to analyze the basic characteristics of users’ behavior pattern data, such as average response time, task success rate, operation frequency, etc.

[0151] B. Machine learning classifier: Build and train a machine learning model. The embodiment of the present application adopts a random forest algorithm to classify emotional states based on the user's behavior pattern data.

[0152] C. Deep learning network: Using deep learning technology, the embodiment of the present application adopts a long short-term memory network (LSTM) to process serialized user behavior pattern data and identify complex emotional states.

[0153] D. Natural Language Processing (NLP): Although tone is not involved in the digital twin simulation environment, the user's text input (such as search queries) can be sentimentally analyzed through NLP technology to identify emotional tendencies.

[0154] E. Sequence model: For the sequence data of user operations, the embodiment of the present application adopts a sequence model long short-term memory network (LSTM) to capture the temporal dependency between operations.

[0155] The embodiment of the present application uses machine learning algorithms and techniques to deeply understand and analyze the user's behavior pattern data, so as to accurately infer the user's emotional state. Specifically, a random forest and a long short-term memory network (LSTM) model are combined to achieve this goal.

[0156] (4.1) Random Forest Model

[0157] In the embodiment of the present application, the application of the random forest model is to accurately predict the emotional state of the digital twin user from the behavior pattern data of the digital twin user. This method takes advantage of ensemble learning to improve the accuracy and robustness of emotion classification by constructing multiple decision trees and combining the prediction results of multiple decision trees. The following is the detailed principle and workflow of the application of the random forest model in the embodiment of the present application:

[0158] The random forest model is an ensemble learning method based on decision trees. Each decision tree is an independent classifier, and the prediction results output by each decision tree are voted or averaged to determine the final prediction result. The advantage of this method is that it can reduce the risk of overfitting of the model and improve its resistance to noisy data.

[0159] Please refer to the attached Figure 4 , attached Figure 4 A flow chart of a method for determining a user's emotional state through a random forest model is shown, and the method includes the following steps.

[0160] Step 1: Feature extraction.

[0161] Data preparation: Collect user behavior pattern data from the digital twin environment. User behavior pattern data is generated by simulating user interaction with the UI.

[0162] Key feature identification: Analyze the behavior patterns of digital twin users based on user behavior pattern data, and extract target behavior pattern data that has a significant impact on emotion classification from the user behavior pattern data. The target behavior pattern data includes operation frequency, response time, task completion rate, and number of errors.

[0163] Feature encoding: Encode the extracted target behavior pattern data to obtain the behavior characteristics of the digital twin user. Encoding methods include normalization or one-hot encoding, which enables the model to better understand the behavior characteristics.

[0164] Step 2: Model training.

[0165] Decision Tree Construction: The random forest model is trained by building multiple decision trees. Each decision tree is trained on a random subset of the dataset, and each split point is selected from a random subset of the features.

[0166] Splitting rule optimization: Each decision tree tries to find the best splitting rule, which is chosen based on information gain, Gini impurity, or other criteria to distinguish between different emotional states.

[0167] Model fusion: After training, random forest improves the overall prediction accuracy by integrating the prediction results of all decision trees.

[0168] Step 3: Emotion classification:

[0169] Prediction execution: Encode and process the newly collected behavior pattern data of digital twin users to obtain the behavior characteristics of digital twin users. Input the behavior characteristics of digital twin users into the trained random forest model, and use the random forest model to classify emotions based on the prediction results of each decision tree.

[0170] Results summary: Since each decision tree gives an emotion classification result, the random forest model determines the final emotion state prediction result based on the emotion classification results of each decision tree by majority voting or average score.

[0171] Emotional state output: The model outputs the final emotional state, such as happiness, frustration, anxiety, etc., to provide a basis for optimizing UI design solutions.

[0172] Through this random forest model-based method, the embodiment of the present application can accurately predict the emotional state from the behavior pattern data of the digital twin user, and can effectively analyze even in the absence of direct emotional expression. This method provides a powerful tool for design teams to help them understand the emotional experience of users when interacting with products, and optimize product design and improve user experience accordingly.

[0173] (4.2) LSTM model

[0174] In the embodiment of the present application, the application of the long short-term memory network (LSTM) model is to deeply analyze the behavior pattern time series of the digital twin user and identify emotion-related patterns from the behavior pattern time series of the digital twin user. The characteristic of the LSTM model is that it can process and remember the long-term dependencies in the behavior pattern time series, which is very important for understanding and predicting the emotional state of the user. The following is the detailed principle and workflow of the LSTM model in the embodiment of the present application:

[0175] The LSTM model is a special recurrent neural network (RNN) that solves the gradient vanishing or exploding problem of traditional RNN when processing long sequences by introducing a gating mechanism. These gating mechanisms include forget gates, input gates, and output gates, which work together to determine the retention, update, and output of information, enabling LSTM to learn long-term dependencies in behavioral pattern time series.

[0176] Please refer to the attached Figure 5 , attached Figure 5 A flow chart of a method for determining a user's emotional state through an LSTM model is shown, and the method includes the following steps.

[0177] Step 1: Sequence Data Processing:

[0178] Data conversion: Convert the behavior pattern data of digital twin users into a time series format suitable for LSTM processing. For example, convert the user’s click events and corresponding timestamps into a time series.

[0179] Sequence construction: Construct time series and ensure that each time series contains time order information, which is crucial for capturing the dynamic changes of user behavior.

[0180] Network structure design for LSTM model.

[0181] Layer setting: Design the layer structure of the LSTM network. The LSTM network includes an input layer, one or more LSTM layers, and an output layer.

[0182] Hyperparameter configuration: Set the hyperparameters of the LSTM model, such as the number of units in the hidden layer, learning rate, and batch size.

[0183] Step 2: Emotion Pattern Recognition:

[0184] Pattern learning: The LSTM model learns patterns in the time series through training, which are related to the user’s emotional state.

[0185] Feature extraction: The LSTM layer can extract emotional features from time series, and emotional features are used for emotion classification.

[0186] Step 3: Capturing Temporal Dependencies:

[0187] Gating mechanism: The gating mechanism of LSTM allows the model to remember or forget information, which enables the model to effectively capture the temporal dependencies between users’ operations.

[0188] Emotional state inference: By analyzing temporal dependencies, the LSTM model is able to infer the user’s emotional state at a specific point in time.

[0189] Step 4: Model Training and Prediction:

[0190] Training execution: Use the behavior sequence of the digital twin user to train the LSTM model until the LSTM model converges.

[0191] Emotion prediction: After training, the LSTM model can predict the emotional state of new user behavior sequence data.

[0192] Step 5: Design Optimization:

[0193] Feedback integration: Integrate the prediction results of the LSTM model into the product design process to identify and improve design elements that may cause negative user emotions.

[0194] Iterative improvement: Continuously iterate the UI design based on the prediction results to improve the user's overall emotional experience.

[0195] By combining digital twin technology and LSTM model, the embodiment of the present application can deeply analyze the time series data of user behavior, identify emotional patterns, and optimize the UI design scheme accordingly. This method not only improves the accuracy of emotional analysis, but also provides a powerful tool for the design team to create products that better meet the emotional needs of users, thereby improving the overall user experience and satisfaction.

[0196] In the technical solution of the embodiment of the present application, the division of labor and collaboration mechanism between key technical components is the basis for ensuring the efficient operation of the entire system. The following is the detailed division of labor, boundaries and collaboration process of key components:

[0197] Large Language Model (LLM): Responsible for interpreting business requirement documents and user feedback. Extract key information from business requirement documents and generate preliminary user experience (UE) process design. Boundary: Focus on natural language processing and text analysis to output structured UE processes.

[0198] Artificial Intelligence Generated Content (AIGC): Generates multiple user interface (UI) design solutions based on the UE process provided by LLM. Boundary: Focuses on creative content generation and the diversity of design solutions, and outputs visual design elements.

[0199] Digital twin technology: simulates real user interaction behaviors and collects interaction data. Boundary: focuses on creating virtual environments and simulating user operations, and outputs interaction behavior pattern data.

[0200] Emotional computing: Analyze the interactive data collected by the digital twin and identify the user's emotional state. Boundary: Focus on emotion recognition and analysis, and output emotion analysis reports and optimization suggestions.

[0201] Please refer to the attached Figure 6 , attached Figure 6 The collaborative process of LLM, AIGC, digital twins, and affective computing is shown, and the collaborative process includes the following steps.

[0202] Step 1: LLM receives the business requirement document and extracts requirements and indicators from the business requirement document through natural language processing technology. LLM generates UE flow charts and test cases to provide a basis for subsequent design.

[0203] Step 2: Design solution generation: AIGC receives the UE process output by LLM as input.

[0204] AIGC automatically generates a variety of UI design solutions based on UE processes and design parameters.

[0205] Step 3: User interaction simulation: Digital twin technology creates a virtual user environment based on the UI design generated by AIGC. Digital twin simulates the interaction between the user and the UI and collects the user's behavior pattern data.

[0206] Step 4: Emotional analysis and optimization suggestions: Receive the interactive data of the digital twin. Analyze the user's emotional state through emotional computing and identify potential problems in the UI design solution. Emotional computing outputs emotional analysis reports and design optimization suggestions.

[0207] Step 5: Design iteration and improvement:

[0208] AIGC adjusts the UI design solution based on the optimization suggestions of emotional computing. The iteration process continues until the UI design solution meets the user's emotional indicators and experience requirements.

[0209] Through this division of labor and collaboration mechanism, the technical solution of the embodiment of the present application can ensure that each component can play its expertise, and the data flow between the components ensures the coordinated work of the entire system. This integrated approach not only improves the efficiency and quality of design, but also ensures that the final product better meets the actual needs and emotional expectations of users.

[0210] In the technical solution of the embodiment of the present application, the data processing flow is used to make the conversion process from raw data to final design optimization efficient and accurate. Figure 7 , attached Figure 7 The data processing flow chart provided in the embodiment of the present application includes the following steps.

[0211] Step 1: Data Collection

[0212] Business document analysis: Collect and analyze business requirement documents, which include product specifications, market positioning, user stories, etc., to provide a basis for interpreting business requirements.

[0213] User feedback collection: Collect direct feedback from users from various channels (such as questionnaires, customer support records, social media, etc.) to understand user needs and pain points.

[0214] Market research: Obtain information on industry trends, competitor analysis, and potential user groups through market research to provide a macro perspective for product design.

[0215] Step 2: Preprocessing

[0216] Data cleaning: Remove noise and inconsistencies in the data, such as removing duplicate records, correcting errors, and filling in missing values.

[0217] Standardization: Convert data into a unified format and unit to facilitate subsequent processing and analysis.

[0218] Transformation: Convert unstructured data (such as text, images) into structured data to facilitate machine learning and model analysis.

[0219] Step 3: LLM analysis: Use the large language model (LLM) to conduct in-depth analysis of text data to extract key requirements and design indicators.

[0220] Step 4: AIGC design generation: Artificial intelligence generated content (AIGC) generates a preliminary user interface (UI) design plan based on the analysis results of LLM.

[0221] Step 5: Digital twin simulation: Digital twin technology simulates the user’s interactive behavior, collects behavioral pattern data, and provides a basis for sentiment analysis.

[0222] Step 6: Emotional computing analysis: Emotional computing technology analyzes user emotions, identifies potential problems in UI design solutions, and provides optimization suggestions.

[0223] Step 7: Iterative Optimization

[0224] Design adjustment: Adjust the UI design based on the feedback from emotional computing to solve the identified problems.

[0225] Verification and testing: Use digital twin technology to verify and test the adjusted UI design to ensure that the new UI design can meet user emotional indicators.

[0226] Continuous iteration: Repeat the iterative optimization process until the design solution reaches the optimal state and meets the user's actual needs and emotional expectations.

[0227] The entire data processing process is a cyclic process from data collection to preprocessing, and then to analysis application and iterative optimization. Through this process, the technical solution of the embodiment of the present application can ensure that design decisions are based on accurate and comprehensive data, thereby improving the quality of UE / UI design and user satisfaction.

[0228] In order to help readers better understand the application scenarios of the embodiments of the present application, a specific example is given below:

[0229] (1) Data Collection Phase

[0230] The business unit submitted a detailed software requirements document that included the target market, expected functionality, key business processes, and security requirements for the new mobile banking application.

[0231] (2) Preprocessing stage

[0232] This software requirement document was pre-processed to extract key information, such as "fast transfer", "multi-factor authentication" and "user account management".

[0233] (3) Analysis and application stage

[0234] LLM analysis: The Large Language Model (LLM) analyzes the preprocessed requirements document to understand and extract core user needs and business metrics, such as "users need to be able to complete transfers within three steps" and "the security authentication process needs to include fingerprint and facial recognition."

[0235] AIGC Design Generation: Artificial Intelligence Generated Content (AIGC) generates a preliminary User Experience (UE) flowchart based on the analysis results of LLM. The UE flowchart describes in detail how users complete various tasks through the application.

[0236] Digital twin simulation: Digital twin technology creates a virtual mobile banking application environment based on the UE flowchart, and simulates the user's operating behaviors in the mobile banking application environment, such as logging in, transferring money, and checking account balances, to obtain operating behavior data.

[0237] Emotional computing analysis: Emotional computing technology analyzes user operation behavior data collected by digital twins to identify signs that users may feel inconvenienced or frustrated during the "multi-factor authentication" process, such as authentication steps that take a long time or are complicated to operate.

[0238] (4) Iterative Optimization Phase

[0239] Based on the analysis results of affective computing, the designers optimized the UE process, simplified the steps of multi-factor authentication, and increased user guidance to reduce user frustration.

[0240] The optimized UE process was simulated again through the digital twin, and the affective computing technology confirmed that the new authentication process received positive feedback from users, making their operations smoother and their emotional satisfaction higher.

[0241] The embodiment of the present application covers the entire process from business demand analysis to UE process design, to UI generation and user operation simulation, and finally to problem analysis and iterative optimization through emotional computing. This process ensures that the design solution not only meets business needs, but is also verified and optimized through simulation and emotional analysis, ultimately improving user experience and satisfaction.

[0242] The technical solution of the embodiment of the present application forms a closed-loop automated UE / UI design and testing process by integrating large language models (LLM), artificial intelligence generated content (AIGC), digital twin technology and emotional computing. The beneficial effects of the embodiment of the present application include the following aspects.

[0243] First, improve design efficiency and quality

[0244] Because LLM can interpret business requirements quickly and accurately, the time and error rate of demand analysis are significantly reduced.

[0245] Since AIGC automatically generates design solutions based on the interpretation results of LLM, the efficiency of the design process is improved while maintaining the diversity and innovation of the design. Combining LLM and AIGC, the embodiment of the present application can provide multiple high-quality UI design solutions in a short time, thereby improving the overall design efficiency and quality.

[0246] Second, optimize user experience

[0247] Because digital twin technology can simulate the operations of real users, it can collect more realistic user behavior pattern data.

[0248] Because affective computing technology can analyze user emotions, it can identify and resolve issues in the design that may cause user discomfort.

[0249] Through the combination of digital twins and emotional computing, the embodiments of the present application can deeply understand user needs and emotional reactions, thereby designing a UI design solution that better meets user expectations and optimizes the user experience.

[0250] Third, reduce development costs and risks

[0251] Since the automated process of the embodiment of the present application reduces manual involvement, the labor cost is reduced. Since the embodiment of the present application can find potential problems in the early stages of design, the risk of later modifications and rework is reduced. By reducing manual errors and solving problems in advance, the embodiment of the present application can effectively reduce development costs and risks.

[0252] Fourth, accelerate time to market

[0253] Because the automated design process of the embodiment of the present application speeds up the design and testing, the product development cycle is shortened.

[0254] Since the embodiments of the present application can quickly iterate and optimize the design, the product's market preparation is accelerated. Combined with rapid design iteration and optimization, the embodiments of the present application can significantly shorten the time from product concept to market and accelerate product market launch.

[0255] Fifth, enhance the market competitiveness of products

[0256] Because the embodiments of the present application can provide higher quality design and better user experience, user satisfaction and loyalty of the product are improved.

[0257] Since the embodiments of the present application can quickly respond to market changes and user needs, the product can better adapt to the market. By providing high-quality design and rapid market response, the embodiments of the present application can enhance the competitiveness of the product in the market and bring greater commercial value to the enterprise.

[0258] The method provided in the embodiment of the present application improves the accuracy and efficiency of demand understanding by utilizing LLM to automatically interpret and analyze business requirements.

[0259] In addition, AIGC technology automatically generates diverse UI design solutions based on the analysis results of LLM, which accelerates the design process and maintains the innovation of the design.

[0260] In addition, digital twin technology simulates the operating behavior of real users, providing a highly realistic testing environment for verifying and optimizing design solutions.

[0261] In addition, the user's emotional state is analyzed through affective computing technology to identify potential problems in the UI design solution, so that the final UI design solution can meet the user's emotional needs.

[0262] The embodiment of the present application effectively integrates LLM, AIGC, digital twin and affective computing technologies to form a closed-loop design and testing process.

[0263] The embodiments of the present application realize an automated design process, including automatic interpretation of requirements, automatic generation of design solutions, user interaction simulation and sentiment analysis, etc.

[0264] The embodiments of the present application implement a design optimization strategy based on data analysis, and in particular utilize user behavior pattern data and sentiment analysis results to guide design iteration and improvement.

[0265] The embodiments of the present application realize the simulation and testing of user experience: a method of simulating and testing user experience using digital twin technology, and optimizing product design through simulation results.

[0266] The embodiments of the present application realize emotion recognition and design feedback: realize the application of emotion computing technology in analyzing user emotions and providing design feedback, and apply these feedbacks to actual design improvements.

[0267] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

[0268] A refers to B, which means that A is the same as B or A is a simple variant of B.

[0269] The above embodiments can be implemented in whole or in part by software, hardware, firmware or any combination thereof. When implemented using software, it can be implemented in whole or in part in the form of a computer program product. The computer program product includes one or more computer instructions. When the computer program instructions are loaded and executed on a computer, the process or function described in accordance with the embodiment of the present application is generated in whole or in part. The computer may be a general-purpose computer, a special-purpose computer, a computer network, or other programmable device. The computer instructions may be stored in a computer-readable storage medium, or transmitted from one computer-readable storage medium to another computer-readable storage medium, for example, the computer instructions may be transmitted from a website site, computer, server or data center by wired (e.g., coaxial cable, optical fiber, digital subscriber line (DSL)) or wireless (e.g., infrared, wireless, microwave, etc.) mode to another website site, computer, server or data center. The computer-readable storage medium may be any available medium that a computer can access or a data storage device such as a server or data center that includes one or more available media integrated. The available medium may be a magnetic medium, (e.g., a floppy disk, a hard disk, a tape), an optical medium (e.g., a DVD), or a semiconductor medium (e.g., a solid-state hard disk SolidState Disk (SSD)), etc.

[0270] The above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, a person skilled in the art should understand that the technical solutions described in the aforementioned embodiments may still be modified, or some of the technical features may be replaced by equivalents. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the scope of the technical solutions of the embodiments of the present application.

Claims

1. A method for generating a UI design scheme, characterized in that: The method comprises: Generate user experience UE process based on business requirement document through large language model LLM; Generate a user interface UI design scheme based on the UE process through artificial intelligence content generation AIGC; Creating a virtual user environment based on the UI design solution through digital twins, simulating the interaction between the user and the UI, and collecting the user's behavior pattern data in the process of simulating the interaction; Performing emotion analysis based on the behavior pattern data to obtain the emotional state of the user; The UI design is updated based on the emotional state.

2. The method according to claim 1, characterized in that The user experience UE process is generated based on the business requirement document by using the large language model LLM, including: Through LLM, user requirements, functional points and design indicators are extracted from the content of the business requirements document through the self-attention mechanism; Based on the user requirements, the functional points and the design indicators, a UE flow chart and test cases are generated.

3. The method according to claim 2, characterized in that The LLM is pre-trained by performing a masked language model task and a next sentence prediction task based on text data; In the process of performing the masked language model task, some words in the input text data of the LLM are randomly masked, and the LLM predicts the original content of the masked words; In the process of performing the next sentence prediction task, the LLM determines whether two sentences appear adjacent to each other in the original text.

4. The method according to claim 1, characterized in that: The method of generating a user interface UI design scheme based on the UE process by artificial intelligence content generation AIGC includes: Generate a user interface UI design scheme based on the UE process through a generative adversarial network GAN, wherein the GAN includes a generator and a discriminator, wherein the generator is used to generate the UI design scheme, and the discriminator is used to discriminate between the generated UI design scheme and the real UI design scheme; or, Encode the UE process into a latent space through a variational autoencoder VAE, and decode from the latent space to generate a new layout or style; or, In the process of performing the conditional generation task, a UI design scheme is generated based on a predetermined color scheme or design style through the model.

5. The method according to claim 1, characterized in that The method of generating a user interface UI design scheme based on the UE process by artificial intelligence content generation AIGC includes: Based on the requirements of the UE process, set the generation parameters of AIGC, the generation parameters including color scheme, font type and layout style; Generate an initial UI design scheme based on the generation parameters and the design knowledge learned from the training data, wherein the initial UI design scheme includes layout, elements and interaction methods; Presenting an initial UI design solution to a user, and receiving evaluation indicators input by the user for the initial UI design solution, wherein the evaluation indicators include aesthetics, usability, and compliance; The generation parameters of the AIGC are adjusted based on the evaluation index, and the UI design scheme is regenerated based on the AIGC with the adjusted generation parameters, until the evaluation index corresponding to the generated UI design scheme meets the user's expectations.

6. The method according to claim 1, characterized in that The method of creating a virtual user environment based on the UI design solution through digital twins to simulate the interaction between the user and the UI includes: The state data of the physical entity is collected in real time by sensors installed on the physical entity, wherein the state data includes temperature, pressure and speed; Transmitting the collected state data to a cloud or a local server via a network, and synchronizing the state data with a digital twin model; creating a digital replica of the physical entity in the virtual user environment based on the state data, the digital replica including geometric structure, material properties, and functional characteristics; The digital twin model simulates the interaction process between the user and the physical entity based on the digital copy of the physical entity.

7. The method according to claim 1, characterized in that The performing emotion analysis based on the behavior pattern data to obtain the emotional state of the user includes: Extracting target behavior pattern data that has an impact on sentiment analysis from the behavior pattern data, the target behavior pattern data including operation frequency, response time, task completion rate, and number of errors; Encoding the target behavior pattern data to obtain behavior characteristics; The behavior features are input into a random forest model, emotion classification results are output for the behavior features through each decision tree in the random forest model, and the emotional state of the user is determined based on the emotion classification results of each decision tree.

8. The method according to claim 1, characterized in that The performing emotion analysis based on the behavior pattern data to obtain the emotional state of the user includes: converting the behavior pattern data into a time series; Extracting emotional features from the time series through an LSTM layer in a long short-term memory network LSTM model, and obtaining temporal dependencies between operations from the time series through a gating mechanism of the LSTM model; Based on the emotional characteristics and the time dependency, the emotional state of the user at a target time point is determined.

9. A UI design scheme generating device, characterized in that: The device comprises: Large Language Model (LLM) unit, used to generate user experience (UE) flow based on business requirement documents through LLM; An artificial intelligence generated content AIGC unit, used to generate a user interface UI design scheme based on the UE process through AIGC; A digital twin unit, used to create a virtual user environment based on the UI design solution through a digital twin, simulate the interaction between the user and the UI, and collect the user's behavior pattern data in the process of simulating the interaction behavior; An emotion analysis unit, configured to perform emotion analysis based on the behavior pattern data to obtain the emotional state of the user; An updating unit is used to update the UI design scheme based on the emotional state.

10. The device according to claim 9, characterized in that The AIGC unit is used to set the generation parameters of the AIGC based on the requirements of the UE process, and the generation parameters include color scheme, font type and layout style; Based on the generation parameters and the design knowledge learned from the training data, an initial UI design scheme is generated, wherein the initial UI design scheme includes layout, elements, and interaction methods; the initial UI design scheme is presented to the user, and evaluation indicators input by the user for the initial UI design scheme are received, wherein the evaluation indicators include aesthetics, usability, and compliance; The generation parameters of the AIGC are adjusted based on the evaluation index, and the UI design scheme is regenerated based on the AIGC with the adjusted generation parameters, until the evaluation index corresponding to the generated UI design scheme meets the user's expectations.

Images