system

Abstract

The system according to this embodiment aims to provide design proposals quickly and support creative work. [Solution] The system according to the embodiment comprises a reception unit, a trend analysis unit, a mockup generation unit, and a provision unit. The reception unit receives input from the user. The trend analysis unit performs trend analysis based on the information received by the reception unit. The mockup generation unit generates a design mockup based on the analysis results obtained by the trend analysis unit. The provision unit provides the design mockup generated by the mockup generation unit to the user.

Description

Technical Field

[0005]

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor, and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a character of the chatbot, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In the conventional technology, there is a problem that it is difficult to obtain inspiration for design and it is difficult to provide a design proposal quickly.

[0005] The system according to the embodiment aims to quickly provide a design proposal and assist in creative work.

Means for Solving the Problems

[0006] The system according to this embodiment comprises a reception unit, a trend analysis unit, a mockup generation unit, and a provision unit. The reception unit receives input from the user. The trend analysis unit performs trend analysis based on the information received by the reception unit. The mockup generation unit generates a design mockup based on the analysis results obtained by the trend analysis unit. The provision unit provides the design mockup generated by the mockup generation unit to the user. [Effects of the Invention]

[0007] The system according to this embodiment can quickly provide design proposals and support creative work. [Brief explanation of the drawing]

[0008] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Modes for carrying out the invention]

[0009] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0010] First, let's explain the terminology used in the following explanation.

[0011] In the following embodiments, the signed processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Furthermore, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include CPU (Central Processing Unit), GPU (Graphics Processing Unit), GPGPU (General-Purpose computing on Graphics Processing Units), APU (Accelerated Processing Unit), or TPU (Tensor Processing Unit).

[0012] In the following embodiments, signed RAM (Random Access Memory) is a memory that temporarily stores information and is used as work memory by the processor.

[0013] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0014] In the following embodiments, the numbered communication I / F (Interface) is an interface including a communication processor, an antenna, and the like. The communication I / F controls communication between a plurality of computers. Examples of communication standards applied to the communication I / F include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B". That is, "A and / or B" means that it may be only A, only B, or a combination of A and B. Also, in this specification, when expressing three or more matters connected by "and / or", the same concept as "A and / or B" is applied.

[0016] [First Embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] As shown in FIG. 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 2 eight, RAM 30, and storage 32 are connected to a bus 34. Also, the database 24 and the communication I / F 26 are connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0020] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, and accepts user input. The touch panel 38A accepts user input via touch by detecting contact with an object (e.g., a pen or finger). The microphone 38B accepts user input via voice by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 (see Figure 2) acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user by outputting the data in a form perceptible to the user (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0023] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0025] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0026] In the smart device 14, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart device 14 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0027] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device having the data generation model 58. The data processing device 12 may also be a server device or a terminal device owned by a user (e.g., a mobile phone, robot, home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example of form 1) The design support system according to an embodiment of the present invention is a system that utilizes generative AI to provide design inspiration and automatically generates design mockups. To solve the problem of difficulty in obtaining conventional design inspiration, the design support system uses generative AI to perform trend analysis and grasp the latest design trends. Next, the design support system automatically generates design mockups based on these trends. This allows designers to quickly obtain design proposals and proceed with creative work efficiently. Specifically, the design support system accepts input from the user and performs trend analysis. For example, if the user inputs "modern interior design," the design support system analyzes the latest trends in modern interior design and generates a design mockup based on that. This mockup is provided to the user to provide design inspiration. The design support system can also learn the user's preferences and past design history and generate individually customized design mockups. This allows users to obtain design proposals that suit their preferences. Furthermore, the design support system supports not only the generation of design mockups but also the entire design process. For example, the design support system guides each step of the design, enabling designers to work efficiently. The design support system also provides design feedback and advice to designers to improve their designs. In this way, design support systems utilize generative AI to provide design inspiration and automatically generate design mockups, thereby supporting creative work and streamlining the design process. This allows design support systems to provide design proposals quickly, enabling more efficient progress in creative work.

[0029] The design support system according to this embodiment comprises a reception unit, a trend analysis unit, a mockup generation unit, and a provision unit. The reception unit receives input from the user. User input includes, but is not limited to, text input, image uploads, and questionnaire responses. For example, the reception unit can receive text input from the user such as "modern interior design." The reception unit can also receive image uploads from the user. Furthermore, the reception unit can also receive questionnaire responses from the user. The trend analysis unit uses a generation AI to perform trend analysis based on the information received by the reception unit. Trend analysis is performed based on, but is not limited to, data sources used, analysis algorithms, evaluation criteria, etc. For example, the trend analysis unit uses a generation AI to analyze the latest design trends. The generation AI uses, for example, a text generation AI (e.g., LLM) to analyze the latest design trends. The trend analysis unit can also use a generation AI to analyze design trends using an image generation AI. Furthermore, the trend analysis unit can also use a generation AI to analyze design trends using a multimodal generation AI. The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. The design mockups are generated based on, for example, image format, resolution, and design elements, but are not limited to these examples. For example, the mockup generation unit uses a generation AI to generate design mockups based on trends. The generation AI uses, for example, a text generation AI (e.g., LLM) to generate design mockups. The mockup generation unit can also use a generation AI to generate design mockups using an image generation AI. Furthermore, the mockup generation unit can also use a generation AI to generate design mockups using a multimodal generation AI. The provisioning unit provides the design mockups generated by the mockup generation unit to the user. The provisioning unit, for example, displays the generated design mockups to the user.The service provider can, for example, provide design mockups to users through web applications or mobile applications. The service provider can also send the generated design mockups to users via email. Furthermore, the service provider can print the generated design mockups using a printer and provide them to users. This allows the design support system according to the embodiment to perform trend analysis based on user input, generate design mockups, and provide them, thereby enabling the rapid provision of design proposals.

[0030] The reception department receives input from users. User input includes, but is not limited to, text input, image uploads, and survey responses. Specifically, if a user enters "modern interior design" as text, the reception department analyzes the text and extracts relevant keywords and concepts. If a user uploads an image, the reception department analyzes the image and identifies the design elements and styles within it. Furthermore, if survey responses are accepted, the reception department presents questions to understand the user's preferences and requests in detail and collects the responses. This allows the reception department to accurately understand the user's specific needs and preferences. The reception department centrally manages this input data and prepares it for subsequent processes such as trend analysis and mockup generation. For example, in the case of text input, natural language processing technology is used to analyze the input content and extract relevant design keywords. In the case of image uploads, image recognition technology is used to identify design elements within the image, and in the case of survey responses, the responses are stored in a database and used for subsequent analysis. This allows the reception desk to efficiently process diverse user inputs and smoothly advance the overall system design support process.

[0031] The Trend Analysis Department uses generative AI to perform trend analysis based on information received by the Reception Department. Trend analysis is performed based on, for example, the data sources used, analysis algorithms, and evaluation criteria, but is not limited to these examples. Specifically, the Trend Analysis Department uses generative AI to analyze the latest design trends. For example, the generative AI uses text generation AI (e.g., LLM) to analyze the latest design trends. The text generation AI collects design-related information from online design-related articles, blogs, and social media posts, and analyzes this information to identify current trends. The Trend Analysis Department can also use generative AI to analyze design trends using image generation AI. The image generation AI analyzes design-related image databases to identify popular design styles, color schemes, layouts, etc. Furthermore, the Trend Analysis Department can also use generative AI to analyze design trends using multimodal generation AI. The multimodal generation AI integrates and analyzes both text and image information to identify more comprehensive design trends. This allows the Trend Analysis Department to accurately grasp the latest design trends based on user input and provide information for subsequent mockup generation departments.

[0032] The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. Design mockups are generated based on, for example, image format, resolution, and design elements, but are not limited to these examples. Specifically, the mockup generation unit uses a generation AI to generate trend-based design mockups. The generation AI uses, for example, a text generation AI (e.g., LLM) to generate design mockups. The text generation AI generates a design overview and layout based on design keywords and concepts provided by the trend analysis unit. The mockup generation unit can also use the generation AI to generate design mockups using an image generation AI. The image generation AI generates specific design images based on design elements provided by the trend analysis unit. Furthermore, the mockup generation unit can also use the generation AI to generate design mockups using a multimodal generation AI. The multimodal generation AI integrates both text and image information to generate more detailed and realistic design mockups. This allows the mockup generation unit to quickly generate high-quality design mockups based on user needs and trends.

[0033] The service provider provides users with design mockups generated by the mockup generation unit. For example, the service provider displays the generated design mockups to users. Specifically, the service provider provides design mockups to users through web applications and mobile applications. Users can view and evaluate the generated design mockups through these applications. The service provider can also send the generated design mockups to users via email. Users can review the design mockups and provide feedback through the received emails. Furthermore, the service provider can print the generated design mockups and provide them to users. The printed design mockups are used by users to visually confirm the actual design. This allows the service provider to provide design mockups to users in various ways, improving user convenience. Additionally, the service provider can continuously improve the accuracy and quality of the entire system by collecting user feedback and providing it to the mockup generation unit and trend analysis unit. This allows the service provider to provide users with fast and accurate design mockups, increasing user satisfaction.

[0034] The trend analysis unit can analyze the latest design trends using generative AI. For example, the trend analysis unit efficiently analyzes the latest design trends using generative AI. For instance, the trend analysis unit can use text generation AI (e.g., LLM) to analyze the latest design trends using generative AI. Furthermore, the trend analysis unit can analyze design trends using image generation AI. Additionally, the trend analysis unit can analyze design trends using multimodal generative AI. This allows for efficient analysis of the latest design trends using generative AI. The generative AI is implemented based, for example, on the machine learning model and training dataset used. Some or all of the above-described processes in the trend analysis unit may be performed using generative AI, or they may not. For example, the trend analysis unit can use text generation AI (e.g., LLM) to analyze the latest design trends using generative AI.

[0035] The mockup generation unit can generate trend-based design mockups using a generation AI. For example, the mockup generation unit efficiently generates trend-based design mockups using a generation AI. For instance, the mockup generation unit can use a text generation AI (e.g., LLM) to generate trend-based design mockups using a generation AI. Furthermore, the mockup generation unit can generate design mockups using an image generation AI. Additionally, the mockup generation unit can generate design mockups using a multimodal generation AI. This allows for the efficient generation of trend-based design mockups using a generation AI. Trend-based design mockups are generated based on, for example, trend evaluation criteria and design element selection methods. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI or not. For example, the mockup generation unit can use a text generation AI (e.g., LLM) to generate trend-based design mockups using a generation AI.

[0036] The service provider can provide the generated design mockups to the user. For example, by providing the generated design mockups to the user, the service provider can quickly provide design proposals. For example, the service provider can display the generated design mockups to the user. The service provider can provide the design mockups to the user, for example, through a web application or a mobile application. The service provider can also send the generated design mockups to the user via email. Furthermore, the service provider can print the generated design mockups and provide them to the user. This allows for the quick provision of design proposals by providing the generated design mockups to the user. Some or all of the above processes in the service provider may be performed using or without a generation AI. For example, the service provider can use a generation AI to provide the generated design mockups to the user.

[0037] The reception unit can learn the user's preferences and past design history. For example, by learning the user's preferences and past design history, the reception unit can generate individually customized design mockups. For example, the reception unit can identify the user's preferences from survey results and past selection history. The reception unit can also obtain the user's past design history and learn from past design selection and usage history. This allows the reception unit to generate individually customized design mockups by learning the user's preferences and past design history. Some or all of the above processing in the reception unit may be performed using a generative AI, or it may be performed without a generative AI. For example, the reception unit can use a generative AI to learn the user's preferences and past design history.

[0038] The mockup generation unit can generate customized design mockups based on user preferences. For example, by generating customized design mockups based on user preferences, the mockup generation unit can provide design proposals that meet user needs. For instance, the mockup generation unit can generate customized design mockups using a method for selecting design elements based on user preferences. This allows for the provision of design proposals that meet user needs by generating customized design mockups based on user preferences. Customized design mockups are generated based, for example, on a method for selecting design elements based on user preferences. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI, or they may be performed without a generation AI. For example, the mockup generation unit can use a generation AI to generate customized design mockups based on user preferences.

[0039] The reception desk can analyze the user's past input history and suggest the optimal input method. For example, the reception desk can automatically display design themes that the user has frequently entered in the past as suggestions. The reception desk can also prioritize suggesting input methods (voice, text, etc.) that the user has used in the past. For example, the reception desk can predict and suggest design themes to be used during a specific time period based on the user's past input history. In this way, the optimal input method can be suggested by analyzing the user's past input history. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to analyze the user's past input history and suggest the optimal input method.

[0040] The input field can filter input content based on the user's current projects and areas of interest. For example, the input field may prioritize displaying design themes related to the user's current projects. The input field may also suggest highly relevant design themes based on the user's areas of interest. The input field may also filter relevant input content based on design themes the user has shown interest in in the past. This allows for the provision of highly relevant input content by filtering input content based on the user's current projects and areas of interest. Some or all of the above processing in the input field may be performed using generative AI, or not. For example, the input field can use generative AI to filter input content based on the user's current projects and areas of interest.

[0041] The reception desk can prioritize receiving highly relevant input content by considering the user's geographical location. For example, if the user is in a specific region, the reception desk will prioritize displaying design themes related to that region. The reception desk can also suggest region-specific design trends based on the user's geographical location. For example, if the user is traveling, the reception desk can prioritize receiving input content related to the culture and design of the destination. This allows for the priority of receiving highly relevant input content by considering the user's geographical location. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to prioritize receiving highly relevant input content by considering the user's geographical location.

[0042] The reception desk can analyze a user's social media activity and suggest relevant input content. For example, the reception desk can suggest relevant design themes based on designs shared by the user on social media. For example, the reception desk can also analyze posts from design influencers followed by the user and suggest relevant input content. For example, the reception desk can predict and suggest design themes that the user might be interested in based on their social media activity. In this way, relevant input content can be suggested by analyzing the user's social media activity. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to analyze a user's social media activity and suggest relevant input content.

[0043] The trend analysis unit can predict current trends by referring to past trend data. For example, the trend analysis unit can analyze design trend data from the past few years to predict current trends. The trend analysis unit can also predict trends suitable for the current season by referring to seasonal design trends. For example, the trend analysis unit can identify periodically repeating trends from past trend data to predict current trends. In this way, current trends can be predicted by referring to past trend data. Past trend data is obtained based on, for example, past market data, user behavior history, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to predict current trends by referring to past trend data.

[0044] The trend analysis unit can apply different trend analysis methods to each design category. For example, it can apply different trend analysis methods to interior design and fashion design. It can also apply different trend analysis methods to graphic design and product design. It can also apply different trend analysis methods to web design and packaging design. By applying different trend analysis methods to each design category, it becomes possible to perform trend analysis appropriate for each category. The trend analysis methods are applied based on, for example, category-specific analysis algorithms and evaluation criteria. Some or all of the above-described processes in the trend analysis unit may be performed using generative AI, or they may not be performed using generative AI. For example, the trend analysis unit can use generative AI to apply different trend analysis methods to each design category.

[0045] The trend analysis unit can analyze changes in trends based on the timing of idea submission. For example, the trend analysis unit can analyze changes in trends based on when ideas were submitted. The trend analysis unit can also analyze changes in trends based on the timing of idea submissions for each season. The trend analysis unit can also analyze changes in trends based on ideas submitted during specific events or festivals. This makes it possible to perform trend analysis according to the time of year by analyzing changes in trends based on the timing of idea submission. The timing of idea submission is determined based on, for example, the submission date and time, the frequency of submission, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to analyze changes in trends based on the timing of idea submission.

[0046] The trend analysis unit can analyze trends by referring to relevant market data. For example, the trend analysis unit can analyze trends by referring to sales data of relevant markets. The trend analysis unit can also analyze trends by referring to consumer behavior data of relevant markets. The trend analysis unit can also analyze trends by referring to design data of competitors in relevant markets. This allows for a more accurate analysis of trends by referring to relevant market data. Relevant market data is obtained based on, for example, market research data, competitor analysis data, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to analyze trends by referring to relevant market data.

[0047] The mockup generation unit can optimize its generation algorithm by referring to past design mockups. For example, the mockup generation unit can analyze previously generated design mockups and optimize its generation algorithm. The mockup generation unit can also customize its generation algorithm by referring to past design mockups used by the user. For example, the mockup generation unit can improve its generation algorithm based on successful examples of past design mockups. This allows for the optimization of the generation algorithm by referring to past design mockups, enabling the generation of more accurate mockups. The generation algorithm is optimized based on, for example, the machine learning model used, the training dataset, etc. Some or all of the above processes in the mockup generation unit may be performed using or without generation AI. For example, the mockup generation unit can use generation AI to optimize its generation algorithm by referring to past design mockups.

[0048] The mockup generation unit can apply different generation algorithms to each design category. For example, it can apply different generation algorithms to interior design and fashion design. It can also apply different generation algorithms to graphic design and product design. It can also apply different generation algorithms to web design and packaging design. By applying different generation algorithms to each design category, it is possible to generate mockups suitable for each category. The generation algorithm is applied based on, for example, a category-specific generation algorithm and evaluation criteria. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI, or they may not be performed using a generation AI. For example, the mockup generation unit can use a generation AI to apply different generation algorithms to each design category.

[0049] The mockup generation unit can generate highly relevant mockups by considering the user's geographical location information. For example, if the user is in a specific region, the mockup generation unit will generate a design mockup related to that region. The mockup generation unit can also generate region-specific design mockups based on the user's geographical location information. For example, if the user is traveling, the mockup generation unit can generate mockups related to the culture and design of the destination. This allows for the generation of highly relevant mockups by considering the user's geographical location information. Geographical location information is obtained, for example, based on GPS data, IP address, etc. Some or all of the above processing in the mockup generation unit may be performed using a generation AI, or not. For example, the mockup generation unit can use a generation AI to generate highly relevant mockups by considering the user's geographical location information.

[0050] The mockup generation unit can improve the accuracy of mockup generation by referring to relevant literature. For example, the mockup generation unit can improve its generation algorithm by referring to the latest design-related literature. The mockup generation unit can also improve the accuracy of mockup generation by referring to literature on design theory. The mockup generation unit can also optimize its generation algorithm by referring to literature on past design projects. In this way, the accuracy of mockup generation can be improved by referring to relevant literature. Relevant literature is obtained based on academic papers, technical reports, etc. Some or all of the above processing in the mockup generation unit may be performed using a generation AI, or not. For example, the mockup generation unit can use a generation AI to improve the accuracy of mockup generation by referring to relevant literature.

[0051] The delivery unit can select the optimal delivery method by referring to past user feedback. For example, the delivery unit may prioritize the delivery method of mockups that users have previously given high ratings to. The delivery unit may also analyze past user feedback and customize the optimal delivery method. For example, the delivery unit may avoid delivery methods that users have previously expressed dissatisfaction with and select improved methods. This allows the delivery unit to select the optimal delivery method by referring to past user feedback. Past feedback is obtained based on, for example, user evaluation comments and feedback frequency. Some or all of the above processing in the delivery unit may be performed using generative AI or not. For example, the delivery unit can use generative AI to select the optimal delivery method by referring to past user feedback.

[0052] The service provider can customize the content offered based on the user's current project status. For example, the service provider may prioritize providing mockups related to the user's ongoing projects. The service provider may also provide appropriate mockups according to the progress of the user's projects. The service provider may also provide customized mockups based on the theme of the user's projects. This allows the service provider to provide highly relevant mockups by customizing the content based on the user's current project status. The current project status is identified, for example, based on the progress of the project, related tasks, etc. Some or all of the above processing in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to customize the content offered based on the user's current project status.

[0053] The service provider can select the optimal service delivery method by considering the user's device information. For example, if the user is using a smartphone, the service provider can provide a display method that matches the screen size. For example, if the user is using a tablet, the service provider can also provide a display method optimized for a larger screen. For example, if the user is using a desktop, the service provider can also provide a display method that includes detailed information. This allows the service provider to select the optimal service delivery method by considering the user's device information. Device information is obtained based on, for example, the device type and OS version. Some or all of the processing described above in the service provider may be performed using a generative AI, or not. For example, the service provider can use a generative AI to select the optimal service delivery method by considering the user's device information.

[0054] The service provider can analyze the user's social media activity and customize the content offered. For example, the service provider can provide relevant mockups based on designs shared by the user on social media. The service provider can also analyze the posts of design influencers followed by the user and provide relevant mockups. The service provider can also predict and provide mockups that the user might be interested in based on their social media activity. This allows the service provider to provide relevant mockups by analyzing the user's social media activity. Social media activity is obtained based on, for example, posts, like history, etc. Some or all of the above processing in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to analyze the user's social media activity and customize the content offered.

[0055] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0056] The design support system can further analyze the user's past design history and generate design mockups based on past successes and failures. For example, it can prioritize incorporating design elements that received high praise in the past, while avoiding those that were unpopular. Furthermore, it can learn the user's preferences from their past design history and generate individually customized design mockups. This allows the system to leverage the user's past design history to provide more accurate design mockups.

[0057] The design support system can also generate design mockups that take the user's geographical location into account. For example, if the user is in a specific region, it can provide designs based on the culture and trends of that region. If the user is traveling, it can generate mockups related to the culture and design of their destination. Furthermore, it can incorporate region-specific design elements based on the user's geographical location. This allows for a more relevant design experience by providing design mockups that consider the user's geographical location.

[0058] The design support system can further analyze users' social media activity and generate relevant design mockups. For example, it can suggest relevant design themes based on designs users have shared on social media. It can also analyze posts from design influencers users follow and provide relevant design mockups. Furthermore, it can predict and suggest design themes that users might be interested in based on their social media activity. This allows for the provision of more personalized design mockups by leveraging users' social media activity.

[0059] The design support system can further generate design mockups while considering the user's device information. For example, if the user is using a smartphone, it can provide a design optimized for the screen size. Similarly, if the user is using a tablet, it can provide a design optimized for the larger screen. Furthermore, if the user is using a desktop, it can provide a design that includes detailed information. By providing design mockups that take the user's device information into account, a more user-friendly design experience can be achieved.

[0060] The following briefly describes the processing flow for example form 1.

[0061] Step 1: The reception desk receives input from users. User input includes text input, image uploads, and survey responses. For example, it accepts users to type "modern interior design," upload images, and answer survey questions. Step 2: The Trend Analysis Department uses generative AI to perform trend analysis based on the information received by the Reception Department. Trend analysis is performed based on the data sources used, analysis algorithms, evaluation criteria, etc. For example, generative AI is used to analyze the latest design trends. Step 3: The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. The design mockups are generated based on image format, resolution, design elements, etc. For example, a design mockup based on trends is generated using the generation AI. Step 4: The provider unit provides the user with the design mockup generated by the mockup generation unit. The provider unit displays the generated design mockup to the user. For example, it can be provided through a web application or a mobile application. It can also be sent via email or printed out.

[0062] (Example of form 2) The design support system according to an embodiment of the present invention is a system that utilizes generative AI to provide design inspiration and automatically generates design mockups. To solve the problem of difficulty in obtaining conventional design inspiration, the design support system uses generative AI to perform trend analysis and grasp the latest design trends. Next, the design support system automatically generates design mockups based on these trends. This allows designers to quickly obtain design proposals and proceed with creative work efficiently. Specifically, the design support system accepts input from the user and performs trend analysis. For example, if the user inputs "modern interior design," the design support system analyzes the latest trends in modern interior design and generates a design mockup based on that. This mockup is provided to the user to provide design inspiration. The design support system can also learn the user's preferences and past design history and generate individually customized design mockups. This allows users to obtain design proposals that suit their preferences. Furthermore, the design support system supports not only the generation of design mockups but also the entire design process. For example, the design support system guides each step of the design, enabling designers to work efficiently. The design support system also provides design feedback and advice to designers to improve their designs. In this way, design support systems utilize generative AI to provide design inspiration and automatically generate design mockups, thereby supporting creative work and streamlining the design process. This allows design support systems to provide design proposals quickly, enabling more efficient progress in creative work.

[0063] The design support system according to this embodiment comprises a reception unit, a trend analysis unit, a mockup generation unit, and a provision unit. The reception unit receives input from the user. User input includes, but is not limited to, text input, image uploads, and questionnaire responses. For example, the reception unit can receive text input from the user such as "modern interior design." The reception unit can also receive image uploads from the user. Furthermore, the reception unit can also receive questionnaire responses from the user. The trend analysis unit uses a generation AI to perform trend analysis based on the information received by the reception unit. Trend analysis is performed based on, but is not limited to, data sources used, analysis algorithms, evaluation criteria, etc. For example, the trend analysis unit uses a generation AI to analyze the latest design trends. The generation AI uses, for example, a text generation AI (e.g., LLM) to analyze the latest design trends. The trend analysis unit can also use a generation AI to analyze design trends using an image generation AI. Furthermore, the trend analysis unit can also use a generation AI to analyze design trends using a multimodal generation AI. The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. The design mockups are generated based on, for example, image format, resolution, and design elements, but are not limited to these examples. For example, the mockup generation unit uses a generation AI to generate design mockups based on trends. The generation AI uses, for example, a text generation AI (e.g., LLM) to generate design mockups. The mockup generation unit can also use a generation AI to generate design mockups using an image generation AI. Furthermore, the mockup generation unit can also use a generation AI to generate design mockups using a multimodal generation AI. The provisioning unit provides the design mockups generated by the mockup generation unit to the user. The provisioning unit, for example, displays the generated design mockups to the user.The service provider can, for example, provide design mockups to users through web applications or mobile applications. The service provider can also send the generated design mockups to users via email. Furthermore, the service provider can print the generated design mockups using a printer and provide them to users. This allows the design support system according to the embodiment to perform trend analysis based on user input, generate design mockups, and provide them, thereby enabling the rapid provision of design proposals.

[0064] The reception department receives input from users. User input includes, but is not limited to, text input, image uploads, and survey responses. Specifically, if a user enters "modern interior design" as text, the reception department analyzes the text and extracts relevant keywords and concepts. If a user uploads an image, the reception department analyzes the image and identifies the design elements and styles within it. Furthermore, if survey responses are accepted, the reception department presents questions to understand the user's preferences and requests in detail and collects the responses. This allows the reception department to accurately understand the user's specific needs and preferences. The reception department centrally manages this input data and prepares it for subsequent processes such as trend analysis and mockup generation. For example, in the case of text input, natural language processing technology is used to analyze the input content and extract relevant design keywords. In the case of image uploads, image recognition technology is used to identify design elements within the image, and in the case of survey responses, the responses are stored in a database and used for subsequent analysis. This allows the reception desk to efficiently process diverse user inputs and smoothly advance the overall system design support process.

[0065] The Trend Analysis Department uses generative AI to perform trend analysis based on information received by the Reception Department. Trend analysis is performed based on, for example, the data sources used, analysis algorithms, and evaluation criteria, but is not limited to these examples. Specifically, the Trend Analysis Department uses generative AI to analyze the latest design trends. For example, the generative AI uses text generation AI (e.g., LLM) to analyze the latest design trends. The text generation AI collects design-related information from online design-related articles, blogs, and social media posts, and analyzes this information to identify current trends. The Trend Analysis Department can also use generative AI to analyze design trends using image generation AI. The image generation AI analyzes design-related image databases to identify popular design styles, color schemes, layouts, etc. Furthermore, the Trend Analysis Department can also use generative AI to analyze design trends using multimodal generation AI. The multimodal generation AI integrates and analyzes both text and image information to identify more comprehensive design trends. This allows the Trend Analysis Department to accurately grasp the latest design trends based on user input and provide information for subsequent mockup generation departments.

[0066] The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. Design mockups are generated based on, for example, image format, resolution, and design elements, but are not limited to these examples. Specifically, the mockup generation unit uses a generation AI to generate trend-based design mockups. The generation AI uses, for example, a text generation AI (e.g., LLM) to generate design mockups. The text generation AI generates a design overview and layout based on design keywords and concepts provided by the trend analysis unit. The mockup generation unit can also use the generation AI to generate design mockups using an image generation AI. The image generation AI generates specific design images based on design elements provided by the trend analysis unit. Furthermore, the mockup generation unit can also use the generation AI to generate design mockups using a multimodal generation AI. The multimodal generation AI integrates both text and image information to generate more detailed and realistic design mockups. This allows the mockup generation unit to quickly generate high-quality design mockups based on user needs and trends.

[0067] The service provider provides users with design mockups generated by the mockup generation unit. For example, the service provider displays the generated design mockups to users. Specifically, the service provider provides design mockups to users through web applications and mobile applications. Users can view and evaluate the generated design mockups through these applications. The service provider can also send the generated design mockups to users via email. Users can review the design mockups and provide feedback through the received emails. Furthermore, the service provider can print the generated design mockups and provide them to users. The printed design mockups are used by users to visually confirm the actual design. This allows the service provider to provide design mockups to users in various ways, improving user convenience. Additionally, the service provider can continuously improve the accuracy and quality of the entire system by collecting user feedback and providing it to the mockup generation unit and trend analysis unit. This allows the service provider to provide users with fast and accurate design mockups, increasing user satisfaction.

[0068] The trend analysis unit can analyze the latest design trends using generative AI. For example, the trend analysis unit efficiently analyzes the latest design trends using generative AI. For instance, the trend analysis unit can use text generation AI (e.g., LLM) to analyze the latest design trends using generative AI. Furthermore, the trend analysis unit can analyze design trends using image generation AI. Additionally, the trend analysis unit can analyze design trends using multimodal generative AI. This allows for efficient analysis of the latest design trends using generative AI. The generative AI is implemented based, for example, on the machine learning model and training dataset used. Some or all of the above-described processes in the trend analysis unit may be performed using generative AI, or they may not. For example, the trend analysis unit can use text generation AI (e.g., LLM) to analyze the latest design trends using generative AI.

[0069] The mockup generation unit can generate trend-based design mockups using a generation AI. For example, the mockup generation unit efficiently generates trend-based design mockups using a generation AI. For instance, the mockup generation unit can use a text generation AI (e.g., LLM) to generate trend-based design mockups using a generation AI. Furthermore, the mockup generation unit can generate design mockups using an image generation AI. Additionally, the mockup generation unit can generate design mockups using a multimodal generation AI. This allows for the efficient generation of trend-based design mockups using a generation AI. Trend-based design mockups are generated based on, for example, trend evaluation criteria and design element selection methods. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI or not. For example, the mockup generation unit can use a text generation AI (e.g., LLM) to generate trend-based design mockups using a generation AI.

[0070] The service provider can provide the generated design mockups to the user. For example, by providing the generated design mockups to the user, the service provider can quickly provide design proposals. For example, the service provider can display the generated design mockups to the user. The service provider can provide the design mockups to the user, for example, through a web application or a mobile application. The service provider can also send the generated design mockups to the user via email. Furthermore, the service provider can print the generated design mockups and provide them to the user. This allows for the quick provision of design proposals by providing the generated design mockups to the user. Some or all of the above processes in the service provider may be performed using or without a generation AI. For example, the service provider can use a generation AI to provide the generated design mockups to the user.

[0071] The reception unit can learn the user's preferences and past design history. For example, by learning the user's preferences and past design history, the reception unit can generate individually customized design mockups. For example, the reception unit can identify the user's preferences from survey results and past selection history. The reception unit can also obtain the user's past design history and learn from past design selection and usage history. This allows the reception unit to generate individually customized design mockups by learning the user's preferences and past design history. Some or all of the above processing in the reception unit may be performed using a generative AI, or it may be performed without a generative AI. For example, the reception unit can use a generative AI to learn the user's preferences and past design history.

[0072] The mockup generation unit can generate customized design mockups based on user preferences. For example, by generating customized design mockups based on user preferences, the mockup generation unit can provide design proposals that meet user needs. For instance, the mockup generation unit can generate customized design mockups using a method for selecting design elements based on user preferences. This allows for the provision of design proposals that meet user needs by generating customized design mockups based on user preferences. Customized design mockups are generated based, for example, on a method for selecting design elements based on user preferences. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI, or they may be performed without a generation AI. For example, the mockup generation unit can use a generation AI to generate customized design mockups based on user preferences.

[0073] The reception unit can estimate the user's emotions and prioritize input content based on the estimated emotions. For example, if the user is stressed, the reception unit can provide a simple interface and minimize the input steps. For example, if the user is relaxed, the reception unit can provide detailed input options and suggest customizable input methods. For example, if the user is in a hurry, the reception unit can prioritize voice input to allow for quick input. This allows for the provision of appropriate input methods tailored to the user's situation by prioritizing input content based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the reception unit may be performed using generative AI or not. For example, the reception unit can use generative AI to estimate the user's emotions and prioritize input content based on the estimated emotions.

[0074] The reception desk can analyze the user's past input history and suggest the optimal input method. For example, the reception desk can automatically display design themes that the user has frequently entered in the past as suggestions. The reception desk can also prioritize suggesting input methods (voice, text, etc.) that the user has used in the past. For example, the reception desk can predict and suggest design themes to be used during a specific time period based on the user's past input history. In this way, the optimal input method can be suggested by analyzing the user's past input history. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to analyze the user's past input history and suggest the optimal input method.

[0075] The input field can filter input content based on the user's current projects and areas of interest. For example, the input field may prioritize displaying design themes related to the user's current projects. The input field may also suggest highly relevant design themes based on the user's areas of interest. The input field may also filter relevant input content based on design themes the user has shown interest in in the past. This allows for the provision of highly relevant input content by filtering input content based on the user's current projects and areas of interest. Some or all of the above processing in the input field may be performed using generative AI, or not. For example, the input field can use generative AI to filter input content based on the user's current projects and areas of interest.

[0076] The reception unit can estimate the user's emotions and adjust how the input content is displayed based on the estimated emotions. For example, if the user is tense, the reception unit can provide an interface with calming colors to reduce visual stress. For example, if the user is having fun, the reception unit can provide an interface with bright colors to make the input process more enjoyable. For example, if the user is tired, the reception unit can provide a simple and highly visible interface to facilitate the input process. By adjusting how the input content is displayed based on the user's emotions, an appropriate display method can be provided according to the user's situation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the reception unit may be performed using generative AI or not. For example, the reception unit can use generative AI to estimate the user's emotions and adjust how the input content is displayed based on the estimated emotions.

[0077] The reception desk can prioritize receiving highly relevant input content by considering the user's geographical location. For example, if the user is in a specific region, the reception desk will prioritize displaying design themes related to that region. The reception desk can also suggest region-specific design trends based on the user's geographical location. For example, if the user is traveling, the reception desk can prioritize receiving input content related to the culture and design of the destination. This allows for the priority of receiving highly relevant input content by considering the user's geographical location. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to prioritize receiving highly relevant input content by considering the user's geographical location.

[0078] The reception desk can analyze a user's social media activity and suggest relevant input content. For example, the reception desk can suggest relevant design themes based on designs shared by the user on social media. For example, the reception desk can also analyze posts from design influencers followed by the user and suggest relevant input content. For example, the reception desk can predict and suggest design themes that the user might be interested in based on their social media activity. In this way, relevant input content can be suggested by analyzing the user's social media activity. Some or all of the above processing in the reception desk may be performed using generative AI, or not. For example, the reception desk can use generative AI to analyze a user's social media activity and suggest relevant input content.

[0079] The trend analysis unit can estimate the user's emotions and adjust the trend analysis criteria based on those estimated emotions. For example, if the user is relaxed, the trend analysis unit can analyze a wide range of design trends to provide inspiration. If the user is in a hurry, for example, the trend analysis unit can focus on analyzing the most popular design trends. If the user is excited, for example, the trend analysis unit can prioritize analyzing visually stimulating design trends. This allows for appropriate trend analysis tailored to the user's situation by adjusting the trend analysis criteria based on the user's emotions. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above-described processes in the trend analysis unit may be performed using generative AI or not. For example, the trend analysis unit can use generative AI to estimate the user's emotions and adjust the trend analysis criteria based on those estimated emotions.

[0080] The trend analysis unit can predict current trends by referring to past trend data. For example, the trend analysis unit can analyze design trend data from the past few years to predict current trends. The trend analysis unit can also predict trends suitable for the current season by referring to seasonal design trends. For example, the trend analysis unit can identify periodically repeating trends from past trend data to predict current trends. In this way, current trends can be predicted by referring to past trend data. Past trend data is obtained based on, for example, past market data, user behavior history, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to predict current trends by referring to past trend data.

[0081] The trend analysis unit can apply different trend analysis methods to each design category. For example, it can apply different trend analysis methods to interior design and fashion design. It can also apply different trend analysis methods to graphic design and product design. It can also apply different trend analysis methods to web design and packaging design. By applying different trend analysis methods to each design category, it becomes possible to perform trend analysis appropriate for each category. The trend analysis methods are applied based on, for example, category-specific analysis algorithms and evaluation criteria. Some or all of the above-described processes in the trend analysis unit may be performed using generative AI, or they may not be performed using generative AI. For example, the trend analysis unit can use generative AI to apply different trend analysis methods to each design category.

[0082] The trend analysis unit can estimate the user's emotions and adjust the display method of the trend analysis results based on the estimated user emotions. For example, if the user is nervous, the trend analysis unit can provide a simple and highly visible display method. For example, if the user is relaxed, the trend analysis unit can also provide a display method that includes detailed information. For example, if the user is in a hurry, the trend analysis unit can also provide a display method that gets straight to the point. In this way, by adjusting the display method of the trend analysis results based on the user's emotions, an appropriate display method can be provided according to the user's situation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the trend analysis unit may be performed using generative AI or not. For example, the trend analysis unit can use generative AI to estimate the user's emotions and adjust the display method of the trend analysis results based on the estimated user emotions.

[0083] The trend analysis unit can analyze changes in trends based on the timing of idea submission. For example, the trend analysis unit can analyze changes in trends based on when ideas were submitted. The trend analysis unit can also analyze changes in trends based on the timing of idea submissions for each season. The trend analysis unit can also analyze changes in trends based on ideas submitted during specific events or festivals. This makes it possible to perform trend analysis according to the time of year by analyzing changes in trends based on the timing of idea submission. The timing of idea submission is determined based on, for example, the submission date and time, the frequency of submission, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to analyze changes in trends based on the timing of idea submission.

[0084] The trend analysis unit can analyze trends by referring to relevant market data. For example, the trend analysis unit can analyze trends by referring to sales data of relevant markets. The trend analysis unit can also analyze trends by referring to consumer behavior data of relevant markets. The trend analysis unit can also analyze trends by referring to design data of competitors in relevant markets. This allows for a more accurate analysis of trends by referring to relevant market data. Relevant market data is obtained based on, for example, market research data, competitor analysis data, etc. Some or all of the above processing in the trend analysis unit may be performed using generative AI, or not. For example, the trend analysis unit can use generative AI to analyze trends by referring to relevant market data.

[0085] The mockup generation unit can estimate the user's emotions and adjust the mockup generation method based on the estimated user emotions. For example, if the user is relaxed, the mockup generation unit can generate a mockup that progresses at a relaxed pace. If the user is in a hurry, the mockup generation unit can also generate a mockup that can be generated in the shortest possible time. If the user is excited, the mockup generation unit can also generate a mockup with visually stimulating effects. In this way, by adjusting the mockup generation method based on the user's emotions, it is possible to generate an appropriate mockup according to the user's situation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the mockup generation unit may be performed using a generative AI or not. For example, the mockup generation unit can use a generative AI to estimate the user's emotions and adjust the mockup generation method based on the estimated user emotions.

[0086] The mockup generation unit can optimize its generation algorithm by referring to past design mockups. For example, the mockup generation unit can analyze previously generated design mockups and optimize its generation algorithm. The mockup generation unit can also customize its generation algorithm by referring to past design mockups used by the user. For example, the mockup generation unit can improve its generation algorithm based on successful examples of past design mockups. This allows for the optimization of the generation algorithm by referring to past design mockups, enabling the generation of more accurate mockups. The generation algorithm is optimized based on, for example, the machine learning model used, the training dataset, etc. Some or all of the above processes in the mockup generation unit may be performed using or without generation AI. For example, the mockup generation unit can use generation AI to optimize its generation algorithm by referring to past design mockups.

[0087] The mockup generation unit can apply different generation algorithms to each design category. For example, it can apply different generation algorithms to interior design and fashion design. It can also apply different generation algorithms to graphic design and product design. It can also apply different generation algorithms to web design and packaging design. By applying different generation algorithms to each design category, it is possible to generate mockups suitable for each category. The generation algorithm is applied based on, for example, a category-specific generation algorithm and evaluation criteria. Some or all of the above-described processes in the mockup generation unit may be performed using a generation AI, or they may not be performed using a generation AI. For example, the mockup generation unit can use a generation AI to apply different generation algorithms to each design category.

[0088] The mockup generation unit can estimate the user's emotions and adjust the display method of the mockup based on the estimated user emotions. For example, if the user is nervous, the mockup generation unit can provide a simple and highly visible display method. For example, if the user is relaxed, the mockup generation unit can also provide a display method that includes detailed information. For example, if the user is in a hurry, the mockup generation unit can also provide a display method that gets straight to the point. In this way, by adjusting the display method of the mockup based on the user's emotions, an appropriate display method can be provided according to the user's situation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or a generative AI. The generative AI is a text generation AI (e.g., LLM) or a multimodal generation AI, but is not limited to such examples. Some or all of the above processing in the mockup generation unit may be performed using a generative AI or not. For example, the mockup generation unit can use a generative AI to estimate the user's emotions and adjust the display method of the mockup based on the estimated user emotions.

[0089] The mockup generation unit can generate highly relevant mockups by considering the user's geographical location information. For example, if the user is in a specific region, the mockup generation unit will generate a design mockup related to that region. The mockup generation unit can also generate region-specific design mockups based on the user's geographical location information. For example, if the user is traveling, the mockup generation unit can generate mockups related to the culture and design of the destination. This allows for the generation of highly relevant mockups by considering the user's geographical location information. Geographical location information is obtained, for example, based on GPS data, IP address, etc. Some or all of the above processing in the mockup generation unit may be performed using a generation AI, or not. For example, the mockup generation unit can use a generation AI to generate highly relevant mockups by considering the user's geographical location information.

[0090] The mockup generation unit can improve the accuracy of mockup generation by referring to relevant literature. For example, the mockup generation unit can improve its generation algorithm by referring to the latest design-related literature. The mockup generation unit can also improve the accuracy of mockup generation by referring to literature on design theory. The mockup generation unit can also optimize its generation algorithm by referring to literature on past design projects. In this way, the accuracy of mockup generation can be improved by referring to relevant literature. Relevant literature is obtained based on academic papers, technical reports, etc. Some or all of the above processing in the mockup generation unit may be performed using a generation AI, or not. For example, the mockup generation unit can use a generation AI to improve the accuracy of mockup generation by referring to relevant literature.

[0091] The service provider can estimate the user's emotions and determine the priority of mockups to offer based on the estimated emotions. For example, if the user is relaxed, the service provider can offer multiple mockups to increase the user's options. If the user is in a hurry, the service provider can prioritize offering the most suitable mockup. If the user is excited, the service provider can prioritize offering a visually stimulating mockup. By prioritizing mockups based on the user's emotions, the service provider can offer appropriate mockups tailored to the user's situation. Emotion estimation is achieved using an emotion estimation function, such as an emotion engine or generative AI. Generative AI includes, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the processing described above in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to estimate the user's emotions and determine the priority of mockups to offer based on the estimated emotions.

[0092] The delivery unit can select the optimal delivery method by referring to past user feedback. For example, the delivery unit may prioritize the delivery method of mockups that users have previously given high ratings to. The delivery unit may also analyze past user feedback and customize the optimal delivery method. For example, the delivery unit may avoid delivery methods that users have previously expressed dissatisfaction with and select improved methods. This allows the delivery unit to select the optimal delivery method by referring to past user feedback. Past feedback is obtained based on, for example, user evaluation comments and feedback frequency. Some or all of the above processing in the delivery unit may be performed using generative AI or not. For example, the delivery unit can use generative AI to select the optimal delivery method by referring to past user feedback.

[0093] The service provider can customize the content offered based on the user's current project status. For example, the service provider may prioritize providing mockups related to the user's ongoing projects. The service provider may also provide appropriate mockups according to the progress of the user's projects. The service provider may also provide customized mockups based on the theme of the user's projects. This allows the service provider to provide highly relevant mockups by customizing the content based on the user's current project status. The current project status is identified, for example, based on the progress of the project, related tasks, etc. Some or all of the above processing in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to customize the content offered based on the user's current project status.

[0094] The service provider can estimate the user's emotions and adjust how the mockups are displayed based on the estimated emotions. For example, if the user is nervous, the service provider can provide a simple and highly visible display method. If the user is relaxed, the service provider can also provide a display method that includes detailed information. If the user is in a hurry, the service provider can also provide a display method that gets straight to the point. By adjusting how the mockups are displayed based on the user's emotions, the service provider can provide an appropriate display method that suits the user's situation. Emotion estimation is achieved using an emotion estimation function, for example, using an emotion engine or generative AI. Generative AI is, but is not limited to, text generation AI (e.g., LLM) or multimodal generation AI. Some or all of the above processing in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to estimate the user's emotions and adjust how the mockups are displayed based on the estimated emotions.

[0095] The service provider can select the optimal service delivery method by considering the user's device information. For example, if the user is using a smartphone, the service provider can provide a display method that matches the screen size. For example, if the user is using a tablet, the service provider can also provide a display method optimized for a larger screen. For example, if the user is using a desktop, the service provider can also provide a display method that includes detailed information. This allows the service provider to select the optimal service delivery method by considering the user's device information. Device information is obtained based on, for example, the device type and OS version. Some or all of the processing described above in the service provider may be performed using a generative AI, or not. For example, the service provider can use a generative AI to select the optimal service delivery method by considering the user's device information.

[0096] The service provider can analyze the user's social media activity and customize the content offered. For example, the service provider can provide relevant mockups based on designs shared by the user on social media. The service provider can also analyze the posts of design influencers followed by the user and provide relevant mockups. The service provider can also predict and provide mockups that the user might be interested in based on their social media activity. This allows the service provider to provide relevant mockups by analyzing the user's social media activity. Social media activity is obtained based on, for example, posts, like history, etc. Some or all of the above processing in the service provider may be performed using generative AI or not. For example, the service provider can use generative AI to analyze the user's social media activity and customize the content offered.

[0097] The system according to the embodiment is not limited to the example described above, and various modifications are possible, for example, as follows.

[0098] The design support system can further estimate the user's emotions and adjust the style of the design mockup based on those emotions. For example, if the user is stressed, it can provide a simple and calming design. If the user is excited, it can provide a vibrant and stimulating design. Furthermore, if the user is relaxed, it can provide a design with soft colors and a relaxing effect. This allows for a more personalized design experience by providing design mockups that respond to the user's emotions.

[0099] The design support system can further analyze the user's past design history and generate design mockups based on past successes and failures. For example, it can prioritize incorporating design elements that received high praise in the past, while avoiding those that were unpopular. Furthermore, it can learn the user's preferences from their past design history and generate individually customized design mockups. This allows the system to leverage the user's past design history to provide more accurate design mockups.

[0100] The design support system can also generate design mockups that take the user's geographical location into account. For example, if the user is in a specific region, it can provide designs based on the culture and trends of that region. If the user is traveling, it can generate mockups related to the culture and design of their destination. Furthermore, it can incorporate region-specific design elements based on the user's geographical location. This allows for a more relevant design experience by providing design mockups that consider the user's geographical location.

[0101] The design support system can further analyze users' social media activity and generate relevant design mockups. For example, it can suggest relevant design themes based on designs users have shared on social media. It can also analyze posts from design influencers users follow and provide relevant design mockups. Furthermore, it can predict and suggest design themes that users might be interested in based on their social media activity. This allows for the provision of more personalized design mockups by leveraging users' social media activity.

[0102] The design support system can further generate design mockups while considering the user's device information. For example, if the user is using a smartphone, it can provide a design optimized for the screen size. Similarly, if the user is using a tablet, it can provide a design optimized for the larger screen. Furthermore, if the user is using a desktop, it can provide a design that includes detailed information. By providing design mockups that take the user's device information into account, a more user-friendly design experience can be achieved.

[0103] Design support systems can further estimate user emotions and adjust the entire design process based on those estimates. For example, if a user is stressed, a simple and intuitive interface can be provided to reduce the burden of work. If a user is relaxed, detailed guides and options can be provided to increase creative freedom. Furthermore, if a user is in a hurry, shortcuts and automation features can be provided to speed up the process. By providing a design process that responds to the user's emotions, a more efficient work environment can be achieved.

[0104] Design support systems can further estimate user emotions and provide design feedback based on those emotions. For example, if a user is feeling anxious, positive feedback can be prioritized to boost their motivation. If a user is confident, constructive criticism can be provided to point out areas for design improvement. Furthermore, if a user is tired, concise and to-the-point feedback can be provided to reduce their workload. In this way, providing design feedback tailored to the user's emotions can support more effective design improvements.

[0105] The design support system can further estimate the user's emotions and adjust how design mockups are presented based on those emotions. For example, if the user is nervous, a simple and highly visible display method can be provided. If the user is relaxed, a display method including detailed information can be provided. Furthermore, if the user is in a hurry, a display method that gets straight to the point can be provided. In this way, by adjusting how design mockups are presented based on the user's emotions, an appropriate display method can be provided according to the user's situation.

[0106] The design support system can further estimate the user's emotions and adjust the design mockup generation algorithm based on those emotions. For example, if the user is relaxed, it can generate a mockup that progresses at a leisurely pace. If the user is in a hurry, it can provide a mockup that can be generated in the shortest possible time. Furthermore, if the user is excited, it can generate a mockup with visually stimulating effects. In this way, by adjusting the design mockup generation algorithm based on the user's emotions, it is possible to provide a mockup appropriate to the user's situation.

[0107] The design support system can further estimate the user's emotions and adjust the evaluation criteria for design mockups based on those emotions. For example, if the user is relaxed, it can evaluate a wide range of design elements and provide inspiration. If the user is in a hurry, it can focus on evaluating the most important design elements. Furthermore, if the user is excited, it can prioritize evaluating visually stimulating design elements. By adjusting the evaluation criteria for design mockups based on the user's emotions, it can provide appropriate evaluations tailored to the user's situation.

[0108] The following briefly describes the processing flow for example form 2.

[0109] Step 1: The reception desk receives input from users. User input includes text input, image uploads, and survey responses. For example, it accepts users to type "modern interior design," upload images, and answer survey questions. Step 2: The Trend Analysis Department uses generative AI to perform trend analysis based on the information received by the Reception Department. Trend analysis is performed based on the data sources used, analysis algorithms, evaluation criteria, etc. For example, generative AI is used to analyze the latest design trends. Step 3: The mockup generation unit uses a generation AI to generate design mockups based on the analysis results obtained by the trend analysis unit. The design mockups are generated based on image format, resolution, design elements, etc. For example, a design mockup based on trends is generated using the generation AI. Step 4: The provider unit provides the user with the design mockup generated by the mockup generation unit. The provider unit displays the generated design mockup to the user. For example, it can be provided through a web application or a mobile application. It can also be sent via email or printed out.

[0110] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0111] Data generation model 58 is a form of so-called generative AI (Artificial Intelligence). An example of data generation model 58 is ChatGPT (registered trademark) (Internet search).<URL: https: / / openai.com / blog / chatgpt> Examples of generative AI include text generation AI, image generation AI, and multimodal generation AI. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats from audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVMs), k-means clustering, convolutional neural networks (CNNs), recurrent neural networks (RNNs), generative adversarial networks (GANs), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each of the above parts is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example.Furthermore, processing performed by AI, including generative AI, may be replaced with rule-based processing, and rule-based processing may be replaced with processing performed by AI, including generative AI.

[0112] Furthermore, the processing performed by the data processing system 10 described above is carried out by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but it may also be carried out by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0113] Each of the multiple elements described above, including the reception unit, trend analysis unit, mockup generation unit, and provision unit, is implemented in at least one of the smart device 14 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the smart device 14 and receives input from the user. The trend analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and performs trend analysis using a generation AI. The mockup generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates a design mockup using a generation AI. The provision unit is implemented by the control unit 46A of the smart device 14 and provides the generated design mockup to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0114] [Second Embodiment] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0115] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0116] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0117] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0118] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0119] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0120] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0121] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing by the processor 28. The storage 32 stores the specific processing program 56.

[0122] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0123] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0124] In the smart glasses 214, specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0125] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0126] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0127] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0128] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart glasses 214 or an external device, and the smart glasses 214 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0129] Each of the multiple elements described above, including the reception unit, trend analysis unit, mockup generation unit, and provision unit, is implemented in at least one of the smart glasses 214 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the smart glasses 214 and receives input from the user. The trend analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and performs trend analysis using generation AI. The mockup generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates a design mockup using generation AI. The provision unit is implemented by the control unit 46A of the smart glasses 214 and provides the generated design mockup to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0130] [Third Embodiment] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0131] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0132] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0133] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0134] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0135] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0136] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0137] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0138] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0139] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0140] In the headset terminal 314, specific processing is performed by the processor 46. The storage 50 stores a specific program 60. The processor 46 reads the specific program 60 from the storage 50 and executes the read specific program 60 on the RAM 48. The specific processing is realized by the processor 46 acting as a control unit 46A according to the specific program 60 executed on the RAM 48. The headset terminal 314 also has a data generation model 58 and an emotion identification model 59, similar to the data generation model and emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0141] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0142] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0143] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0144] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset terminal 314, but may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset terminal 314. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the headset terminal 314 or an external device, and the headset terminal 314 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0145] Each of the multiple elements described above, including the reception unit, trend analysis unit, mockup generation unit, and provision unit, is implemented in at least one of the headset terminal 314 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the headset terminal 314 and receives input from the user. The trend analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and performs trend analysis using generation AI. The mockup generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates a design mockup using generation AI. The provision unit is implemented by the control unit 46A of the headset terminal 314 and provides the generated design mockup to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0146] [Fourth Embodiment] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[0147] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0148] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN and / or LAN.

[0149] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[0150] The microphone 238 receives voice signals from the user and accepts instructions from the user. The microphone 238 captures the voice signals from the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0151] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS image sensor or CCD image sensor, which captures images of the area around the user (for example, an imaging range defined by a field of view equivalent to the field of vision of a typical healthy person).

[0152] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0153] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. The robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[0154] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0155] The processor 28 reads a specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 acting as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0156] Storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate the user's emotions using the emotion identification model 59 and perform identification processing using the user's emotions. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotions, including but not limited to these examples. Furthermore, emotion estimation and prediction also include, for example, emotion analysis.

[0157] In robot 414, specific processing is performed by processor 46. A specific program 60 is stored in storage 50. Processor 46 reads the specific program 60 from storage 50 and executes it on RAM 48. The specific processing is achieved by processor 46 acting as a control unit 46A according to the specific program 60 executed on RAM 48. Robot 414 also has data generation model 58 and emotion identification model 59, similar to those of the robot, and can perform processing similar to that of the specific processing unit 290 using these models.

[0158] Furthermore, other devices besides the data processing device 12 may also have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 obtains processing results (such as prediction results) using the data generation model 58 by communicating with the server device that has the data generation model 58. Also, the data processing device 12 may be a server device or a terminal device owned by the user (for example, a mobile phone, robot, home appliance, etc.).

[0159] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0160] The data generation model 58 is a so-called generative AI. An example of a data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and inference data such as audio data representing speech, text data representing text, and image data representing images (e.g., still image data or video data). The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference result in one or more data formats such as audio data, text data, and image data. The data generation model 58 includes, for example, text generation AI, image generation AI, and multimodal generation AI. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specific processing unit 290 performs the specific processing described above using the data generation model 58. The data generation model 58 may be a fine-tuned model that outputs inference results from prompts that do not contain instructions, in which case the data generation model 58 can output inference results from prompts that do not contain instructions. In the data processing device 12, etc., there are multiple types of data generation models 58, and the data generation model 58 includes AI other than generative AI. AI other than generative AI includes, for example, linear regression, logistic regression, decision trees, random forests, support vector machines (SVM), k-means clustering, convolutional neural networks (CNN), recurrent neural networks (RNN), generative adversarial networks (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. Also, the AI ​​may be an AI agent. Furthermore, when the processing of each part described above is performed by the AI, the processing may be performed by the AI ​​in part or in whole, but is not limited to this example. Also, processing performed by an AI including a generative AI may be replaced by rule-based processing, and rule-based processing may be replaced by processing performed by an AI including a generative AI.

[0161] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is performed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but it may also be performed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. In addition, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the robot 414 or an external device, and the robot 414 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0162] Each of the multiple elements described above, including the reception unit, trend analysis unit, mockup generation unit, and provision unit, is implemented in at least one of the robot 414 and the data processing unit 12. For example, the reception unit is implemented by the control unit 46A of the robot 414 and receives input from the user. The trend analysis unit is implemented by the specific processing unit 290 of the data processing unit 12 and performs trend analysis using a generation AI. The mockup generation unit is implemented by the specific processing unit 290 of the data processing unit 12 and generates a design mockup using a generation AI. The provision unit is implemented by the control unit 46A of the robot 414 and provides the generated design mockup to the user. The correspondence between each unit and the device or control unit is not limited to the example described above and can be modified in various ways.

[0163] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0164] Figure 9 shows the emotion map 400, in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[0165] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[0166] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[0167] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, and motorcycles, emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated based, for example, on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[0168] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[0169] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[0170] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing method for the specific process may be used, which includes computer 22 and multiple other computers.

[0171] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[0172] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0173] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[0174] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[0175] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[0176] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[0177] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[0178] Furthermore, although the above-described examples were divided into four embodiments, some or all of these embodiments may be combined. Also, the smart device 14, smart glasses 214, headset terminal 314, and robot 414 are just examples, and they may be combined, or other devices may be used. Also, although the above-described examples were divided into two embodiments, Embodiment 1 and Embodiment 2, these may be combined.

[0179] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and other things that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[0180] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[0181] (Note 1) A reception area that receives input from users, A trend analysis unit performs trend analysis based on the information received by the aforementioned reception unit, A mockup generation unit generates a design mockup based on the analysis results obtained by the trend analysis unit, The system includes a provisioning unit that provides the user with the design mockup generated by the mockup generation unit. A system characterized by the following features. (Note 2) The aforementioned trend analysis department, Using generative AI to analyze the latest design trends The system described in Appendix 1, characterized by the features described herein. (Note 3) The aforementioned mockup generation unit is, Generating AI to create trend-based design mockups The system described in Appendix 1, characterized by the features described herein. (Note 4) The aforementioned supply unit is, Provide the generated design mockup to the user. The system described in Appendix 1, characterized by the features described herein. (Note 5) The aforementioned reception unit is Learns user preferences and past design history. The system described in Appendix 1, characterized by the features described herein. (Note 6) The aforementioned mockup generation unit is, Generate customized design mockups based on user preferences. The system described in Appendix 1, characterized by the features described herein. (Note 7) The aforementioned reception unit is It estimates the user's emotions and prioritizes input content based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 8) The aforementioned reception unit is It analyzes the user's past input history and suggests the optimal input method. The system described in Appendix 1, characterized by the features described herein. (Note 9) The aforementioned reception unit is Filter input based on the user's current projects and areas of interest. The system described in Appendix 1, characterized by the features described herein. (Note 10) The aforementioned reception unit is It estimates the user's emotions and adjusts how the input content is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 11) The aforementioned reception unit is The system prioritizes accepting input that is highly relevant, taking into account the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 12) The aforementioned reception unit is Analyzes users' social media activity and suggests relevant inputs. The system described in Appendix 1, characterized by the features described herein. (Note 13) The aforementioned trend analysis department, We estimate user sentiment and adjust the criteria for trend analysis based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 14) The aforementioned trend analysis department, Predicting current trends by referring to past trend data The system described in Appendix 1, characterized by the features described herein. (Note 15) The aforementioned trend analysis department, Applying different trend analysis methods to each design category. The system described in Appendix 1, characterized by the features described herein. (Note 16) The aforementioned trend analysis department, It estimates user sentiment and adjusts how trend analysis results are displayed based on the estimated user sentiment. The system described in Appendix 1, characterized by the features described herein. (Note 17) The aforementioned trend analysis department, Analyze trend changes based on the timing of idea submission. The system described in Appendix 1, characterized by the features described herein. (Note 18) The aforementioned trend analysis department, Analyze trends by referring to relevant market data. The system described in Appendix 1, characterized by the features described herein. (Note 19) The aforementioned mockup generation unit is, We estimate the user's emotions and adjust the mockup generation method based on the estimated user emotions. The system described in Appendix 1, characterized by the features described herein. (Note 20) The aforementioned mockup generation unit is, Optimize the generation algorithm by referring to past design mockups. The system described in Appendix 1, characterized by the features described herein. (Note 21) The aforementioned mockup generation unit is, Applying different generation algorithms to each design category The system described in Appendix 1, characterized by the features described herein. (Note 22) The aforementioned mockup generation unit is, It estimates the user's emotions and adjusts how the mockup is displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 23) The aforementioned mockup generation unit is, Generate highly relevant mockups considering the user's geographical location. The system described in Appendix 1, characterized by the features described herein. (Note 24) The aforementioned mockup generation unit is, Refer to related literature to improve the accuracy of mockup generation. The system described in Appendix 1, characterized by the features described herein. (Note 25) The aforementioned supply unit is, We estimate the user's emotions and determine the priority of the mockups to provide based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 26) The aforementioned supply unit is, We select the optimal delivery method by referring to past user feedback. The system described in Appendix 1, characterized by the features described herein. (Note 27) The aforementioned supply unit is, Customize the offerings based on the user's current project status. The system described in Appendix 1, characterized by the features described herein. (Note 28) The aforementioned supply unit is, It estimates the user's emotions and adjusts how the provided mockups are displayed based on those estimated emotions. The system described in Appendix 1, characterized by the features described herein. (Note 29) The aforementioned supply unit is, The optimal delivery method is selected considering the user's device information. The system described in Appendix 1, characterized by the features described herein. (Note 30) The aforementioned supply unit is, Analyze users' social media activity to customize the content offered. The system described in Appendix 1, characterized by the features described herein. [Explanation of symbols]

[0182] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots

Claims

1. A reception area that receives input from users, A trend analysis unit performs trend analysis based on the information received by the aforementioned reception unit, A mockup generation unit generates a design mockup based on the analysis results obtained by the trend analysis unit, The system includes a provisioning unit that provides the user with the design mockup generated by the mockup generation unit. A system characterized by the following features.

2. The aforementioned trend analysis department, Using generative AI to analyze the latest design trends. The system according to feature 1.

3. The aforementioned mockup generation unit is: The AI ​​generates design mockups based on current trends. The system according to feature 1.

4. The aforementioned supply unit is, Provide the generated design mockup to the user. The system according to feature 1.

5. The aforementioned reception unit is Learns user preferences and past design history. The system according to feature 1.

6. The aforementioned mockup generation unit is: Generate customized design mockups based on user preferences. The system according to feature 1.

7. The aforementioned reception unit is It estimates the user's emotions and prioritizes input content based on those estimated emotions. The system according to feature 1.

8. The aforementioned reception unit is It analyzes the user's past input history and suggests the optimal input method. The system according to feature 1.

Images