system

Abstract

We provide the system. [Solution] An input means for the user to input design instructions to the system in natural language, A natural language processing means that analyzes natural language instructions input through an input means, An instruction interpretation means interprets specific design requirements and placement conditions based on instructions analyzed by a natural language processing means, An image generation means that automatically generates design drawings based on the design requirements obtained by the instruction interpretation means, A means for verifying the results, which includes checking the quality of the generated design drawings and verifying their consistency with the instructions, An output means for providing the user with the final confirmed design drawings, A system that includes this.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a persona chatbot control method performed by at least one processor, including steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance as a response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When manually drawing a design drawing, it takes a huge amount of time, and the number of personnel with specialized skills is limited, which has become a major problem in many industries. Also, the high cost of drawing tools and their operation costs are issues. Furthermore, in the design of wireless devices, it is difficult to accurately calculate the radio wave coverage range and perform the layout across multiple floors. There is a need to efficiently solve these problems and create high-quality design drawings in an automated manner.

Means for Solving the Problems

[0005] The present invention solves the above problems by providing a system including an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, and an output means. In this system, the user inputs instructions in natural language, and the natural language analysis means analyzes them. The analyzed instructions are converted into specific design requirements by the instruction interpretation means. Subsequently, a design drawing is generated using the image generation means, and its quality is confirmed by the result confirmation means. Finally, the design drawing is provided to the user through the output means. This process improves the efficiency of design drawing creation, enabling significant reductions in time and cost.

[0006] An "input method" is an interface that allows the user to input design instructions to the system in natural language.

[0007] "Natural language analysis means" refers to a function that analyzes natural language instructions input through an input means and performs processing such as syntactic analysis and tokenization.

[0008] "Instruction interpretation means" refers to a function that interprets specific design requirements and placement conditions based on instructions analyzed by natural language processing means.

[0009] The "image generation means" is an image generation engine for automatically generating design drawings based on the design requirements obtained by the instruction interpretation means.

[0010] A "result verification means" is a function for checking the quality of the generated design drawings and verifying that they match the instructions.

[0011] The "output means" is an interface for providing the user with the final, confirmed design drawings. [Brief explanation of the drawing]

[0012] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2]This is a conceptual diagram showing an example of the essential functions of a data processing device and a smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10] This shows an emotion map where multiple emotions are mapped. [Figure 11] This is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] This is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] This is a sequence diagram showing the processing flow of the data processing system in Example 2, which incorporates an emotion engine. [Figure 14] This is a sequence diagram showing the processing flow of the data processing system in Application Example 2, which combines an emotion engine. [Modes for carrying out the invention]

[0013] Hereinafter, an example of an embodiment of the system relating to the technology of this disclosure will be described with reference to the attached drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a processor with a reference numeral (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0016] In the following embodiments, a RAM (Random Access Memory) with a reference numeral is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0017] In the following embodiments, a storage with a reference numeral is one or more non-volatile storage devices that store various programs and various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes, and the like.

[0018] In the following embodiments, a communication I / F (Interface) with a reference numeral is an interface including a communication processor and an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark), and the like.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0020] [First Embodiment]

[0021] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0022] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0023] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0024] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0025] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0026] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0027] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0028] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0029] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0030] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0031] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0032] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0033] This invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, and an output means, and aims to streamline and automate the process of creating design drawings.

[0034] System Configuration

[0035] 1. User input

[0036] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0037] 2. Sending instructions

[0038] The terminal sends natural language instructions entered by the user to the server, using an appropriate communication protocol (e.g., HTTP POST request).

[0039] 3. Analysis of Instructions

[0040] The server is equipped with natural language processing capabilities to analyze received instructions. Specifically, it uses a natural language processing engine to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "5G wireless device," "office building," "3 devices per floor," and "8 rooms."

[0041] 4. Interpretation of instructions and extraction of requirements

[0042] The server-side instruction interpretation mechanism concretizes the design requirements based on extracted keywords and phrases. This process defines the optimal placement of wireless devices and the floor coverage area, among other things.

[0043] 5. Generating blueprints

[0044] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. For generation, an existing image generation engine (e.g., DALL-E) is used to create wireless equipment layout diagrams for each floor.

[0045] 6. Check the generation results

[0046] The result verification mechanism checks the generated design drawings and verifies that they match the user's instructions. An automatic correction function is also included, and further adjustments to the layout are made as needed.

[0047] 7. Submit the results

[0048] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0049] Specific example

[0050] 1. User input

[0051] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0052] 2. Sending instructions

[0053] The terminal sends this instruction to the server.

[0054] 3. Analysis of Instructions

[0055] The natural language processing system installed on the server analyzes the instructions and extracts elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0056] 4. Interpretation of instructions and extraction of requirements

[0057] The server's instruction interpretation mechanism uses this information to specify the design requirements for the placement of wireless equipment on each floor. The optimal placement of wireless equipment is determined based on the floor area and room layout.

[0058] 5. Generating blueprints

[0059] The image generation means generates design drawings based on the above design requirements. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0060] 6. Check the generation results

[0061] The result verification mechanism checks for consistency between the generated design drawings and the instructions, and provides feedback for corrections if necessary.

[0062] 7. Submit the results

[0063] The server sends the verified blueprint to the terminal. The user reviews it and enters further instructions if necessary.

[0064] This system allows users to obtain high-quality blueprints quickly without requiring advanced expertise or expensive tools.

[0065] The following describes the processing flow.

[0066] Step 1:

[0067] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0068] Step 2:

[0069] Terminal: Sends the input natural language instructions to the server. This data is sent using an HTTP POST request.

[0070] Step 3:

[0071] Server: Activates natural language processing equipment to analyze the received natural language instructions. First, the instructions are tokenized, and the sentence structure is analyzed.

[0072] Step 4:

[0073] Server: Extracts necessary keys and phrases from the text analyzed by the natural language processing system. For example, it identifies elements such as "3-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[0074] Step 5:

[0075] Server: Based on the extracted keywords for instruction interpretation, specific design requirements are created. In this process, the area of ​​each floor and the layout of the rooms are considered to determine the optimal placement of the wireless equipment.

[0076] Step 6:

[0077] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using the image generation engine (e.g., DALL-E).

[0078] Step 7:

[0079] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0080] Step 8:

[0081] Server: Sends the verified design drawings to the terminal. The design drawings are sent in PDF or PNG format.

[0082] Step 9:

[0083] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0084] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge.

[0085] (Example 1)

[0086] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0087] Traditional design drawing processes require advanced expertise and expensive software tools, resulting in significant costs and time investment. Furthermore, manual design work is prone to human error, leading to inconsistent quality in the final product. Additionally, design changes and modifications require further time and effort to address.

[0088] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0089] In this invention, the server includes an input means for the user to input specific instructions regarding the design drawing in natural language, a natural language analysis means for analyzing the input natural language instructions, an instruction interpretation means for concretizing the design requirements based on extracted keywords and phrases, an image generation means for generating a visual design drawing based on the design requirements, a result confirmation means for confirming that the generated design drawing matches the instruction content, and an output means for transmitting the confirmed design drawing to a terminal. This makes it possible to automatically create high-quality design drawings in a short time without specialized knowledge, enabling efficient and highly accurate design work.

[0090] An "input means" is a device or interface that allows a user to input specific instructions regarding a design drawing in natural language.

[0091] A "natural language processing tool" is an engine or software that analyzes input natural language instructions and extracts important keywords and phrases.

[0092] A "direction interpretation means" is a processing device or algorithm for concretizing design requirements based on extracted keywords or phrases and determining the requirements of the design drawing.

[0093] "Image generation means" refers to an engine or software for generating visual design drawings based on design requirements.

[0094] A "result verification means" is a device or system that has the function of checking whether the generated design drawings match the user's instructions and making corrections as necessary.

[0095] "Output means" refers to a device or interface that transmits the verified design drawings to the user's terminal, allowing the user to review the design drawings.

[0096] This invention is a system for streamlining and automating the process of creating design drawings. This system includes multiple means for consistently processing everything from user input to the generation of design drawings. Specific embodiments of this system are described below.

[0097] System Configuration

[0098] This system uses the following hardware and software:

[0099] Hardware: User devices (e.g., PCs, tablets), servers

[0100] software:

[0101] Input means

[0102] Natural language processing tools (e.g., natural language processing engines such as GPT-3)

[0103] Instruction interpretation means

[0104] Image generation means (e.g., image generation engine such as DALL-E)

[0105] Means of checking results

[0106] Output means

[0107] Program Processing Description

[0108] 1. User input

[0109] Users input specific instructions regarding the design plans in natural language via their own devices. For example, they might input, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This allows the system to receive detailed requirements based on the user's needs.

[0110] 2. Sending instructions

[0111] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. This transmission is packaged in an appropriate format, and a simple communication protocol is also implemented to check for communication errors.

[0112] 3. Analysis of Instructions

[0113] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The server tokenizes the instructions to understand the context and extracts important keywords and phrases. For example, it analyzes information such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0114] 4. Interpretation of instructions and extraction of requirements

[0115] The server's instruction interpretation mechanism concretizes design requirements based on extracted keywords and phrases. This process determines details such as the optimal placement of radios and floor coverage. For example, the optimal placement of radios is calculated considering the area of ​​each floor and the layout of the rooms.

[0116] 5. Generating blueprints

[0117] The server uses an image generation engine (e.g., DALL-E) to generate design drawings based on the design requirements. The image generation engine creates a visual design drawing based on input prompts, and generates a layout diagram that visually shows the location and coverage area of ​​the radio equipment.

[0118] 6. Check the generation results

[0119] The server's result verification mechanism checks whether the generated design matches the user's instructions. An automatic correction function also operates, readjusting the previously generated design as needed. During this process, a detailed check is performed to ensure it meets the user's requirements.

[0120] 7. Submit the results

[0121] The server sends the verified blueprint to the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0122] Examples of specific actions

[0123] 1. User input

[0124] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0125] 2. Sending instructions

[0126] The device sends this instruction to the server via an HTTP POST request.

[0127] 3. Analysis of Instructions

[0128] The server's natural language processing engine (such as GPT-3) breaks down the instruction sentence into keywords such as "three-story building," "office building," "Wi-Fi wireless device," and "four devices on each floor" and analyzes them.

[0129] 4. Interpretation of instructions and extraction of requirements

[0130] The server calculates the optimal location for the Wi-Fi wireless router, taking into account the area of ​​each floor and the layout of the rooms.

[0131] 5. Generating blueprints

[0132] The server uses an image generation engine to draw a layout diagram of the radio equipment, creating a visual design drawing.

[0133] 6. Check the generation results

[0134] The result verification mechanism checks the generated design drawings to confirm whether they meet the user's requirements.

[0135] 7. Submit the results

[0136] The server sends the verified blueprints to the terminal, and the user confirms them.

[0137] This system allows users to obtain high-quality blueprints quickly, without requiring advanced expertise or expensive tools.

[0138] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0139] Step 1: User input

[0140] The user uses a terminal to input specific instructions regarding the design drawings in natural language. An example of input would be a prompt message such as, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This instruction would be the input. Through this operation, the user communicates the design requirements in detail to the system.

[0141] Step 2: Sending instructions

[0142] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. During this process, the input instructions are packaged into an appropriate format. The sent prompt text becomes the output. The terminal also checks for transmission errors through this operation.

[0143] Step 3: Analyzing the instructions

[0144] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The input to this operation is the sent prompt sentence, and the output is tokenized keywords and phrases. Specifically, the instruction sentence is broken down into important keywords such as "three-story building," "office building," "Wi-Fi radio," and "four units per floor."

[0145] Step 4: Interpretation of instructions and requirements extraction

[0146] The server's instruction interpretation mechanism concretizes design requirements based on tokenized keywords and phrases. The input to this operation is the extracted keywords and phrases, and the output is the specific content of the design requirements. Specifically, it calculates the optimal placement of wireless devices, taking into account the area of ​​each floor and the layout of the rooms.

[0147] Step 5: Generate the blueprint

[0148] The server uses an image generation means (e.g., DALL-E) to generate design drawings based on the design requirements. The input to this operation is the specified design requirements, and the output is a visual design drawing. The image generation engine creates a visual layout diagram showing the location and coverage area of ​​the radio equipment.

[0149] Step 6: Check the generation results

[0150] The server's result verification mechanism checks the generated blueprint against the user's instructions. The input for this operation is the generated visual blueprint, and the output is the verified blueprint. Furthermore, an automatic correction function is also in operation, including specific actions to readjust the blueprint as needed.

[0151] Step 7: Submit the results

[0152] The server sends the verified blueprint to the user's terminal. The input for this operation is the verified blueprint, and the output is the blueprint displayed on the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0153] (Application Example 1)

[0154] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0155] Designing equipment layouts and production lines within a factory is a task that requires advanced expertise and time, making it difficult to perform efficiently and accurately. Furthermore, there are limited systems that allow users to input instructions in natural language and automatically generate design drawings based on those instructions. A system is needed to solve this problem and enable rapid and accurate equipment layout and production line design within factories.

[0156] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0157] In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and means for automatically generating the layout of factory equipment and production line designs. This makes it possible to efficiently and accurately automatically generate the layout of factory equipment and production line designs based on instructions entered by the user in natural language.

[0158] An "input method" refers to a device or interface that allows a user to input instructions in natural language.

[0159] "Natural language processing tools" is a general term for engines and algorithms that analyze input natural language instructions and perform tokenization and keyword extraction.

[0160] The "instruction interpretation means" is a function that concretizes design requirements based on analyzed keywords and phrases and extracts the information necessary for generating design drawings.

[0161] "Image generation means" is a general term for engines and algorithms that automatically generate design drawings based on design requirements obtained from instruction interpretation means.

[0162] The "result verification means" is a function for confirming that the generated design drawings match the user's instructions and provides feedback for corrections as needed.

[0163] "Output means" refers to the means of providing users with confirmed design drawings, and includes functions for saving and displaying them as digital files.

[0164] "Means for automatically generating equipment layouts and production line designs within a factory" refers to a general term for engines and algorithms that automatically generate design drawings based on the factory layout, required robot types, and work areas.

[0165] This invention provides a system for automating the layout of equipment and production line design within a factory. The system includes an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result verification means, an output means, and means for automatically generating the layout of equipment and production line design within the factory.

[0166] Explain the program's processing in natural language.

[0167] The entire system operates as follows:

[0168] 1. User input instructions:

[0169] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, an instruction might be, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square."

[0170] 2. Sending instructions:

[0171] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request.

[0172] 3. Analysis of instructions:

[0173] The natural language processing (NLP) system installed on the server analyzes the received instructions. Specifically, it uses a natural language processing engine (e.g., GPT-4) to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square."

[0174] 4. Interpretation of instructions and extraction of requirements:

[0175] The server-side instruction interpretation mechanism concretizes the design requirements based on the analyzed keywords and phrases. In this process, the optimal design is created by considering factors such as the factory layout, the type of robots required, and the work area.

[0176] 5. Generating blueprints:

[0177] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E 2) is used to draw design drawings of the equipment layout and production lines within the factory.

[0178] 6. Check the generation results:

[0179] The results verification system checks whether the generated design drawings match the user's instructions. If necessary, it also provides an automatic correction function and offers feedback as needed.

[0180] 7. Submit the results:

[0181] The server sends the verified design drawings to the terminal. The user can then review the design drawings via the terminal and enter further modification instructions as needed.

[0182] Examples of specific cases and prompt statements

[0183] Specific example:

[0184] The user enters the following: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0185] Examples of prompts to input into a generative AI model:

[0186] Natural Language Processing: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0187] This system allows users to efficiently and accurately generate factory equipment layouts and production line designs automatically, even without advanced expertise.

[0188] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0189] Step 1:

[0190] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, they might input instructions such as, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square." The input data is in natural language text format.

[0191] Step 2:

[0192] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request. The input is the natural language instruction entered by the user, and the output is the transmission of the instruction to the server.

[0193] Step 3:

[0194] The server's built-in natural language processing system analyzes the received instructions. Specifically, it uses the generative AI model GPT-4 to tokenize the instructions and extract important keywords and phrases. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square." The input is the user's natural language instructions, and the output is the analyzed keywords and phrases.

[0195] Step 4:

[0196] The server's instruction interpretation mechanism concretizes design requirements based on analyzed keywords and phrases. This process considers factors such as the factory layout, required robot types, and work areas to create an optimal design. The input consists of keywords and phrases from the natural language processing mechanism, while the output is the concretized design requirements.

[0197] Step 5:

[0198] Using the server's image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine such as DALL-E 2 is used to draw design drawings of factory equipment layouts and production lines. The input is the design requirements from the instruction interpretation means, and the output is the generated design drawing.

[0199] Step 6:

[0200] The result verification mechanism verifies that the generated design drawing matches the user's instructions. If necessary, it provides an automatic correction function and offers feedback as needed. The input is the generated design drawing, and the output is the design drawing verified through verification and correction.

[0201] Step 7:

[0202] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input is the verified blueprint, and the output is the blueprint provided to the user.

[0203] In this way, the system quickly and accurately automatically generates factory equipment layouts and production line designs based on the user's natural language instructions.

[0204] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0205] The present invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and an emotion engine, and aims to streamline and automate the design drawing creation process, as well as to recognize and adapt to the user's emotions.

[0206] System Configuration

[0207] 1. User input

[0208] The user inputs specific instructions regarding the design drawings using natural language via a terminal. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0209] 2. Recognition of emotions

[0210] The terminal is equipped with an emotion engine that recognizes the user's emotions in real time from their facial expressions and voice as they input design instructions. This emotion information is sent to the server along with the natural language instructions.

[0211] 3. Sending instructions and emotional data

[0212] The device sends natural language instructions entered by the user and emotion data recognized by the emotion engine to the server. This data is sent using an HTTP POST request.

[0213] 4. Analysis of Instructions

[0214] The server is equipped with natural language processing capabilities to analyze received instructions and sentiment data. It uses a natural language processing engine to tokenize instructions, analyze the sentence structure, and analyze sentiment data.

[0215] 5. Interpretation of instructions and extraction of requirements

[0216] The server-side instruction interpretation mechanism creates specific design requirements based on extracted keywords, phrases, and sentiment data. During this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[0217] 6. Generating blueprints

[0218] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E) is used for generation to create wireless equipment layout diagrams for each floor.

[0219] 7. Check the generation results

[0220] The result verification mechanism verifies the generated design drawings. It checks whether the design drawings match the instructions and whether quality is maintained. If necessary, it performs automated corrections, taking sentiment data into consideration.

[0221] 8. Submit the results

[0222] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0223] Specific example

[0224] 1. User input

[0225] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0226] 2. Recognition of emotions

[0227] The emotion engine recognizes whether the user is stressed or relaxed based on their facial expressions and voice during input.

[0228] 3. Sending instructions and emotional data

[0229] The device sends these instructions and emotional data to the server.

[0230] 4. Analysis of Instructions

[0231] The server analyzes the received instructions using natural language processing to identify elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four devices per floor."

[0232] 5. Interpretation of instructions and extraction of requirements

[0233] The server's instruction interpretation mechanism uses this information to concretize design requirements while considering emotional data. For example, if the user is experiencing stress, the response is simplified to reduce the burden.

[0234] 6. Generating blueprints

[0235] The image generation means generates design drawings based on the design requirements created by the instruction interpretation means. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0236] 7. Check the generation results

[0237] The result verification method confirms that the generated design drawings and instructions match, and makes corrections if necessary, taking emotional data into consideration.

[0238] 8. Submit the results

[0239] The server sends the verified blueprint to the terminal. The user reviews the blueprint and enters further instructions if necessary.

[0240] Through this process, users can create efficient and high-quality blueprints, and the introduction of an emotion engine improves the user experience.

[0241] The following describes the processing flow.

[0242] Step 1:

[0243] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0244] Step 2:

[0245] Terminal: Analyzes the user's facial expressions and voice in real time while they are inputting instructions, and uses an emotion engine to recognize the user's emotional state. For example, it can determine whether the user is stressed or relaxed.

[0246] Step 3:

[0247] Terminal: Sends input natural language instructions and recognized emotion data to the server. Data is sent using an HTTP POST request.

[0248] Step 4:

[0249] Server: Activates natural language processing (NLP) to analyze received natural language instructions and sentiment data. First, it tokenizes the instructions and extracts important keywords and phrases necessary to understand the context. For example, it extracts information such as "3-story building," "office building," "Wi-Fi radio," and "4 units per floor."

[0250] Step 5:

[0251] Server: Analyzes emotional data to understand the user's emotional state. For example, if the user is stressed, it will simplify the response to reduce the burden on the user.

[0252] Step 6:

[0253] Server: Based on the keywords, phrases, and sentiment data extracted by the instruction interpretation system, specific design requirements are created. In this process, the optimal placement of wireless devices and the floor coverage area are defined.

[0254] Step 7:

[0255] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using an image generation engine (e.g., DALL-E). The design drawing includes a wireless equipment layout diagram for each floor.

[0256] Step 8:

[0257] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0258] Step 9:

[0259] Server: Sends the verified blueprints to the terminal. The blueprints are sent in PDF or PNG format.

[0260] Step 10:

[0261] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0262] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge. Furthermore, the introduction of an emotional engine improves the user experience and reduces stress.

[0263] (Example 2)

[0264] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0265] Existing design drawing processes are inefficient because users have to repeatedly input detailed instructions. Furthermore, the lack of consideration for user emotions during the design process leads to increased stress and hinders improvements in the user experience. Additionally, manual quality checks and corrections of design drawings increase the likelihood of errors.

[0266] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, an emotion recognition means, and a communication means. As a result, the user can input instructions in natural language, and based on those instructions and emotion data, a design drawing can be automatically and efficiently generated and confirmed, thereby improving the efficiency of the entire design process and enhancing the user experience.

[0267] "Input means" refers to a device or system for a user to input specific instructions regarding a design drawing in natural language.

[0268] A "natural language processing device" is a device or system that analyzes natural language instructions entered by a user and extracts the necessary information.

[0269] "Instruction interpretation means" refers to a device or system for creating specific design requirements based on information extracted by natural language processing means.

[0270] "Image generation means" refers to a device or system for generating design drawings based on design requirements created by the instruction interpretation means.

[0271] A "result verification means" is a device or system for verifying whether the generated design drawings match the user's instructions and making corrections as necessary.

[0272] "Output means" refers to a device or system for providing generated blueprints to a user or other system.

[0273] An "emotion recognition means" is a device or system for recognizing a user's emotions from facial expressions, voice, etc., and acquiring that information.

[0274] "Communication means" refers to a device or system for transmitting user instruction data and emotional data to a server, and for transmitting the generated blueprint to the user's terminal.

[0275] This invention is a system that allows users to input instructions regarding design drawings in natural language, and then automatically and efficiently generates design drawings based on those instructions and emotional data. This system automates the entire design process and, by taking user emotions into consideration, provides user-friendly and high-quality design drawings.

[0276] System Configuration

[0277] Overview of the design drawing creation system

[0278] This system consists of input means from a terminal, natural language processing means, instruction interpretation means, image generation means, result confirmation means, output means, emotion recognition means, and communication means.

[0279] User input

[0280] Using a device (e.g., PC, tablet, smartphone), the user inputs instructions regarding the design plan in natural language. For example, they might input specific instructions such as, "Please create a design plan for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms." These instructions are saved on the device as text in natural language format.

[0281] Recognition of emotions

[0282] The terminal is equipped with emotion recognition means (e.g., Affectiva SDK), and when the user inputs an instruction, the emotion engine evaluates in real-time the facial expression data and voice data obtained through the camera and microphone. The emotion engine recognizes the user's emotional state (e.g., stress state, relaxation state) and generates the result as emotion data.

[0283] Transmission of Instruction and Emotion Data

[0284] The terminal uses an HTTP POST request to send the natural language instruction input by the user and the emotion data recognized by the emotion recognition means to the server. This data includes the input instruction text and emotion data (e.g., emotion scoring) indicating the user's emotional state.

[0285] Analysis of Instruction

[0286] The server is equipped with natural language analysis means (e.g., GPT-4), which tokenizes the received instruction and analyzes its structure. Specifically, it separates the necessary keywords and phrases (e.g., "5G-compatible wireless device", "office building", "each floor", "3 units") from the instruction text and simultaneously processes the analyzed emotion data.

[0287] Interpretation of Instruction and Requirement Extraction

[0288] The server's instruction interpretation means creates specific design requirements based on the analyzed keywords, phrases, and emotion data. For example, when the user is feeling stressed, it generates design requirements to simplify the response text and reduce the burden. This realizes a more adaptable and user-friendly system.

[0289] Generation of Design Drawing

[0290] The server's image generation means generates a design drawing based on the design requirements created by the instruction interpretation means. Specifically, it uses an image generation engine (e.g., DALL-E) to automatically generate a specific design drawing such as arranging 3 wireless devices on each floor.

[0291] Checking the generation results

[0292] The server's result verification mechanism verifies the generated blueprint to confirm that it matches the user's instructions. It also automatically makes adjustments as needed based on emotional data. For example, if the user is experiencing stress, it will output a simpler, easier-to-understand blueprint.

[0293] Send results

[0294] The server sends the verified design drawings to the terminal. The user can review the design drawings through the terminal and enter further modification instructions as needed. For example, if the user re-enters "Change the placement of the radios to four per floor," the same processing steps will be followed to generate the revised design drawings.

[0295] Specific example

[0296] 1. User input

[0297] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a five-story office building."

[0298] 2. Recognition of emotions

[0299] The emotion engine recognizes that the user is relaxed based on their facial expressions and voice during input.

[0300] 3. Sending instructions and emotional data

[0301] The device sends instructions and emotional data to the server via an HTTP POST request.

[0302] 4. Analysis of Instructions

[0303] The server analyzes the instructions using natural language processing and extracts elements such as "5-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[0304] 5. Interpretation of Instructions and Requirement Extraction

[0305] Based on this information, the server generates specific design requirements and creates a response text considering the sentiment data.

[0306] 6. Generation of Design Drawings

[0307] The image generation engine (DALL-E) generates design drawings based on the design requirements. For example, it arranges 4 Wi-Fi radios on each floor.

[0308] 7. Confirmation of Generation Results

[0309] The server evaluates the quality of the design drawings and makes necessary corrections considering the sentiment data.

[0310] 8. Transmission of Results

[0311] The server transmits the confirmed design drawings to the terminal for the user to check.

[0312] Through this process, the user can automatically obtain efficient and high-quality design drawings, and through the emotion recognition function, a more comfortable and less stressful design process is realized.

[0313] The flow of specific processing in Example 2 will be described using FIG. 13.

[0314] Step 1:

[0315] The user uses the terminal to input specific instructions regarding the design drawing in natural language. For example, the user inputs an instruction such as "Please create a design drawing for installing 3 5G-compatible radios on each floor of the office building." This input is saved as text data in the terminal. The input data is an instruction sentence in natural language.

[0316] Step 2:

[0317] The device's emotion recognition system recognizes the user's facial expressions and voice in real time as they input instructions. Using the camera and microphone, it collects facial and voice data, which an emotion engine analyzes to evaluate the user's emotional state. This analysis generates emotion data such as "the user is stressed" or "the user is relaxed." Input data consists of facial and voice data, while output data is the result of the emotion evaluation.

[0318] Step 3:

[0319] The device sends the user's natural language instructions and sentiment data to the server using an HTTP POST request. Specifically, the user's instructions and sentiment data obtained by the sentiment engine are sent to the server. The input data consists of the instructions and sentiment data, while the output data is the request sent to the server.

[0320] Step 4:

[0321] The server uses natural language processing to tokenize the received instruction sentence and analyze its structure. It extracts the analyzed tokens and phrases (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and identifies the necessary information. Simultaneously, it also analyzes sentiment data and records the results. The input data consists of the instruction sentence and sentiment data, while the output data consists of the analyzed tokens and phrases.

[0322] Step 5:

[0323] The server's instruction interpretation mechanism creates specific design requirements based on the analyzed tokens and phrases. Sentimental data is also taken into consideration during this process. For example, if the user is experiencing stress, the response may be made more concise. The input data consists of analyzed tokens and sentimental data, while the output data consists of the specific design requirements.

[0324] Step 6:

[0325] The server's image generation mechanism generates a design drawing based on the design requirements created by the instruction interpretation mechanism. For generation, an image generation engine (e.g., DALL-E) is used to output a specific design drawing as an image. For example, a design drawing of an office building with three wireless transceivers on each floor is generated. The input data is the design requirements, and the output data is an image of the design drawing.

[0326] Step 7:

[0327] The server's result verification mechanism evaluates the quality of the generated blueprint. It checks whether the blueprint matches the user's instructions and meets specific quality standards, and makes corrections as needed based on sentiment data. For example, if the user is experiencing stress, the blueprint is adjusted to be visually simpler. The input data consists of the blueprint and sentiment data, and the output data is the corrected blueprint.

[0328] Step 8:

[0329] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input data is the verified blueprint, and the output data is the blueprint displayed on the terminal.

[0330] (Application Example 2)

[0331] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0332] In modern virtual store design, it is not enough to simply generate designs efficiently based on user instructions; it is also necessary to recognize the user's emotional state in real time and incorporate that information into the design. However, current systems simply design according to instructions without considering the user's emotions, which makes it difficult to improve customer satisfaction.

[0333] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an emotion recognition means, an instruction interpretation means, an image generation means, and a result confirmation means. This makes it possible to extract design requirements based on instructions and generate a design that reflects the user's emotions.

[0334] An "input means" is a device or interface that allows a user to input instructions to a system in natural language.

[0335] A "natural language processing device" is a processing device that analyzes natural language instructions input by a user and extracts the necessary information.

[0336] An "emotion recognition tool" is a device or software that recognizes a user's emotions in real time from their facial expressions and voice, and generates information based on those emotions.

[0337] A "direction interpretation means" is a device or function that creates specific design requirements based on information extracted by a natural language processing means and emotional data obtained by an emotion recognition means.

[0338] "Image generation means" refers to an engine or software that generates design drawings or designs based on the design requirements obtained by the instruction interpretation means.

[0339] A "result verification means" refers to a device or function that checks whether the generated images or design drawings match the instructions, verifies their quality, and makes corrections as necessary.

[0340] "Output means" refers to devices or interfaces for providing users with the final confirmed design drawings or images.

[0341] This invention is a system for automatically generating an optimal design for virtual stores by analyzing the instructions entered by the user when they design a virtual store, and by recognizing and considering the emotions the user was feeling at the time of input.

[0342] System Configuration

[0343] This system includes the following main means:

[0344] 1. Input Method: Users input instructions for the virtual store design in natural language using a smartphone or smart glasses. For example, they might input instructions such as, "Please change the layout of the product shelves to a more relaxing one."

[0345] 2. Natural Language Processing Method: Analyzes user-inputted instructions and extracts necessary information. Specifically, it uses a natural language processing engine (e.g., BERT, GPT-3) to tokenize the instruction sentences and analyze their structure.

[0346] 3. Emotion Recognition Method: Emotions are recognized in real time from the user's facial expressions and voice during input. This information is sent to the server along with natural language instructions. The technologies used include facial recognition algorithms and voice emotion recognition engines (e.g., OpenFace, IBM Watson Tone Analyzer).

[0347] 4. Instruction Interpretation Method: Based on data obtained from natural language processing and emotion recognition methods, specific design requirements are extracted. In this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[0348] 5. Image generation means: Generates a specific virtual store design based on the design requirements. An image generation engine (e.g., DALL-E) is used to generate an image based on the design requirements obtained from the instruction interpretation means.

[0349] 6. Result Verification Method: Verify that the generated design matches the instructions and that quality is maintained. If necessary, perform automated corrections that also take sentiment data into consideration.

[0350] 7. Output method: The final confirmed design is provided to the user. The user can review the design drawings and input further instructions as needed.

[0351] Server Processing

[0352] The server processes data by combining a series of methods. First, it receives natural language instructions and sentiment data from the user via an HTTP POST request. The received data is analyzed by a natural language analysis tool, and the content of the instructions is tokenized and structurally analyzed. Furthermore, considering the user's emotions recognized by the sentiment recognition tool, the instruction interpretation tool extracts specific design requirements. After that, the design generated using the image generation tool is verified by the results verification tool, and finally sent back to the user.

[0353] Specific example

[0354] Example 1: When a user inputs, "Please rearrange the books to create a relaxing corner," the system recognizes the user's relaxed state from their facial expressions and voice. Based on this, it generates a layout design that promotes relaxation.

[0355] Examples of prompts for generative AI models:

[0356] Prompt: "You have instructed the user to create a relaxing space in a corner of a coffee shop. The user is feeling stressed."

[0357] In this way, the present invention can combine user instructions and emotions to provide an optimal virtual store design and improve the user experience.

[0358] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0359] Step 1:

[0360] Users input design instructions for a virtual store using natural language via their smartphones or smart glasses. For example, they might give instructions such as, "Please create a book corner with a relaxing atmosphere." The input instructions are then sent to the server as initial data via the device.

[0361] Step 2:

[0362] The terminal analyzes the user's facial expressions and voice during input using emotion recognition technology, and acquires the user's emotion data in real time. The acquired emotion data is sent to the server along with natural language instructions. The specific operations of emotion recognition utilize facial recognition algorithms and voice emotion recognition engines.

[0363] Step 3:

[0364] The server receives input natural language instructions and sentiment data via an HTTP POST request. The received data is first passed to a natural language processing system, where the instruction sentences are tokenized and their structure is analyzed. For example, specific keywords and phrases such as "a book corner with a relaxing atmosphere" are extracted.

[0365] Step 4:

[0366] The server passes the data obtained by the natural language processing means to the instruction interpretation means. The instruction interpretation means extracts specific design requirements based on the analyzed instructions and emotion data. In this process, the design requirements are adjusted according to the user's emotions, thereby realizing an emotionally sensitive design process.

[0367] Step 5:

[0368] The server uses the design requirements extracted by the instruction interpretation means to generate a concrete design drawing using the image generation means. An image generation engine (e.g., DALL-E) is used to draw the layout of the virtual store according to the design instructions. During this process, the design requirements are input to the image generation engine as prompts.

[0369] Step 6:

[0370] The server verifies the generated blueprint using a results verification mechanism. It checks whether the blueprint matches the instructions and whether quality is maintained, and automatically makes corrections as needed, taking emotional data into consideration. For example, it checks whether the relaxation effect is properly achieved in response to an instruction requesting relaxation.

[0371] Step 7:

[0372] The server sends the final, verified blueprint to the terminal. The user can review the blueprint through the terminal and make further modifications by entering instructions if necessary. In this way, the blueprint is continuously adjusted to improve the user experience.

[0373] This series of processing steps enables efficient and user-friendly virtual store design by combining natural language input and emotion recognition.

[0374] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0375] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0376] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0377] [Second Embodiment]

[0378] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0379] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0380] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0381] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0382] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0383] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0384] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0385] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0386] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0387] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0388] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0389] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0390] This invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, and an output means, and aims to streamline and automate the process of creating design drawings.

[0391] System Configuration

[0392] 1. User input

[0393] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0394] 2. Sending instructions

[0395] The terminal sends natural language instructions entered by the user to the server, using an appropriate communication protocol (e.g., HTTP POST request).

[0396] 3. Analysis of Instructions

[0397] The server is equipped with natural language processing capabilities to analyze received instructions. Specifically, it uses a natural language processing engine to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "5G wireless device," "office building," "3 devices per floor," and "8 rooms."

[0398] 4. Interpretation of instructions and extraction of requirements

[0399] The server-side instruction interpretation mechanism concretizes the design requirements based on extracted keywords and phrases. This process defines the optimal placement of wireless devices and the floor coverage area, among other things.

[0400] 5. Generating blueprints

[0401] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. For generation, an existing image generation engine (e.g., DALL-E) is used to create wireless equipment layout diagrams for each floor.

[0402] 6. Check the generation results

[0403] The result verification mechanism checks the generated design drawings and verifies that they match the user's instructions. An automatic correction function is also included, and further adjustments to the layout are made as needed.

[0404] 7. Submit the results

[0405] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0406] Specific example

[0407] 1. User input

[0408] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0409] 2. Sending instructions

[0410] The terminal sends this instruction to the server.

[0411] 3. Analysis of Instructions

[0412] The natural language processing system installed on the server analyzes the instructions and extracts elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0413] 4. Interpretation of instructions and extraction of requirements

[0414] The server's instruction interpretation mechanism uses this information to specify the design requirements for the placement of wireless equipment on each floor. The optimal placement of wireless equipment is determined based on the floor area and room layout.

[0415] 5. Generating blueprints

[0416] The image generation means generates design drawings based on the above design requirements. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0417] 6. Check the generation results

[0418] The result verification mechanism checks for consistency between the generated design drawings and the instructions, and provides feedback for corrections if necessary.

[0419] 7. Submit the results

[0420] The server sends the verified blueprint to the terminal. The user reviews it and enters further instructions if necessary.

[0421] This system allows users to obtain high-quality blueprints quickly without requiring advanced expertise or expensive tools.

[0422] The following describes the processing flow.

[0423] Step 1:

[0424] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0425] Step 2:

[0426] Terminal: Sends the input natural language instructions to the server. This data is sent using an HTTP POST request.

[0427] Step 3:

[0428] Server: Activates natural language processing equipment to analyze the received natural language instructions. First, the instructions are tokenized, and the sentence structure is analyzed.

[0429] Step 4:

[0430] Server: Extracts necessary keys and phrases from the text analyzed by the natural language processing system. For example, it identifies elements such as "3-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[0431] Step 5:

[0432] Server: Based on the extracted keywords for instruction interpretation, specific design requirements are created. In this process, the area of ​​each floor and the layout of the rooms are considered to determine the optimal placement of the wireless equipment.

[0433] Step 6:

[0434] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using the image generation engine (e.g., DALL-E).

[0435] Step 7:

[0436] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0437] Step 8:

[0438] Server: Sends the verified design drawings to the terminal. The design drawings are sent in PDF or PNG format.

[0439] Step 9:

[0440] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0441] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge.

[0442] (Example 1)

[0443] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0444] Traditional design drawing processes require advanced expertise and expensive software tools, resulting in significant costs and time investment. Furthermore, manual design work is prone to human error, leading to inconsistent quality in the final product. Additionally, design changes and modifications require further time and effort to address.

[0445] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0446] In this invention, the server includes an input means for the user to input specific instructions regarding the design drawing in natural language, a natural language analysis means for analyzing the input natural language instructions, an instruction interpretation means for concretizing the design requirements based on extracted keywords and phrases, an image generation means for generating a visual design drawing based on the design requirements, a result confirmation means for confirming that the generated design drawing matches the instruction content, and an output means for transmitting the confirmed design drawing to a terminal. This makes it possible to automatically create high-quality design drawings in a short time without specialized knowledge, enabling efficient and highly accurate design work.

[0447] An "input means" is a device or interface that allows a user to input specific instructions regarding a design drawing in natural language.

[0448] A "natural language processing tool" is an engine or software that analyzes input natural language instructions and extracts important keywords and phrases.

[0449] A "direction interpretation means" is a processing device or algorithm for concretizing design requirements based on extracted keywords or phrases and determining the requirements of the design drawing.

[0450] "Image generation means" refers to an engine or software for generating visual design drawings based on design requirements.

[0451] A "result verification means" is a device or system that has the function of checking whether the generated design drawings match the user's instructions and making corrections as necessary.

[0452] "Output means" refers to a device or interface that transmits the verified design drawings to the user's terminal, allowing the user to review the design drawings.

[0453] This invention is a system for streamlining and automating the process of creating design drawings. This system includes multiple means for consistently processing everything from user input to the generation of design drawings. Specific embodiments of this system are described below.

[0454] System Configuration

[0455] This system uses the following hardware and software:

[0456] Hardware: User devices (e.g., PCs, tablets), servers

[0457] software:

[0458] Input means

[0459] Natural language processing tools (e.g., natural language processing engines such as GPT-3)

[0460] Instruction interpretation means

[0461] Image generation means (e.g., image generation engine such as DALL-E)

[0462] Means of checking results

[0463] Output means

[0464] Program Processing Description

[0465] 1. User input

[0466] Users input specific instructions regarding the design plans in natural language via their own devices. For example, they might input, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This allows the system to receive detailed requirements based on the user's needs.

[0467] 2. Sending instructions

[0468] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. This transmission is packaged in an appropriate format, and a simple communication protocol is also implemented to check for communication errors.

[0469] 3. Analysis of Instructions

[0470] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The server tokenizes the instructions to understand the context and extracts important keywords and phrases. For example, it analyzes information such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0471] 4. Interpretation of instructions and extraction of requirements

[0472] The server's instruction interpretation mechanism concretizes design requirements based on extracted keywords and phrases. This process determines details such as the optimal placement of radios and floor coverage. For example, the optimal placement of radios is calculated considering the area of ​​each floor and the layout of the rooms.

[0473] 5. Generating blueprints

[0474] The server uses an image generation engine (e.g., DALL-E) to generate design drawings based on the design requirements. The image generation engine creates a visual design drawing based on input prompts, and generates a layout diagram that visually shows the location and coverage area of ​​the radio equipment.

[0475] 6. Check the generation results

[0476] The server's result verification mechanism checks whether the generated design matches the user's instructions. An automatic correction function also operates, readjusting the previously generated design as needed. During this process, a detailed check is performed to ensure it meets the user's requirements.

[0477] 7. Submit the results

[0478] The server sends the verified blueprint to the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0479] Examples of specific actions

[0480] 1. User input

[0481] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0482] 2. Sending instructions

[0483] The device sends this instruction to the server via an HTTP POST request.

[0484] 3. Analysis of Instructions

[0485] The server's natural language processing engine (such as GPT-3) breaks down the instruction sentence into keywords such as "three-story building," "office building," "Wi-Fi wireless device," and "four devices on each floor" and analyzes them.

[0486] 4. Interpretation of instructions and extraction of requirements

[0487] The server calculates the optimal location for the Wi-Fi wireless router, taking into account the area of ​​each floor and the layout of the rooms.

[0488] 5. Generating blueprints

[0489] The server uses an image generation engine to draw a layout diagram of the radio equipment, creating a visual design drawing.

[0490] 6. Check the generation results

[0491] The result verification mechanism checks the generated design drawings to confirm whether they meet the user's requirements.

[0492] 7. Submit the results

[0493] The server sends the verified blueprints to the terminal, and the user confirms them.

[0494] This system allows users to obtain high-quality blueprints quickly, without requiring advanced expertise or expensive tools.

[0495] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0496] Step 1: User input

[0497] The user uses a terminal to input specific instructions regarding the design drawings in natural language. An example of input would be a prompt message such as, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This instruction would be the input. Through this operation, the user communicates the design requirements in detail to the system.

[0498] Step 2: Sending instructions

[0499] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. During this process, the input instructions are packaged into an appropriate format. The sent prompt text becomes the output. The terminal also checks for transmission errors through this operation.

[0500] Step 3: Analyzing the instructions

[0501] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The input to this operation is the sent prompt sentence, and the output is tokenized keywords and phrases. Specifically, the instruction sentence is broken down into important keywords such as "three-story building," "office building," "Wi-Fi radio," and "four units per floor."

[0502] Step 4: Interpretation of instructions and requirements extraction

[0503] The server's instruction interpretation mechanism concretizes design requirements based on tokenized keywords and phrases. The input to this operation is the extracted keywords and phrases, and the output is the specific content of the design requirements. Specifically, it calculates the optimal placement of wireless devices, taking into account the area of ​​each floor and the layout of the rooms.

[0504] Step 5: Generate the blueprint

[0505] The server uses an image generation means (e.g., DALL-E) to generate design drawings based on the design requirements. The input to this operation is the specified design requirements, and the output is a visual design drawing. The image generation engine creates a visual layout diagram showing the location and coverage area of ​​the radio equipment.

[0506] Step 6: Check the generation results

[0507] The server's result verification mechanism checks the generated blueprint against the user's instructions. The input for this operation is the generated visual blueprint, and the output is the verified blueprint. Furthermore, an automatic correction function is also in operation, including specific actions to readjust the blueprint as needed.

[0508] Step 7: Submit the results

[0509] The server sends the verified blueprint to the user's terminal. The input for this operation is the verified blueprint, and the output is the blueprint displayed on the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0510] (Application Example 1)

[0511] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0512] Designing equipment layouts and production lines within a factory is a task that requires advanced expertise and time, making it difficult to perform efficiently and accurately. Furthermore, there are limited systems that allow users to input instructions in natural language and automatically generate design drawings based on those instructions. A system is needed to solve this problem and enable rapid and accurate equipment layout and production line design within factories.

[0513] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0514] In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and means for automatically generating the layout of factory equipment and production line designs. This makes it possible to efficiently and accurately automatically generate the layout of factory equipment and production line designs based on instructions entered by the user in natural language.

[0515] An "input method" refers to a device or interface that allows a user to input instructions in natural language.

[0516] "Natural language processing tools" is a general term for engines and algorithms that analyze input natural language instructions and perform tokenization and keyword extraction.

[0517] The "instruction interpretation means" is a function that concretizes design requirements based on analyzed keywords and phrases and extracts the information necessary for generating design drawings.

[0518] "Image generation means" is a general term for engines and algorithms that automatically generate design drawings based on design requirements obtained from instruction interpretation means.

[0519] The "result verification means" is a function for confirming that the generated design drawings match the user's instructions and provides feedback for corrections as needed.

[0520] "Output means" refers to the means of providing users with confirmed design drawings, and includes functions for saving and displaying them as digital files.

[0521] "Means for automatically generating equipment layouts and production line designs within a factory" refers to a general term for engines and algorithms that automatically generate design drawings based on the factory layout, required robot types, and work areas.

[0522] This invention provides a system for automating the layout of equipment and production line design within a factory. The system includes an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result verification means, an output means, and means for automatically generating the layout of equipment and production line design within the factory.

[0523] Explain the program's processing in natural language.

[0524] The entire system operates as follows:

[0525] 1. User input instructions:

[0526] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, an instruction might be, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square."

[0527] 2. Sending instructions:

[0528] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request.

[0529] 3. Analysis of instructions:

[0530] The natural language processing (NLP) system installed on the server analyzes the received instructions. Specifically, it uses a natural language processing engine (e.g., GPT-4) to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square."

[0531] 4. Interpretation of instructions and extraction of requirements:

[0532] The server-side instruction interpretation mechanism concretizes the design requirements based on the analyzed keywords and phrases. In this process, the optimal design is created by considering factors such as the factory layout, the type of robots required, and the work area.

[0533] 5. Generating blueprints:

[0534] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E 2) is used to draw design drawings of the equipment layout and production lines within the factory.

[0535] 6. Check the generation results:

[0536] The results verification system checks whether the generated design drawings match the user's instructions. If necessary, it also provides an automatic correction function and offers feedback as needed.

[0537] 7. Submit the results:

[0538] The server sends the verified design drawings to the terminal. The user can then review the design drawings via the terminal and enter further modification instructions as needed.

[0539] Examples of specific cases and prompt statements

[0540] Specific example:

[0541] The user enters the following: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0542] Examples of prompts to input into a generative AI model:

[0543] Natural Language Processing: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0544] This system allows users to efficiently and accurately generate factory equipment layouts and production line designs automatically, even without advanced expertise.

[0545] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0546] Step 1:

[0547] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, they might input instructions such as, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square." The input data is in natural language text format.

[0548] Step 2:

[0549] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request. The input is the natural language instruction entered by the user, and the output is the transmission of the instruction to the server.

[0550] Step 3:

[0551] The server's built-in natural language processing system analyzes the received instructions. Specifically, it uses the generative AI model GPT-4 to tokenize the instructions and extract important keywords and phrases. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square." The input is the user's natural language instructions, and the output is the analyzed keywords and phrases.

[0552] Step 4:

[0553] The server's instruction interpretation mechanism concretizes design requirements based on analyzed keywords and phrases. This process considers factors such as the factory layout, required robot types, and work areas to create an optimal design. The input consists of keywords and phrases from the natural language processing mechanism, while the output is the concretized design requirements.

[0554] Step 5:

[0555] Using the server's image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine such as DALL-E 2 is used to draw design drawings of factory equipment layouts and production lines. The input is the design requirements from the instruction interpretation means, and the output is the generated design drawing.

[0556] Step 6:

[0557] The result verification mechanism verifies that the generated design drawing matches the user's instructions. If necessary, it provides an automatic correction function and offers feedback as needed. The input is the generated design drawing, and the output is the design drawing verified through verification and correction.

[0558] Step 7:

[0559] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input is the verified blueprint, and the output is the blueprint provided to the user.

[0560] In this way, the system quickly and accurately automatically generates factory equipment layouts and production line designs based on the user's natural language instructions.

[0561] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0562] The present invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and an emotion engine, and aims to streamline and automate the design drawing creation process, as well as to recognize and adapt to the user's emotions.

[0563] System Configuration

[0564] 1. User input

[0565] The user inputs specific instructions regarding the design drawings using natural language via a terminal. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0566] 2. Recognition of emotions

[0567] The terminal is equipped with an emotion engine that recognizes the user's emotions in real time from their facial expressions and voice as they input design instructions. This emotion information is sent to the server along with the natural language instructions.

[0568] 3. Sending instructions and emotional data

[0569] The device sends natural language instructions entered by the user and emotion data recognized by the emotion engine to the server. This data is sent using an HTTP POST request.

[0570] 4. Analysis of Instructions

[0571] The server is equipped with natural language processing capabilities to analyze received instructions and sentiment data. It uses a natural language processing engine to tokenize instructions, analyze the sentence structure, and analyze sentiment data.

[0572] 5. Interpretation of instructions and extraction of requirements

[0573] The server-side instruction interpretation mechanism creates specific design requirements based on extracted keywords, phrases, and sentiment data. During this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[0574] 6. Generating blueprints

[0575] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E) is used for generation to create wireless equipment layout diagrams for each floor.

[0576] 7. Check the generation results

[0577] The result verification mechanism verifies the generated design drawings. It checks whether the design drawings match the instructions and whether quality is maintained. If necessary, it performs automated corrections, taking sentiment data into consideration.

[0578] 8. Submit the results

[0579] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0580] Specific example

[0581] 1. User input

[0582] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0583] 2. Recognition of emotions

[0584] The emotion engine recognizes whether the user is stressed or relaxed based on their facial expressions and voice during input.

[0585] 3. Sending instructions and emotional data

[0586] The device sends these instructions and emotional data to the server.

[0587] 4. Analysis of Instructions

[0588] The server analyzes the received instructions using natural language processing to identify elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four devices per floor."

[0589] 5. Interpretation of instructions and extraction of requirements

[0590] The server's instruction interpretation mechanism uses this information to concretize design requirements while considering emotional data. For example, if the user is experiencing stress, the response is simplified to reduce the burden.

[0591] 6. Generating blueprints

[0592] The image generation means generates design drawings based on the design requirements created by the instruction interpretation means. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0593] 7. Check the generation results

[0594] The result verification method confirms that the generated design drawings and instructions match, and makes corrections if necessary, taking emotional data into consideration.

[0595] 8. Submit the results

[0596] The server sends the verified blueprint to the terminal. The user reviews the blueprint and enters further instructions if necessary.

[0597] Through this process, users can create efficient and high-quality blueprints, and the introduction of an emotion engine improves the user experience.

[0598] The following describes the processing flow.

[0599] Step 1:

[0600] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0601] Step 2:

[0602] Terminal: Analyzes the user's facial expressions and voice in real time while they are inputting instructions, and uses an emotion engine to recognize the user's emotional state. For example, it can determine whether the user is stressed or relaxed.

[0603] Step 3:

[0604] Terminal: Sends input natural language instructions and recognized emotion data to the server. Data is sent using an HTTP POST request.

[0605] Step 4:

[0606] Server: Activates natural language processing (NLP) to analyze received natural language instructions and sentiment data. First, it tokenizes the instructions and extracts important keywords and phrases necessary to understand the context. For example, it extracts information such as "3-story building," "office building," "Wi-Fi radio," and "4 units per floor."

[0607] Step 5:

[0608] Server: Analyzes emotional data to understand the user's emotional state. For example, if the user is stressed, it will simplify the response to reduce the burden on the user.

[0609] Step 6:

[0610] Server: Based on the keywords, phrases, and sentiment data extracted by the instruction interpretation system, specific design requirements are created. In this process, the optimal placement of wireless devices and the floor coverage area are defined.

[0611] Step 7:

[0612] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using an image generation engine (e.g., DALL-E). The design drawing includes a wireless equipment layout diagram for each floor.

[0613] Step 8:

[0614] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0615] Step 9:

[0616] Server: Sends the verified blueprints to the terminal. The blueprints are sent in PDF or PNG format.

[0617] Step 10:

[0618] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0619] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge. Furthermore, the introduction of an emotional engine improves the user experience and reduces stress.

[0620] (Example 2)

[0621] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0622] Existing design drawing processes are inefficient because users have to repeatedly input detailed instructions. Furthermore, the lack of consideration for user emotions during the design process leads to increased stress and hinders improvements in the user experience. Additionally, manual quality checks and corrections of design drawings increase the likelihood of errors.

[0623] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, an emotion recognition means, and a communication means. As a result, the user can input instructions in natural language, and based on those instructions and emotion data, a design drawing can be automatically and efficiently generated and confirmed, thereby improving the efficiency of the entire design process and enhancing the user experience.

[0624] "Input means" refers to a device or system for a user to input specific instructions regarding a design drawing in natural language.

[0625] A "natural language processing device" is a device or system that analyzes natural language instructions entered by a user and extracts the necessary information.

[0626] "Instruction interpretation means" refers to a device or system for creating specific design requirements based on information extracted by natural language processing means.

[0627] "Image generation means" refers to a device or system for generating design drawings based on design requirements created by the instruction interpretation means.

[0628] A "result verification means" is a device or system for verifying whether the generated design drawings match the user's instructions and making corrections as necessary.

[0629] "Output means" refers to a device or system for providing generated blueprints to a user or other system.

[0630] An "emotion recognition means" is a device or system for recognizing a user's emotions from facial expressions, voice, etc., and acquiring that information.

[0631] "Communication means" refers to a device or system for transmitting user instruction data and emotional data to a server, and for transmitting the generated blueprint to the user's terminal.

[0632] This invention is a system that allows users to input instructions regarding design drawings in natural language, and then automatically and efficiently generates design drawings based on those instructions and emotional data. This system automates the entire design process and, by taking user emotions into consideration, provides user-friendly and high-quality design drawings.

[0633] System Configuration

[0634] Overview of the design drawing creation system

[0635] This system consists of input means from a terminal, natural language processing means, instruction interpretation means, image generation means, result confirmation means, output means, emotion recognition means, and communication means.

[0636] User input

[0637] Using a device (e.g., PC, tablet, smartphone), the user inputs instructions regarding the design plan in natural language. For example, they might input specific instructions such as, "Please create a design plan for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms." These instructions are saved on the device as text in natural language format.

[0638] Recognition of emotions

[0639] The device is equipped with an emotion recognition system (e.g., Affectiva SDK), and when the user inputs instructions, the emotion engine evaluates the facial expression data and voice data acquired through the camera and microphone in real time. The emotion engine recognizes the user's emotional state (e.g., stressed state, relaxed state) and generates emotion data based on the result.

[0640] Sending instructions and emotional data

[0641] The device uses an HTTP POST request to send natural language instructions entered by the user and sentiment data recognized by the sentiment recognition system to the server. This data includes the entered instruction text and sentiment data indicating the user's emotional state (e.g., sentiment scoring).

[0642] Analysis of instructions

[0643] The server is equipped with natural language processing tools (e.g., GPT-4) that tokenize received instructions and analyze their structure. Specifically, it separates necessary keywords and phrases from the instruction text (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and simultaneously processes the analyzed sentiment data.

[0644] Interpretation of instructions and extraction of requirements

[0645] The server's instruction interpretation mechanism creates specific design requirements based on analyzed keywords, phrases, and sentiment data. For example, if a user is experiencing stress, it generates design requirements that simplify response statements and reduce the burden. This results in a more adaptive and user-friendly system.

[0646] Design drawing generation

[0647] The server's image generation means generates a design drawing based on the design requirements created by the instruction interpretation means. Specifically, it uses an image generation engine (e.g., DALL-E) to automatically generate a detailed design drawing, such as placing three wireless devices on each floor.

[0648] Checking the generation results

[0649] The server's result verification mechanism verifies the generated blueprint to confirm that it matches the user's instructions. It also automatically makes adjustments as needed based on emotional data. For example, if the user is experiencing stress, it will output a simpler, easier-to-understand blueprint.

[0650] Send results

[0651] The server sends the verified design drawings to the terminal. The user can review the design drawings through the terminal and enter further modification instructions as needed. For example, if the user re-enters "Change the placement of the radios to four per floor," the same processing steps will be followed to generate the revised design drawings.

[0652] Specific example

[0653] 1. User input

[0654] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a five-story office building."

[0655] 2. Recognition of emotions

[0656] The emotion engine recognizes that the user is relaxed based on their facial expressions and voice during input.

[0657] 3. Sending instructions and emotional data

[0658] The device sends instructions and emotional data to the server via an HTTP POST request.

[0659] 4. Analysis of Instructions

[0660] The server analyzes the instructions using natural language processing and extracts elements such as "5-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[0661] 5. Interpretation of instructions and extraction of requirements

[0662] The server uses this information to generate specific design requirements and creates response messages that take sentiment data into consideration.

[0663] 6. Generating blueprints

[0664] The image generation engine (DALL-E) generates design drawings based on the design requirements, for example, by placing four Wi-Fi wireless devices on each floor.

[0665] 7. Check the generation results

[0666] The server evaluates the quality of the design blueprint and makes necessary corrections, taking emotional data into consideration.

[0667] 8. Submit the results

[0668] The server sends the verified blueprints to the terminal, and the user confirms them.

[0669] This process allows users to automatically obtain efficient and high-quality blueprints, and through emotion recognition, a more comfortable and less stressful design process is achieved.

[0670] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0671] Step 1:

[0672] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building." This input is saved as text data on the terminal. The input data is a natural language instruction.

[0673] Step 2:

[0674] The device's emotion recognition system recognizes the user's facial expressions and voice in real time as they input instructions. Using the camera and microphone, it collects facial and voice data, which an emotion engine analyzes to evaluate the user's emotional state. This analysis generates emotion data such as "the user is stressed" or "the user is relaxed." Input data consists of facial and voice data, while output data is the result of the emotion evaluation.

[0675] Step 3:

[0676] The device sends the user's natural language instructions and sentiment data to the server using an HTTP POST request. Specifically, the user's instructions and sentiment data obtained by the sentiment engine are sent to the server. The input data consists of the instructions and sentiment data, while the output data is the request sent to the server.

[0677] Step 4:

[0678] The server uses natural language processing to tokenize the received instruction sentence and analyze its structure. It extracts the analyzed tokens and phrases (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and identifies the necessary information. Simultaneously, it also analyzes sentiment data and records the results. The input data consists of the instruction sentence and sentiment data, while the output data consists of the analyzed tokens and phrases.

[0679] Step 5:

[0680] The server's instruction interpretation mechanism creates specific design requirements based on the analyzed tokens and phrases. Sentimental data is also taken into consideration during this process. For example, if the user is experiencing stress, the response may be made more concise. The input data consists of analyzed tokens and sentimental data, while the output data consists of the specific design requirements.

[0681] Step 6:

[0682] The server's image generation mechanism generates a design drawing based on the design requirements created by the instruction interpretation mechanism. For generation, an image generation engine (e.g., DALL-E) is used to output a specific design drawing as an image. For example, a design drawing of an office building with three wireless transceivers on each floor is generated. The input data is the design requirements, and the output data is an image of the design drawing.

[0683] Step 7:

[0684] The server's result verification mechanism evaluates the quality of the generated blueprint. It checks whether the blueprint matches the user's instructions and meets specific quality standards, and makes corrections as needed based on sentiment data. For example, if the user is experiencing stress, the blueprint is adjusted to be visually simpler. The input data consists of the blueprint and sentiment data, and the output data is the corrected blueprint.

[0685] Step 8:

[0686] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input data is the verified blueprint, and the output data is the blueprint displayed on the terminal.

[0687] (Application Example 2)

[0688] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0689] In modern virtual store design, it is not enough to simply generate designs efficiently based on user instructions; it is also necessary to recognize the user's emotional state in real time and incorporate that information into the design. However, current systems simply design according to instructions without considering the user's emotions, which makes it difficult to improve customer satisfaction.

[0690] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an emotion recognition means, an instruction interpretation means, an image generation means, and a result confirmation means. This makes it possible to extract design requirements based on instructions and generate a design that reflects the user's emotions.

[0691] An "input means" is a device or interface that allows a user to input instructions to a system in natural language.

[0692] A "natural language processing device" is a processing device that analyzes natural language instructions input by a user and extracts the necessary information.

[0693] An "emotion recognition tool" is a device or software that recognizes a user's emotions in real time from their facial expressions and voice, and generates information based on those emotions.

[0694] A "direction interpretation means" is a device or function that creates specific design requirements based on information extracted by a natural language processing means and emotional data obtained by an emotion recognition means.

[0695] "Image generation means" refers to an engine or software that generates design drawings or designs based on the design requirements obtained by the instruction interpretation means.

[0696] A "result verification means" refers to a device or function that checks whether the generated images or design drawings match the instructions, verifies their quality, and makes corrections as necessary.

[0697] "Output means" refers to devices or interfaces for providing users with the final confirmed design drawings or images.

[0698] This invention is a system for automatically generating an optimal design for virtual stores by analyzing the instructions entered by the user when they design a virtual store, and by recognizing and considering the emotions the user was feeling at the time of input.

[0699] System Configuration

[0700] This system includes the following main means:

[0701] 1. Input Method: Users input instructions for the virtual store design in natural language using a smartphone or smart glasses. For example, they might input instructions such as, "Please change the layout of the product shelves to a more relaxing one."

[0702] 2. Natural Language Processing Method: Analyzes user-inputted instructions and extracts necessary information. Specifically, it uses a natural language processing engine (e.g., BERT, GPT-3) to tokenize the instruction sentences and analyze their structure.

[0703] 3. Emotion Recognition Method: Emotions are recognized in real time from the user's facial expressions and voice during input. This information is sent to the server along with natural language instructions. The technologies used include facial recognition algorithms and voice emotion recognition engines (e.g., OpenFace, IBM Watson Tone Analyzer).

[0704] 4. Instruction Interpretation Method: Based on data obtained from natural language processing and emotion recognition methods, specific design requirements are extracted. In this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[0705] 5. Image generation means: Generates a specific virtual store design based on the design requirements. An image generation engine (e.g., DALL-E) is used to generate an image based on the design requirements obtained from the instruction interpretation means.

[0706] 6. Result Verification Method: Verify that the generated design matches the instructions and that quality is maintained. If necessary, perform automated corrections that also take sentiment data into consideration.

[0707] 7. Output method: The final confirmed design is provided to the user. The user can review the design drawings and input further instructions as needed.

[0708] Server Processing

[0709] The server processes data by combining a series of methods. First, it receives natural language instructions and sentiment data from the user via an HTTP POST request. The received data is analyzed by a natural language analysis tool, and the content of the instructions is tokenized and structurally analyzed. Furthermore, considering the user's emotions recognized by the sentiment recognition tool, the instruction interpretation tool extracts specific design requirements. After that, the design generated using the image generation tool is verified by the results verification tool, and finally sent back to the user.

[0710] Specific example

[0711] Example 1: When a user inputs, "Please rearrange the books to create a relaxing corner," the system recognizes the user's relaxed state from their facial expressions and voice. Based on this, it generates a layout design that promotes relaxation.

[0712] Examples of prompts for generative AI models:

[0713] Prompt: "You have instructed the user to create a relaxing space in a corner of a coffee shop. The user is feeling stressed."

[0714] In this way, the present invention can combine user instructions and emotions to provide an optimal virtual store design and improve the user experience.

[0715] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0716] Step 1:

[0717] Users input design instructions for a virtual store using natural language via their smartphones or smart glasses. For example, they might give instructions such as, "Please create a book corner with a relaxing atmosphere." The input instructions are then sent to the server as initial data via the device.

[0718] Step 2:

[0719] The terminal analyzes the user's facial expressions and voice during input using emotion recognition technology, and acquires the user's emotion data in real time. The acquired emotion data is sent to the server along with natural language instructions. The specific operations of emotion recognition utilize facial recognition algorithms and voice emotion recognition engines.

[0720] Step 3:

[0721] The server receives input natural language instructions and sentiment data via an HTTP POST request. The received data is first passed to a natural language processing system, where the instruction sentences are tokenized and their structure is analyzed. For example, specific keywords and phrases such as "a book corner with a relaxing atmosphere" are extracted.

[0722] Step 4:

[0723] The server passes the data obtained by the natural language processing means to the instruction interpretation means. The instruction interpretation means extracts specific design requirements based on the analyzed instructions and emotion data. In this process, the design requirements are adjusted according to the user's emotions, thereby realizing an emotionally sensitive design process.

[0724] Step 5:

[0725] The server uses the design requirements extracted by the instruction interpretation means to generate a concrete design drawing using the image generation means. An image generation engine (e.g., DALL-E) is used to draw the layout of the virtual store according to the design instructions. During this process, the design requirements are input to the image generation engine as prompts.

[0726] Step 6:

[0727] The server verifies the generated blueprint using a results verification mechanism. It checks whether the blueprint matches the instructions and whether quality is maintained, and automatically makes corrections as needed, taking emotional data into consideration. For example, it checks whether the relaxation effect is properly achieved in response to an instruction requesting relaxation.

[0728] Step 7:

[0729] The server sends the final, verified blueprint to the terminal. The user can review the blueprint through the terminal and make further modifications by entering instructions if necessary. In this way, the blueprint is continuously adjusted to improve the user experience.

[0730] This series of processing steps enables efficient and user-friendly virtual store design by combining natural language input and emotion recognition.

[0731] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0732] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0733] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0734] [Third Embodiment]

[0735] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0736] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0737] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0738] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0739] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0740] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0741] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0742] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0743] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0744] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0745] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0746] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0747] This invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, and an output means, and aims to streamline and automate the process of creating design drawings.

[0748] System Configuration

[0749] 1. User input

[0750] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0751] 2. Sending instructions

[0752] The terminal sends natural language instructions entered by the user to the server, using an appropriate communication protocol (e.g., HTTP POST request).

[0753] 3. Analysis of Instructions

[0754] The server is equipped with natural language processing capabilities to analyze received instructions. Specifically, it uses a natural language processing engine to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "5G wireless device," "office building," "3 devices per floor," and "8 rooms."

[0755] 4. Interpretation of instructions and extraction of requirements

[0756] The server-side instruction interpretation mechanism concretizes the design requirements based on extracted keywords and phrases. This process defines the optimal placement of wireless devices and the floor coverage area, among other things.

[0757] 5. Generating blueprints

[0758] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. For generation, an existing image generation engine (e.g., DALL-E) is used to create wireless equipment layout diagrams for each floor.

[0759] 6. Check the generation results

[0760] The result verification mechanism checks the generated design drawings and verifies that they match the user's instructions. An automatic correction function is also included, and further adjustments to the layout are made as needed.

[0761] 7. Submit the results

[0762] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0763] Specific example

[0764] 1. User input

[0765] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0766] 2. Sending instructions

[0767] The terminal sends this instruction to the server.

[0768] 3. Analysis of Instructions

[0769] The natural language processing system installed on the server analyzes the instructions and extracts elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0770] 4. Interpretation of instructions and extraction of requirements

[0771] The server's instruction interpretation mechanism uses this information to specify the design requirements for the placement of wireless equipment on each floor. The optimal placement of wireless equipment is determined based on the floor area and room layout.

[0772] 5. Generating blueprints

[0773] The image generation means generates design drawings based on the above design requirements. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0774] 6. Check the generation results

[0775] The result verification mechanism checks for consistency between the generated design drawings and the instructions, and provides feedback for corrections if necessary.

[0776] 7. Submit the results

[0777] The server sends the verified blueprint to the terminal. The user reviews it and enters further instructions if necessary.

[0778] This system allows users to obtain high-quality blueprints quickly without requiring advanced expertise or expensive tools.

[0779] The following describes the processing flow.

[0780] Step 1:

[0781] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0782] Step 2:

[0783] Terminal: Sends the input natural language instructions to the server. This data is sent using an HTTP POST request.

[0784] Step 3:

[0785] Server: Activates natural language processing equipment to analyze the received natural language instructions. First, the instructions are tokenized, and the sentence structure is analyzed.

[0786] Step 4:

[0787] Server: Extracts necessary keys and phrases from the text analyzed by the natural language processing system. For example, it identifies elements such as "3-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[0788] Step 5:

[0789] Server: Based on the extracted keywords for instruction interpretation, specific design requirements are created. In this process, the area of ​​each floor and the layout of the rooms are considered to determine the optimal placement of the wireless equipment.

[0790] Step 6:

[0791] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using the image generation engine (e.g., DALL-E).

[0792] Step 7:

[0793] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0794] Step 8:

[0795] Server: Sends the verified design drawings to the terminal. The design drawings are sent in PDF or PNG format.

[0796] Step 9:

[0797] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0798] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge.

[0799] (Example 1)

[0800] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0801] Traditional design drawing processes require advanced expertise and expensive software tools, resulting in significant costs and time investment. Furthermore, manual design work is prone to human error, leading to inconsistent quality in the final product. Additionally, design changes and modifications require further time and effort to address.

[0802] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0803] In this invention, the server includes an input means for the user to input specific instructions regarding the design drawing in natural language, a natural language analysis means for analyzing the input natural language instructions, an instruction interpretation means for concretizing the design requirements based on extracted keywords and phrases, an image generation means for generating a visual design drawing based on the design requirements, a result confirmation means for confirming that the generated design drawing matches the instruction content, and an output means for transmitting the confirmed design drawing to a terminal. This makes it possible to automatically create high-quality design drawings in a short time without specialized knowledge, enabling efficient and highly accurate design work.

[0804] An "input means" is a device or interface that allows a user to input specific instructions regarding a design drawing in natural language.

[0805] A "natural language processing tool" is an engine or software that analyzes input natural language instructions and extracts important keywords and phrases.

[0806] A "direction interpretation means" is a processing device or algorithm for concretizing design requirements based on extracted keywords or phrases and determining the requirements of the design drawing.

[0807] "Image generation means" refers to an engine or software for generating visual design drawings based on design requirements.

[0808] A "result verification means" is a device or system that has the function of checking whether the generated design drawings match the user's instructions and making corrections as necessary.

[0809] "Output means" refers to a device or interface that transmits the verified design drawings to the user's terminal, allowing the user to review the design drawings.

[0810] This invention is a system for streamlining and automating the process of creating design drawings. This system includes multiple means for consistently processing everything from user input to the generation of design drawings. Specific embodiments of this system are described below.

[0811] System Configuration

[0812] This system uses the following hardware and software:

[0813] Hardware: User devices (e.g., PCs, tablets), servers

[0814] software:

[0815] Input means

[0816] Natural language processing tools (e.g., natural language processing engines such as GPT-3)

[0817] Instruction interpretation means

[0818] Image generation means (e.g., image generation engine such as DALL-E)

[0819] Means of checking results

[0820] Output means

[0821] Program Processing Description

[0822] 1. User input

[0823] Users input specific instructions regarding the design plans in natural language via their own devices. For example, they might input, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This allows the system to receive detailed requirements based on the user's needs.

[0824] 2. Sending instructions

[0825] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. This transmission is packaged in an appropriate format, and a simple communication protocol is also implemented to check for communication errors.

[0826] 3. Analysis of Instructions

[0827] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The server tokenizes the instructions to understand the context and extracts important keywords and phrases. For example, it analyzes information such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[0828] 4. Interpretation of instructions and extraction of requirements

[0829] The server's instruction interpretation mechanism concretizes design requirements based on extracted keywords and phrases. This process determines details such as the optimal placement of radios and floor coverage. For example, the optimal placement of radios is calculated considering the area of ​​each floor and the layout of the rooms.

[0830] 5. Generating blueprints

[0831] The server uses an image generation engine (e.g., DALL-E) to generate design drawings based on the design requirements. The image generation engine creates a visual design drawing based on input prompts, and generates a layout diagram that visually shows the location and coverage area of ​​the radio equipment.

[0832] 6. Check the generation results

[0833] The server's result verification mechanism checks whether the generated design matches the user's instructions. An automatic correction function also operates, readjusting the previously generated design as needed. During this process, a detailed check is performed to ensure it meets the user's requirements.

[0834] 7. Submit the results

[0835] The server sends the verified blueprint to the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0836] Examples of specific actions

[0837] 1. User input

[0838] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0839] 2. Sending instructions

[0840] The device sends this instruction to the server via an HTTP POST request.

[0841] 3. Analysis of Instructions

[0842] The server's natural language processing engine (such as GPT-3) breaks down the instruction sentence into keywords such as "three-story building," "office building," "Wi-Fi wireless device," and "four devices on each floor" and analyzes them.

[0843] 4. Interpretation of instructions and extraction of requirements

[0844] The server calculates the optimal location for the Wi-Fi wireless router, taking into account the area of ​​each floor and the layout of the rooms.

[0845] 5. Generating blueprints

[0846] The server uses an image generation engine to draw a layout diagram of the radio equipment, creating a visual design drawing.

[0847] 6. Check the generation results

[0848] The result verification mechanism checks the generated design drawings to confirm whether they meet the user's requirements.

[0849] 7. Submit the results

[0850] The server sends the verified blueprints to the terminal, and the user confirms them.

[0851] This system allows users to obtain high-quality blueprints quickly, without requiring advanced expertise or expensive tools.

[0852] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0853] Step 1: User input

[0854] The user uses a terminal to input specific instructions regarding the design drawings in natural language. An example of input would be a prompt message such as, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This instruction would be the input. Through this operation, the user communicates the design requirements in detail to the system.

[0855] Step 2: Sending instructions

[0856] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. During this process, the input instructions are packaged into an appropriate format. The sent prompt text becomes the output. The terminal also checks for transmission errors through this operation.

[0857] Step 3: Analyzing the instructions

[0858] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The input to this operation is the sent prompt sentence, and the output is tokenized keywords and phrases. Specifically, the instruction sentence is broken down into important keywords such as "three-story building," "office building," "Wi-Fi radio," and "four units per floor."

[0859] Step 4: Interpretation of instructions and requirements extraction

[0860] The server's instruction interpretation mechanism concretizes design requirements based on tokenized keywords and phrases. The input to this operation is the extracted keywords and phrases, and the output is the specific content of the design requirements. Specifically, it calculates the optimal placement of wireless devices, taking into account the area of ​​each floor and the layout of the rooms.

[0861] Step 5: Generate the blueprint

[0862] The server uses an image generation means (e.g., DALL-E) to generate design drawings based on the design requirements. The input to this operation is the specified design requirements, and the output is a visual design drawing. The image generation engine creates a visual layout diagram showing the location and coverage area of ​​the radio equipment.

[0863] Step 6: Check the generation results

[0864] The server's result verification mechanism checks the generated blueprint against the user's instructions. The input for this operation is the generated visual blueprint, and the output is the verified blueprint. Furthermore, an automatic correction function is also in operation, including specific actions to readjust the blueprint as needed.

[0865] Step 7: Submit the results

[0866] The server sends the verified blueprint to the user's terminal. The input for this operation is the verified blueprint, and the output is the blueprint displayed on the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[0867] (Application Example 1)

[0868] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0869] Designing equipment layouts and production lines within a factory is a task that requires advanced expertise and time, making it difficult to perform efficiently and accurately. Furthermore, there are limited systems that allow users to input instructions in natural language and automatically generate design drawings based on those instructions. A system is needed to solve this problem and enable rapid and accurate equipment layout and production line design within factories.

[0870] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0871] In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and means for automatically generating the layout of factory equipment and production line designs. This makes it possible to efficiently and accurately automatically generate the layout of factory equipment and production line designs based on instructions entered by the user in natural language.

[0872] An "input method" refers to a device or interface that allows a user to input instructions in natural language.

[0873] "Natural language processing tools" is a general term for engines and algorithms that analyze input natural language instructions and perform tokenization and keyword extraction.

[0874] The "instruction interpretation means" is a function that concretizes design requirements based on analyzed keywords and phrases and extracts the information necessary for generating design drawings.

[0875] "Image generation means" is a general term for engines and algorithms that automatically generate design drawings based on design requirements obtained from instruction interpretation means.

[0876] The "result verification means" is a function for confirming that the generated design drawings match the user's instructions and provides feedback for corrections as needed.

[0877] "Output means" refers to the means of providing users with confirmed design drawings, and includes functions for saving and displaying them as digital files.

[0878] "Means for automatically generating equipment layouts and production line designs within a factory" refers to a general term for engines and algorithms that automatically generate design drawings based on the factory layout, required robot types, and work areas.

[0879] This invention provides a system for automating the layout of equipment and production line design within a factory. The system includes an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result verification means, an output means, and means for automatically generating the layout of equipment and production line design within the factory.

[0880] Explain the program's processing in natural language.

[0881] The entire system operates as follows:

[0882] 1. User input instructions:

[0883] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, an instruction might be, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square."

[0884] 2. Sending instructions:

[0885] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request.

[0886] 3. Analysis of instructions:

[0887] The natural language processing (NLP) system installed on the server analyzes the received instructions. Specifically, it uses a natural language processing engine (e.g., GPT-4) to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square."

[0888] 4. Interpretation of instructions and extraction of requirements:

[0889] The server-side instruction interpretation mechanism concretizes the design requirements based on the analyzed keywords and phrases. In this process, the optimal design is created by considering factors such as the factory layout, the type of robots required, and the work area.

[0890] 5. Generating blueprints:

[0891] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E 2) is used to draw design drawings of the equipment layout and production lines within the factory.

[0892] 6. Check the generation results:

[0893] The results verification system checks whether the generated design drawings match the user's instructions. If necessary, it also provides an automatic correction function and offers feedback as needed.

[0894] 7. Submit the results:

[0895] The server sends the verified design drawings to the terminal. The user can then review the design drawings via the terminal and enter further modification instructions as needed.

[0896] Examples of specific cases and prompt statements

[0897] Specific example:

[0898] The user enters the following: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0899] Examples of prompts to input into a generative AI model:

[0900] Natural Language Processing: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[0901] This system allows users to efficiently and accurately generate factory equipment layouts and production line designs automatically, even without advanced expertise.

[0902] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0903] Step 1:

[0904] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, they might input instructions such as, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square." The input data is in natural language text format.

[0905] Step 2:

[0906] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request. The input is the natural language instruction entered by the user, and the output is the transmission of the instruction to the server.

[0907] Step 3:

[0908] The server's built-in natural language processing system analyzes the received instructions. Specifically, it uses the generative AI model GPT-4 to tokenize the instructions and extract important keywords and phrases. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square." The input is the user's natural language instructions, and the output is the analyzed keywords and phrases.

[0909] Step 4:

[0910] The server's instruction interpretation mechanism concretizes design requirements based on analyzed keywords and phrases. This process considers factors such as the factory layout, required robot types, and work areas to create an optimal design. The input consists of keywords and phrases from the natural language processing mechanism, while the output is the concretized design requirements.

[0911] Step 5:

[0912] Using the server's image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine such as DALL-E 2 is used to draw design drawings of factory equipment layouts and production lines. The input is the design requirements from the instruction interpretation means, and the output is the generated design drawing.

[0913] Step 6:

[0914] The result verification mechanism verifies that the generated design drawing matches the user's instructions. If necessary, it provides an automatic correction function and offers feedback as needed. The input is the generated design drawing, and the output is the design drawing verified through verification and correction.

[0915] Step 7:

[0916] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input is the verified blueprint, and the output is the blueprint provided to the user.

[0917] In this way, the system quickly and accurately automatically generates factory equipment layouts and production line designs based on the user's natural language instructions.

[0918] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0919] The present invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and an emotion engine, and aims to streamline and automate the design drawing creation process, as well as to recognize and adapt to the user's emotions.

[0920] System Configuration

[0921] 1. User input

[0922] The user inputs specific instructions regarding the design drawings using natural language via a terminal. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[0923] 2. Recognition of emotions

[0924] The terminal is equipped with an emotion engine that recognizes the user's emotions in real time from their facial expressions and voice as they input design instructions. This emotion information is sent to the server along with the natural language instructions.

[0925] 3. Sending instructions and emotional data

[0926] The device sends natural language instructions entered by the user and emotion data recognized by the emotion engine to the server. This data is sent using an HTTP POST request.

[0927] 4. Analysis of Instructions

[0928] The server is equipped with natural language processing capabilities to analyze received instructions and sentiment data. It uses a natural language processing engine to tokenize instructions, analyze the sentence structure, and analyze sentiment data.

[0929] 5. Interpretation of instructions and extraction of requirements

[0930] The server-side instruction interpretation mechanism creates specific design requirements based on extracted keywords, phrases, and sentiment data. During this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[0931] 6. Generating blueprints

[0932] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E) is used for generation to create wireless equipment layout diagrams for each floor.

[0933] 7. Check the generation results

[0934] The result verification mechanism verifies the generated design drawings. It checks whether the design drawings match the instructions and whether quality is maintained. If necessary, it performs automated corrections, taking sentiment data into consideration.

[0935] 8. Submit the results

[0936] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[0937] Specific example

[0938] 1. User input

[0939] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[0940] 2. Recognition of emotions

[0941] The emotion engine recognizes whether the user is stressed or relaxed based on their facial expressions and voice during input.

[0942] 3. Sending instructions and emotional data

[0943] The device sends these instructions and emotional data to the server.

[0944] 4. Analysis of Instructions

[0945] The server analyzes the received instructions using natural language processing to identify elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four devices per floor."

[0946] 5. Interpretation of instructions and extraction of requirements

[0947] The server's instruction interpretation mechanism uses this information to concretize design requirements while considering emotional data. For example, if the user is experiencing stress, the response is simplified to reduce the burden.

[0948] 6. Generating blueprints

[0949] The image generation means generates design drawings based on the design requirements created by the instruction interpretation means. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[0950] 7. Check the generation results

[0951] The result verification method confirms that the generated design drawings and instructions match, and makes corrections if necessary, taking emotional data into consideration.

[0952] 8. Submit the results

[0953] The server sends the verified blueprint to the terminal. The user reviews the blueprint and enters further instructions if necessary.

[0954] Through this process, users can create efficient and high-quality blueprints, and the introduction of an emotion engine improves the user experience.

[0955] The following describes the processing flow.

[0956] Step 1:

[0957] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[0958] Step 2:

[0959] Terminal: Analyzes the user's facial expressions and voice in real time while they are inputting instructions, and uses an emotion engine to recognize the user's emotional state. For example, it can determine whether the user is stressed or relaxed.

[0960] Step 3:

[0961] Terminal: Sends input natural language instructions and recognized emotion data to the server. Data is sent using an HTTP POST request.

[0962] Step 4:

[0963] Server: Activates natural language processing (NLP) to analyze received natural language instructions and sentiment data. First, it tokenizes the instructions and extracts important keywords and phrases necessary to understand the context. For example, it extracts information such as "3-story building," "office building," "Wi-Fi radio," and "4 units per floor."

[0964] Step 5:

[0965] Server: Analyzes emotional data to understand the user's emotional state. For example, if the user is stressed, it will simplify the response to reduce the burden on the user.

[0966] Step 6:

[0967] Server: Based on the keywords, phrases, and sentiment data extracted by the instruction interpretation system, specific design requirements are created. In this process, the optimal placement of wireless devices and the floor coverage area are defined.

[0968] Step 7:

[0969] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using an image generation engine (e.g., DALL-E). The design drawing includes a wireless equipment layout diagram for each floor.

[0970] Step 8:

[0971] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[0972] Step 9:

[0973] Server: Sends the verified blueprints to the terminal. The blueprints are sent in PDF or PNG format.

[0974] Step 10:

[0975] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[0976] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge. Furthermore, the introduction of an emotional engine improves the user experience and reduces stress.

[0977] (Example 2)

[0978] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0979] Existing design drawing processes are inefficient because users have to repeatedly input detailed instructions. Furthermore, the lack of consideration for user emotions during the design process leads to increased stress and hinders improvements in the user experience. Additionally, manual quality checks and corrections of design drawings increase the likelihood of errors.

[0980] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, an emotion recognition means, and a communication means. As a result, the user can input instructions in natural language, and based on those instructions and emotion data, a design drawing can be automatically and efficiently generated and confirmed, thereby improving the efficiency of the entire design process and enhancing the user experience.

[0981] "Input means" refers to a device or system for a user to input specific instructions regarding a design drawing in natural language.

[0982] A "natural language processing device" is a device or system that analyzes natural language instructions entered by a user and extracts the necessary information.

[0983] "Instruction interpretation means" refers to a device or system for creating specific design requirements based on information extracted by natural language processing means.

[0984] "Image generation means" refers to a device or system for generating design drawings based on design requirements created by the instruction interpretation means.

[0985] A "result verification means" is a device or system for verifying whether the generated design drawings match the user's instructions and making corrections as necessary.

[0986] "Output means" refers to a device or system for providing generated blueprints to a user or other system.

[0987] An "emotion recognition means" is a device or system for recognizing a user's emotions from facial expressions, voice, etc., and acquiring that information.

[0988] "Communication means" refers to a device or system for transmitting user instruction data and emotional data to a server, and for transmitting the generated blueprint to the user's terminal.

[0989] This invention is a system that allows users to input instructions regarding design drawings in natural language, and then automatically and efficiently generates design drawings based on those instructions and emotional data. This system automates the entire design process and, by taking user emotions into consideration, provides user-friendly and high-quality design drawings.

[0990] System Configuration

[0991] Overview of the design drawing creation system

[0992] This system consists of input means from a terminal, natural language processing means, instruction interpretation means, image generation means, result confirmation means, output means, emotion recognition means, and communication means.

[0993] User input

[0994] Using a device (e.g., PC, tablet, smartphone), the user inputs instructions regarding the design plan in natural language. For example, they might input specific instructions such as, "Please create a design plan for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms." These instructions are saved on the device as text in natural language format.

[0995] Recognition of emotions

[0996] The device is equipped with an emotion recognition system (e.g., Affectiva SDK), and when the user inputs instructions, the emotion engine evaluates the facial expression data and voice data acquired through the camera and microphone in real time. The emotion engine recognizes the user's emotional state (e.g., stressed state, relaxed state) and generates emotion data based on the result.

[0997] Sending instructions and emotional data

[0998] The device uses an HTTP POST request to send natural language instructions entered by the user and sentiment data recognized by the sentiment recognition system to the server. This data includes the entered instruction text and sentiment data indicating the user's emotional state (e.g., sentiment scoring).

[0999] Analysis of instructions

[1000] The server is equipped with natural language processing tools (e.g., GPT-4) that tokenize received instructions and analyze their structure. Specifically, it separates necessary keywords and phrases from the instruction text (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and simultaneously processes the analyzed sentiment data.

[1001] Interpretation of instructions and extraction of requirements

[1002] The server's instruction interpretation mechanism creates specific design requirements based on analyzed keywords, phrases, and sentiment data. For example, if a user is experiencing stress, it generates design requirements that simplify response statements and reduce the burden. This results in a more adaptive and user-friendly system.

[1003] Design drawing generation

[1004] The server's image generation means generates a design drawing based on the design requirements created by the instruction interpretation means. Specifically, it uses an image generation engine (e.g., DALL-E) to automatically generate a detailed design drawing, such as placing three wireless devices on each floor.

[1005] Checking the generation results

[1006] The server's result verification mechanism verifies the generated blueprint to confirm that it matches the user's instructions. It also automatically makes adjustments as needed based on emotional data. For example, if the user is experiencing stress, it will output a simpler, easier-to-understand blueprint.

[1007] Send results

[1008] The server sends the verified design drawings to the terminal. The user can review the design drawings through the terminal and enter further modification instructions as needed. For example, if the user re-enters "Change the placement of the radios to four per floor," the same processing steps will be followed to generate the revised design drawings.

[1009] Specific example

[1010] 1. User input

[1011] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a five-story office building."

[1012] 2. Recognition of emotions

[1013] The emotion engine recognizes that the user is relaxed based on their facial expressions and voice during input.

[1014] 3. Sending instructions and emotional data

[1015] The device sends instructions and emotional data to the server via an HTTP POST request.

[1016] 4. Analysis of Instructions

[1017] The server analyzes the instructions using natural language processing and extracts elements such as "5-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[1018] 5. Interpretation of instructions and extraction of requirements

[1019] The server uses this information to generate specific design requirements and creates response messages that take sentiment data into consideration.

[1020] 6. Generating blueprints

[1021] The image generation engine (DALL-E) generates design drawings based on the design requirements, for example, by placing four Wi-Fi wireless devices on each floor.

[1022] 7. Check the generation results

[1023] The server evaluates the quality of the design blueprint and makes necessary corrections, taking emotional data into consideration.

[1024] 8. Submit the results

[1025] The server sends the verified blueprints to the terminal, and the user confirms them.

[1026] This process allows users to automatically obtain efficient and high-quality blueprints, and through emotion recognition, a more comfortable and less stressful design process is achieved.

[1027] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1028] Step 1:

[1029] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building." This input is saved as text data on the terminal. The input data is a natural language instruction.

[1030] Step 2:

[1031] The device's emotion recognition system recognizes the user's facial expressions and voice in real time as they input instructions. Using the camera and microphone, it collects facial and voice data, which an emotion engine analyzes to evaluate the user's emotional state. This analysis generates emotion data such as "the user is stressed" or "the user is relaxed." Input data consists of facial and voice data, while output data is the result of the emotion evaluation.

[1032] Step 3:

[1033] The device sends the user's natural language instructions and sentiment data to the server using an HTTP POST request. Specifically, the user's instructions and sentiment data obtained by the sentiment engine are sent to the server. The input data consists of the instructions and sentiment data, while the output data is the request sent to the server.

[1034] Step 4:

[1035] The server uses natural language processing to tokenize the received instruction sentence and analyze its structure. It extracts the analyzed tokens and phrases (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and identifies the necessary information. Simultaneously, it also analyzes sentiment data and records the results. The input data consists of the instruction sentence and sentiment data, while the output data consists of the analyzed tokens and phrases.

[1036] Step 5:

[1037] The server's instruction interpretation mechanism creates specific design requirements based on the analyzed tokens and phrases. Sentimental data is also taken into consideration during this process. For example, if the user is experiencing stress, the response may be made more concise. The input data consists of analyzed tokens and sentimental data, while the output data consists of the specific design requirements.

[1038] Step 6:

[1039] The server's image generation mechanism generates a design drawing based on the design requirements created by the instruction interpretation mechanism. For generation, an image generation engine (e.g., DALL-E) is used to output a specific design drawing as an image. For example, a design drawing of an office building with three wireless transceivers on each floor is generated. The input data is the design requirements, and the output data is an image of the design drawing.

[1040] Step 7:

[1041] The server's result verification mechanism evaluates the quality of the generated blueprint. It checks whether the blueprint matches the user's instructions and meets specific quality standards, and makes corrections as needed based on sentiment data. For example, if the user is experiencing stress, the blueprint is adjusted to be visually simpler. The input data consists of the blueprint and sentiment data, and the output data is the corrected blueprint.

[1042] Step 8:

[1043] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input data is the verified blueprint, and the output data is the blueprint displayed on the terminal.

[1044] (Application Example 2)

[1045] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1046] In modern virtual store design, it is not enough to simply generate designs efficiently based on user instructions; it is also necessary to recognize the user's emotional state in real time and incorporate that information into the design. However, current systems simply design according to instructions without considering the user's emotions, which makes it difficult to improve customer satisfaction.

[1047] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an emotion recognition means, an instruction interpretation means, an image generation means, and a result confirmation means. This makes it possible to extract design requirements based on instructions and generate a design that reflects the user's emotions.

[1048] An "input means" is a device or interface that allows a user to input instructions to a system in natural language.

[1049] A "natural language processing device" is a processing device that analyzes natural language instructions input by a user and extracts the necessary information.

[1050] An "emotion recognition tool" is a device or software that recognizes a user's emotions in real time from their facial expressions and voice, and generates information based on those emotions.

[1051] A "direction interpretation means" is a device or function that creates specific design requirements based on information extracted by a natural language processing means and emotional data obtained by an emotion recognition means.

[1052] "Image generation means" refers to an engine or software that generates design drawings or designs based on the design requirements obtained by the instruction interpretation means.

[1053] A "result verification means" refers to a device or function that checks whether the generated images or design drawings match the instructions, verifies their quality, and makes corrections as necessary.

[1054] "Output means" refers to devices or interfaces for providing users with the final confirmed design drawings or images.

[1055] This invention is a system for automatically generating an optimal design for virtual stores by analyzing the instructions entered by the user when they design a virtual store, and by recognizing and considering the emotions the user was feeling at the time of input.

[1056] System Configuration

[1057] This system includes the following main means:

[1058] 1. Input Method: Users input instructions for the virtual store design in natural language using a smartphone or smart glasses. For example, they might input instructions such as, "Please change the layout of the product shelves to a more relaxing one."

[1059] 2. Natural Language Processing Method: Analyzes user-inputted instructions and extracts necessary information. Specifically, it uses a natural language processing engine (e.g., BERT, GPT-3) to tokenize the instruction sentences and analyze their structure.

[1060] 3. Emotion Recognition Method: Emotions are recognized in real time from the user's facial expressions and voice during input. This information is sent to the server along with natural language instructions. The technologies used include facial recognition algorithms and voice emotion recognition engines (e.g., OpenFace, IBM Watson Tone Analyzer).

[1061] 4. Instruction Interpretation Method: Based on data obtained from natural language processing and emotion recognition methods, specific design requirements are extracted. In this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[1062] 5. Image generation means: Generates a specific virtual store design based on the design requirements. An image generation engine (e.g., DALL-E) is used to generate an image based on the design requirements obtained from the instruction interpretation means.

[1063] 6. Result Verification Method: Verify that the generated design matches the instructions and that quality is maintained. If necessary, perform automated corrections that also take sentiment data into consideration.

[1064] 7. Output method: The final confirmed design is provided to the user. The user can review the design drawings and input further instructions as needed.

[1065] Server Processing

[1066] The server processes data by combining a series of methods. First, it receives natural language instructions and sentiment data from the user via an HTTP POST request. The received data is analyzed by a natural language analysis tool, and the content of the instructions is tokenized and structurally analyzed. Furthermore, considering the user's emotions recognized by the sentiment recognition tool, the instruction interpretation tool extracts specific design requirements. After that, the design generated using the image generation tool is verified by the results verification tool, and finally sent back to the user.

[1067] Specific example

[1068] Example 1: When a user inputs, "Please rearrange the books to create a relaxing corner," the system recognizes the user's relaxed state from their facial expressions and voice. Based on this, it generates a layout design that promotes relaxation.

[1069] Examples of prompts for generative AI models:

[1070] Prompt: "You have instructed the user to create a relaxing space in a corner of a coffee shop. The user is feeling stressed."

[1071] In this way, the present invention can combine user instructions and emotions to provide an optimal virtual store design and improve the user experience.

[1072] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1073] Step 1:

[1074] Users input design instructions for a virtual store using natural language via their smartphones or smart glasses. For example, they might give instructions such as, "Please create a book corner with a relaxing atmosphere." The input instructions are then sent to the server as initial data via the device.

[1075] Step 2:

[1076] The terminal analyzes the user's facial expressions and voice during input using emotion recognition technology, and acquires the user's emotion data in real time. The acquired emotion data is sent to the server along with natural language instructions. The specific operations of emotion recognition utilize facial recognition algorithms and voice emotion recognition engines.

[1077] Step 3:

[1078] The server receives input natural language instructions and sentiment data via an HTTP POST request. The received data is first passed to a natural language processing system, where the instruction sentences are tokenized and their structure is analyzed. For example, specific keywords and phrases such as "a book corner with a relaxing atmosphere" are extracted.

[1079] Step 4:

[1080] The server passes the data obtained by the natural language processing means to the instruction interpretation means. The instruction interpretation means extracts specific design requirements based on the analyzed instructions and emotion data. In this process, the design requirements are adjusted according to the user's emotions, thereby realizing an emotionally sensitive design process.

[1081] Step 5:

[1082] The server uses the design requirements extracted by the instruction interpretation means to generate a concrete design drawing using the image generation means. An image generation engine (e.g., DALL-E) is used to draw the layout of the virtual store according to the design instructions. During this process, the design requirements are input to the image generation engine as prompts.

[1083] Step 6:

[1084] The server verifies the generated blueprint using a results verification mechanism. It checks whether the blueprint matches the instructions and whether quality is maintained, and automatically makes corrections as needed, taking emotional data into consideration. For example, it checks whether the relaxation effect is properly achieved in response to an instruction requesting relaxation.

[1085] Step 7:

[1086] The server sends the final, verified blueprint to the terminal. The user can review the blueprint through the terminal and make further modifications by entering instructions if necessary. In this way, the blueprint is continuously adjusted to improve the user experience.

[1087] This series of processing steps enables efficient and user-friendly virtual store design by combining natural language input and emotion recognition.

[1088] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1089] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1090] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1091] [Fourth Embodiment]

[1092] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1093] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1094] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1095] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1096] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1097] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1098] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1099] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1100] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1101] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1102] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1103] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1104] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1105] This invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, and an output means, and aims to streamline and automate the process of creating design drawings.

[1106] System Configuration

[1107] 1. User input

[1108] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[1109] 2. Sending instructions

[1110] The terminal sends natural language instructions entered by the user to the server, using an appropriate communication protocol (e.g., HTTP POST request).

[1111] 3. Analysis of Instructions

[1112] The server is equipped with natural language processing capabilities to analyze received instructions. Specifically, it uses a natural language processing engine to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "5G wireless device," "office building," "3 devices per floor," and "8 rooms."

[1113] 4. Interpretation of instructions and extraction of requirements

[1114] The server-side instruction interpretation mechanism concretizes the design requirements based on extracted keywords and phrases. This process defines the optimal placement of wireless devices and the floor coverage area, among other things.

[1115] 5. Generating blueprints

[1116] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. For generation, an existing image generation engine (e.g., DALL-E) is used to create wireless equipment layout diagrams for each floor.

[1117] 6. Check the generation results

[1118] The result verification mechanism checks the generated design drawings and verifies that they match the user's instructions. An automatic correction function is also included, and further adjustments to the layout are made as needed.

[1119] 7. Submit the results

[1120] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[1121] Specific example

[1122] 1. User input

[1123] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[1124] 2. Sending instructions

[1125] The terminal sends this instruction to the server.

[1126] 3. Analysis of Instructions

[1127] The natural language processing system installed on the server analyzes the instructions and extracts elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[1128] 4. Interpretation of instructions and extraction of requirements

[1129] The server's instruction interpretation mechanism uses this information to specify the design requirements for the placement of wireless equipment on each floor. The optimal placement of wireless equipment is determined based on the floor area and room layout.

[1130] 5. Generating blueprints

[1131] The image generation means generates design drawings based on the above design requirements. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[1132] 6. Check the generation results

[1133] The result verification mechanism checks for consistency between the generated design drawings and the instructions, and provides feedback for corrections if necessary.

[1134] 7. Submit the results

[1135] The server sends the verified blueprint to the terminal. The user reviews it and enters further instructions if necessary.

[1136] This system allows users to obtain high-quality blueprints quickly without requiring advanced expertise or expensive tools.

[1137] The following describes the processing flow.

[1138] Step 1:

[1139] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[1140] Step 2:

[1141] Terminal: Sends the input natural language instructions to the server. This data is sent using an HTTP POST request.

[1142] Step 3:

[1143] Server: Activates natural language processing equipment to analyze the received natural language instructions. First, the instructions are tokenized, and the sentence structure is analyzed.

[1144] Step 4:

[1145] Server: Extracts necessary keys and phrases from the text analyzed by the natural language processing system. For example, it identifies elements such as "3-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[1146] Step 5:

[1147] Server: Based on the extracted keywords for instruction interpretation, specific design requirements are created. In this process, the area of ​​each floor and the layout of the rooms are considered to determine the optimal placement of the wireless equipment.

[1148] Step 6:

[1149] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using the image generation engine (e.g., DALL-E).

[1150] Step 7:

[1151] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[1152] Step 8:

[1153] Server: Sends the verified design drawings to the terminal. The design drawings are sent in PDF or PNG format.

[1154] Step 9:

[1155] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[1156] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge.

[1157] (Example 1)

[1158] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1159] Traditional design drawing processes require advanced expertise and expensive software tools, resulting in significant costs and time investment. Furthermore, manual design work is prone to human error, leading to inconsistent quality in the final product. Additionally, design changes and modifications require further time and effort to address.

[1160] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1161] In this invention, the server includes an input means for the user to input specific instructions regarding the design drawing in natural language, a natural language analysis means for analyzing the input natural language instructions, an instruction interpretation means for concretizing the design requirements based on extracted keywords and phrases, an image generation means for generating a visual design drawing based on the design requirements, a result confirmation means for confirming that the generated design drawing matches the instruction content, and an output means for transmitting the confirmed design drawing to a terminal. This makes it possible to automatically create high-quality design drawings in a short time without specialized knowledge, enabling efficient and highly accurate design work.

[1162] An "input means" is a device or interface that allows a user to input specific instructions regarding a design drawing in natural language.

[1163] A "natural language processing tool" is an engine or software that analyzes input natural language instructions and extracts important keywords and phrases.

[1164] A "direction interpretation means" is a processing device or algorithm for concretizing design requirements based on extracted keywords or phrases and determining the requirements of the design drawing.

[1165] "Image generation means" refers to an engine or software for generating visual design drawings based on design requirements.

[1166] A "result verification means" is a device or system that has the function of checking whether the generated design drawings match the user's instructions and making corrections as necessary.

[1167] "Output means" refers to a device or interface that transmits the verified design drawings to the user's terminal, allowing the user to review the design drawings.

[1168] This invention is a system for streamlining and automating the process of creating design drawings. This system includes multiple means for consistently processing everything from user input to the generation of design drawings. Specific embodiments of this system are described below.

[1169] System Configuration

[1170] This system uses the following hardware and software:

[1171] Hardware: User devices (e.g., PCs, tablets), servers

[1172] software:

[1173] Input means

[1174] Natural language processing tools (e.g., natural language processing engines such as GPT-3)

[1175] Instruction interpretation means

[1176] Image generation means (e.g., image generation engine such as DALL-E)

[1177] Means of checking results

[1178] Output means

[1179] Program Processing Description

[1180] 1. User input

[1181] Users input specific instructions regarding the design plans in natural language via their own devices. For example, they might input, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This allows the system to receive detailed requirements based on the user's needs.

[1182] 2. Sending instructions

[1183] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. This transmission is packaged in an appropriate format, and a simple communication protocol is also implemented to check for communication errors.

[1184] 3. Analysis of Instructions

[1185] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The server tokenizes the instructions to understand the context and extracts important keywords and phrases. For example, it analyzes information such as "three-story building," "office building," "Wi-Fi wireless router," and "four units per floor."

[1186] 4. Interpretation of instructions and extraction of requirements

[1187] The server's instruction interpretation mechanism concretizes design requirements based on extracted keywords and phrases. This process determines details such as the optimal placement of radios and floor coverage. For example, the optimal placement of radios is calculated considering the area of ​​each floor and the layout of the rooms.

[1188] 5. Generating blueprints

[1189] The server uses an image generation engine (e.g., DALL-E) to generate design drawings based on the design requirements. The image generation engine creates a visual design drawing based on input prompts, and generates a layout diagram that visually shows the location and coverage area of ​​the radio equipment.

[1190] 6. Check the generation results

[1191] The server's result verification mechanism checks whether the generated design matches the user's instructions. An automatic correction function also operates, readjusting the previously generated design as needed. During this process, a detailed check is performed to ensure it meets the user's requirements.

[1192] 7. Submit the results

[1193] The server sends the verified blueprint to the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[1194] Examples of specific actions

[1195] 1. User input

[1196] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[1197] 2. Sending instructions

[1198] The device sends this instruction to the server via an HTTP POST request.

[1199] 3. Analysis of Instructions

[1200] The server's natural language processing engine (such as GPT-3) breaks down the instruction sentence into keywords such as "three-story building," "office building," "Wi-Fi wireless device," and "four devices on each floor" and analyzes them.

[1201] 4. Interpretation of instructions and extraction of requirements

[1202] The server calculates the optimal location for the Wi-Fi wireless router, taking into account the area of ​​each floor and the layout of the rooms.

[1203] 5. Generating blueprints

[1204] The server uses an image generation engine to draw a layout diagram of the radio equipment, creating a visual design drawing.

[1205] 6. Check the generation results

[1206] The result verification mechanism checks the generated design drawings to confirm whether they meet the user's requirements.

[1207] 7. Submit the results

[1208] The server sends the verified blueprints to the terminal, and the user confirms them.

[1209] This system allows users to obtain high-quality blueprints quickly, without requiring advanced expertise or expensive tools.

[1210] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1211] Step 1: User input

[1212] The user uses a terminal to input specific instructions regarding the design drawings in natural language. An example of input would be a prompt message such as, "Install four Wi-Fi wireless devices on each floor of a three-story office building." This instruction would be the input. Through this operation, the user communicates the design requirements in detail to the system.

[1213] Step 2: Sending instructions

[1214] The terminal sends natural language instructions entered by the user to the server using an HTTP POST request. During this process, the input instructions are packaged into an appropriate format. The sent prompt text becomes the output. The terminal also checks for transmission errors through this operation.

[1215] Step 3: Analyzing the instructions

[1216] The server analyzes the received instructions using a natural language processing engine (e.g., GPT-3). The input to this operation is the sent prompt sentence, and the output is tokenized keywords and phrases. Specifically, the instruction sentence is broken down into important keywords such as "three-story building," "office building," "Wi-Fi radio," and "four units per floor."

[1217] Step 4: Interpretation of instructions and requirements extraction

[1218] The server's instruction interpretation mechanism concretizes design requirements based on tokenized keywords and phrases. The input to this operation is the extracted keywords and phrases, and the output is the specific content of the design requirements. Specifically, it calculates the optimal placement of wireless devices, taking into account the area of ​​each floor and the layout of the rooms.

[1219] Step 5: Generate the blueprint

[1220] The server uses an image generation means (e.g., DALL-E) to generate design drawings based on the design requirements. The input to this operation is the specified design requirements, and the output is a visual design drawing. The image generation engine creates a visual layout diagram showing the location and coverage area of ​​the radio equipment.

[1221] Step 6: Check the generation results

[1222] The server's result verification mechanism checks the generated blueprint against the user's instructions. The input for this operation is the generated visual blueprint, and the output is the verified blueprint. Furthermore, an automatic correction function is also in operation, including specific actions to readjust the blueprint as needed.

[1223] Step 7: Submit the results

[1224] The server sends the verified blueprint to the user's terminal. The input for this operation is the verified blueprint, and the output is the blueprint displayed on the user's terminal. Once the user's terminal receives the blueprint, the user can review it and enter correction instructions as needed.

[1225] (Application Example 1)

[1226] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1227] Designing equipment layouts and production lines within a factory is a task that requires advanced expertise and time, making it difficult to perform efficiently and accurately. Furthermore, there are limited systems that allow users to input instructions in natural language and automatically generate design drawings based on those instructions. A system is needed to solve this problem and enable rapid and accurate equipment layout and production line design within factories.

[1228] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1229] In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and means for automatically generating the layout of factory equipment and production line designs. This makes it possible to efficiently and accurately automatically generate the layout of factory equipment and production line designs based on instructions entered by the user in natural language.

[1230] An "input method" refers to a device or interface that allows a user to input instructions in natural language.

[1231] "Natural language processing tools" is a general term for engines and algorithms that analyze input natural language instructions and perform tokenization and keyword extraction.

[1232] The "instruction interpretation means" is a function that concretizes design requirements based on analyzed keywords and phrases and extracts the information necessary for generating design drawings.

[1233] "Image generation means" is a general term for engines and algorithms that automatically generate design drawings based on design requirements obtained from instruction interpretation means.

[1234] The "result verification means" is a function for confirming that the generated design drawings match the user's instructions and provides feedback for corrections as needed.

[1235] "Output means" refers to the means of providing users with confirmed design drawings, and includes functions for saving and displaying them as digital files.

[1236] "Means for automatically generating equipment layouts and production line designs within a factory" refers to a general term for engines and algorithms that automatically generate design drawings based on the factory layout, required robot types, and work areas.

[1237] This invention provides a system for automating the layout of equipment and production line design within a factory. The system includes an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result verification means, an output means, and means for automatically generating the layout of equipment and production line design within the factory.

[1238] Explain the program's processing in natural language.

[1239] The entire system operates as follows:

[1240] 1. User input instructions:

[1241] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, an instruction might be, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square."

[1242] 2. Sending instructions:

[1243] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request.

[1244] 3. Analysis of instructions:

[1245] The natural language processing (NLP) system installed on the server analyzes the received instructions. Specifically, it uses a natural language processing engine (e.g., GPT-4) to tokenize the instructions and extract important keywords and phrases necessary to understand the context. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square."

[1246] 4. Interpretation of instructions and extraction of requirements:

[1247] The server-side instruction interpretation mechanism concretizes the design requirements based on the analyzed keywords and phrases. In this process, the optimal design is created by considering factors such as the factory layout, the type of robots required, and the work area.

[1248] 5. Generating blueprints:

[1249] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E 2) is used to draw design drawings of the equipment layout and production lines within the factory.

[1250] 6. Check the generation results:

[1251] The results verification system checks whether the generated design drawings match the user's instructions. If necessary, it also provides an automatic correction function and offers feedback as needed.

[1252] 7. Submit the results:

[1253] The server sends the verified design drawings to the terminal. The user can then review the design drawings via the terminal and enter further modification instructions as needed.

[1254] Examples of specific cases and prompt statements

[1255] Specific example:

[1256] The user enters the following: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[1257] Examples of prompts to input into a generative AI model:

[1258] Natural Language Processing: "Install three new welding robots in the factory's manufacturing area. Each robot has a working area of ​​2 meters square."

[1259] This system allows users to efficiently and accurately generate factory equipment layouts and production line designs automatically, even without advanced expertise.

[1260] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1261] Step 1:

[1262] Users input specific instructions regarding the factory layout, required robot types, and work areas using natural language via devices such as smartphones and tablets. For example, they might input instructions such as, "Install three new welding robots in the factory's manufacturing area. Each robot's work area should be 2 meters square." The input data is in natural language text format.

[1263] Step 2:

[1264] The terminal sends natural language instructions entered by the user to the server. This is done using an appropriate communication protocol, such as an HTTP POST request. The input is the natural language instruction entered by the user, and the output is the transmission of the instruction to the server.

[1265] Step 3:

[1266] The server's built-in natural language processing system analyzes the received instructions. Specifically, it uses the generative AI model GPT-4 to tokenize the instructions and extract important keywords and phrases. For example, it extracts information such as "welding robot," "manufacturing area," "3 units," and "2 meters square." The input is the user's natural language instructions, and the output is the analyzed keywords and phrases.

[1267] Step 4:

[1268] The server's instruction interpretation mechanism concretizes design requirements based on analyzed keywords and phrases. This process considers factors such as the factory layout, required robot types, and work areas to create an optimal design. The input consists of keywords and phrases from the natural language processing mechanism, while the output is the concretized design requirements.

[1269] Step 5:

[1270] Using the server's image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine such as DALL-E 2 is used to draw design drawings of factory equipment layouts and production lines. The input is the design requirements from the instruction interpretation means, and the output is the generated design drawing.

[1271] Step 6:

[1272] The result verification mechanism verifies that the generated design drawing matches the user's instructions. If necessary, it provides an automatic correction function and offers feedback as needed. The input is the generated design drawing, and the output is the design drawing verified through verification and correction.

[1273] Step 7:

[1274] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input is the verified blueprint, and the output is the blueprint provided to the user.

[1275] In this way, the system quickly and accurately automatically generates factory equipment layouts and production line designs based on the user's natural language instructions.

[1276] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1277] The present invention is a system including an input means, a natural language processing means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, and an emotion engine, and aims to streamline and automate the design drawing creation process, as well as to recognize and adapt to the user's emotions.

[1278] System Configuration

[1279] 1. User input

[1280] The user inputs specific instructions regarding the design drawings using natural language via a terminal. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms."

[1281] 2. Recognition of emotions

[1282] The terminal is equipped with an emotion engine that recognizes the user's emotions in real time from their facial expressions and voice as they input design instructions. This emotion information is sent to the server along with the natural language instructions.

[1283] 3. Sending instructions and emotional data

[1284] The device sends natural language instructions entered by the user and emotion data recognized by the emotion engine to the server. This data is sent using an HTTP POST request.

[1285] 4. Analysis of Instructions

[1286] The server is equipped with natural language processing capabilities to analyze received instructions and sentiment data. It uses a natural language processing engine to tokenize instructions, analyze the sentence structure, and analyze sentiment data.

[1287] 5. Interpretation of instructions and extraction of requirements

[1288] The server-side instruction interpretation mechanism creates specific design requirements based on extracted keywords, phrases, and sentiment data. During this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[1289] 6. Generating blueprints

[1290] Using an image generation means, design drawings are generated based on the design requirements obtained from the instruction interpretation means. An image generation engine (e.g., DALL-E) is used for generation to create wireless equipment layout diagrams for each floor.

[1291] 7. Check the generation results

[1292] The result verification mechanism verifies the generated design drawings. It checks whether the design drawings match the instructions and whether quality is maintained. If necessary, it performs automated corrections, taking sentiment data into consideration.

[1293] 8. Submit the results

[1294] The server sends the verified design drawings to the terminal. The user can then review the design drawings through the terminal and enter further modification instructions as needed.

[1295] Specific example

[1296] 1. User input

[1297] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a three-story office building."

[1298] 2. Recognition of emotions

[1299] The emotion engine recognizes whether the user is stressed or relaxed based on their facial expressions and voice during input.

[1300] 3. Sending instructions and emotional data

[1301] The device sends these instructions and emotional data to the server.

[1302] 4. Analysis of Instructions

[1303] The server analyzes the received instructions using natural language processing to identify elements such as "three-story building," "office building," "Wi-Fi wireless router," and "four devices per floor."

[1304] 5. Interpretation of instructions and extraction of requirements

[1305] The server's instruction interpretation mechanism uses this information to concretize design requirements while considering emotional data. For example, if the user is experiencing stress, the response is simplified to reduce the burden.

[1306] 6. Generating blueprints

[1307] The image generation means generates design drawings based on the design requirements created by the instruction interpretation means. An image generation engine such as DALL-E is used to draw the layout diagram of the radio equipment.

[1308] 7. Check the generation results

[1309] The result verification method confirms that the generated design drawings and instructions match, and makes corrections if necessary, taking emotional data into consideration.

[1310] 8. Submit the results

[1311] The server sends the verified blueprint to the terminal. The user reviews the blueprint and enters further instructions if necessary.

[1312] Through this process, users can create efficient and high-quality blueprints, and the introduction of an emotion engine improves the user experience.

[1313] The following describes the processing flow.

[1314] Step 1:

[1315] User: Use the terminal's input interface to enter specific instructions regarding the design drawing in natural language. For example, "I want to install four Wi-Fi wireless devices on each floor of a three-story office building."

[1316] Step 2:

[1317] Terminal: Analyzes the user's facial expressions and voice in real time while they are inputting instructions, and uses an emotion engine to recognize the user's emotional state. For example, it can determine whether the user is stressed or relaxed.

[1318] Step 3:

[1319] Terminal: Sends input natural language instructions and recognized emotion data to the server. Data is sent using an HTTP POST request.

[1320] Step 4:

[1321] Server: Activates natural language processing (NLP) to analyze received natural language instructions and sentiment data. First, it tokenizes the instructions and extracts important keywords and phrases necessary to understand the context. For example, it extracts information such as "3-story building," "office building," "Wi-Fi radio," and "4 units per floor."

[1322] Step 5:

[1323] Server: Analyzes emotional data to understand the user's emotional state. For example, if the user is stressed, it will simplify the response to reduce the burden on the user.

[1324] Step 6:

[1325] Server: Based on the keywords, phrases, and sentiment data extracted by the instruction interpretation system, specific design requirements are created. In this process, the optimal placement of wireless devices and the floor coverage area are defined.

[1326] Step 7:

[1327] Server: The image generation means receives the design requirements created by the instruction interpretation means and generates a design drawing using an image generation engine (e.g., DALL-E). The design drawing includes a wireless equipment layout diagram for each floor.

[1328] Step 8:

[1329] Server: Verify the generated blueprints using the results verification method. Check whether the blueprints match the instructions and whether quality is maintained. Perform automatic corrections as needed.

[1330] Step 9:

[1331] Server: Sends the verified blueprints to the terminal. The blueprints are sent in PDF or PNG format.

[1332] Step 10:

[1333] Terminal: Displays the received blueprint to the user. The user can review the blueprint and enter further instructions as needed.

[1334] Through this process, users can obtain blueprints quickly and accurately, without requiring manual work or specialized knowledge. Furthermore, the introduction of an emotional engine improves the user experience and reduces stress.

[1335] (Example 2)

[1336] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1337] Existing design drawing processes are inefficient because users have to repeatedly input detailed instructions. Furthermore, the lack of consideration for user emotions during the design process leads to increased stress and hinders improvements in the user experience. Additionally, manual quality checks and corrections of design drawings increase the likelihood of errors.

[1338] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an instruction interpretation means, an image generation means, a result confirmation means, an output means, an emotion recognition means, and a communication means. As a result, the user can input instructions in natural language, and based on those instructions and emotion data, a design drawing can be automatically and efficiently generated and confirmed, thereby improving the efficiency of the entire design process and enhancing the user experience.

[1339] "Input means" refers to a device or system for a user to input specific instructions regarding a design drawing in natural language.

[1340] A "natural language processing device" is a device or system that analyzes natural language instructions entered by a user and extracts the necessary information.

[1341] "Instruction interpretation means" refers to a device or system for creating specific design requirements based on information extracted by natural language processing means.

[1342] "Image generation means" refers to a device or system for generating design drawings based on design requirements created by the instruction interpretation means.

[1343] A "result verification means" is a device or system for verifying whether the generated design drawings match the user's instructions and making corrections as necessary.

[1344] "Output means" refers to a device or system for providing generated blueprints to a user or other system.

[1345] An "emotion recognition means" is a device or system for recognizing a user's emotions from facial expressions, voice, etc., and acquiring that information.

[1346] "Communication means" refers to a device or system for transmitting user instruction data and emotional data to a server, and for transmitting the generated blueprint to the user's terminal.

[1347] This invention is a system that allows users to input instructions regarding design drawings in natural language, and then automatically and efficiently generates design drawings based on those instructions and emotional data. This system automates the entire design process and, by taking user emotions into consideration, provides user-friendly and high-quality design drawings.

[1348] System Configuration

[1349] Overview of the design drawing creation system

[1350] This system consists of input means from a terminal, natural language processing means, instruction interpretation means, image generation means, result confirmation means, output means, emotion recognition means, and communication means.

[1351] User input

[1352] Using a device (e.g., PC, tablet, smartphone), the user inputs instructions regarding the design plan in natural language. For example, they might input specific instructions such as, "Please create a design plan for installing three 5G-compatible wireless devices on each floor of an office building. Each floor has eight rooms." These instructions are saved on the device as text in natural language format.

[1353] Recognition of emotions

[1354] The device is equipped with an emotion recognition system (e.g., Affectiva SDK), and when the user inputs instructions, the emotion engine evaluates the facial expression data and voice data acquired through the camera and microphone in real time. The emotion engine recognizes the user's emotional state (e.g., stressed state, relaxed state) and generates emotion data based on the result.

[1355] Sending instructions and emotional data

[1356] The device uses an HTTP POST request to send natural language instructions entered by the user and sentiment data recognized by the sentiment recognition system to the server. This data includes the entered instruction text and sentiment data indicating the user's emotional state (e.g., sentiment scoring).

[1357] Analysis of instructions

[1358] The server is equipped with natural language processing tools (e.g., GPT-4) that tokenize received instructions and analyze their structure. Specifically, it separates necessary keywords and phrases from the instruction text (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and simultaneously processes the analyzed sentiment data.

[1359] Interpretation of instructions and extraction of requirements

[1360] The server's instruction interpretation mechanism creates specific design requirements based on analyzed keywords, phrases, and sentiment data. For example, if a user is experiencing stress, it generates design requirements that simplify response statements and reduce the burden. This results in a more adaptive and user-friendly system.

[1361] Design drawing generation

[1362] The server's image generation means generates a design drawing based on the design requirements created by the instruction interpretation means. Specifically, it uses an image generation engine (e.g., DALL-E) to automatically generate a detailed design drawing, such as placing three wireless devices on each floor.

[1363] Checking the generation results

[1364] The server's result verification mechanism verifies the generated blueprint to confirm that it matches the user's instructions. It also automatically makes adjustments as needed based on emotional data. For example, if the user is experiencing stress, it will output a simpler, easier-to-understand blueprint.

[1365] Send results

[1366] The server sends the verified design drawings to the terminal. The user can review the design drawings through the terminal and enter further modification instructions as needed. For example, if the user re-enters "Change the placement of the radios to four per floor," the same processing steps will be followed to generate the revised design drawings.

[1367] Specific example

[1368] 1. User input

[1369] The user uses a terminal to input, "Please install four Wi-Fi wireless devices on each floor of a five-story office building."

[1370] 2. Recognition of emotions

[1371] The emotion engine recognizes that the user is relaxed based on their facial expressions and voice during input.

[1372] 3. Sending instructions and emotional data

[1373] The device sends instructions and emotional data to the server via an HTTP POST request.

[1374] 4. Analysis of Instructions

[1375] The server analyzes the instructions using natural language processing and extracts elements such as "5-story building," "office building," "Wi-Fi wireless router," and "4 units per floor."

[1376] 5. Interpretation of instructions and extraction of requirements

[1377] The server uses this information to generate specific design requirements and creates response messages that take sentiment data into consideration.

[1378] 6. Generating blueprints

[1379] The image generation engine (DALL-E) generates design drawings based on the design requirements, for example, by placing four Wi-Fi wireless devices on each floor.

[1380] 7. Check the generation results

[1381] The server evaluates the quality of the design blueprint and makes necessary corrections, taking emotional data into consideration.

[1382] 8. Submit the results

[1383] The server sends the verified blueprints to the terminal, and the user confirms them.

[1384] This process allows users to automatically obtain efficient and high-quality blueprints, and through emotion recognition, a more comfortable and less stressful design process is achieved.

[1385] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1386] Step 1:

[1387] The user uses a terminal to input specific instructions regarding the design drawings in natural language. For example, they might input instructions such as, "Please create a design drawing for installing three 5G-compatible wireless devices on each floor of an office building." This input is saved as text data on the terminal. The input data is a natural language instruction.

[1388] Step 2:

[1389] The device's emotion recognition system recognizes the user's facial expressions and voice in real time as they input instructions. Using the camera and microphone, it collects facial and voice data, which an emotion engine analyzes to evaluate the user's emotional state. This analysis generates emotion data such as "the user is stressed" or "the user is relaxed." Input data consists of facial and voice data, while output data is the result of the emotion evaluation.

[1390] Step 3:

[1391] The device sends the user's natural language instructions and sentiment data to the server using an HTTP POST request. Specifically, the user's instructions and sentiment data obtained by the sentiment engine are sent to the server. The input data consists of the instructions and sentiment data, while the output data is the request sent to the server.

[1392] Step 4:

[1393] The server uses natural language processing to tokenize the received instruction sentence and analyze its structure. It extracts the analyzed tokens and phrases (e.g., "5G-compatible wireless device," "office building," "each floor," "3 units") and identifies the necessary information. Simultaneously, it also analyzes sentiment data and records the results. The input data consists of the instruction sentence and sentiment data, while the output data consists of the analyzed tokens and phrases.

[1394] Step 5:

[1395] The server's instruction interpretation mechanism creates specific design requirements based on the analyzed tokens and phrases. Sentimental data is also taken into consideration during this process. For example, if the user is experiencing stress, the response may be made more concise. The input data consists of analyzed tokens and sentimental data, while the output data consists of the specific design requirements.

[1396] Step 6:

[1397] The server's image generation mechanism generates a design drawing based on the design requirements created by the instruction interpretation mechanism. For generation, an image generation engine (e.g., DALL-E) is used to output a specific design drawing as an image. For example, a design drawing of an office building with three wireless transceivers on each floor is generated. The input data is the design requirements, and the output data is an image of the design drawing.

[1398] Step 7:

[1399] The server's result verification mechanism evaluates the quality of the generated blueprint. It checks whether the blueprint matches the user's instructions and meets specific quality standards, and makes corrections as needed based on sentiment data. For example, if the user is experiencing stress, the blueprint is adjusted to be visually simpler. The input data consists of the blueprint and sentiment data, and the output data is the corrected blueprint.

[1400] Step 8:

[1401] The server sends the verified blueprint to the terminal. The user can review the blueprint through the terminal and enter further modification instructions as needed. The input data is the verified blueprint, and the output data is the blueprint displayed on the terminal.

[1402] (Application Example 2)

[1403] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1404] In modern virtual store design, it is not enough to simply generate designs efficiently based on user instructions; it is also necessary to recognize the user's emotional state in real time and incorporate that information into the design. However, current systems simply design according to instructions without considering the user's emotions, which makes it difficult to improve customer satisfaction.

[1405] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes an input means, a natural language analysis means, an emotion recognition means, an instruction interpretation means, an image generation means, and a result confirmation means. This makes it possible to extract design requirements based on instructions and generate a design that reflects the user's emotions.

[1406] An "input means" is a device or interface that allows a user to input instructions to a system in natural language.

[1407] A "natural language processing device" is a processing device that analyzes natural language instructions input by a user and extracts the necessary information.

[1408] An "emotion recognition tool" is a device or software that recognizes a user's emotions in real time from their facial expressions and voice, and generates information based on those emotions.

[1409] A "direction interpretation means" is a device or function that creates specific design requirements based on information extracted by a natural language processing means and emotional data obtained by an emotion recognition means.

[1410] "Image generation means" refers to an engine or software that generates design drawings or designs based on the design requirements obtained by the instruction interpretation means.

[1411] A "result verification means" refers to a device or function that checks whether the generated images or design drawings match the instructions, verifies their quality, and makes corrections as necessary.

[1412] "Output means" refers to devices or interfaces for providing users with the final confirmed design drawings or images.

[1413] This invention is a system for automatically generating an optimal design for virtual stores by analyzing the instructions entered by the user when they design a virtual store, and by recognizing and considering the emotions the user was feeling at the time of input.

[1414] System Configuration

[1415] This system includes the following main means:

[1416] 1. Input Method: Users input instructions for the virtual store design in natural language using a smartphone or smart glasses. For example, they might input instructions such as, "Please change the layout of the product shelves to a more relaxing one."

[1417] 2. Natural Language Processing Method: Analyzes user-inputted instructions and extracts necessary information. Specifically, it uses a natural language processing engine (e.g., BERT, GPT-3) to tokenize the instruction sentences and analyze their structure.

[1418] 3. Emotion Recognition Method: Emotions are recognized in real time from the user's facial expressions and voice during input. This information is sent to the server along with natural language instructions. The technologies used include facial recognition algorithms and voice emotion recognition engines (e.g., OpenFace, IBM Watson Tone Analyzer).

[1419] 4. Instruction Interpretation Method: Based on data obtained from natural language processing and emotion recognition methods, specific design requirements are extracted. In this process, user emotions are taken into consideration, and the design process is adjusted to minimize stress.

[1420] 5. Image generation means: Generates a specific virtual store design based on the design requirements. An image generation engine (e.g., DALL-E) is used to generate an image based on the design requirements obtained from the instruction interpretation means.

[1421] 6. Result Verification Method: Verify that the generated design matches the instructions and that quality is maintained. If necessary, perform automated corrections that also take sentiment data into consideration.

[1422] 7. Output method: The final confirmed design is provided to the user. The user can review the design drawings and input further instructions as needed.

[1423] Server Processing

[1424] The server processes data by combining a series of methods. First, it receives natural language instructions and sentiment data from the user via an HTTP POST request. The received data is analyzed by a natural language analysis tool, and the content of the instructions is tokenized and structurally analyzed. Furthermore, considering the user's emotions recognized by the sentiment recognition tool, the instruction interpretation tool extracts specific design requirements. After that, the design generated using the image generation tool is verified by the results verification tool, and finally sent back to the user.

[1425] Specific example

[1426] Example 1: When a user inputs, "Please rearrange the books to create a relaxing corner," the system recognizes the user's relaxed state from their facial expressions and voice. Based on this, it generates a layout design that promotes relaxation.

[1427] Examples of prompts for generative AI models:

[1428] Prompt: "You have instructed the user to create a relaxing space in a corner of a coffee shop. The user is feeling stressed."

[1429] In this way, the present invention can combine user instructions and emotions to provide an optimal virtual store design and improve the user experience.

[1430] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1431] Step 1:

[1432] Users input design instructions for a virtual store using natural language via their smartphones or smart glasses. For example, they might give instructions such as, "Please create a book corner with a relaxing atmosphere." The input instructions are then sent to the server as initial data via the device.

[1433] Step 2:

[1434] The terminal analyzes the user's facial expressions and voice during input using emotion recognition technology, and acquires the user's emotion data in real time. The acquired emotion data is sent to the server along with natural language instructions. The specific operations of emotion recognition utilize facial recognition algorithms and voice emotion recognition engines.

[1435] Step 3:

[1436] The server receives input natural language instructions and sentiment data via an HTTP POST request. The received data is first passed to a natural language processing system, where the instruction sentences are tokenized and their structure is analyzed. For example, specific keywords and phrases such as "a book corner with a relaxing atmosphere" are extracted.

[1437] Step 4:

[1438] The server passes the data obtained by the natural language processing means to the instruction interpretation means. The instruction interpretation means extracts specific design requirements based on the analyzed instructions and emotion data. In this process, the design requirements are adjusted according to the user's emotions, thereby realizing an emotionally sensitive design process.

[1439] Step 5:

[1440] The server uses the design requirements extracted by the instruction interpretation means to generate a concrete design drawing using the image generation means. An image generation engine (e.g., DALL-E) is used to draw the layout of the virtual store according to the design instructions. During this process, the design requirements are input to the image generation engine as prompts.

[1441] Step 6:

[1442] The server verifies the generated blueprint using a results verification mechanism. It checks whether the blueprint matches the instructions and whether quality is maintained, and automatically makes corrections as needed, taking emotional data into consideration. For example, it checks whether the relaxation effect is properly achieved in response to an instruction requesting relaxation.

[1443] Step 7:

[1444] The server sends the final, verified blueprint to the terminal. The user can review the blueprint through the terminal and make further modifications by entering instructions if necessary. In this way, the blueprint is continuously adjusted to improve the user experience.

[1445] This series of processing steps enables efficient and user-friendly virtual store design by combining natural language input and emotion recognition.

[1446] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1447] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1448] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1449] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1450] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1451] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1452] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1453] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1454] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1455] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1456] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1457] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1458] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1459] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1460] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1461] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1462] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1463] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1464] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it does not deviate from the main purpose.

[1465] The descriptions and illustrations presented above are detailed explanations of the technical aspects of this disclosure and are merely examples of the technical aspects. For example, the above descriptions of the structure, function, operation, and effect are examples of the structure, function, operation, and effect of the technical aspects of this disclosure. Therefore, it goes without saying that you may delete unnecessary parts, add new elements, or replace elements in the descriptions and illustrations presented above, as long as you do not deviate from the essence of the technical aspects of this disclosure. Furthermore, in order to avoid confusion and facilitate understanding of the technical aspects of this disclosure, explanations of common technical knowledge and the like that do not require special explanation to enable the implementation of the technical aspects of this disclosure have been omitted from the descriptions and illustrations presented above.

[1466] All documents, patent applications, and technical standards described herein are incorporated by reference to the same extent as if each individual document, patent application, and technical standard were specifically and individually noted to be incorporated by reference.

[1467] The following is further disclosed regarding the embodiments described above.

[1468] (Claim 1)

[1469] Input means and

[1470] Natural language processing tools,

[1471] Means of interpreting instructions,

[1472] Image generation means,

[1473] Means for confirming results,

[1474] Output means and

[1475] A system that includes this.

[1476] (Claim 2)

[1477] The system according to claim 1, which uses a natural language processing means for analyzing instructions input in natural language.

[1478] (Claim 3)

[1479] The system according to claim 1, wherein the instruction interpretation means extracts design requirements and the image generation means generates a design drawing.

[1480] "Example 1"

[1481] (Claim 1)

[1482] An input method for users to input specific instructions regarding design drawings in natural language,

[1483] A natural language processing means for analyzing input natural language instructions,

[1484] An instruction interpretation means that concretizes design requirements based on extracted keywords and phrases,

[1485] Image generation means for generating a visual design drawing based on design requirements,

[1486] A means for verifying the results to confirm that the generated design drawings match the instructions,

[1487] An output means for transmitting the confirmed design drawings to the terminal,

[1488] A system that includes this.

[1489] (Claim 2)

[1490] The system according to claim 1, which uses a natural language processing engine that analyzes instructions input in natural language.

[1491] (Claim 3)

[1492] The system according to claim 1, wherein an instruction interpretation means extracts design requirements and an image generation engine generates a design drawing.

[1493] "Application Example 1"

[1494] (Claim 1)

[1495] Input means and

[1496] Natural language processing tools,

[1497] Means of interpreting instructions,

[1498] Image generation means,

[1499] Means for confirming results,

[1500] Output means and

[1501] A means for automatically generating the layout of equipment and production line designs within a factory,

[1502] A system that includes this.

[1503] (Claim 2)

[1504] The system according to claim 1, which uses a natural language processing means for analyzing instructions input in natural language.

[1505] (Claim 3)

[1506] The system according to claim 1, wherein the instruction interpretation means extracts design requirements and the image generation means generates a design drawing.

[1507] (Claim 4)

[1508] The system according to claim 1, wherein a natural language processing means analyzes instructions regarding the factory layout, the type of robots required, and the work area.

[1509] (Claim 5)

[1510] The system according to claim 1, further comprising a result confirmation means that allows a user to confirm and modify a design drawing generated by an image generation means.

[1511] "Example 2 of combining an emotion engine"

[1512] (Claim 1)

[1513] Input means and

[1514] Natural language processing tools,

[1515] Means of interpreting instructions,

[1516] Image generation means,

[1517] Means for confirming results,

[1518] Output means and

[1519] Means of recognizing emotions,

[1520] Communication methods,

[1521] A system that includes this.

[1522] (Claim 2)

[1523] The system according to claim 1, which uses a natural language processing means for analyzing instructions input in natural language.

[1524] (Claim 3)

[1525] The system according to claim 1, wherein the instruction interpretation means extracts design requirements and the image generation means generates a design drawing.

[1526] (Claim 4)

[1527] The system according to claim 1, wherein an emotion recognition means recognizes the user's emotions, and an instruction interpretation means adjusts the design requirements based on that data.

[1528] (Claim 5)

[1529] The system according to claim 1, which transmits user instruction data and emotion data to a server using communication means.

[1530] (Claim 6)

[1531] The system according to claim 1, wherein the result verification means evaluates the quality of the design drawings and makes corrections based on sentiment data as necessary.

[1532] "Application example 2 when combining with an emotional engine"

[1533] (Claim 1)

[1534] Input means and

[1535] Natural language processing tools,

[1536] Means of recognizing emotions,

[1537] Means of interpreting instructions,

[1538] Image generation means,

[1539] Means for confirming results,

[1540] Output means and

[1541] A system that includes this.

[1542] (Claim 2)

[1543] The system according to claim 1, which uses a natural language processing means for analyzing instructions input in natural language.

[1544] (Claim 3)

[1545] The system according to claim 1, wherein an emotion recognition means recognizes the user's emotions, and an instruction interpretation means extracts design requirements based on that data. [Explanation of symbols]

[1546] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Devices 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robots< / url:> < / url:> < / url:> < / url:>

Claims

[Claim 1] An input means for users to input design instructions to the system in natural language, A natural language processing means that analyzes natural language instructions input through an input means, An instruction interpretation means interprets specific design requirements and placement conditions based on instructions analyzed by a natural language processing means, An image generation means that automatically generates design drawings based on the design requirements obtained by the instruction interpretation means, A means for verifying the results, which includes checking the quality of the generated design drawings and verifying their consistency with the instructions, An output means for providing the user with the final confirmed design drawings, A system that includes this.

Images