system

Abstract

Provide a system. 【Solution means】Means for receiving and analyzing request data from users, Means for generating a design drawing based on the analyzed data, Means for presenting the generated design drawing to the user and obtaining confirmation, Means for selecting the optimal real producer from the craftsman database, Means for conducting conditional negotiations with the selected real producer, Means for monitoring the progress of real production, Means for solving problems occurring during real production, Means for presenting the completed real product to the user and obtaining confirmation, A system including means for delivering the finished product to the user.

Description

Technical Field

[0001] The technology of the present disclosure relates to a system.

Background Art

[0002] Patent Document 1 discloses a method for controlling a persona chatbot, which is performed by at least one processor and includes steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to an explanation of a chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] The present invention aims to solve the problem of matching people who want to create original products but have no craftsmanship skills with craftsmen who have skills but no work, and creating high - quality original crafts at low cost and in a short period. In conventional cloud - sourcing services, there is a problem that corresponding knowledge and skills are required on the requester side, making it difficult to use. Furthermore, when a company creates a prototype of a new product in small lots at low cost, the requester is often burdened.

Means for Solving the Problems

[0005] The present invention solves the aforementioned problems by providing a system that includes means for receiving and analyzing request data from users, means for generating blueprints based on the analyzed data, means for presenting the generated blueprints to the user and obtaining their confirmation, means for selecting the most suitable real-world creator from a craftsman database, means for negotiating terms with the selected real-world creator, means for monitoring the progress of real-world creation, means for resolving problems that arise during real-world creation, means for presenting the completed real-world product to the user and obtaining their confirmation, and means for delivering the finished product to the user. With this system, users do not need to design specific products or select creators, and the AI ​​supervises all processes, making it easy to create original crafts. Furthermore, it becomes easier for craftsmen to acquire new customers.

[0006] A "user" is an individual or legal entity that provides request data to the system and wishes to have original crafts created for them.

[0007] "Request data" refers to data that includes design and specification information that the user designs and uploads to the system.

[0008] "Analysis" refers to the process of extracting important information from the data provided by the user and converting it into a format necessary for generating design drawings.

[0009] A "design drawing" is a detailed drawing or plan that includes specific instructions such as dimensions, materials, and manufacturing methods.

[0010] A "optimal real-world creator" is an individual or group selected from a database of artisans who possess the skills and experience to perform the requested craftwork.

[0011] "Negotiating terms" is the process of reaching an agreement with the real-world creator regarding the work schedule and costs.

[0012] "Monitoring progress" is the process of checking the progress at each stage of real-time generation and verifying the status in real time to ensure quality.

[0013] "Solving a problem" refers to quickly addressing unexpected troubles or delays that occur during real-time generation and implementing a recovery plan.

[0014] A "finished product" is the final product created by a real-world generator based on the requested data and design drawings.

[0015] "Delivery" refers to the process of promptly delivering the finished product to the location specified by the user. [Brief explanation of the drawing]

[0016] [Figure 1] This is a conceptual diagram showing an example of the configuration of a data processing system according to the first embodiment. [Figure 2] This is a conceptual diagram showing an example of the essential functions of the data processing device and smart device according to the first embodiment. [Figure 3] This is a conceptual diagram showing an example of the configuration of a data processing system according to the second embodiment. [Figure 4] This is a conceptual diagram showing an example of the main functions of a data processing device and smart glasses according to the second embodiment. [Figure 5] This is a conceptual diagram showing an example of the configuration of a data processing system according to the third embodiment. [Figure 6] This is a conceptual diagram showing an example of the main functions of a data processing device and a headset-type terminal according to the third embodiment. [Figure 7] This is a conceptual diagram showing an example of the configuration of a data processing system according to the fourth embodiment. [Figure 8] This is a conceptual diagram showing an example of the main functions of a data processing device and a robot according to the fourth embodiment. [Figure 9] This shows an emotion map where multiple emotions are mapped. [Figure 10]Shows an emotion map to which a plurality of emotions are mapped. [Figure 11] It is a sequence diagram showing the processing flow of the data processing system in Example 1. [Figure 12] It is a sequence diagram showing the processing flow of the data processing system in Application Example 1. [Figure 13] It is a sequence diagram showing the processing flow of the data processing system in Example 2 when an emotion engine is combined. [Figure 14] It is a sequence diagram showing the processing flow of the data processing system in Application Example 2 when an emotion engine is combined.

Mode for Carrying Out the Invention

[0017] Hereinafter, an example of an embodiment of a system according to the technology of the present disclosure will be described according to the accompanying drawings.

[0018] First, the language used in the following description will be explained.

[0019] In the following embodiments, a numbered processor (hereinafter simply referred to as "processor") may be a single arithmetic unit or a combination of multiple arithmetic units. Also, the processor may be a single type of arithmetic unit or a combination of multiple types of arithmetic units. Examples of arithmetic units include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), and the like.

[0020] In the following embodiments, a numbered RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a work memory by the processor.

[0021] In the following embodiments, the signed storage is one or more non-volatile storage devices that store various programs and various parameters. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), or magnetic tapes.

[0022] In the following embodiments, the signed communication interface (I / F) is an interface that includes a communication processor and an antenna, etc. The communication interface manages communication between multiple computers. Examples of communication standards applicable to the communication interface include wireless communication standards such as 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), or Bluetooth (registered trademark).

[0023] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." That is, "A and / or B" means that it may be A alone, or B alone, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" applies when expressing three or more things linked by "and / or."

[0024] [First Embodiment]

[0025] Figure 1 shows an example of the configuration of the data processing system 10 according to the first embodiment.

[0026] As shown in Figure 1, the data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0027] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0028] The smart device 14 comprises a computer 36, a reception device 38, an output device 40, a camera 42, and a communication interface 44. The computer 36 comprises a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The reception device 38, output device 40, and camera 42 are also connected to the bus 52.

[0029] The reception device 38 is equipped with a touch panel 38A and a microphone 38B, etc., and receives user input. The touch panel 38A receives user input by detecting contact with an object (e.g., a pen or finger). The microphone 38B receives user input by detecting the user's voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0030] The output device 40 includes a display 40A and a speaker 40B, and presents data to the user 20 by outputting the data in a form perceptible to the user 20 (e.g., audio and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs audio according to instructions from the processor 46. The camera 42 is a small digital camera equipped with an optical system such as a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0031] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various types of information between processor 46 and processor 28 via network 54.

[0032] Figure 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0033] As shown in Figure 2, in the data processing device 12, a specific processing is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" related to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 according to the specific processing program 56 executed on the RAM 30.

[0034] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0035] In the smart device 14, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The reception output program 60 is used in conjunction with a specific processing program 56 by the data processing system 10. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0036] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0037] This invention provides a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans). This system includes the following main functions:

[0038] 1. Data entry and analysis

[0039] Users upload their original designs and ideas to the system. The uploaded data may include text, images, or videos, and the server performs an initial analysis based on this data.

[0040] The server extracts important keywords from the input data and analyzes the shape, color, and pattern of image data. It extracts dimensions and requirements from text data and requests additional information from the user if needed.

[0041] 2. Design drawing generation

[0042] The server uses design AI to generate detailed blueprints based on the analyzed data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods, enabling the creation of perfect crafts.

[0043] The generated design drawings are presented to the user for review and feedback. This ensures that the design drawings meet the user's requirements in a final confirmation.

[0044] 3. Selection and negotiation of real-world creators

[0045] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. The selection criteria are diverse, including past performance, area of ​​expertise, and reviews.

[0046] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[0047] 4. Monitoring and management of real-world generation

[0048] The server monitors the progress of the real-world generation in real time. It checks the progress at each stage and verifies the quality based on reports from the craftsmen.

[0049] In the event of a problem, the server immediately provides a recovery plan and instructs the craftsman. This ensures that the work proceeds smoothly and that high-quality products are completed.

[0050] 5. Confirmation of finished product and delivery

[0051] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the product is promptly delivered to the user.

[0052] After receiving the provided product, users contribute to improving future services by providing feedback to the server.

[0053] Specific example

[0054] The following are specific examples.

[0055] 1. Examples of custom-made clothing

[0056] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. The design AI generates a detailed design drawing, which is then presented to the user for confirmation.

[0057] Subsequently, the AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. Progress is monitored in real time, and any problems that arise are addressed immediately. The finished garments are then sent to the user for final confirmation, and after confirmation, they are promptly delivered.

[0058] 2. Examples of custom-made furniture

[0059] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. The blueprints are then presented to the user, and feedback is incorporated.

[0060] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and takes countermeasures if problems arise. The completed furniture is then requested to be confirmed by the user, and after confirmation, it is delivered.

[0061] This allows users to obtain high-quality, original products at a low cost and in a short time. Furthermore, artisans gain the opportunity to connect with a new customer base.

[0062] The following describes the processing flow.

[0063] Step 1:

[0064] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded via their device.

[0065] Step 2:

[0066] The server receives the uploaded request data and begins analyzing it. For text data, it uses natural language processing techniques to extract important keywords and requirements. For image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[0067] Step 3:

[0068] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[0069] Step 4:

[0070] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback as needed. Based on this feedback, the blueprint is revised again.

[0071] Step 5:

[0072] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[0073] Step 6:

[0074] The server will contact the selected real-world generator and negotiate the terms regarding the work schedule and costs. If an agreement is reached, the work progress schedule will be finalized.

[0075] Step 7:

[0076] The server prompts the real-world generators to begin their actual work. It monitors the progress of the real-world generation in real time, checking the progress at each stage. Based on reports and status updates from the craftsmen, it checks whether the work is progressing appropriately.

[0077] Step 8:

[0078] If a problem occurs during real-time generation, the server will quickly provide a recovery plan and take appropriate action, such as providing additional resources or adjusting work procedures.

[0079] Step 9:

[0080] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then performs the final confirmation and provides feedback, reporting any necessary corrections.

[0081] Step 10:

[0082] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[0083] (Example 1)

[0084] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0085] In traditional custom-made product manufacturing processes, it was difficult to adequately reflect the specific requests of users, and coordination with manufacturers and management of progress were cumbersome. This increased the likelihood of delivery delays, reduced quality, and decreased user satisfaction.

[0086] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0087] In this invention, the server includes means for receiving and analyzing design requests and ideas from users; means for generating blueprints based on the analyzed designs and ideas; means for presenting the generated blueprints to the user and receiving feedback; means for selecting the most suitable manufacturer from a database; means for negotiating working conditions with the selected manufacturer; means for monitoring the progress of manufacturing in real time; means for providing a recovery plan to resolve problems that arise during manufacturing; means for presenting the completed product to the user and obtaining final confirmation; and means for delivering the completed product to the user after final confirmation. This enables the fulfillment of user requests with high accuracy and consistent management of the manufacturing process.

[0088] "Requested designs and ideas" refer to information about the design of products that users provide to the system, and include formats such as text, images, and videos.

[0089] A "server" is a computer system that performs a series of processes such as analysis, design drawing generation, manufacturer selection, progress monitoring, and quality checks.

[0090] "Means of analysis" refers to software and algorithms used to extract and understand information about designs and ideas provided by users.

[0091] "Means for generating design drawings" refers to software or tools used to create specific product design drawings based on analyzed designs and ideas.

[0092] "Means of receiving feedback" refers to an interface that presents the generated blueprint to the user and collects additional requests and suggestions from the user.

[0093] "Means of selecting a manufacturer" refers to the procedures or algorithms used to select the most suitable manufacturer from a database.

[0094] "Means of negotiating working conditions" refers to means of contacting selected manufacturers and discussing matters such as delivery dates, costs, and specifications.

[0095] "Means of monitoring progress in real time" refers to software and tools used to track the manufacturing process in real time and monitor progress and problems.

[0096] "Means of providing a recovery plan" refers to software or procedures that provide quick solutions when problems occur during the manufacturing process.

[0097] "A means of presenting the finished product and obtaining final confirmation" refers to an interface used to present the completed product to the user and obtain their final approval.

[0098] "Delivery method" refers to the procedures and systems for quickly delivering finished products to the user's specified location.

[0099] This invention provides a comprehensive system for analyzing users' original designs and ideas, generating blueprints, and collaborating with manufacturers (craftsmen) to create concrete products. This system includes means for exchanging data between users, servers, and manufacturers to achieve an optimal manufacturing process.

[0100] Data entry and upload

[0101] Users upload their designs and ideas to the system. These designs and ideas are provided in various formats, including text, images, and videos. Users enter the data using a dedicated web form or smartphone app and click the upload button.

[0102] Data analysis and extraction

[0103] The server analyzes uploaded designs and ideas. For image data, it uses image analysis libraries such as OpenCV and TensorFlow to analyze shape, color, and pattern. For text data, it uses NLTK and spaCy to extract dimensions, materials, and other requirements. If necessary, the server will request additional information from the user.

[0104] Design drawing generation and verification

[0105] The server generates design drawings using design AI based on the analyzed designs and ideas. The design AI used includes the AutoCAD API and SolidWorks API. The generated design drawings include detailed information such as dimensions, materials, and manufacturing methods. The server then presents the design drawings to the user and receives feedback.

[0106] Selection and negotiation of manufacturers

[0107] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI employs a Python-based machine learning model. It selects manufacturers based on information such as past performance, areas of expertise, and reviews from a database. The server then contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications.

[0108] Manufacturing monitoring and control

[0109] The server uses cloud-based monitoring tools such as AWS CloudWatch to monitor the progress of manufacturing in real time. It receives progress reports from craftsmen and checks quality. If a problem occurs, it immediately provides a recovery plan and directs the corrective actions.

[0110] Confirmation of finished products and delivery

[0111] The server photographs the finished product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the server initiates procedures to quickly deliver the finished product to the user.

[0112] Specific example

[0113] The following are examples of custom-made products.

[0114] Examples of custom-made clothing

[0115] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. For example, the prompt might look like this:

[0116] Please generate detailed blueprints for the following design:

[0117] Item: Dress

[0118] Color: Red

[0119] Waist size: 70cm

[0120] Material: Silk

[0121] The server analyzes the data, extracts the necessary information, and then uses design AI to generate detailed blueprints, which are presented to the user. After user confirmation, the procurement AI selects the most suitable manufacturer, negotiates terms, and then begins work. Progress is monitored in real time, and the completed garments are sent to the user for final confirmation before being promptly delivered.

[0122] Examples of custom-made furniture

[0123] Users upload sketches, dimensions, and material details of their original furniture designs. For example, the prompt might look like this:

[0124] Please generate detailed blueprints for the following design:

[0125] Item: Table

[0126] Material: wood

[0127] Height: 75cm

[0128] Width: 120cm

[0129] Color: Brown

[0130] The server analyzes the data, generates blueprints, presents them to the user, and receives feedback. The procurement AI selects the most suitable manufacturer, negotiates terms, and then requests the manufacturer to perform the work. Progress is monitored in real time, and countermeasures are taken if problems arise. The completed furniture is asked to be confirmed by the user, and after confirmation, it is delivered.

[0131] This system allows users to quickly obtain high-quality, custom-made products, while also providing manufacturers with access to a new customer base.

[0132] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0133] Step 1: Data entry and upload

[0134] Users upload designs and ideas to the system using a dedicated web form or smartphone app. The data entered by users can be in the form of text, images, or videos, providing the system with user requests. Examples of input data include "red dress," "silk material," and "waist size 70cm." Users click the "Upload" button to submit the data. The server then receives this data.

[0135] Step 2: Data Analysis and Extraction

[0136] The server begins analyzing the data received from the user. Based on the input data, the server performs the following analyses:

[0137] For image data, we use OpenCV or TensorFlow to analyze shape, color, and pattern.

[0138] For text data, use NLTK or spaCy to extract dimensions, materials, and other requirements.

[0139] For example, features such as "shape: dress," "color: red," and "pattern: solid" are extracted from the image data. The server retrieves the necessary information as analysis results and notifies the user if additional information is required.

[0140] Step 3: Design drawing generation and verification

[0141] The server uses design AI to generate detailed design drawings based on the analyzed data. The design AI used is either the AutoCAD API or the SolidWorks API. It receives analysis results (dimensions, materials, design features, etc.) as input, and the design AI generates detailed design drawings based on this. The design drawings include details such as "length and width dimensions," "materials used," and "manufacturing methods."

[0142] The generated blueprints are presented to the user via the server. The user reviews the blueprints and provides feedback as needed. For example, they might submit requests such as "I want to widen the shoulders a little" or "I want to change the fabric color." The server receives this feedback and modifies the blueprints.

[0143] Step 4: Selecting and negotiating with manufacturers

[0144] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI uses a Python-based machine learning model. Based on user and product requirements data (past performance, expertise, reviews, etc.), it selects the most suitable manufacturer. The server contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications. For example, it might negotiate conditions such as "2 weeks for production, cost of 50,000 yen, and use of silk fabric."

[0145] Step 5: Monitoring and Controlling Production

[0146] The server monitors the manufacturing progress in real time. It uses cloud-based monitoring tools such as AWS CloudWatch to receive progress reports from manufacturers. For example, it retrieves progress information such as "30% sewn" or "dyeing process started." The server manages this progress information and checks quality. If a problem occurs, the server provides a recovery plan and gives appropriate instructions to the manufacturer.

[0147] Step 6: Check and ship the finished product.

[0148] The server photographs the finished product and sends the image to the user. The user reviews the image and makes a final confirmation, for example, by giving approval such as "It's just as pictured, no problems." After the final confirmation, the server proceeds with the shipping process. The server interacts with the shipping company's API to determine the shipping schedule for the user's address. For example, it generates information such as "Scheduled delivery date: October 15th" and "Shipping company: XX Transportation."

[0149] Through the steps outlined above, this system can realize users' original designs with high precision and deliver products efficiently and quickly.

[0150] (Application Example 1)

[0151] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart device 14 will be referred to as the "terminal."

[0152] When bringing user-designed original products to life, challenges exist, such as inadequate communication of detailed design instructions, inappropriate selection of real-world product generators, insufficient monitoring of progress, and slow problem resolution. Furthermore, as the diversity and accuracy of user-uploaded design data increase, there is a need for a system that can smoothly analyze this data and generate prompts.

[0153] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0154] In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, means for presenting the generated design drawing to the user and obtaining their confirmation, means for selecting the most suitable real-world generator from a database of skilled craftsmen, means for negotiating terms with the selected real-world generator, means for monitoring the progress of real-world generation in real time, means for resolving problems that arise during real-world generation, means for presenting the completed real-world product to the user and obtaining their confirmation, means for delivering the finished product to the user, means for extracting dimensions and important information from the user's design images and text, and means for generating prompt sentences based on the generated design drawing and inputting them into a generation AI model to obtain more detailed instructions. This makes it possible to analyze the user's design in detail, select the most suitable real-world generator, and quickly monitor the progress and resolve problems in real time. Furthermore, by using a generation AI model, the user's design can be further optimized and specific instructions can be obtained quickly.

[0155] "Requested data" refers to digital data in which a user expresses their original design or idea, and includes images, text, or video formats.

[0156] A "design drawing" is a detailed drawing generated from the analyzed request data, containing specific instructions including the dimensions, materials, and manufacturing methods of the product desired by the user.

[0157] A "user" is an individual or organization that uses this system to request a custom-made product.

[0158] The "Craftsman Database" is a database containing information on a variety of craftsmen, including each craftsman's area of ​​expertise, achievements, reviews, and other relevant information.

[0159] A "real-world creator" is a craftsman or artisan who produces a concrete product based on a design requested by a user.

[0160] "Negotiating terms" refers to the process of agreeing on terms such as delivery date, cost, and specifications with the real-world creator regarding the request.

[0161] "Real-world generation" refers to the process of creating real-world products based on user request data.

[0162] "Real-time monitoring of progress" is a process of checking the progress of work at each stage of real-time generation in real time and giving instructions or making corrections as needed.

[0163] A "prompt message" is a phrase used as input for the AI ​​generation model, and it is generated by analyzing the user's design.

[0164] A "generative AI model" is a model that uses artificial intelligence to analyze a design and generate more detailed instructions.

[0165] The means of extracting "dimensions and important information" refers to a function that extracts necessary information from uploaded design images and text.

[0166] "A means of inputting into a generative AI model to obtain more detailed instructions" refers to the process of inputting prompt sentences into a generative AI model based on the analysis results, and generating specific instructions.

[0167] The system for implementing this invention is built based on the following program.

[0168] First, users upload their original designs or ideas as request data using a smartphone or tablet application. This request data can include images, text, or videos.

[0169] Data Analysis

[0170] The server analyzes the uploaded request data and extracts important keywords, shapes, colors, patterns, dimensions, etc. Image data is converted to grayscale using OpenCV, thresholded, and then subjected to shape analysis. Text data is extracted using Pytesseract to obtain dimensions and important information.

[0171] Design drawing generation

[0172] The server generates detailed blueprints based on the extracted data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. Furthermore, prompt statements are generated based on these blueprints and input into the generating AI model to obtain even more detailed instructions.

[0173] Selection of craftsmen and negotiation of terms

[0174] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications, and then finalizes the work schedule once an agreement is reached.

[0175] Monitoring and management of real-world generation

[0176] The server monitors the progress of real-time generation, checking the craftsman's work process. If a problem occurs, the server immediately provides a recovery plan and instructs the craftsman. This ensures that work proceeds smoothly and high-quality products are completed.

[0177] Confirmation of finished products and delivery

[0178] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving confirmation from the user, the product is promptly shipped. After receiving the product, the user provides feedback to the server, contributing to improvements in future services.

[0179] Hardware and software to be used

[0180] Hardware: Smartphones, tablets, servers

[0181] Software: Python 3, OpenCV, Pytesseract, Keras

[0182] Specific example

[0183] 1. Uploading the design: The user uploads an image of their original costume design named "example_design.jpg" to the application.

[0184] 2. Data Analysis: The server analyzes the uploaded design images and uses OpenCV to extract important information such as shape and dimensions. If text data is included, Pytesseract is used to extract the text and obtain dimensions and other important information.

[0185] 3. Prompt message generation: Prompt messages are generated based on the extracted data and input into the generation AI model. This allows for more detailed instructions to be obtained.

[0186] Example of a prompt

[0187] "Please analyze the user-uploaded design (e.g., example_design.jpg) and extract information about its shape and dimensions."

[0188] This allows for detailed analysis of the user's design, selection of the most suitable craftsman, real-time monitoring of progress, and rapid problem resolution. Furthermore, by using generative AI models, the user's design can be further optimized, and specific instructions can be obtained quickly.

[0189] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0190] Step 1:

[0191] Users upload designs.

[0192] Users upload their own designed images, text, or video request data using a smartphone or tablet application.

[0193] Input: Image file (e.g., example_design.jpg), text file

[0194] Output: Design data saved on the server

[0195] Step 2:

[0196] Analysis of requested data

[0197] The server receives the uploaded request data and begins analysis. For image data, it uses OpenCV to convert it to grayscale and recognize its shape and color. For text data, it uses Pytesseract to extract the text.

[0198] Input: Design data stored on the server

[0199] Output: Analytical data such as extracted shape information, color information, and dimensions.

[0200] Step 3:

[0201] Design drawing generation

[0202] The server generates a detailed blueprint based on the analyzed data. This blueprint includes dimensions, materials, and manufacturing methods. It also generates prompt statements based on the generated blueprint and inputs them into the AI ​​model to obtain more specific instructions.

[0203] Input: Analysis data

[0204] Output: Detailed blueprint, generated prompts

[0205] Step 4:

[0206] Selection and negotiation of terms

[0207] The server uses a procurement AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications.

[0208] Input: Detailed blueprints, craftsman database

[0209] Output: Selected real generators, agreed conditions

[0210] Step 5:

[0211] Monitoring and management of real-world generation

[0212] The server monitors the progress of the real-time generation process. During this process, it checks the craftsman's work process and immediately provides a recovery plan if any problems occur.

[0213] Input: Real-time generation progress data

[0214] Output: Progress report, recovery plan

[0215] Step 6:

[0216] Confirmation of finished products and delivery

[0217] The server photographs the completed physical product and presents the image to the user for final confirmation. After confirmation, the user requests delivery of the product to the store or their home.

[0218] Input: Image of the finished product

[0219] Output: User's final confirmation and delivery instructions

[0220] Examples of specific actions

[0221] 1. Upload your design:

[0222] The user takes a photo of the clothing design "example_design.jpg" with their smartphone and uploads the image using the application.

[0223] 2. Data Analysis:

[0224] The server converts the uploaded "example_design.jpg" to grayscale using OpenCV, applies thresholding, and analyzes its shape and color information. If text is present, Pytesseract is used to extract dimensions and specifications as text.

[0225] 3. Generating blueprints:

[0226] The server inputs prompt messages into the generated AI model based on the analysis data, receiving instructions such as, "Please describe the specific dimensions and manufacturing method of this design." Based on these instructions, a detailed design drawing is created.

[0227] 4. Selection and Negotiation of Terms:

[0228] The server uses an AI-powered scheduling system to search a database of artisans, select the most suitable one, and contact them. For example, it might find an artisan who has previously produced similar garments and reconfirm the delivery time and cost requirements.

[0229] 5. Monitoring and management of real-world generation:

[0230] The server instructs the craftsman to report progress in real time, and progress reports are received as they occur. If a problem arises, alternative solutions are provided promptly.

[0231] 6. Confirmation of finished product and delivery:

[0232] The craftsman photographs the finished garment and sends the image to the server. The server then presents the image to the user and requests final confirmation. After the user confirms, the shipping process begins.

[0233] These steps allow users to efficiently bring their original designs to life and receive high-quality products.

[0234] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0235] This invention combines a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans), with an emotion engine that recognizes user emotions, to further enhance the user experience. This system includes the following key functions:

[0236] 1. Data entry and analysis

[0237] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded from the user's device.

[0238] The server receives the uploaded request data and begins analysis. For text data, natural language processing techniques are used to extract important keywords and requirements, while for image and video data, image recognition techniques are used to analyze shapes, colors, patterns, etc.

[0239] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[0240] 2. User sentiment analysis using an emotion engine

[0241] When analyzing request data, the server uses an emotion engine to analyze the user's emotions based on user input and feedback. This emotion data is used to generate blueprints and optimize feedback.

[0242] 3. Design drawing generation and feedback

[0243] The server presents the generated blueprint to the user and receives feedback while monitoring the user's emotional state. If the user expresses a specific emotion (e.g., satisfaction, dissatisfaction) towards the blueprint, the server modifies the blueprint accordingly.

[0244] 4. Selection and negotiation of real-world creators

[0245] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[0246] The server contacts the selected real-world generator, negotiates working conditions (delivery date, cost, specifications), and confirms the work schedule once an agreement is reached.

[0247] 5. Monitoring and Management of Real-Generation

[0248] The server monitors the progress of the real-time creators in real time and receives progress reports. It also monitors the users' emotional states and provides emotional data to the real-time creators as needed to optimize their work.

[0249] In the event of a problem, the server immediately provides a recovery plan and gives appropriate instructions to the real-world creators. This ensures that work proceeds smoothly and high-quality products are produced.

[0250] 6. Confirmation of finished products and delivery

[0251] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then makes a final confirmation and reports any necessary corrections.

[0252] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[0253] Specific example

[0254] 1. Examples of custom-made clothing

[0255] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. A design AI generates a detailed blueprint, presents it to the user while analyzing their emotions, and obtains their confirmation.

[0256] The AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. It monitors the progress in real time and communicates appropriately based on the user's feelings. The user is asked to give final confirmation of the finished garment, and after confirmation, it is promptly delivered.

[0257] 2. Examples of custom-made furniture

[0258] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. It presents the blueprints while analyzing the user's emotions and incorporates their feedback.

[0259] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and resolves issues based on the user's emotions. The completed furniture is then requested to be confirmed by the user, and after confirmation, delivery is arranged.

[0260] This allows users to obtain high-quality, original products at low cost and in a short timeframe, while also providing a personalized experience powered by an emotional engine. Furthermore, artisans gain the opportunity to connect with a new customer base.

[0261] The following describes the processing flow.

[0262] Step 1:

[0263] Users upload their original designs and ideas to the system in text, image, and video formats. This data is then sent from the user's device to the server.

[0264] Step 2:

[0265] The server receives the uploaded request data and begins initial analysis. For text data, it uses natural language processing techniques to extract important keywords and requirements, while for image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[0266] Step 3:

[0267] The server uses an emotion engine to analyze user emotions from their input and feedback. Emotional data is extracted based on information obtained from the user's text, voice, images, and videos.

[0268] Step 4:

[0269] The server uses design AI based on the analysis results to generate detailed blueprints. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. The design AI also takes user sentiment data into consideration to generate blueprints that enhance user satisfaction.

[0270] Step 5:

[0271] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback. At this stage, the emotion engine analyzes the user's emotional state in real time, reading emotions from the user's facial expressions and tone of voice.

[0272] Step 6:

[0273] The server revises its design based on user feedback and asks the user for final confirmation. This process can be repeated until the user is satisfied.

[0274] Step 7:

[0275] The server uses a scheduling AI to select the most suitable real-world creator from a database of artisans. Selection criteria include past performance, area of ​​expertise, reviews, and user sentiment data.

[0276] Step 8:

[0277] The server contacts the selected real-world generator and negotiates the terms of work (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[0278] Step 9:

[0279] The server monitors the progress of real production in real time and receives progress reports. The emotion engine also monitors the user's emotional state in parallel and provides emotion data to the real producer as needed. This personalizes the work process and improves quality.

[0280] Step 10:

[0281] If a problem occurs during real production, the server immediately provides a recovery plan and gives appropriate instructions to the craftsman. For example, providing additional resources or adjusting the work procedure are considered.

[0282] Step 11:

[0283] The server takes a photo of the completed real product, presents the image to the user, and requests final confirmation. The emotion engine also analyzes the user's emotional state at the time of final confirmation and makes final adjustments if necessary.

[0284] Step 12:

[0285] After receiving the final confirmation from the user, the server arranges to quickly deliver the finished product to the user. It coordinates with the delivery provider to safely and quickly deliver the finished product to the user.

[0286] Step 13:

[0287] After receiving the provided product, the user contributes to the improvement of future services by providing feedback to the server. The emotion engine also extracts emotion data from this feedback and uses it to improve the entire system.

[0288] Through the above processing steps, it is possible to provide an original product with high satisfaction to the user, and the craftsman can also obtain an opportunity to connect with a new customer base.

[0289] (Example 2)

[0290] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart device 14 as the "terminal".

[0291] Traditional systems for translating original designs into actual products were time-consuming and costly, and struggled to accurately reflect user intentions and emotions. Furthermore, the process of negotiating terms with craftsmen, monitoring the progress of the real-world product creation, and resolving problems was cumbersome, resulting in a poor user experience.

[0292] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for analyzing the request data using natural language processing or image recognition technology according to the format of the request data, means for generating a detailed design drawing using design AI based on the analyzed data, means for analyzing the user's emotional state and modifying the design drawing based on feedback, means for selecting the optimal real-world producer from a craftsman database using arrangement AI, means for negotiating terms with the selected real-world producer, means for monitoring the progress of real-world production in real time and providing emotional data as needed, means for immediately providing a recovery plan for problems that occur during real-world production, means for presenting the completed real-world product to the user and obtaining confirmation, and means for delivering the finished product to the user. This enables the rapid, low-cost, and high-quality realization of the user's designs and ideas into original products, and allows for a personalized experience that takes the user's emotions into consideration.

[0293] "Request data" refers to information provided by the user to the system, including original designs and ideas, and can be in text, image, or video format.

[0294] "Natural language processing" is a technique for extracting important keywords and requirements from text data, and involves using models such as BERT and GPT-3.

[0295] "Image recognition technology" is a technique that analyzes features such as color, shape, and pattern from image and video data, and involves using software such as the Google Cloud Vision API.

[0296] "Design AI" refers to an artificial intelligence system that automatically generates detailed design drawings based on user request data, and includes tools such as AutoCAD and Blender.

[0297] An "emotion engine" is a system that analyzes emotions from user input and feedback, using technologies such as IBM Watson Tone Analyzer.

[0298] "Arrangement AI" is an artificial intelligence system that selects the most suitable real-world creator from a database of skilled craftsmen.

[0299] A "real-world creator" is a craftsman or business that actually produces goods based on a user's request.

[0300] A "recovery plan" is a set of countermeasures to solve problems that occur during real-time generation.

[0301] This invention enhances the user experience by combining a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans) with an emotion engine that recognizes the user's emotions. This system includes the following main functions:

[0302] Data entry and analysis

[0303] Users upload their original designs and ideas to the system in text, image, and video formats. Data is sent from the user's device to the server. Specifically, a user uploads a sketch of a garment they designed from their device and sends the data by pressing the "Upload" button.

[0304] The server receives the uploaded request data and starts analysis. In the case of text data, important keywords and requirements are extracted using natural language processing technologies (e.g., BERT or GPT-3). In the case of image or video data, image recognition technologies (e.g., Google Cloud Vision API) are used to analyze shapes, colors, patterns, etc. For example, a sketch image of a piece of clothing is sent to the Google Cloud Vision API to recognize color codes and shapes.

[0305] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings. These design drawings contain specific instructions such as dimensions, materials, and processing methods. Specifically, Blender is used to generate a 3D model of the clothing, and dimension and material information is attached based on it.

[0306] User sentiment analysis by the sentiment engine

[0307] When analyzing the user's input and feedback, the server uses a sentiment engine (e.g., IBM Watson Tone Analyzer) to analyze the user's sentiment. This sentiment data is used for generating design drawings and optimizing feedback. When the user provides text feedback such as "satisfied" or "dissatisfied" with a sketch, the sentiment is analyzed.

[0308] Design drawing generation and feedback

[0309] The server presents the generated design drawing to the user and accepts feedback based on the emotional state. For example, when the user sends feedback such as "I want more red in this design", the server uses the IBM Watson Tone Analyzer to analyze the sentiment and modify the design drawing.

[0310] Selection of real producers and condition negotiation

[0311] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. Specifically, the server queries the craftsman database and requests the appropriate craftsman.

[0312] The server contacts the selected real-world generators and negotiates working conditions (delivery date, cost, specifications). For example, it contacts craftsmen via email to reach an agreement on delivery date and cost.

[0313] Monitoring and management of real-world generation

[0314] The server monitors the progress of real-time creators and receives progress reports. It also monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives progress images from the creators periodically and shares the results of the user's emotional analysis with them.

[0315] The server will immediately provide a recovery plan and give appropriate instructions to the real-world generator in the event of a problem. For example, it will suggest alternative materials if a material shortage occurs.

[0316] Confirmation of finished products and delivery

[0317] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user then reviews the product and reports any necessary corrections. For example, the user might send an image of the finished garment and obtain final approval.

[0318] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[0319] Specific example

[0320] Examples of custom-made clothing

[0321] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their designed clothing to the system. The server analyzes this data and extracts the necessary information. A design AI generates detailed blueprints, presents them to the user while analyzing their emotions, and obtains their confirmation. A procurement AI selects the most suitable craftsman, negotiates terms, and begins work. The system monitors the progress in real time and communicates appropriately based on the user's emotions. The completed clothing is then asked for final confirmation from the user, and after confirmation, it is promptly delivered.

[0322] Example of a prompt

[0323] "I've uploaded an original sketch of a garment I designed. The material is cotton, the color is red, and the size is M. Please create a detailed blueprint for this design, analyze my feelings, and provide feedback."

[0324] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0325] Step 1: Data Entry

[0326] Users can upload their original designs and ideas to the system in image, text, and video formats. Generally, users send data to the server using their device. When the "Upload" button is pressed, the selected design data is sent. Specifically, users can upload sketches of clothing or detailed product specifications (material, color, size, etc.) from their device.

[0327] Input: Original designs and ideas from users (images, text, videos)

[0328] Output: Design data sent to the server

[0329] Step 2: Data Analysis

[0330] The server receives the uploaded design data and analyzes it according to its format. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shape, color, and patterns. For video data, it performs image recognition frame by frame.

[0331] Input: Design data sent to the server

[0332] Output: Analysis results (important keywords, shapes, colors, patterns, etc.)

[0333] Step 3: Generate blueprints

[0334] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings based on the analyzed data. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. For example, Blender is used to generate a 3D model of clothing, and then dimensions and material information is added based on that model.

[0335] Input: Analysis results (important keywords, shapes, colors, patterns, etc.)

[0336] Output: Detailed design drawings (dimensions, materials, manufacturing methods, etc.)

[0337] Step 4: Sentiment Analysis

[0338] The server uses an emotion engine (for example, IBM Watson Tone Analyzer) to analyze emotions based on user input data and feedback. When a user provides text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, the server analyzes that emotion.

[0339] Input: User feedback (text format)

[0340] Output: Emotion analysis results

[0341] Step 5: Feedback and design revisions

[0342] The server presents the generated blueprint to the user and accepts feedback based on the sentiment analysis results. It then modifies the blueprint based on the user's feedback. For example, if a user submits a comment such as, "I want more red in this design," the server analyzes that feedback and adjusts the blueprint again.

[0343] Input: Sentiment analysis results, user feedback

[0344] Output: Revised blueprint

[0345] Step 6: Selecting a Real Generator

[0346] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. For example, the server queries the craftsman database and lists the most suitable craftsmen.

[0347] Input: Modified blueprint

[0348] Output: Selected real generators

[0349] Step 7: Negotiating Terms

[0350] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). Specifically, negotiations are conducted via email and telephone until an agreement is reached.

[0351] Input: Selected real generator

[0352] Output: Agreed working conditions (delivery date, cost, specifications)

[0353] Step 8: Monitoring the progress of real generation

[0354] The server monitors the progress of real-time creators and receives progress reports. It also simultaneously monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives regular progress reports from the creators and shares the results of the user's emotional analysis.

[0355] Input: Agreed working conditions

[0356] Output: Progress report, sentiment analysis results

[0357] Step 9: Recovery in case of problems

[0358] If a problem occurs, the server will immediately provide a recovery plan and give appropriate instructions to the real-time generator. For example, if there is a shortage of materials, it will suggest alternative materials or adjust delivery dates.

[0359] Input: Progress report, sentiment analysis results

[0360] Output: Recovery Plan

[0361] Step 10: Check and ship the finished product.

[0362] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user makes the final confirmation and reports any necessary corrections. After the user's final confirmation, the server arranges for the product to be delivered to the user as quickly as possible. Specifically, it arranges for a delivery company to ensure the product is delivered safely and promptly.

[0363] Input: Completed real product

[0364] Output: User's final confirmation, shipping arrangements

[0365] These processing steps enable users to quickly and cost-effectively translate their designs and ideas into high-quality original products, creating personalized experiences that take user emotions into consideration.

[0366] (Application Example 2)

[0367] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as a "server" and the smart device 14 as a "terminal".

[0368] This invention aims to solve the problem of providing high-quality, custom-made products that reflect the user's emotions in a short period of time and at low cost, particularly in the process of bringing original designs and ideas to life in collaboration with real-world creators (craftsmen). Conventional technologies lack a mechanism to adequately reflect the user's emotional feedback, and there is a lack of means to efficiently adjust the progress of real-world creation while monitoring it in real time. As a result, there have been problems in ensuring sufficient user satisfaction and quality for customized orders.

[0369] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, and emotion analysis means for analyzing the user's emotional state. This makes it possible to grasp the user's emotions in real time, generate a design drawing that reflects those emotions, and monitor the progress.

[0370] "Requested data" refers to information provided by users, including designs and ideas, and can take the form of text, images, videos, and other data.

[0371] "Analysis methods" refer to techniques for analyzing requested data and extracting important keywords, shapes, and patterns.

[0372] "Design drawing generation means" refers to a technology that creates specific design drawings based on analyzed data, generating instructions that include dimensions, materials, and processing methods.

[0373] "Sentiment analysis methods" refer to technologies that extract emotional data from user input and feedback and use it to generate and modify design blueprints.

[0374] "Verification methods" refer to technologies that present generated design drawings or finished products to users and receive their verification and feedback.

[0375] A "craftsman database" is a database that accumulates data about craftsmen, including information such as past achievements, areas of expertise, and reviews.

[0376] A "real-world creator" refers to a craftsman who works to bring a user's original design to life.

[0377] "Negotiation methods" refer to techniques for negotiating working conditions such as delivery date, cost, and specifications with selected real-world generators.

[0378] "Progress monitoring means" refers to technology that monitors the work progress of the real-time creator in real time and makes adjustments or recoveries as needed.

[0379] "Problem-solving methods" refer to technologies that identify problems that arise during real-world generation and provide appropriate solutions.

[0380] A "smartphone application interface" is an interface that allows users to input designs and feedback via their smartphones and check the progress of their work.

[0381] System Program Overview

[0382] The program for the system that implements this application example will be explained using the following method.

[0383] Overall flow

[0384] 1. Users upload designs and ideas as request data through a smartphone application interface.

[0385] 2. The server analyzes this request data and generates a design drawing.

[0386] 3. The server analyzes the user's emotional state using emotion analysis tools.

[0387] 4. Present the generated design to the user and receive their feedback.

[0388] 5. The server selects the most suitable real-world generator from the craftsman database and negotiates terms.

[0389] 6. The real-world creator begins work and monitors its progress.

[0390] 7. In the event of a problem, the server will provide a timely recovery solution.

[0391] 8. After the work is completed, present the finished product to the user and obtain their confirmation.

[0392] 9. After final confirmation, the finished product is delivered to the user.

[0393] Hardware and software to be used

[0394] Hardware: Smartphones, business servers, emotion measurement devices (cameras and voice recognition devices)

[0395] Software: Python-based emotion recognition engine (emotion_engine), design AI (design_ai), job handling AI (job_handling_ai)

[0396] Examples of data processing and calculations

[0397] Users upload their designs and ideas as text, images, or videos using a smartphone application interface. This data is analyzed on a server; text data is analyzed using natural language processing to extract key keywords, and image and video data is analyzed using image recognition technology to determine shape, color, and pattern. A design AI generates detailed blueprints including dimensions, materials, and manufacturing methods, and these blueprints are then fed back to the user based on sentiment analysis results from an emotion engine.

[0398] Specific example

[0399] scenario

[0400] Users open the jewelry shop app and upload images and text descriptions of their jewelry designs. The server analyzes this data and uses design AI to generate detailed blueprints. Simultaneously, an emotion engine analyzes the emotions the user is feeling at the time of upload and reflects them in the blueprints. The generated blueprints are presented to the user, who is asked to review and provide feedback via the smartphone interface. Based on the feedback, the server makes necessary improvements and finalizes the blueprints.

[0401] Example of a prompt

[0402] "Please upload your original jewelry design. What materials and manufacturing methods would you prefer for that design? Also, please tell us how you feel about creating this jewelry."

[0403] This system allows users to efficiently request customizable, bespoke products utilizing sentiment analysis and receive high-quality finished products. Furthermore, real-world creators can understand the user's specific requests and emotions as they work, resulting in a mutually satisfying process.

[0404] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0405] Step 1:

[0406] Users upload their designs and ideas as request data in text, image, or video format using a smartphone application interface. This sends the request data from the user to the server. The input is the user's design data, and the output is the transmission of data to the server. Specifically, the user opens the application, clicks the "Upload New Design" button, selects a file, and uploads it.

[0407] Step 2:

[0408] The server receives the uploaded request data and performs analysis using natural language processing techniques for text data and image recognition techniques for image and video data. The input is design data submitted by the user, and the output is the analysis results (such as important keywords and shape patterns). Specifically, the analysis algorithm running on the server processes the data and passes the analysis results to the next step.

[0409] Step 3:

[0410] The server uses design AI based on the analysis results to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. The input is the analysis results, and the output is the detailed design drawing. Specifically, the design AI determines design parameters based on the analysis results, generates the design drawing, and saves it.

[0411] Step 4:

[0412] The server uses an emotion engine to analyze the user's emotional state and incorporate this into the design blueprint generation. Inputs are user request data and feedback data, while output is the emotion analysis result. Specifically, the emotion engine executes an emotion recognition algorithm, extracts the user's emotional data, and incorporates it into the blueprint.

[0413] Step 5:

[0414] The generated blueprint is presented to the user from the server, and they are asked to review it via a smartphone application. The input is the detailed blueprint, and the output is the user's feedback. Specifically, the blueprint is displayed in the application interface, and the user presses either the "Confirm" button or the "Request Revision" button.

[0415] Step 6:

[0416] Based on user feedback, the server modifies the design drawings as needed. The input is the user feedback, and the output is the modified design drawings. Specifically, the design AI generates and re-presents the design drawings based on the feedback.

[0417] Step 7:

[0418] The server selects the most suitable real-world craftsman from the craftsman database and negotiates terms. The input is a detailed blueprint, and the output is the work conditions agreed upon with the selected real-world craftsman. Specifically, the sourcing AI searches the database, selects the most suitable craftsman, and saves the results of the negotiation.

[0419] Step 8:

[0420] A selected real-time generator begins work, and the server monitors the progress in real time. Input is work start information, and output is progress data. Specifically, the generator sends progress reports to the server, which processes them.

[0421] Step 9:

[0422] If a problem occurs during the process, the server will analyze it and provide a timely recovery plan. The input is data about the problem that occurred, and the output is the recovery plan. Specifically, the server detects the problem, executes the recovery algorithm, and issues instructions.

[0423] Step 10:

[0424] After the work is completed, the finished product is presented to the user for confirmation. The input is information about the finished product, and the output is the user's final confirmation. Specifically, the server sends an image of the finished product to the user and requests confirmation.

[0425] Step 11:

[0426] After final confirmation, the server arranges for the finished product to be delivered to the user. Input is the user's delivery information, and output is a notification of delivery completion. Specifically, the server sends instructions to the delivery company and notifies the user of the delivery status.

[0427] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0428] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0429] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart device 14.

[0430] [Second Embodiment]

[0431] Figure 3 shows an example of the configuration of the data processing system 210 according to the second embodiment.

[0432] As shown in Figure 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0433] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0434] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication interface 44. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, and camera 42 are also connected to the bus 52.

[0435] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0436] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0437] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0438] Figure 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Figure 4, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0439] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0440] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0441] In the smart glasses 214, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0442] Next, the identification processing performed by the identification processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0443] This invention provides a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans). This system includes the following main functions:

[0444] 1. Data entry and analysis

[0445] Users upload their original designs and ideas to the system. The uploaded data may include text, images, or videos, and the server performs an initial analysis based on this data.

[0446] The server extracts important keywords from the input data and analyzes the shape, color, and pattern of image data. It extracts dimensions and requirements from text data and requests additional information from the user if needed.

[0447] 2. Design drawing generation

[0448] The server uses design AI to generate detailed blueprints based on the analyzed data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods, enabling the creation of perfect crafts.

[0449] The generated design drawings are presented to the user for review and feedback. This ensures that the design drawings meet the user's requirements in a final confirmation.

[0450] 3. Selection and negotiation of real-world creators

[0451] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. The selection criteria are diverse, including past performance, area of ​​expertise, and reviews.

[0452] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[0453] 4. Monitoring and management of real-world generation

[0454] The server monitors the progress of the real-world generation in real time. It checks the progress at each stage and verifies the quality based on reports from the craftsmen.

[0455] In the event of a problem, the server immediately provides a recovery plan and instructs the craftsman. This ensures that the work proceeds smoothly and that high-quality products are completed.

[0456] 5. Confirmation of finished product and delivery

[0457] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the product is promptly delivered to the user.

[0458] After receiving the provided product, users contribute to improving future services by providing feedback to the server.

[0459] Specific example

[0460] The following are specific examples.

[0461] 1. Examples of custom-made clothing

[0462] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. The design AI generates a detailed design drawing, which is then presented to the user for confirmation.

[0463] Subsequently, the AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. Progress is monitored in real time, and any problems that arise are addressed immediately. The finished garments are then sent to the user for final confirmation, and after confirmation, they are promptly delivered.

[0464] 2. Examples of custom-made furniture

[0465] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. The blueprints are then presented to the user, and feedback is incorporated.

[0466] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and takes countermeasures if problems arise. The completed furniture is then requested to be confirmed by the user, and after confirmation, it is delivered.

[0467] This allows users to obtain high-quality, original products at a low cost and in a short time. Furthermore, artisans gain the opportunity to connect with a new customer base.

[0468] The following describes the processing flow.

[0469] Step 1:

[0470] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded via their device.

[0471] Step 2:

[0472] The server receives the uploaded request data and begins analyzing it. For text data, it uses natural language processing techniques to extract important keywords and requirements. For image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[0473] Step 3:

[0474] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[0475] Step 4:

[0476] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback as needed. Based on this feedback, the blueprint is revised again.

[0477] Step 5:

[0478] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[0479] Step 6:

[0480] The server will contact the selected real-world generator and negotiate the terms regarding the work schedule and costs. If an agreement is reached, the work progress schedule will be finalized.

[0481] Step 7:

[0482] The server prompts the real-world generators to begin their actual work. It monitors the progress of the real-world generation in real time, checking the progress at each stage. Based on reports and status updates from the craftsmen, it checks whether the work is progressing appropriately.

[0483] Step 8:

[0484] If a problem occurs during real-time generation, the server will quickly provide a recovery plan and take appropriate action, such as providing additional resources or adjusting work procedures.

[0485] Step 9:

[0486] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then performs the final confirmation and provides feedback, reporting any necessary corrections.

[0487] Step 10:

[0488] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[0489] (Example 1)

[0490] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0491] In traditional custom-made product manufacturing processes, it was difficult to adequately reflect the specific requests of users, and coordination with manufacturers and management of progress were cumbersome. This increased the likelihood of delivery delays, reduced quality, and decreased user satisfaction.

[0492] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0493] In this invention, the server includes means for receiving and analyzing design requests and ideas from users; means for generating blueprints based on the analyzed designs and ideas; means for presenting the generated blueprints to the user and receiving feedback; means for selecting the most suitable manufacturer from a database; means for negotiating working conditions with the selected manufacturer; means for monitoring the progress of manufacturing in real time; means for providing a recovery plan to resolve problems that arise during manufacturing; means for presenting the completed product to the user and obtaining final confirmation; and means for delivering the completed product to the user after final confirmation. This enables the fulfillment of user requests with high accuracy and consistent management of the manufacturing process.

[0494] "Requested designs and ideas" refer to information about the design of products that users provide to the system, and include formats such as text, images, and videos.

[0495] A "server" is a computer system that performs a series of processes such as analysis, design drawing generation, manufacturer selection, progress monitoring, and quality checks.

[0496] "Means of analysis" refers to software and algorithms used to extract and understand information about designs and ideas provided by users.

[0497] "Means for generating design drawings" refers to software or tools used to create specific product design drawings based on analyzed designs and ideas.

[0498] "Means of receiving feedback" refers to an interface that presents the generated blueprint to the user and collects additional requests and suggestions from the user.

[0499] "Means of selecting a manufacturer" refers to the procedures or algorithms used to select the most suitable manufacturer from a database.

[0500] "Means of negotiating working conditions" refers to means of contacting selected manufacturers and discussing matters such as delivery dates, costs, and specifications.

[0501] "Means of monitoring progress in real time" refers to software and tools used to track the manufacturing process in real time and monitor progress and problems.

[0502] "Means of providing a recovery plan" refers to software or procedures that provide quick solutions when problems occur during the manufacturing process.

[0503] "A means of presenting the finished product and obtaining final confirmation" refers to an interface used to present the completed product to the user and obtain their final approval.

[0504] "Delivery method" refers to the procedures and systems for quickly delivering finished products to the user's specified location.

[0505] This invention provides a comprehensive system for analyzing users' original designs and ideas, generating blueprints, and collaborating with manufacturers (craftsmen) to create concrete products. This system includes means for exchanging data between users, servers, and manufacturers to achieve an optimal manufacturing process.

[0506] Data entry and upload

[0507] Users upload their designs and ideas to the system. These designs and ideas are provided in various formats, including text, images, and videos. Users enter the data using a dedicated web form or smartphone app and click the upload button.

[0508] Data analysis and extraction

[0509] The server analyzes uploaded designs and ideas. For image data, it uses image analysis libraries such as OpenCV and TensorFlow to analyze shape, color, and pattern. For text data, it uses NLTK and spaCy to extract dimensions, materials, and other requirements. If necessary, the server will request additional information from the user.

[0510] Design drawing generation and verification

[0511] The server generates design drawings using design AI based on the analyzed designs and ideas. The design AI used includes the AutoCAD API and SolidWorks API. The generated design drawings include detailed information such as dimensions, materials, and manufacturing methods. The server then presents the design drawings to the user and receives feedback.

[0512] Selection and negotiation of manufacturers

[0513] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI employs a Python-based machine learning model. It selects manufacturers based on information such as past performance, areas of expertise, and reviews from a database. The server then contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications.

[0514] Manufacturing monitoring and control

[0515] The server uses cloud-based monitoring tools such as AWS CloudWatch to monitor the progress of manufacturing in real time. It receives progress reports from craftsmen and checks quality. If a problem occurs, it immediately provides a recovery plan and directs the corrective actions.

[0516] Confirmation of finished products and delivery

[0517] The server photographs the finished product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the server initiates procedures to quickly deliver the finished product to the user.

[0518] Specific example

[0519] The following are examples of custom-made products.

[0520] Examples of custom-made clothing

[0521] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. For example, the prompt might look like this:

[0522] Please generate detailed blueprints for the following design:

[0523] Item: Dress

[0524] Color: Red

[0525] Waist size: 70cm

[0526] Material: Silk

[0527] The server analyzes the data, extracts the necessary information, and then uses design AI to generate detailed blueprints, which are presented to the user. After user confirmation, the procurement AI selects the most suitable manufacturer, negotiates terms, and then begins work. Progress is monitored in real time, and the completed garments are sent to the user for final confirmation before being promptly delivered.

[0528] Examples of custom-made furniture

[0529] Users upload sketches, dimensions, and material details of their original furniture designs. For example, the prompt might look like this:

[0530] Please generate detailed blueprints for the following design:

[0531] Item: Table

[0532] Material: wood

[0533] Height: 75cm

[0534] Width: 120cm

[0535] Color: Brown

[0536] The server analyzes the data, generates blueprints, presents them to the user, and receives feedback. The procurement AI selects the most suitable manufacturer, negotiates terms, and then requests the manufacturer to perform the work. Progress is monitored in real time, and countermeasures are taken if problems arise. The completed furniture is asked to be confirmed by the user, and after confirmation, it is delivered.

[0537] This system allows users to quickly obtain high-quality, custom-made products, while also providing manufacturers with access to a new customer base.

[0538] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0539] Step 1: Data entry and upload

[0540] Users upload designs and ideas to the system using a dedicated web form or smartphone app. The data entered by users can be in the form of text, images, or videos, providing the system with user requests. Examples of input data include "red dress," "silk material," and "waist size 70cm." Users click the "Upload" button to submit the data. The server then receives this data.

[0541] Step 2: Data Analysis and Extraction

[0542] The server begins analyzing the data received from the user. Based on the input data, the server performs the following analyses:

[0543] For image data, we use OpenCV or TensorFlow to analyze shape, color, and pattern.

[0544] For text data, use NLTK or spaCy to extract dimensions, materials, and other requirements.

[0545] For example, features such as "shape: dress," "color: red," and "pattern: solid" are extracted from the image data. The server retrieves the necessary information as analysis results and notifies the user if additional information is required.

[0546] Step 3: Design drawing generation and verification

[0547] The server uses design AI to generate detailed design drawings based on the analyzed data. The design AI used is either the AutoCAD API or the SolidWorks API. It receives analysis results (dimensions, materials, design features, etc.) as input, and the design AI generates detailed design drawings based on this. The design drawings include details such as "length and width dimensions," "materials used," and "manufacturing methods."

[0548] The generated blueprints are presented to the user via the server. The user reviews the blueprints and provides feedback as needed. For example, they might submit requests such as "I want to widen the shoulders a little" or "I want to change the fabric color." The server receives this feedback and modifies the blueprints.

[0549] Step 4: Selecting and negotiating with manufacturers

[0550] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI uses a Python-based machine learning model. Based on user and product requirements data (past performance, expertise, reviews, etc.), it selects the most suitable manufacturer. The server contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications. For example, it might negotiate conditions such as "2 weeks for production, cost of 50,000 yen, and use of silk fabric."

[0551] Step 5: Monitoring and Controlling Production

[0552] The server monitors the manufacturing progress in real time. It uses cloud-based monitoring tools such as AWS CloudWatch to receive progress reports from manufacturers. For example, it retrieves progress information such as "30% sewn" or "dyeing process started." The server manages this progress information and checks quality. If a problem occurs, the server provides a recovery plan and gives appropriate instructions to the manufacturer.

[0553] Step 6: Check and ship the finished product.

[0554] The server photographs the finished product and sends the image to the user. The user reviews the image and makes a final confirmation, for example, by giving approval such as "It's just as pictured, no problems." After the final confirmation, the server proceeds with the shipping process. The server interacts with the shipping company's API to determine the shipping schedule for the user's address. For example, it generates information such as "Scheduled delivery date: October 15th" and "Shipping company: XX Transportation."

[0555] Through the steps outlined above, this system can realize users' original designs with high precision and deliver products efficiently and quickly.

[0556] (Application Example 1)

[0557] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0558] When bringing user-designed original products to life, challenges exist, such as inadequate communication of detailed design instructions, inappropriate selection of real-world product generators, insufficient monitoring of progress, and slow problem resolution. Furthermore, as the diversity and accuracy of user-uploaded design data increase, there is a need for a system that can smoothly analyze this data and generate prompts.

[0559] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0560] In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, means for presenting the generated design drawing to the user and obtaining their confirmation, means for selecting the most suitable real-world generator from a database of skilled craftsmen, means for negotiating terms with the selected real-world generator, means for monitoring the progress of real-world generation in real time, means for resolving problems that arise during real-world generation, means for presenting the completed real-world product to the user and obtaining their confirmation, means for delivering the finished product to the user, means for extracting dimensions and important information from the user's design images and text, and means for generating prompt sentences based on the generated design drawing and inputting them into a generation AI model to obtain more detailed instructions. This makes it possible to analyze the user's design in detail, select the most suitable real-world generator, and quickly monitor the progress and resolve problems in real time. Furthermore, by using a generation AI model, the user's design can be further optimized and specific instructions can be obtained quickly.

[0561] "Requested data" refers to digital data in which a user expresses their original design or idea, and includes images, text, or video formats.

[0562] A "design drawing" is a detailed drawing generated from the analyzed request data, containing specific instructions including the dimensions, materials, and manufacturing methods of the product desired by the user.

[0563] A "user" is an individual or organization that uses this system to request a custom-made product.

[0564] The "Craftsman Database" is a database containing information on a variety of craftsmen, including each craftsman's area of ​​expertise, achievements, reviews, and other relevant information.

[0565] A "real-world creator" is a craftsman or artisan who produces a concrete product based on a design requested by a user.

[0566] "Negotiating terms" refers to the process of agreeing on terms such as delivery date, cost, and specifications with the real-world creator regarding the request.

[0567] "Real-world generation" refers to the process of creating real-world products based on user request data.

[0568] "Real-time monitoring of progress" is a process of checking the progress of work at each stage of real-time generation in real time and giving instructions or making corrections as needed.

[0569] A "prompt message" is a phrase used as input for the AI ​​generation model, and it is generated by analyzing the user's design.

[0570] A "generative AI model" is a model that uses artificial intelligence to analyze a design and generate more detailed instructions.

[0571] The means of extracting "dimensions and important information" refers to a function that extracts necessary information from uploaded design images and text.

[0572] "A means of inputting into a generative AI model to obtain more detailed instructions" refers to the process of inputting prompt sentences into a generative AI model based on the analysis results, and generating specific instructions.

[0573] The system for implementing this invention is built based on the following program.

[0574] First, users upload their original designs or ideas as request data using a smartphone or tablet application. This request data can include images, text, or videos.

[0575] Data Analysis

[0576] The server analyzes the uploaded request data and extracts important keywords, shapes, colors, patterns, dimensions, etc. Image data is converted to grayscale using OpenCV, thresholded, and then subjected to shape analysis. Text data is extracted using Pytesseract to obtain dimensions and important information.

[0577] Design drawing generation

[0578] The server generates detailed blueprints based on the extracted data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. Furthermore, prompt statements are generated based on these blueprints and input into the generating AI model to obtain even more detailed instructions.

[0579] Selection of craftsmen and negotiation of terms

[0580] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications, and then finalizes the work schedule once an agreement is reached.

[0581] Monitoring and management of real-world generation

[0582] The server monitors the progress of real-time generation, checking the craftsman's work process. If a problem occurs, the server immediately provides a recovery plan and instructs the craftsman. This ensures that work proceeds smoothly and high-quality products are completed.

[0583] Confirmation of finished products and delivery

[0584] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving confirmation from the user, the product is promptly shipped. After receiving the product, the user provides feedback to the server, contributing to improvements in future services.

[0585] Hardware and software to be used

[0586] Hardware: Smartphones, tablets, servers

[0587] Software: Python 3, OpenCV, Pytesseract, Keras

[0588] Specific example

[0589] 1. Uploading the design: The user uploads an image of their original costume design named "example_design.jpg" to the application.

[0590] 2. Data Analysis: The server analyzes the uploaded design images and uses OpenCV to extract important information such as shape and dimensions. If text data is included, Pytesseract is used to extract the text and obtain dimensions and other important information.

[0591] 3. Prompt message generation: Prompt messages are generated based on the extracted data and input into the generation AI model. This allows for more detailed instructions to be obtained.

[0592] Example of a prompt

[0593] "Please analyze the user-uploaded design (e.g., example_design.jpg) and extract information about its shape and dimensions."

[0594] This allows for detailed analysis of the user's design, selection of the most suitable craftsman, real-time monitoring of progress, and rapid problem resolution. Furthermore, by using generative AI models, the user's design can be further optimized, and specific instructions can be obtained quickly.

[0595] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[0596] Step 1:

[0597] Users upload designs.

[0598] Users upload their own designed images, text, or video request data using a smartphone or tablet application.

[0599] Input: Image file (e.g., example_design.jpg), text file

[0600] Output: Design data saved on the server

[0601] Step 2:

[0602] Analysis of requested data

[0603] The server receives the uploaded request data and begins analysis. For image data, it uses OpenCV to convert it to grayscale and recognize its shape and color. For text data, it uses Pytesseract to extract the text.

[0604] Input: Design data stored on the server

[0605] Output: Analytical data such as extracted shape information, color information, and dimensions.

[0606] Step 3:

[0607] Design drawing generation

[0608] The server generates a detailed blueprint based on the analyzed data. This blueprint includes dimensions, materials, and manufacturing methods. It also generates prompt statements based on the generated blueprint and inputs them into the AI ​​model to obtain more specific instructions.

[0609] Input: Analysis data

[0610] Output: Detailed blueprint, generated prompts

[0611] Step 4:

[0612] Selection and negotiation of terms

[0613] The server uses a procurement AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications.

[0614] Input: Detailed blueprints, craftsman database

[0615] Output: Selected real generators, agreed conditions

[0616] Step 5:

[0617] Monitoring and management of real-world generation

[0618] The server monitors the progress of the real-time generation process. During this process, it checks the craftsman's work process and immediately provides a recovery plan if any problems occur.

[0619] Input: Real-time generation progress data

[0620] Output: Progress report, recovery plan

[0621] Step 6:

[0622] Confirmation of finished products and delivery

[0623] The server photographs the completed physical product and presents the image to the user for final confirmation. After confirmation, the user requests delivery of the product to the store or their home.

[0624] Input: Image of the finished product

[0625] Output: User's final confirmation and delivery instructions

[0626] Examples of specific actions

[0627] 1. Upload your design:

[0628] The user takes a photo of the clothing design "example_design.jpg" with their smartphone and uploads the image using the application.

[0629] 2. Data Analysis:

[0630] The server converts the uploaded "example_design.jpg" to grayscale using OpenCV, applies thresholding, and analyzes its shape and color information. If text is present, Pytesseract is used to extract dimensions and specifications as text.

[0631] 3. Generating blueprints:

[0632] The server inputs prompt messages into the generated AI model based on the analysis data, receiving instructions such as, "Please describe the specific dimensions and manufacturing method of this design." Based on these instructions, a detailed design drawing is created.

[0633] 4. Selection and Negotiation of Terms:

[0634] The server uses an AI-powered scheduling system to search a database of artisans, select the most suitable one, and contact them. For example, it might find an artisan who has previously produced similar garments and reconfirm the delivery time and cost requirements.

[0635] 5. Monitoring and management of real-world generation:

[0636] The server instructs the craftsman to report progress in real time, and progress reports are received as they occur. If a problem arises, alternative solutions are provided promptly.

[0637] 6. Confirmation of finished product and delivery:

[0638] The craftsman photographs the finished garment and sends the image to the server. The server then presents the image to the user and requests final confirmation. After the user confirms, the shipping process begins.

[0639] These steps allow users to efficiently bring their original designs to life and receive high-quality products.

[0640] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[0641] This invention combines a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans), with an emotion engine that recognizes user emotions, to further enhance the user experience. This system includes the following key functions:

[0642] 1. Data entry and analysis

[0643] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded from the user's device.

[0644] The server receives the uploaded request data and begins analysis. For text data, natural language processing techniques are used to extract important keywords and requirements, while for image and video data, image recognition techniques are used to analyze shapes, colors, patterns, etc.

[0645] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[0646] 2. User sentiment analysis using an emotion engine

[0647] When analyzing request data, the server uses an emotion engine to analyze the user's emotions based on user input and feedback. This emotion data is used to generate blueprints and optimize feedback.

[0648] 3. Design drawing generation and feedback

[0649] The server presents the generated blueprint to the user and receives feedback while monitoring the user's emotional state. If the user expresses a specific emotion (e.g., satisfaction, dissatisfaction) towards the blueprint, the server modifies the blueprint accordingly.

[0650] 4. Selection and negotiation of real-world creators

[0651] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[0652] The server contacts the selected real-world generator, negotiates working conditions (delivery date, cost, specifications), and confirms the work schedule once an agreement is reached.

[0653] 5. Monitoring and Management of Real-Generation

[0654] The server monitors the progress of the real-time creators in real time and receives progress reports. It also monitors the users' emotional states and provides emotional data to the real-time creators as needed to optimize their work.

[0655] In the event of a problem, the server immediately provides a recovery plan and gives appropriate instructions to the real-world creators. This ensures that work proceeds smoothly and high-quality products are produced.

[0656] 6. Confirmation of finished products and delivery

[0657] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then makes a final confirmation and reports any necessary corrections.

[0658] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[0659] Specific example

[0660] 1. Examples of custom-made clothing

[0661] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. A design AI generates a detailed blueprint, presents it to the user while analyzing their emotions, and obtains their confirmation.

[0662] The AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. It monitors the progress in real time and communicates appropriately based on the user's feelings. The user is asked to give final confirmation of the finished garment, and after confirmation, it is promptly delivered.

[0663] 2. Examples of custom-made furniture

[0664] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. It presents the blueprints while analyzing the user's emotions and incorporates their feedback.

[0665] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and resolves issues based on the user's emotions. The completed furniture is then requested to be confirmed by the user, and after confirmation, delivery is arranged.

[0666] This allows users to obtain high-quality, original products at low cost and in a short timeframe, while also providing a personalized experience powered by an emotional engine. Furthermore, artisans gain the opportunity to connect with a new customer base.

[0667] The following describes the processing flow.

[0668] Step 1:

[0669] Users upload their original designs and ideas to the system in text, image, and video formats. This data is then sent from the user's device to the server.

[0670] Step 2:

[0671] The server receives the uploaded request data and begins initial analysis. For text data, it uses natural language processing techniques to extract important keywords and requirements, while for image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[0672] Step 3:

[0673] The server uses an emotion engine to analyze user emotions from their input and feedback. Emotional data is extracted based on information obtained from the user's text, voice, images, and videos.

[0674] Step 4:

[0675] The server uses design AI based on the analysis results to generate detailed blueprints. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. The design AI also takes user sentiment data into consideration to generate blueprints that enhance user satisfaction.

[0676] Step 5:

[0677] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback. At this stage, the emotion engine analyzes the user's emotional state in real time, reading emotions from the user's facial expressions and tone of voice.

[0678] Step 6:

[0679] The server revises its design based on user feedback and asks the user for final confirmation. This process can be repeated until the user is satisfied.

[0680] Step 7:

[0681] The server uses a scheduling AI to select the most suitable real-world creator from a database of artisans. Selection criteria include past performance, area of ​​expertise, reviews, and user sentiment data.

[0682] Step 8:

[0683] The server contacts the selected real-world generator and negotiates the terms of work (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[0684] Step 9:

[0685] The server monitors the progress of the real-time generation process and receives progress reports. The emotion engine also monitors the user's emotional state in parallel and provides emotional data to the real-time generator as needed. This personalizes the work process and improves quality.

[0686] Step 10:

[0687] If a problem occurs during real-time generation, the server will immediately provide a recovery plan and give appropriate instructions to the craftsmen. For example, this may include providing additional resources or adjusting work procedures.

[0688] Step 11:

[0689] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The emotion engine also analyzes the user's emotional state at the time of final confirmation and makes final adjustments if necessary.

[0690] Step 12:

[0691] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[0692] Step 13:

[0693] After receiving the product, users contribute to improving future services by providing feedback to the server. The emotion engine extracts emotion data from this feedback and uses it to improve the entire system.

[0694] Through the above processing steps, we can provide users with highly satisfying original products, and craftsmen can also gain opportunities to connect with a new customer base.

[0695] (Example 2)

[0696] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the smart glasses 214 will be referred to as the "terminal".

[0697] Traditional systems for translating original designs into actual products were time-consuming and costly, and struggled to accurately reflect user intentions and emotions. Furthermore, the process of negotiating terms with craftsmen, monitoring the progress of the real-world product creation, and resolving problems was cumbersome, resulting in a poor user experience.

[0698] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for analyzing the request data using natural language processing or image recognition technology according to the format of the request data, means for generating a detailed design drawing using design AI based on the analyzed data, means for analyzing the user's emotional state and modifying the design drawing based on feedback, means for selecting the optimal real-world producer from a craftsman database using arrangement AI, means for negotiating terms with the selected real-world producer, means for monitoring the progress of real-world production in real time and providing emotional data as needed, means for immediately providing a recovery plan for problems that occur during real-world production, means for presenting the completed real-world product to the user and obtaining confirmation, and means for delivering the finished product to the user. This enables the rapid, low-cost, and high-quality realization of the user's designs and ideas into original products, and allows for a personalized experience that takes the user's emotions into consideration.

[0699] "Request data" refers to information provided by the user to the system, including original designs and ideas, and can be in text, image, or video format.

[0700] "Natural language processing" is a technique for extracting important keywords and requirements from text data, and involves using models such as BERT and GPT-3.

[0701] "Image recognition technology" is a technique that analyzes features such as color, shape, and pattern from image and video data, and involves using software such as the Google Cloud Vision API.

[0702] "Design AI" refers to an artificial intelligence system that automatically generates detailed design drawings based on user request data, and includes tools such as AutoCAD and Blender.

[0703] An "emotion engine" is a system that analyzes emotions from user input and feedback, using technologies such as IBM Watson Tone Analyzer.

[0704] "Arrangement AI" is an artificial intelligence system that selects the most suitable real-world creator from a database of skilled craftsmen.

[0705] A "real-world creator" is a craftsman or business that actually produces goods based on a user's request.

[0706] A "recovery plan" is a set of countermeasures to solve problems that occur during real-time generation.

[0707] This invention enhances the user experience by combining a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans) with an emotion engine that recognizes the user's emotions. This system includes the following main functions:

[0708] Data entry and analysis

[0709] Users upload their original designs and ideas to the system in text, image, and video formats. Data is sent from the user's device to the server. Specifically, a user uploads a sketch of a garment they designed from their device and sends the data by pressing the "Upload" button.

[0710] The server receives the uploaded request data and begins analysis. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image and video data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shapes, colors, patterns, etc. For example, it sends a sketch image of clothing to the Google Cloud Vision API to recognize its color code and shape.

[0711] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. Specifically, Blender is used to generate a 3D model of the clothing, and then dimension and material information is attached based on that model.

[0712] User sentiment analysis using an emotion engine

[0713] The server analyzes user emotions using an emotion engine (e.g., IBM Watson Tone Analyzer) when analyzing user input and feedback. This emotion data is used to generate blueprints and optimize feedback. When users provide text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, that emotion is analyzed.

[0714] Design drawing generation and feedback

[0715] The server presents the generated blueprint to the user and accepts feedback based on their emotional state. For example, if a user submits feedback such as "I want more red in this design," the server uses IBM Watson Tone Analyzer to analyze the emotion and revise the blueprint.

[0716] Selection of real-world creators and negotiation of terms

[0717] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. Specifically, the server queries the craftsman database and requests the appropriate craftsman.

[0718] The server contacts the selected real-world generators and negotiates working conditions (delivery date, cost, specifications). For example, it contacts craftsmen via email to reach an agreement on delivery date and cost.

[0719] Monitoring and management of real-world generation

[0720] The server monitors the progress of real-time creators and receives progress reports. It also monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives progress images from the creators periodically and shares the results of the user's emotional analysis with them.

[0721] The server will immediately provide a recovery plan and give appropriate instructions to the real-world generator in the event of a problem. For example, it will suggest alternative materials if a material shortage occurs.

[0722] Confirmation of finished products and delivery

[0723] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user then reviews the product and reports any necessary corrections. For example, the user might send an image of the finished garment and obtain final approval.

[0724] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[0725] Specific example

[0726] Examples of custom-made clothing

[0727] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their designed clothing to the system. The server analyzes this data and extracts the necessary information. A design AI generates detailed blueprints, presents them to the user while analyzing their emotions, and obtains their confirmation. A procurement AI selects the most suitable craftsman, negotiates terms, and begins work. The system monitors the progress in real time and communicates appropriately based on the user's emotions. The completed clothing is then asked for final confirmation from the user, and after confirmation, it is promptly delivered.

[0728] Example of a prompt

[0729] "I've uploaded an original sketch of a garment I designed. The material is cotton, the color is red, and the size is M. Please create a detailed blueprint for this design, analyze my feelings, and provide feedback."

[0730] The flow of the specific processing in Example 2 will be explained using Figure 13.

[0731] Step 1: Data Entry

[0732] Users can upload their original designs and ideas to the system in image, text, and video formats. Generally, users send data to the server using their device. When the "Upload" button is pressed, the selected design data is sent. Specifically, users can upload sketches of clothing or detailed product specifications (material, color, size, etc.) from their device.

[0733] Input: Original designs and ideas from users (images, text, videos)

[0734] Output: Design data sent to the server

[0735] Step 2: Data Analysis

[0736] The server receives the uploaded design data and analyzes it according to its format. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shape, color, and patterns. For video data, it performs image recognition frame by frame.

[0737] Input: Design data sent to the server

[0738] Output: Analysis results (important keywords, shapes, colors, patterns, etc.)

[0739] Step 3: Generate blueprints

[0740] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings based on the analyzed data. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. For example, Blender is used to generate a 3D model of clothing, and then dimensions and material information is added based on that model.

[0741] Input: Analysis results (important keywords, shapes, colors, patterns, etc.)

[0742] Output: Detailed design drawings (dimensions, materials, manufacturing methods, etc.)

[0743] Step 4: Sentiment Analysis

[0744] The server uses an emotion engine (for example, IBM Watson Tone Analyzer) to analyze emotions based on user input data and feedback. When a user provides text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, the server analyzes that emotion.

[0745] Input: User feedback (text format)

[0746] Output: Emotion analysis results

[0747] Step 5: Feedback and design revisions

[0748] The server presents the generated blueprint to the user and accepts feedback based on the sentiment analysis results. It then modifies the blueprint based on the user's feedback. For example, if a user submits a comment such as, "I want more red in this design," the server analyzes that feedback and adjusts the blueprint again.

[0749] Input: Sentiment analysis results, user feedback

[0750] Output: Revised blueprint

[0751] Step 6: Selecting a Real Generator

[0752] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. For example, the server queries the craftsman database and lists the most suitable craftsmen.

[0753] Input: Modified blueprint

[0754] Output: Selected real generators

[0755] Step 7: Negotiating Terms

[0756] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). Specifically, negotiations are conducted via email and telephone until an agreement is reached.

[0757] Input: Selected real generator

[0758] Output: Agreed working conditions (delivery date, cost, specifications)

[0759] Step 8: Monitoring the progress of real generation

[0760] The server monitors the progress of real-time creators and receives progress reports. It also simultaneously monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives regular progress reports from the creators and shares the results of the user's emotional analysis.

[0761] Input: Agreed working conditions

[0762] Output: Progress report, sentiment analysis results

[0763] Step 9: Recovery in case of problems

[0764] If a problem occurs, the server will immediately provide a recovery plan and give appropriate instructions to the real-time generator. For example, if there is a shortage of materials, it will suggest alternative materials or adjust delivery dates.

[0765] Input: Progress report, sentiment analysis results

[0766] Output: Recovery Plan

[0767] Step 10: Check and ship the finished product.

[0768] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user makes the final confirmation and reports any necessary corrections. After the user's final confirmation, the server arranges for the product to be delivered to the user as quickly as possible. Specifically, it arranges for a delivery company to ensure the product is delivered safely and promptly.

[0769] Input: Completed real product

[0770] Output: User's final confirmation, shipping arrangements

[0771] These processing steps enable users to quickly and cost-effectively translate their designs and ideas into high-quality original products, creating personalized experiences that take user emotions into consideration.

[0772] (Application Example 2)

[0773] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the smart glasses 214 will be referred to as the "terminal."

[0774] This invention aims to solve the problem of providing high-quality, custom-made products that reflect the user's emotions in a short period of time and at low cost, particularly in the process of bringing original designs and ideas to life in collaboration with real-world creators (craftsmen). Conventional technologies lack a mechanism to adequately reflect the user's emotional feedback, and there is a lack of means to efficiently adjust the progress of real-world creation while monitoring it in real time. As a result, there have been problems in ensuring sufficient user satisfaction and quality for customized orders.

[0775] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, and emotion analysis means for analyzing the user's emotional state. This makes it possible to grasp the user's emotions in real time, generate a design drawing that reflects those emotions, and monitor the progress.

[0776] "Requested data" refers to information provided by users, including designs and ideas, and can take the form of text, images, videos, and other data.

[0777] "Analysis methods" refer to techniques for analyzing requested data and extracting important keywords, shapes, and patterns.

[0778] "Design drawing generation means" refers to a technology that creates specific design drawings based on analyzed data, generating instructions that include dimensions, materials, and processing methods.

[0779] "Sentiment analysis methods" refer to technologies that extract emotional data from user input and feedback and use it to generate and modify design blueprints.

[0780] "Verification methods" refer to technologies that present generated design drawings or finished products to users and receive their verification and feedback.

[0781] A "craftsman database" is a database that accumulates data about craftsmen, including information such as past achievements, areas of expertise, and reviews.

[0782] A "real-world creator" refers to a craftsman who works to bring a user's original design to life.

[0783] "Negotiation methods" refer to techniques for negotiating working conditions such as delivery date, cost, and specifications with selected real-world generators.

[0784] "Progress monitoring means" refers to technology that monitors the work progress of the real-time creator in real time and makes adjustments or recoveries as needed.

[0785] "Problem-solving methods" refer to technologies that identify problems that arise during real-world generation and provide appropriate solutions.

[0786] A "smartphone application interface" is an interface that allows users to input designs and feedback via their smartphones and check the progress of their work.

[0787] System Program Overview

[0788] The program for the system that implements this application example will be explained using the following method.

[0789] Overall flow

[0790] 1. Users upload designs and ideas as request data through a smartphone application interface.

[0791] 2. The server analyzes this request data and generates a design drawing.

[0792] 3. The server analyzes the user's emotional state using emotion analysis tools.

[0793] 4. Present the generated design to the user and receive their feedback.

[0794] 5. The server selects the most suitable real-world generator from the craftsman database and negotiates terms.

[0795] 6. The real-world creator begins work and monitors its progress.

[0796] 7. In the event of a problem, the server will provide a timely recovery solution.

[0797] 8. After the work is completed, present the finished product to the user and obtain their confirmation.

[0798] 9. After final confirmation, the finished product is delivered to the user.

[0799] Hardware and software to be used

[0800] Hardware: Smartphones, business servers, emotion measurement devices (cameras and voice recognition devices)

[0801] Software: Python-based emotion recognition engine (emotion_engine), design AI (design_ai), job handling AI (job_handling_ai)

[0802] Examples of data processing and calculations

[0803] Users upload their designs and ideas as text, images, or videos using a smartphone application interface. This data is analyzed on a server; text data is analyzed using natural language processing to extract key keywords, and image and video data is analyzed using image recognition technology to determine shape, color, and pattern. A design AI generates detailed blueprints including dimensions, materials, and manufacturing methods, and these blueprints are then fed back to the user based on sentiment analysis results from an emotion engine.

[0804] Specific example

[0805] scenario

[0806] Users open the jewelry shop app and upload images and text descriptions of their jewelry designs. The server analyzes this data and uses design AI to generate detailed blueprints. Simultaneously, an emotion engine analyzes the emotions the user is feeling at the time of upload and reflects them in the blueprints. The generated blueprints are presented to the user, who is asked to review and provide feedback via the smartphone interface. Based on the feedback, the server makes necessary improvements and finalizes the blueprints.

[0807] Example of a prompt

[0808] "Please upload your original jewelry design. What materials and manufacturing methods would you prefer for that design? Also, please tell us how you feel about creating this jewelry."

[0809] This system allows users to efficiently request customizable, bespoke products utilizing sentiment analysis and receive high-quality finished products. Furthermore, real-world creators can understand the user's specific requests and emotions as they work, resulting in a mutually satisfying process.

[0810] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[0811] Step 1:

[0812] Users upload their designs and ideas as request data in text, image, or video format using a smartphone application interface. This sends the request data from the user to the server. The input is the user's design data, and the output is the transmission of data to the server. Specifically, the user opens the application, clicks the "Upload New Design" button, selects a file, and uploads it.

[0813] Step 2:

[0814] The server receives the uploaded request data and performs analysis using natural language processing techniques for text data and image recognition techniques for image and video data. The input is design data submitted by the user, and the output is the analysis results (such as important keywords and shape patterns). Specifically, the analysis algorithm running on the server processes the data and passes the analysis results to the next step.

[0815] Step 3:

[0816] The server uses design AI based on the analysis results to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. The input is the analysis results, and the output is the detailed design drawing. Specifically, the design AI determines design parameters based on the analysis results, generates the design drawing, and saves it.

[0817] Step 4:

[0818] The server uses an emotion engine to analyze the user's emotional state and incorporate this into the design blueprint generation. Inputs are user request data and feedback data, while output is the emotion analysis result. Specifically, the emotion engine executes an emotion recognition algorithm, extracts the user's emotional data, and incorporates it into the blueprint.

[0819] Step 5:

[0820] The generated blueprint is presented to the user from the server, and they are asked to review it via a smartphone application. The input is the detailed blueprint, and the output is the user's feedback. Specifically, the blueprint is displayed in the application interface, and the user presses either the "Confirm" button or the "Request Revision" button.

[0821] Step 6:

[0822] Based on user feedback, the server modifies the design drawings as needed. The input is the user feedback, and the output is the modified design drawings. Specifically, the design AI generates and re-presents the design drawings based on the feedback.

[0823] Step 7:

[0824] The server selects the most suitable real-world craftsman from the craftsman database and negotiates terms. The input is a detailed blueprint, and the output is the work conditions agreed upon with the selected real-world craftsman. Specifically, the sourcing AI searches the database, selects the most suitable craftsman, and saves the results of the negotiation.

[0825] Step 8:

[0826] A selected real-time generator begins work, and the server monitors the progress in real time. Input is work start information, and output is progress data. Specifically, the generator sends progress reports to the server, which processes them.

[0827] Step 9:

[0828] If a problem occurs during the process, the server will analyze it and provide a timely recovery plan. The input is data about the problem that occurred, and the output is the recovery plan. Specifically, the server detects the problem, executes the recovery algorithm, and issues instructions.

[0829] Step 10:

[0830] After the work is completed, the finished product is presented to the user for confirmation. The input is information about the finished product, and the output is the user's final confirmation. Specifically, the server sends an image of the finished product to the user and requests confirmation.

[0831] Step 11:

[0832] After final confirmation, the server arranges for the finished product to be delivered to the user. Input is the user's delivery information, and output is a notification of delivery completion. Specifically, the server sends instructions to the delivery company and notifies the user of the delivery status.

[0833] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[0834] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0835] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the smart glasses 214.

[0836] [Third Embodiment]

[0837] Figure 5 shows an example of the configuration of the data processing system 310 according to the third embodiment.

[0838] As shown in Figure 5, the data processing system 310 includes a data processing device 12 and a headset terminal 314. An example of the data processing device 12 is a server.

[0839] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0840] The headset terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a display 343. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and display 343 are also connected to the bus 52.

[0841] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[0842] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[0843] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[0844] Figure 6 shows an example of the main functions of the data processing device 12 and the headset terminal 314. As shown in Figure 6, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[0845] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0846] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[0847] In the headset terminal 314, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[0848] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the headset terminal 314 will be referred to as the "terminal".

[0849] This invention provides a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans). This system includes the following main functions:

[0850] 1. Data entry and analysis

[0851] Users upload their original designs and ideas to the system. The uploaded data may include text, images, or videos, and the server performs an initial analysis based on this data.

[0852] The server extracts important keywords from the input data and analyzes the shape, color, and pattern of image data. It extracts dimensions and requirements from text data and requests additional information from the user if needed.

[0853] 2. Design drawing generation

[0854] The server uses design AI to generate detailed blueprints based on the analyzed data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods, enabling the creation of perfect crafts.

[0855] The generated design drawings are presented to the user for review and feedback. This ensures that the design drawings meet the user's requirements in a final confirmation.

[0856] 3. Selection and negotiation of real-world creators

[0857] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. The selection criteria are diverse, including past performance, area of ​​expertise, and reviews.

[0858] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[0859] 4. Monitoring and management of real-world generation

[0860] The server monitors the progress of the real-world generation in real time. It checks the progress at each stage and verifies the quality based on reports from the craftsmen.

[0861] In the event of a problem, the server immediately provides a recovery plan and instructs the craftsman. This ensures that the work proceeds smoothly and that high-quality products are completed.

[0862] 5. Confirmation of finished product and delivery

[0863] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the product is promptly delivered to the user.

[0864] After receiving the provided product, users contribute to improving future services by providing feedback to the server.

[0865] Specific example

[0866] The following are specific examples.

[0867] 1. Examples of custom-made clothing

[0868] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. The design AI generates a detailed design drawing, which is then presented to the user for confirmation.

[0869] Subsequently, the AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. Progress is monitored in real time, and any problems that arise are addressed immediately. The finished garments are then sent to the user for final confirmation, and after confirmation, they are promptly delivered.

[0870] 2. Examples of custom-made furniture

[0871] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. The blueprints are then presented to the user, and feedback is incorporated.

[0872] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and takes countermeasures if problems arise. The completed furniture is then requested to be confirmed by the user, and after confirmation, it is delivered.

[0873] This allows users to obtain high-quality, original products at a low cost and in a short time. Furthermore, artisans gain the opportunity to connect with a new customer base.

[0874] The following describes the processing flow.

[0875] Step 1:

[0876] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded via their device.

[0877] Step 2:

[0878] The server receives the uploaded request data and begins analyzing it. For text data, it uses natural language processing techniques to extract important keywords and requirements. For image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[0879] Step 3:

[0880] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[0881] Step 4:

[0882] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback as needed. Based on this feedback, the blueprint is revised again.

[0883] Step 5:

[0884] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[0885] Step 6:

[0886] The server will contact the selected real-world generator and negotiate the terms regarding the work schedule and costs. If an agreement is reached, the work progress schedule will be finalized.

[0887] Step 7:

[0888] The server prompts the real-world generators to begin their actual work. It monitors the progress of the real-world generation in real time, checking the progress at each stage. Based on reports and status updates from the craftsmen, it checks whether the work is progressing appropriately.

[0889] Step 8:

[0890] If a problem occurs during real-time generation, the server will quickly provide a recovery plan and take appropriate action, such as providing additional resources or adjusting work procedures.

[0891] Step 9:

[0892] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then performs the final confirmation and provides feedback, reporting any necessary corrections.

[0893] Step 10:

[0894] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[0895] (Example 1)

[0896] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0897] In traditional custom-made product manufacturing processes, it was difficult to adequately reflect the specific requests of users, and coordination with manufacturers and management of progress were cumbersome. This increased the likelihood of delivery delays, reduced quality, and decreased user satisfaction.

[0898] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[0899] In this invention, the server includes means for receiving and analyzing design requests and ideas from users; means for generating blueprints based on the analyzed designs and ideas; means for presenting the generated blueprints to the user and receiving feedback; means for selecting the most suitable manufacturer from a database; means for negotiating working conditions with the selected manufacturer; means for monitoring the progress of manufacturing in real time; means for providing a recovery plan to resolve problems that arise during manufacturing; means for presenting the completed product to the user and obtaining final confirmation; and means for delivering the completed product to the user after final confirmation. This enables the fulfillment of user requests with high accuracy and consistent management of the manufacturing process.

[0900] "Requested designs and ideas" refer to information about the design of products that users provide to the system, and include formats such as text, images, and videos.

[0901] A "server" is a computer system that performs a series of processes such as analysis, design drawing generation, manufacturer selection, progress monitoring, and quality checks.

[0902] "Means of analysis" refers to software and algorithms used to extract and understand information about designs and ideas provided by users.

[0903] "Means for generating design drawings" refers to software or tools used to create specific product design drawings based on analyzed designs and ideas.

[0904] "Means of receiving feedback" refers to an interface that presents the generated blueprint to the user and collects additional requests and suggestions from the user.

[0905] "Means of selecting a manufacturer" refers to the procedures or algorithms used to select the most suitable manufacturer from a database.

[0906] "Means of negotiating working conditions" refers to means of contacting selected manufacturers and discussing matters such as delivery dates, costs, and specifications.

[0907] "Means of monitoring progress in real time" refers to software and tools used to track the manufacturing process in real time and monitor progress and problems.

[0908] "Means of providing a recovery plan" refers to software or procedures that provide quick solutions when problems occur during the manufacturing process.

[0909] "A means of presenting the finished product and obtaining final confirmation" refers to an interface used to present the completed product to the user and obtain their final approval.

[0910] "Delivery method" refers to the procedures and systems for quickly delivering finished products to the user's specified location.

[0911] This invention provides a comprehensive system for analyzing users' original designs and ideas, generating blueprints, and collaborating with manufacturers (craftsmen) to create concrete products. This system includes means for exchanging data between users, servers, and manufacturers to achieve an optimal manufacturing process.

[0912] Data entry and upload

[0913] Users upload their designs and ideas to the system. These designs and ideas are provided in various formats, including text, images, and videos. Users enter the data using a dedicated web form or smartphone app and click the upload button.

[0914] Data analysis and extraction

[0915] The server analyzes uploaded designs and ideas. For image data, it uses image analysis libraries such as OpenCV and TensorFlow to analyze shape, color, and pattern. For text data, it uses NLTK and spaCy to extract dimensions, materials, and other requirements. If necessary, the server will request additional information from the user.

[0916] Design drawing generation and verification

[0917] The server generates design drawings using design AI based on the analyzed designs and ideas. The design AI used includes the AutoCAD API and SolidWorks API. The generated design drawings include detailed information such as dimensions, materials, and manufacturing methods. The server then presents the design drawings to the user and receives feedback.

[0918] Selection and negotiation of manufacturers

[0919] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI employs a Python-based machine learning model. It selects manufacturers based on information such as past performance, areas of expertise, and reviews from a database. The server then contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications.

[0920] Manufacturing monitoring and control

[0921] The server uses cloud-based monitoring tools such as AWS CloudWatch to monitor the progress of manufacturing in real time. It receives progress reports from craftsmen and checks quality. If a problem occurs, it immediately provides a recovery plan and directs the corrective actions.

[0922] Confirmation of finished products and delivery

[0923] The server photographs the finished product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the server initiates procedures to quickly deliver the finished product to the user.

[0924] Specific example

[0925] The following are examples of custom-made products.

[0926] Examples of custom-made clothing

[0927] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. For example, the prompt might look like this:

[0928] Please generate detailed blueprints for the following design:

[0929] Item: Dress

[0930] Color: Red

[0931] Waist size: 70cm

[0932] Material: Silk

[0933] The server analyzes the data, extracts the necessary information, and then uses design AI to generate detailed blueprints, which are presented to the user. After user confirmation, the procurement AI selects the most suitable manufacturer, negotiates terms, and then begins work. Progress is monitored in real time, and the completed garments are sent to the user for final confirmation before being promptly delivered.

[0934] Examples of custom-made furniture

[0935] Users upload sketches, dimensions, and material details of their original furniture designs. For example, the prompt might look like this:

[0936] Please generate detailed blueprints for the following design:

[0937] Item: Table

[0938] Material: wood

[0939] Height: 75cm

[0940] Width: 120cm

[0941] Color: Brown

[0942] The server analyzes the data, generates blueprints, presents them to the user, and receives feedback. The procurement AI selects the most suitable manufacturer, negotiates terms, and then requests the manufacturer to perform the work. Progress is monitored in real time, and countermeasures are taken if problems arise. The completed furniture is asked to be confirmed by the user, and after confirmation, it is delivered.

[0943] This system allows users to quickly obtain high-quality, custom-made products, while also providing manufacturers with access to a new customer base.

[0944] The flow of the specific processing in Example 1 will be explained using Figure 11.

[0945] Step 1: Data entry and upload

[0946] Users upload designs and ideas to the system using a dedicated web form or smartphone app. The data entered by users can be in the form of text, images, or videos, providing the system with user requests. Examples of input data include "red dress," "silk material," and "waist size 70cm." Users click the "Upload" button to submit the data. The server then receives this data.

[0947] Step 2: Data Analysis and Extraction

[0948] The server begins analyzing the data received from the user. Based on the input data, the server performs the following analyses:

[0949] For image data, we use OpenCV or TensorFlow to analyze shape, color, and pattern.

[0950] For text data, use NLTK or spaCy to extract dimensions, materials, and other requirements.

[0951] For example, features such as "shape: dress," "color: red," and "pattern: solid" are extracted from the image data. The server retrieves the necessary information as analysis results and notifies the user if additional information is required.

[0952] Step 3: Design drawing generation and verification

[0953] The server uses design AI to generate detailed design drawings based on the analyzed data. The design AI used is either the AutoCAD API or the SolidWorks API. It receives analysis results (dimensions, materials, design features, etc.) as input, and the design AI generates detailed design drawings based on this. The design drawings include details such as "length and width dimensions," "materials used," and "manufacturing methods."

[0954] The generated blueprints are presented to the user via the server. The user reviews the blueprints and provides feedback as needed. For example, they might submit requests such as "I want to widen the shoulders a little" or "I want to change the fabric color." The server receives this feedback and modifies the blueprints.

[0955] Step 4: Selecting and negotiating with manufacturers

[0956] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI uses a Python-based machine learning model. Based on user and product requirements data (past performance, expertise, reviews, etc.), it selects the most suitable manufacturer. The server contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications. For example, it might negotiate conditions such as "2 weeks for production, cost of 50,000 yen, and use of silk fabric."

[0957] Step 5: Monitoring and Controlling Production

[0958] The server monitors the manufacturing progress in real time. It uses cloud-based monitoring tools such as AWS CloudWatch to receive progress reports from manufacturers. For example, it retrieves progress information such as "30% sewn" or "dyeing process started." The server manages this progress information and checks quality. If a problem occurs, the server provides a recovery plan and gives appropriate instructions to the manufacturer.

[0959] Step 6: Check and ship the finished product.

[0960] The server photographs the finished product and sends the image to the user. The user reviews the image and makes a final confirmation, for example, by giving approval such as "It's just as pictured, no problems." After the final confirmation, the server proceeds with the shipping process. The server interacts with the shipping company's API to determine the shipping schedule for the user's address. For example, it generates information such as "Scheduled delivery date: October 15th" and "Shipping company: XX Transportation."

[0961] Through the steps outlined above, this system can realize users' original designs with high precision and deliver products efficiently and quickly.

[0962] (Application Example 1)

[0963] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[0964] When bringing user-designed original products to life, challenges exist, such as inadequate communication of detailed design instructions, inappropriate selection of real-world product generators, insufficient monitoring of progress, and slow problem resolution. Furthermore, as the diversity and accuracy of user-uploaded design data increase, there is a need for a system that can smoothly analyze this data and generate prompts.

[0965] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[0966] In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, means for presenting the generated design drawing to the user and obtaining their confirmation, means for selecting the most suitable real-world generator from a database of skilled craftsmen, means for negotiating terms with the selected real-world generator, means for monitoring the progress of real-world generation in real time, means for resolving problems that arise during real-world generation, means for presenting the completed real-world product to the user and obtaining their confirmation, means for delivering the finished product to the user, means for extracting dimensions and important information from the user's design images and text, and means for generating prompt sentences based on the generated design drawing and inputting them into a generation AI model to obtain more detailed instructions. This makes it possible to analyze the user's design in detail, select the most suitable real-world generator, and quickly monitor the progress and resolve problems in real time. Furthermore, by using a generation AI model, the user's design can be further optimized and specific instructions can be obtained quickly.

[0967] "Requested data" refers to digital data in which a user expresses their original design or idea, and includes images, text, or video formats.

[0968] A "design drawing" is a detailed drawing generated from the analyzed request data, containing specific instructions including the dimensions, materials, and manufacturing methods of the product desired by the user.

[0969] A "user" is an individual or organization that uses this system to request a custom-made product.

[0970] The "Craftsman Database" is a database containing information on a variety of craftsmen, including each craftsman's area of ​​expertise, achievements, reviews, and other relevant information.

[0971] A "real-world creator" is a craftsman or artisan who produces a concrete product based on a design requested by a user.

[0972] "Negotiating terms" refers to the process of agreeing on terms such as delivery date, cost, and specifications with the real-world creator regarding the request.

[0973] "Real-world generation" refers to the process of creating real-world products based on user request data.

[0974] "Real-time monitoring of progress" is a process of checking the progress of work at each stage of real-time generation in real time and giving instructions or making corrections as needed.

[0975] A "prompt message" is a phrase used as input for the AI ​​generation model, and it is generated by analyzing the user's design.

[0976] A "generative AI model" is a model that uses artificial intelligence to analyze a design and generate more detailed instructions.

[0977] The means of extracting "dimensions and important information" refers to a function that extracts necessary information from uploaded design images and text.

[0978] "A means of inputting into a generative AI model to obtain more detailed instructions" refers to the process of inputting prompt sentences into a generative AI model based on the analysis results, and generating specific instructions.

[0979] The system for implementing this invention is built based on the following program.

[0980] First, users upload their original designs or ideas as request data using a smartphone or tablet application. This request data can include images, text, or videos.

[0981] Data Analysis

[0982] The server analyzes the uploaded request data and extracts important keywords, shapes, colors, patterns, dimensions, etc. Image data is converted to grayscale using OpenCV, thresholded, and then subjected to shape analysis. Text data is extracted using Pytesseract to obtain dimensions and important information.

[0983] Design drawing generation

[0984] The server generates detailed blueprints based on the extracted data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. Furthermore, prompt statements are generated based on these blueprints and input into the generating AI model to obtain even more detailed instructions.

[0985] Selection of craftsmen and negotiation of terms

[0986] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications, and then finalizes the work schedule once an agreement is reached.

[0987] Monitoring and management of real-world generation

[0988] The server monitors the progress of real-time generation, checking the craftsman's work process. If a problem occurs, the server immediately provides a recovery plan and instructs the craftsman. This ensures that work proceeds smoothly and high-quality products are completed.

[0989] Confirmation of finished products and delivery

[0990] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving confirmation from the user, the product is promptly shipped. After receiving the product, the user provides feedback to the server, contributing to improvements in future services.

[0991] Hardware and software to be used

[0992] Hardware: Smartphones, tablets, servers

[0993] Software: Python 3, OpenCV, Pytesseract, Keras

[0994] Specific example

[0995] 1. Uploading the design: The user uploads an image of their original costume design named "example_design.jpg" to the application.

[0996] 2. Data Analysis: The server analyzes the uploaded design images and uses OpenCV to extract important information such as shape and dimensions. If text data is included, Pytesseract is used to extract the text and obtain dimensions and other important information.

[0997] 3. Prompt message generation: Prompt messages are generated based on the extracted data and input into the generation AI model. This allows for more detailed instructions to be obtained.

[0998] Example of a prompt

[0999] "Please analyze the user-uploaded design (e.g., example_design.jpg) and extract information about its shape and dimensions."

[1000] This allows for detailed analysis of the user's design, selection of the most suitable craftsman, real-time monitoring of progress, and rapid problem resolution. Furthermore, by using generative AI models, the user's design can be further optimized, and specific instructions can be obtained quickly.

[1001] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1002] Step 1:

[1003] Users upload designs.

[1004] Users upload their own designed images, text, or video request data using a smartphone or tablet application.

[1005] Input: Image file (e.g., example_design.jpg), text file

[1006] Output: Design data saved on the server

[1007] Step 2:

[1008] Analysis of requested data

[1009] The server receives the uploaded request data and begins analysis. For image data, it uses OpenCV to convert it to grayscale and recognize its shape and color. For text data, it uses Pytesseract to extract the text.

[1010] Input: Design data stored on the server

[1011] Output: Analytical data such as extracted shape information, color information, and dimensions.

[1012] Step 3:

[1013] Design drawing generation

[1014] The server generates a detailed blueprint based on the analyzed data. This blueprint includes dimensions, materials, and manufacturing methods. It also generates prompt statements based on the generated blueprint and inputs them into the AI ​​model to obtain more specific instructions.

[1015] Input: Analysis data

[1016] Output: Detailed blueprint, generated prompts

[1017] Step 4:

[1018] Selection and negotiation of terms

[1019] The server uses a procurement AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications.

[1020] Input: Detailed blueprints, craftsman database

[1021] Output: Selected real generators, agreed conditions

[1022] Step 5:

[1023] Monitoring and management of real-world generation

[1024] The server monitors the progress of the real-time generation process. During this process, it checks the craftsman's work process and immediately provides a recovery plan if any problems occur.

[1025] Input: Real-time generation progress data

[1026] Output: Progress report, recovery plan

[1027] Step 6:

[1028] Confirmation of finished products and delivery

[1029] The server photographs the completed physical product and presents the image to the user for final confirmation. After confirmation, the user requests delivery of the product to the store or their home.

[1030] Input: Image of the finished product

[1031] Output: User's final confirmation and delivery instructions

[1032] Examples of specific actions

[1033] 1. Upload your design:

[1034] The user takes a photo of the clothing design "example_design.jpg" with their smartphone and uploads the image using the application.

[1035] 2. Data Analysis:

[1036] The server converts the uploaded "example_design.jpg" to grayscale using OpenCV, applies thresholding, and analyzes its shape and color information. If text is present, Pytesseract is used to extract dimensions and specifications as text.

[1037] 3. Generating blueprints:

[1038] The server inputs prompt messages into the generated AI model based on the analysis data, receiving instructions such as, "Please describe the specific dimensions and manufacturing method of this design." Based on these instructions, a detailed design drawing is created.

[1039] 4. Selection and Negotiation of Terms:

[1040] The server uses an AI-powered scheduling system to search a database of artisans, select the most suitable one, and contact them. For example, it might find an artisan who has previously produced similar garments and reconfirm the delivery time and cost requirements.

[1041] 5. Monitoring and management of real-world generation:

[1042] The server instructs the craftsman to report progress in real time, and progress reports are received as they occur. If a problem arises, alternative solutions are provided promptly.

[1043] 6. Confirmation of finished product and delivery:

[1044] The craftsman photographs the finished garment and sends the image to the server. The server then presents the image to the user and requests final confirmation. After the user confirms, the shipping process begins.

[1045] These steps allow users to efficiently bring their original designs to life and receive high-quality products.

[1046] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1047] This invention combines a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans), with an emotion engine that recognizes user emotions, to further enhance the user experience. This system includes the following key functions:

[1048] 1. Data entry and analysis

[1049] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded from the user's device.

[1050] The server receives the uploaded request data and begins analysis. For text data, natural language processing techniques are used to extract important keywords and requirements, while for image and video data, image recognition techniques are used to analyze shapes, colors, patterns, etc.

[1051] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[1052] 2. User sentiment analysis using an emotion engine

[1053] When analyzing request data, the server uses an emotion engine to analyze the user's emotions based on user input and feedback. This emotion data is used to generate blueprints and optimize feedback.

[1054] 3. Design drawing generation and feedback

[1055] The server presents the generated blueprint to the user and receives feedback while monitoring the user's emotional state. If the user expresses a specific emotion (e.g., satisfaction, dissatisfaction) towards the blueprint, the server modifies the blueprint accordingly.

[1056] 4. Selection and negotiation of real-world creators

[1057] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[1058] The server contacts the selected real-world generator, negotiates working conditions (delivery date, cost, specifications), and confirms the work schedule once an agreement is reached.

[1059] 5. Monitoring and Management of Real-Generation

[1060] The server monitors the progress of the real-time creators in real time and receives progress reports. It also monitors the users' emotional states and provides emotional data to the real-time creators as needed to optimize their work.

[1061] In the event of a problem, the server immediately provides a recovery plan and gives appropriate instructions to the real-world creators. This ensures that work proceeds smoothly and high-quality products are produced.

[1062] 6. Confirmation of finished products and delivery

[1063] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then makes a final confirmation and reports any necessary corrections.

[1064] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[1065] Specific example

[1066] 1. Examples of custom-made clothing

[1067] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. A design AI generates a detailed blueprint, presents it to the user while analyzing their emotions, and obtains their confirmation.

[1068] The AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. It monitors the progress in real time and communicates appropriately based on the user's feelings. The user is asked to give final confirmation of the finished garment, and after confirmation, it is promptly delivered.

[1069] 2. Examples of custom-made furniture

[1070] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. It presents the blueprints while analyzing the user's emotions and incorporates their feedback.

[1071] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and resolves issues based on the user's emotions. The completed furniture is then requested to be confirmed by the user, and after confirmation, delivery is arranged.

[1072] This allows users to obtain high-quality, original products at low cost and in a short timeframe, while also providing a personalized experience powered by an emotional engine. Furthermore, artisans gain the opportunity to connect with a new customer base.

[1073] The following describes the processing flow.

[1074] Step 1:

[1075] Users upload their original designs and ideas to the system in text, image, and video formats. This data is then sent from the user's device to the server.

[1076] Step 2:

[1077] The server receives the uploaded request data and begins initial analysis. For text data, it uses natural language processing techniques to extract important keywords and requirements, while for image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[1078] Step 3:

[1079] The server uses an emotion engine to analyze user emotions from their input and feedback. Emotional data is extracted based on information obtained from the user's text, voice, images, and videos.

[1080] Step 4:

[1081] The server uses design AI based on the analysis results to generate detailed blueprints. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. The design AI also takes user sentiment data into consideration to generate blueprints that enhance user satisfaction.

[1082] Step 5:

[1083] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback. At this stage, the emotion engine analyzes the user's emotional state in real time, reading emotions from the user's facial expressions and tone of voice.

[1084] Step 6:

[1085] The server revises its design based on user feedback and asks the user for final confirmation. This process can be repeated until the user is satisfied.

[1086] Step 7:

[1087] The server uses a scheduling AI to select the most suitable real-world creator from a database of artisans. Selection criteria include past performance, area of ​​expertise, reviews, and user sentiment data.

[1088] Step 8:

[1089] The server contacts the selected real-world generator and negotiates the terms of work (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[1090] Step 9:

[1091] The server monitors the progress of the real-time generation process and receives progress reports. The emotion engine also monitors the user's emotional state in parallel and provides emotional data to the real-time generator as needed. This personalizes the work process and improves quality.

[1092] Step 10:

[1093] If a problem occurs during real-time generation, the server will immediately provide a recovery plan and give appropriate instructions to the craftsmen. For example, this may include providing additional resources or adjusting work procedures.

[1094] Step 11:

[1095] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The emotion engine also analyzes the user's emotional state at the time of final confirmation and makes final adjustments if necessary.

[1096] Step 12:

[1097] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[1098] Step 13:

[1099] After receiving the product, users contribute to improving future services by providing feedback to the server. The emotion engine extracts emotion data from this feedback and uses it to improve the entire system.

[1100] Through the above processing steps, we can provide users with highly satisfying original products, and craftsmen can also gain opportunities to connect with a new customer base.

[1101] (Example 2)

[1102] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1103] Traditional systems for translating original designs into actual products were time-consuming and costly, and struggled to accurately reflect user intentions and emotions. Furthermore, the process of negotiating terms with craftsmen, monitoring the progress of the real-world product creation, and resolving problems was cumbersome, resulting in a poor user experience.

[1104] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for analyzing the request data using natural language processing or image recognition technology according to the format of the request data, means for generating a detailed design drawing using design AI based on the analyzed data, means for analyzing the user's emotional state and modifying the design drawing based on feedback, means for selecting the optimal real-world producer from a craftsman database using arrangement AI, means for negotiating terms with the selected real-world producer, means for monitoring the progress of real-world production in real time and providing emotional data as needed, means for immediately providing a recovery plan for problems that occur during real-world production, means for presenting the completed real-world product to the user and obtaining confirmation, and means for delivering the finished product to the user. This enables the rapid, low-cost, and high-quality realization of the user's designs and ideas into original products, and allows for a personalized experience that takes the user's emotions into consideration.

[1105] "Request data" refers to information provided by the user to the system, including original designs and ideas, and can be in text, image, or video format.

[1106] "Natural language processing" is a technique for extracting important keywords and requirements from text data, and involves using models such as BERT and GPT-3.

[1107] "Image recognition technology" is a technique that analyzes features such as color, shape, and pattern from image and video data, and involves using software such as the Google Cloud Vision API.

[1108] "Design AI" refers to an artificial intelligence system that automatically generates detailed design drawings based on user request data, and includes tools such as AutoCAD and Blender.

[1109] An "emotion engine" is a system that analyzes emotions from user input and feedback, using technologies such as IBM Watson Tone Analyzer.

[1110] "Arrangement AI" is an artificial intelligence system that selects the most suitable real-world creator from a database of skilled craftsmen.

[1111] A "real-world creator" is a craftsman or business that actually produces goods based on a user's request.

[1112] A "recovery plan" is a set of countermeasures to solve problems that occur during real-time generation.

[1113] This invention enhances the user experience by combining a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans) with an emotion engine that recognizes the user's emotions. This system includes the following main functions:

[1114] Data entry and analysis

[1115] Users upload their original designs and ideas to the system in text, image, and video formats. Data is sent from the user's device to the server. Specifically, a user uploads a sketch of a garment they designed from their device and sends the data by pressing the "Upload" button.

[1116] The server receives the uploaded request data and begins analysis. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image and video data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shapes, colors, patterns, etc. For example, it sends a sketch image of clothing to the Google Cloud Vision API to recognize its color code and shape.

[1117] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. Specifically, Blender is used to generate a 3D model of the clothing, and then dimension and material information is attached based on that model.

[1118] User sentiment analysis using an emotion engine

[1119] The server analyzes user emotions using an emotion engine (e.g., IBM Watson Tone Analyzer) when analyzing user input and feedback. This emotion data is used to generate blueprints and optimize feedback. When users provide text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, that emotion is analyzed.

[1120] Design drawing generation and feedback

[1121] The server presents the generated blueprint to the user and accepts feedback based on their emotional state. For example, if a user submits feedback such as "I want more red in this design," the server uses IBM Watson Tone Analyzer to analyze the emotion and revise the blueprint.

[1122] Selection of real-world creators and negotiation of terms

[1123] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. Specifically, the server queries the craftsman database and requests the appropriate craftsman.

[1124] The server contacts the selected real-world generators and negotiates working conditions (delivery date, cost, specifications). For example, it contacts craftsmen via email to reach an agreement on delivery date and cost.

[1125] Monitoring and management of real-world generation

[1126] The server monitors the progress of real-time creators and receives progress reports. It also monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives progress images from the creators periodically and shares the results of the user's emotional analysis with them.

[1127] The server will immediately provide a recovery plan and give appropriate instructions to the real-world generator in the event of a problem. For example, it will suggest alternative materials if a material shortage occurs.

[1128] Confirmation of finished products and delivery

[1129] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user then reviews the product and reports any necessary corrections. For example, the user might send an image of the finished garment and obtain final approval.

[1130] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[1131] Specific example

[1132] Examples of custom-made clothing

[1133] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their designed clothing to the system. The server analyzes this data and extracts the necessary information. A design AI generates detailed blueprints, presents them to the user while analyzing their emotions, and obtains their confirmation. A procurement AI selects the most suitable craftsman, negotiates terms, and begins work. The system monitors the progress in real time and communicates appropriately based on the user's emotions. The completed clothing is then asked for final confirmation from the user, and after confirmation, it is promptly delivered.

[1134] Example of a prompt

[1135] "I've uploaded an original sketch of a garment I designed. The material is cotton, the color is red, and the size is M. Please create a detailed blueprint for this design, analyze my feelings, and provide feedback."

[1136] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1137] Step 1: Data Entry

[1138] Users can upload their original designs and ideas to the system in image, text, and video formats. Generally, users send data to the server using their device. When the "Upload" button is pressed, the selected design data is sent. Specifically, users can upload sketches of clothing or detailed product specifications (material, color, size, etc.) from their device.

[1139] Input: Original designs and ideas from users (images, text, videos)

[1140] Output: Design data sent to the server

[1141] Step 2: Data Analysis

[1142] The server receives the uploaded design data and analyzes it according to its format. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shape, color, and patterns. For video data, it performs image recognition frame by frame.

[1143] Input: Design data sent to the server

[1144] Output: Analysis results (important keywords, shapes, colors, patterns, etc.)

[1145] Step 3: Generate blueprints

[1146] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings based on the analyzed data. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. For example, Blender is used to generate a 3D model of clothing, and then dimensions and material information is added based on that model.

[1147] Input: Analysis results (important keywords, shapes, colors, patterns, etc.)

[1148] Output: Detailed design drawings (dimensions, materials, manufacturing methods, etc.)

[1149] Step 4: Sentiment Analysis

[1150] The server uses an emotion engine (for example, IBM Watson Tone Analyzer) to analyze emotions based on user input data and feedback. When a user provides text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, the server analyzes that emotion.

[1151] Input: User feedback (text format)

[1152] Output: Emotion analysis results

[1153] Step 5: Feedback and design revisions

[1154] The server presents the generated blueprint to the user and accepts feedback based on the sentiment analysis results. It then modifies the blueprint based on the user's feedback. For example, if a user submits a comment such as, "I want more red in this design," the server analyzes that feedback and adjusts the blueprint again.

[1155] Input: Sentiment analysis results, user feedback

[1156] Output: Revised blueprint

[1157] Step 6: Selecting a Real Generator

[1158] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. For example, the server queries the craftsman database and lists the most suitable craftsmen.

[1159] Input: Modified blueprint

[1160] Output: Selected real generators

[1161] Step 7: Negotiating Terms

[1162] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). Specifically, negotiations are conducted via email and telephone until an agreement is reached.

[1163] Input: Selected real generator

[1164] Output: Agreed working conditions (delivery date, cost, specifications)

[1165] Step 8: Monitoring the progress of real generation

[1166] The server monitors the progress of real-time creators and receives progress reports. It also simultaneously monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives regular progress reports from the creators and shares the results of the user's emotional analysis.

[1167] Input: Agreed working conditions

[1168] Output: Progress report, sentiment analysis results

[1169] Step 9: Recovery in case of problems

[1170] If a problem occurs, the server will immediately provide a recovery plan and give appropriate instructions to the real-time generator. For example, if there is a shortage of materials, it will suggest alternative materials or adjust delivery dates.

[1171] Input: Progress report, sentiment analysis results

[1172] Output: Recovery Plan

[1173] Step 10: Check and ship the finished product.

[1174] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user makes the final confirmation and reports any necessary corrections. After the user's final confirmation, the server arranges for the product to be delivered to the user as quickly as possible. Specifically, it arranges for a delivery company to ensure the product is delivered safely and promptly.

[1175] Input: Completed real product

[1176] Output: User's final confirmation, shipping arrangements

[1177] These processing steps enable users to quickly and cost-effectively translate their designs and ideas into high-quality original products, creating personalized experiences that take user emotions into consideration.

[1178] (Application Example 2)

[1179] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server," and the headset-type terminal 314 will be referred to as the "terminal."

[1180] This invention aims to solve the problem of providing high-quality, custom-made products that reflect the user's emotions in a short period of time and at low cost, particularly in the process of bringing original designs and ideas to life in collaboration with real-world creators (craftsmen). Conventional technologies lack a mechanism to adequately reflect the user's emotional feedback, and there is a lack of means to efficiently adjust the progress of real-world creation while monitoring it in real time. As a result, there have been problems in ensuring sufficient user satisfaction and quality for customized orders.

[1181] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, and emotion analysis means for analyzing the user's emotional state. This makes it possible to grasp the user's emotions in real time, generate a design drawing that reflects those emotions, and monitor the progress.

[1182] "Requested data" refers to information provided by users, including designs and ideas, and can take the form of text, images, videos, and other data.

[1183] "Analysis methods" refer to techniques for analyzing requested data and extracting important keywords, shapes, and patterns.

[1184] "Design drawing generation means" refers to a technology that creates specific design drawings based on analyzed data, generating instructions that include dimensions, materials, and processing methods.

[1185] "Sentiment analysis methods" refer to technologies that extract emotional data from user input and feedback and use it to generate and modify design blueprints.

[1186] "Verification methods" refer to technologies that present generated design drawings or finished products to users and receive their verification and feedback.

[1187] A "craftsman database" is a database that accumulates data about craftsmen, including information such as past achievements, areas of expertise, and reviews.

[1188] A "real-world creator" refers to a craftsman who works to bring a user's original design to life.

[1189] "Negotiation methods" refer to techniques for negotiating working conditions such as delivery date, cost, and specifications with selected real-world generators.

[1190] "Progress monitoring means" refers to technology that monitors the work progress of the real-time creator in real time and makes adjustments or recoveries as needed.

[1191] "Problem-solving methods" refer to technologies that identify problems that arise during real-world generation and provide appropriate solutions.

[1192] A "smartphone application interface" is an interface that allows users to input designs and feedback via their smartphones and check the progress of their work.

[1193] System Program Overview

[1194] The program for the system that implements this application example will be explained using the following method.

[1195] Overall flow

[1196] 1. Users upload designs and ideas as request data through a smartphone application interface.

[1197] 2. The server analyzes this request data and generates a design drawing.

[1198] 3. The server analyzes the user's emotional state using emotion analysis tools.

[1199] 4. Present the generated design to the user and receive their feedback.

[1200] 5. The server selects the most suitable real-world generator from the craftsman database and negotiates terms.

[1201] 6. The real-world creator begins work and monitors its progress.

[1202] 7. In the event of a problem, the server will provide a timely recovery solution.

[1203] 8. After the work is completed, present the finished product to the user and obtain their confirmation.

[1204] 9. After final confirmation, the finished product is delivered to the user.

[1205] Hardware and software to be used

[1206] Hardware: Smartphones, business servers, emotion measurement devices (cameras and voice recognition devices)

[1207] Software: Python-based emotion recognition engine (emotion_engine), design AI (design_ai), job handling AI (job_handling_ai)

[1208] Examples of data processing and calculations

[1209] Users upload their designs and ideas as text, images, or videos using a smartphone application interface. This data is analyzed on a server; text data is analyzed using natural language processing to extract key keywords, and image and video data is analyzed using image recognition technology to determine shape, color, and pattern. A design AI generates detailed blueprints including dimensions, materials, and manufacturing methods, and these blueprints are then fed back to the user based on sentiment analysis results from an emotion engine.

[1210] Specific example

[1211] scenario

[1212] Users open the jewelry shop app and upload images and text descriptions of their jewelry designs. The server analyzes this data and uses design AI to generate detailed blueprints. Simultaneously, an emotion engine analyzes the emotions the user is feeling at the time of upload and reflects them in the blueprints. The generated blueprints are presented to the user, who is asked to review and provide feedback via the smartphone interface. Based on the feedback, the server makes necessary improvements and finalizes the blueprints.

[1213] Example of a prompt

[1214] "Please upload your original jewelry design. What materials and manufacturing methods would you prefer for that design? Also, please tell us how you feel about creating this jewelry."

[1215] This system allows users to efficiently request customizable, bespoke products utilizing sentiment analysis and receive high-quality finished products. Furthermore, real-world creators can understand the user's specific requests and emotions as they work, resulting in a mutually satisfying process.

[1216] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1217] Step 1:

[1218] Users upload their designs and ideas as request data in text, image, or video format using a smartphone application interface. This sends the request data from the user to the server. The input is the user's design data, and the output is the transmission of data to the server. Specifically, the user opens the application, clicks the "Upload New Design" button, selects a file, and uploads it.

[1219] Step 2:

[1220] The server receives the uploaded request data and performs analysis using natural language processing techniques for text data and image recognition techniques for image and video data. The input is design data submitted by the user, and the output is the analysis results (such as important keywords and shape patterns). Specifically, the analysis algorithm running on the server processes the data and passes the analysis results to the next step.

[1221] Step 3:

[1222] The server uses design AI based on the analysis results to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. The input is the analysis results, and the output is the detailed design drawing. Specifically, the design AI determines design parameters based on the analysis results, generates the design drawing, and saves it.

[1223] Step 4:

[1224] The server uses an emotion engine to analyze the user's emotional state and incorporate this into the design blueprint generation. Inputs are user request data and feedback data, while output is the emotion analysis result. Specifically, the emotion engine executes an emotion recognition algorithm, extracts the user's emotional data, and incorporates it into the blueprint.

[1225] Step 5:

[1226] The generated blueprint is presented to the user from the server, and they are asked to review it via a smartphone application. The input is the detailed blueprint, and the output is the user's feedback. Specifically, the blueprint is displayed in the application interface, and the user presses either the "Confirm" button or the "Request Revision" button.

[1227] Step 6:

[1228] Based on user feedback, the server modifies the design drawings as needed. The input is the user feedback, and the output is the modified design drawings. Specifically, the design AI generates and re-presents the design drawings based on the feedback.

[1229] Step 7:

[1230] The server selects the most suitable real-world craftsman from the craftsman database and negotiates terms. The input is a detailed blueprint, and the output is the work conditions agreed upon with the selected real-world craftsman. Specifically, the sourcing AI searches the database, selects the most suitable craftsman, and saves the results of the negotiation.

[1231] Step 8:

[1232] A selected real-time generator begins work, and the server monitors the progress in real time. Input is work start information, and output is progress data. Specifically, the generator sends progress reports to the server, which processes them.

[1233] Step 9:

[1234] If a problem occurs during the process, the server will analyze it and provide a timely recovery plan. The input is data about the problem that occurred, and the output is the recovery plan. Specifically, the server detects the problem, executes the recovery algorithm, and issues instructions.

[1235] Step 10:

[1236] After the work is completed, the finished product is presented to the user for confirmation. The input is information about the finished product, and the output is the user's final confirmation. Specifically, the server sends an image of the finished product to the user and requests confirmation.

[1237] Step 11:

[1238] After final confirmation, the server arranges for the finished product to be delivered to the user. Input is the user's delivery information, and output is a notification of delivery completion. Specifically, the server sends instructions to the delivery company and notifies the user of the delivery status.

[1239] The specific processing unit 290 transmits the result of the specific processing to the headset terminal 314. In the headset terminal 314, the control unit 46A causes the speaker 240 and display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1240] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1241] In the above embodiment, an example was given in which specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and specific processing may also be performed by the headset terminal 314.

[1242] [Fourth Embodiment]

[1243] Figure 7 shows an example of the configuration of the data processing system 410 according to the fourth embodiment.

[1244] As shown in Figure 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[1245] The data processing device 12 comprises a computer 22, a database 24, and a communication interface 26. The computer 22 is an example of a "computer" related to the technology of this disclosure. The computer 22 comprises a processor 28, RAM 30, and storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication interface 26 are also connected to the bus 34. The communication interface 26 is connected to a network 54. An example of the network 54 is a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[1246] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication interface 44, and a controlled object 443. The computer 36 includes a processor 46, RAM 48, and storage 50. The processor 46, RAM 48, and storage 50 are connected to a bus 52. The microphone 238, speaker 240, camera 42, and controlled object 443 are also connected to the bus 52.

[1247] The microphone 238 receives voice signals from the user 20 and receives instructions from the user 20. The microphone 238 captures the voice signals from the user 20, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to the instructions from the processor 46.

[1248] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an image sensor such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the area around the user 20 (for example, an imaging range defined by a field of view equivalent to the width of a typical healthy person's field of vision).

[1249] Communication interface 44 is connected to network 54. Communication interfaces 44 and 26 are responsible for the exchange of various information between processor 46 and processor 28 via network 54. The exchange of various information between processor 46 and processor 28 using communication interfaces 44 and 26 is performed in a secure manner.

[1250] The controlled object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the robot 414's emotions can be expressed by controlling these motors. Furthermore, the robot 414's facial expressions can also be expressed by controlling the illumination state of the LEDs in its eyes.

[1251] Figure 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Figure 8, the data processing device 12 performs specific processing using the processor 28. The storage 32 stores the specific processing program 56.

[1252] The specific processing program 56 is an example of a "program" relating to the technology of this disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[1253] The storage 32 stores the data generation model 58 and the emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290.

[1254] In robot 414, the processor 46 performs the reception output processing. The storage 50 stores the reception output program 60. The processor 46 reads the reception output program 60 from the storage 50 and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as a control unit 46A according to the reception output program 60 executed on the RAM 48.

[1255] Next, the specific processing performed by the specific processing unit 290 of the data processing device 12 will be described. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1256] This invention provides a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans). This system includes the following main functions:

[1257] 1. Data entry and analysis

[1258] Users upload their original designs and ideas to the system. The uploaded data may include text, images, or videos, and the server performs an initial analysis based on this data.

[1259] The server extracts important keywords from the input data and analyzes the shape, color, and pattern of image data. It extracts dimensions and requirements from text data and requests additional information from the user if needed.

[1260] 2. Design drawing generation

[1261] The server uses design AI to generate detailed blueprints based on the analyzed data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods, enabling the creation of perfect crafts.

[1262] The generated design drawings are presented to the user for review and feedback. This ensures that the design drawings meet the user's requirements in a final confirmation.

[1263] 3. Selection and negotiation of real-world creators

[1264] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. The selection criteria are diverse, including past performance, area of ​​expertise, and reviews.

[1265] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[1266] 4. Monitoring and management of real-world generation

[1267] The server monitors the progress of the real-world generation in real time. It checks the progress at each stage and verifies the quality based on reports from the craftsmen.

[1268] In the event of a problem, the server immediately provides a recovery plan and instructs the craftsman. This ensures that the work proceeds smoothly and that high-quality products are completed.

[1269] 5. Confirmation of finished product and delivery

[1270] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the product is promptly delivered to the user.

[1271] After receiving the provided product, users contribute to improving future services by providing feedback to the server.

[1272] Specific example

[1273] The following are specific examples.

[1274] 1. Examples of custom-made clothing

[1275] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. The design AI generates a detailed design drawing, which is then presented to the user for confirmation.

[1276] Subsequently, the AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. Progress is monitored in real time, and any problems that arise are addressed immediately. The finished garments are then sent to the user for final confirmation, and after confirmation, they are promptly delivered.

[1277] 2. Examples of custom-made furniture

[1278] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. The blueprints are then presented to the user, and feedback is incorporated.

[1279] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and takes countermeasures if problems arise. The completed furniture is then requested to be confirmed by the user, and after confirmation, it is delivered.

[1280] This allows users to obtain high-quality, original products at a low cost and in a short time. Furthermore, artisans gain the opportunity to connect with a new customer base.

[1281] The following describes the processing flow.

[1282] Step 1:

[1283] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded via their device.

[1284] Step 2:

[1285] The server receives the uploaded request data and begins analyzing it. For text data, it uses natural language processing techniques to extract important keywords and requirements. For image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[1286] Step 3:

[1287] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[1288] Step 4:

[1289] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback as needed. Based on this feedback, the blueprint is revised again.

[1290] Step 5:

[1291] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[1292] Step 6:

[1293] The server will contact the selected real-world generator and negotiate the terms regarding the work schedule and costs. If an agreement is reached, the work progress schedule will be finalized.

[1294] Step 7:

[1295] The server prompts the real-world generators to begin their actual work. It monitors the progress of the real-world generation in real time, checking the progress at each stage. Based on reports and status updates from the craftsmen, it checks whether the work is progressing appropriately.

[1296] Step 8:

[1297] If a problem occurs during real-time generation, the server will quickly provide a recovery plan and take appropriate action, such as providing additional resources or adjusting work procedures.

[1298] Step 9:

[1299] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then performs the final confirmation and provides feedback, reporting any necessary corrections.

[1300] Step 10:

[1301] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[1302] (Example 1)

[1303] Next, we will describe Example 1. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1304] In traditional custom-made product manufacturing processes, it was difficult to adequately reflect the specific requests of users, and coordination with manufacturers and management of progress were cumbersome. This increased the likelihood of delivery delays, reduced quality, and decreased user satisfaction.

[1305] The identification process performed by the identification processing unit 290 of the data processing device 12 in Example 1 is realized by the following means.

[1306] In this invention, the server includes means for receiving and analyzing design requests and ideas from users; means for generating blueprints based on the analyzed designs and ideas; means for presenting the generated blueprints to the user and receiving feedback; means for selecting the most suitable manufacturer from a database; means for negotiating working conditions with the selected manufacturer; means for monitoring the progress of manufacturing in real time; means for providing a recovery plan to resolve problems that arise during manufacturing; means for presenting the completed product to the user and obtaining final confirmation; and means for delivering the completed product to the user after final confirmation. This enables the fulfillment of user requests with high accuracy and consistent management of the manufacturing process.

[1307] "Requested designs and ideas" refer to information about the design of products that users provide to the system, and include formats such as text, images, and videos.

[1308] A "server" is a computer system that performs a series of processes such as analysis, design drawing generation, manufacturer selection, progress monitoring, and quality checks.

[1309] "Means of analysis" refers to software and algorithms used to extract and understand information about designs and ideas provided by users.

[1310] "Means for generating design drawings" refers to software or tools used to create specific product design drawings based on analyzed designs and ideas.

[1311] "Means of receiving feedback" refers to an interface that presents the generated blueprint to the user and collects additional requests and suggestions from the user.

[1312] "Means of selecting a manufacturer" refers to the procedures or algorithms used to select the most suitable manufacturer from a database.

[1313] "Means of negotiating working conditions" refers to means of contacting selected manufacturers and discussing matters such as delivery dates, costs, and specifications.

[1314] "Means of monitoring progress in real time" refers to software and tools used to track the manufacturing process in real time and monitor progress and problems.

[1315] "Means of providing a recovery plan" refers to software or procedures that provide quick solutions when problems occur during the manufacturing process.

[1316] "A means of presenting the finished product and obtaining final confirmation" refers to an interface used to present the completed product to the user and obtain their final approval.

[1317] "Delivery method" refers to the procedures and systems for quickly delivering finished products to the user's specified location.

[1318] This invention provides a comprehensive system for analyzing users' original designs and ideas, generating blueprints, and collaborating with manufacturers (craftsmen) to create concrete products. This system includes means for exchanging data between users, servers, and manufacturers to achieve an optimal manufacturing process.

[1319] Data entry and upload

[1320] Users upload their designs and ideas to the system. These designs and ideas are provided in various formats, including text, images, and videos. Users enter the data using a dedicated web form or smartphone app and click the upload button.

[1321] Data analysis and extraction

[1322] The server analyzes uploaded designs and ideas. For image data, it uses image analysis libraries such as OpenCV and TensorFlow to analyze shape, color, and pattern. For text data, it uses NLTK and spaCy to extract dimensions, materials, and other requirements. If necessary, the server will request additional information from the user.

[1323] Design drawing generation and verification

[1324] The server generates design drawings using design AI based on the analyzed designs and ideas. The design AI used includes the AutoCAD API and SolidWorks API. The generated design drawings include detailed information such as dimensions, materials, and manufacturing methods. The server then presents the design drawings to the user and receives feedback.

[1325] Selection and negotiation of manufacturers

[1326] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI employs a Python-based machine learning model. It selects manufacturers based on information such as past performance, areas of expertise, and reviews from a database. The server then contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications.

[1327] Manufacturing monitoring and control

[1328] The server uses cloud-based monitoring tools such as AWS CloudWatch to monitor the progress of manufacturing in real time. It receives progress reports from craftsmen and checks quality. If a problem occurs, it immediately provides a recovery plan and directs the corrective actions.

[1329] Confirmation of finished products and delivery

[1330] The server photographs the finished product and presents the image to the user for final confirmation. After receiving final confirmation from the user, the server initiates procedures to quickly deliver the finished product to the user.

[1331] Specific example

[1332] The following are examples of custom-made products.

[1333] Examples of custom-made clothing

[1334] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. For example, the prompt might look like this:

[1335] Please generate detailed blueprints for the following design:

[1336] Item: Dress

[1337] Color: Red

[1338] Waist size: 70cm

[1339] Material: Silk

[1340] The server analyzes the data, extracts the necessary information, and then uses design AI to generate detailed blueprints, which are presented to the user. After user confirmation, the procurement AI selects the most suitable manufacturer, negotiates terms, and then begins work. Progress is monitored in real time, and the completed garments are sent to the user for final confirmation before being promptly delivered.

[1341] Examples of custom-made furniture

[1342] Users upload sketches, dimensions, and material details of their original furniture designs. For example, the prompt might look like this:

[1343] Please generate detailed blueprints for the following design:

[1344] Item: Table

[1345] Material: wood

[1346] Height: 75cm

[1347] Width: 120cm

[1348] Color: Brown

[1349] The server analyzes the data, generates blueprints, presents them to the user, and receives feedback. The procurement AI selects the most suitable manufacturer, negotiates terms, and then requests the manufacturer to perform the work. Progress is monitored in real time, and countermeasures are taken if problems arise. The completed furniture is asked to be confirmed by the user, and after confirmation, it is delivered.

[1350] This system allows users to quickly obtain high-quality, custom-made products, while also providing manufacturers with access to a new customer base.

[1351] The flow of the specific processing in Example 1 will be explained using Figure 11.

[1352] Step 1: Data entry and upload

[1353] Users upload designs and ideas to the system using a dedicated web form or smartphone app. The data entered by users can be in the form of text, images, or videos, providing the system with user requests. Examples of input data include "red dress," "silk material," and "waist size 70cm." Users click the "Upload" button to submit the data. The server then receives this data.

[1354] Step 2: Data Analysis and Extraction

[1355] The server begins analyzing the data received from the user. Based on the input data, the server performs the following analyses:

[1356] For image data, we use OpenCV or TensorFlow to analyze shape, color, and pattern.

[1357] For text data, use NLTK or spaCy to extract dimensions, materials, and other requirements.

[1358] For example, features such as "shape: dress," "color: red," and "pattern: solid" are extracted from the image data. The server retrieves the necessary information as analysis results and notifies the user if additional information is required.

[1359] Step 3: Design drawing generation and verification

[1360] The server uses design AI to generate detailed design drawings based on the analyzed data. The design AI used is either the AutoCAD API or the SolidWorks API. It receives analysis results (dimensions, materials, design features, etc.) as input, and the design AI generates detailed design drawings based on this. The design drawings include details such as "length and width dimensions," "materials used," and "manufacturing methods."

[1361] The generated blueprints are presented to the user via the server. The user reviews the blueprints and provides feedback as needed. For example, they might submit requests such as "I want to widen the shoulders a little" or "I want to change the fabric color." The server receives this feedback and modifies the blueprints.

[1362] Step 4: Selecting and negotiating with manufacturers

[1363] The server uses a procurement AI to select the most suitable manufacturer. The procurement AI uses a Python-based machine learning model. Based on user and product requirements data (past performance, expertise, reviews, etc.), it selects the most suitable manufacturer. The server contacts the selected manufacturer and negotiates regarding delivery time, cost, and specifications. For example, it might negotiate conditions such as "2 weeks for production, cost of 50,000 yen, and use of silk fabric."

[1364] Step 5: Monitoring and Controlling Production

[1365] The server monitors the manufacturing progress in real time. It uses cloud-based monitoring tools such as AWS CloudWatch to receive progress reports from manufacturers. For example, it retrieves progress information such as "30% sewn" or "dyeing process started." The server manages this progress information and checks quality. If a problem occurs, the server provides a recovery plan and gives appropriate instructions to the manufacturer.

[1366] Step 6: Check and ship the finished product.

[1367] The server photographs the finished product and sends the image to the user. The user reviews the image and makes a final confirmation, for example, by giving approval such as "It's just as pictured, no problems." After the final confirmation, the server proceeds with the shipping process. The server interacts with the shipping company's API to determine the shipping schedule for the user's address. For example, it generates information such as "Scheduled delivery date: October 15th" and "Shipping company: XX Transportation."

[1368] Through the steps outlined above, this system can realize users' original designs with high precision and deliver products efficiently and quickly.

[1369] (Application Example 1)

[1370] Next, we will explain Application Example 1. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1371] When bringing user-designed original products to life, challenges exist, such as inadequate communication of detailed design instructions, inappropriate selection of real-world product generators, insufficient monitoring of progress, and slow problem resolution. Furthermore, as the diversity and accuracy of user-uploaded design data increase, there is a need for a system that can smoothly analyze this data and generate prompts.

[1372] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 1 is realized by the following means.

[1373] In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, means for presenting the generated design drawing to the user and obtaining their confirmation, means for selecting the most suitable real-world generator from a database of skilled craftsmen, means for negotiating terms with the selected real-world generator, means for monitoring the progress of real-world generation in real time, means for resolving problems that arise during real-world generation, means for presenting the completed real-world product to the user and obtaining their confirmation, means for delivering the finished product to the user, means for extracting dimensions and important information from the user's design images and text, and means for generating prompt sentences based on the generated design drawing and inputting them into a generation AI model to obtain more detailed instructions. This makes it possible to analyze the user's design in detail, select the most suitable real-world generator, and quickly monitor the progress and resolve problems in real time. Furthermore, by using a generation AI model, the user's design can be further optimized and specific instructions can be obtained quickly.

[1374] "Requested data" refers to digital data in which a user expresses their original design or idea, and includes images, text, or video formats.

[1375] A "design drawing" is a detailed drawing generated from the analyzed request data, containing specific instructions including the dimensions, materials, and manufacturing methods of the product desired by the user.

[1376] A "user" is an individual or organization that uses this system to request a custom-made product.

[1377] The "Craftsman Database" is a database containing information on a variety of craftsmen, including each craftsman's area of ​​expertise, achievements, reviews, and other relevant information.

[1378] A "real-world creator" is a craftsman or artisan who produces a concrete product based on a design requested by a user.

[1379] "Negotiating terms" refers to the process of agreeing on terms such as delivery date, cost, and specifications with the real-world creator regarding the request.

[1380] "Real-world generation" refers to the process of creating real-world products based on user request data.

[1381] "Real-time monitoring of progress" is a process of checking the progress of work at each stage of real-time generation in real time and giving instructions or making corrections as needed.

[1382] A "prompt message" is a phrase used as input for the AI ​​generation model, and it is generated by analyzing the user's design.

[1383] A "generative AI model" is a model that uses artificial intelligence to analyze a design and generate more detailed instructions.

[1384] The means of extracting "dimensions and important information" refers to a function that extracts necessary information from uploaded design images and text.

[1385] "A means of inputting into a generative AI model to obtain more detailed instructions" refers to the process of inputting prompt sentences into a generative AI model based on the analysis results, and generating specific instructions.

[1386] The system for implementing this invention is built based on the following program.

[1387] First, users upload their original designs or ideas as request data using a smartphone or tablet application. This request data can include images, text, or videos.

[1388] Data Analysis

[1389] The server analyzes the uploaded request data and extracts important keywords, shapes, colors, patterns, dimensions, etc. Image data is converted to grayscale using OpenCV, thresholded, and then subjected to shape analysis. Text data is extracted using Pytesseract to obtain dimensions and important information.

[1390] Design drawing generation

[1391] The server generates detailed blueprints based on the extracted data. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. Furthermore, prompt statements are generated based on these blueprints and input into the generating AI model to obtain even more detailed instructions.

[1392] Selection of craftsmen and negotiation of terms

[1393] The server uses a scheduling AI to select the most suitable real-world creator from a database of skilled craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications, and then finalizes the work schedule once an agreement is reached.

[1394] Monitoring and management of real-world generation

[1395] The server monitors the progress of real-time generation, checking the craftsman's work process. If a problem occurs, the server immediately provides a recovery plan and instructs the craftsman. This ensures that work proceeds smoothly and high-quality products are completed.

[1396] Confirmation of finished products and delivery

[1397] The server photographs the completed physical product and presents the image to the user for final confirmation. After receiving confirmation from the user, the product is promptly shipped. After receiving the product, the user provides feedback to the server, contributing to improvements in future services.

[1398] Hardware and software to be used

[1399] Hardware: Smartphones, tablets, servers

[1400] Software: Python 3, OpenCV, Pytesseract, Keras

[1401] Specific example

[1402] 1. Uploading the design: The user uploads an image of their original costume design named "example_design.jpg" to the application.

[1403] 2. Data Analysis: The server analyzes the uploaded design images and uses OpenCV to extract important information such as shape and dimensions. If text data is included, Pytesseract is used to extract the text and obtain dimensions and other important information.

[1404] 3. Prompt message generation: Prompt messages are generated based on the extracted data and input into the generation AI model. This allows for more detailed instructions to be obtained.

[1405] Example of a prompt

[1406] "Please analyze the user-uploaded design (e.g., example_design.jpg) and extract information about its shape and dimensions."

[1407] This allows for detailed analysis of the user's design, selection of the most suitable craftsman, real-time monitoring of progress, and rapid problem resolution. Furthermore, by using generative AI models, the user's design can be further optimized, and specific instructions can be obtained quickly.

[1408] The flow of a specific process in Application Example 1 will be explained using Figure 12.

[1409] Step 1:

[1410] Users upload designs.

[1411] Users upload their own designed images, text, or video request data using a smartphone or tablet application.

[1412] Input: Image file (e.g., example_design.jpg), text file

[1413] Output: Design data saved on the server

[1414] Step 2:

[1415] Analysis of requested data

[1416] The server receives the uploaded request data and begins analysis. For image data, it uses OpenCV to convert it to grayscale and recognize its shape and color. For text data, it uses Pytesseract to extract the text.

[1417] Input: Design data stored on the server

[1418] Output: Analytical data such as extracted shape information, color information, and dimensions.

[1419] Step 3:

[1420] Design drawing generation

[1421] The server generates a detailed blueprint based on the analyzed data. This blueprint includes dimensions, materials, and manufacturing methods. It also generates prompt statements based on the generated blueprint and inputs them into the AI ​​model to obtain more specific instructions.

[1422] Input: Analysis data

[1423] Output: Detailed blueprint, generated prompts

[1424] Step 4:

[1425] Selection and negotiation of terms

[1426] The server uses a procurement AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. After selection, the server contacts the real-world creator to negotiate terms regarding delivery time, cost, and specifications.

[1427] Input: Detailed blueprints, craftsman database

[1428] Output: Selected real generators, agreed conditions

[1429] Step 5:

[1430] Monitoring and management of real-world generation

[1431] The server monitors the progress of the real-time generation process. During this process, it checks the craftsman's work process and immediately provides a recovery plan if any problems occur.

[1432] Input: Real-time generation progress data

[1433] Output: Progress report, recovery plan

[1434] Step 6:

[1435] Confirmation of finished products and delivery

[1436] The server photographs the completed physical product and presents the image to the user for final confirmation. After confirmation, the user requests delivery of the product to the store or their home.

[1437] Input: Image of the finished product

[1438] Output: User's final confirmation and delivery instructions

[1439] Examples of specific actions

[1440] 1. Upload your design:

[1441] The user takes a photo of the clothing design "example_design.jpg" with their smartphone and uploads the image using the application.

[1442] 2. Data Analysis:

[1443] The server converts the uploaded "example_design.jpg" to grayscale using OpenCV, applies thresholding, and analyzes its shape and color information. If text is present, Pytesseract is used to extract dimensions and specifications as text.

[1444] 3. Generating blueprints:

[1445] The server inputs prompt messages into the generated AI model based on the analysis data, receiving instructions such as, "Please describe the specific dimensions and manufacturing method of this design." Based on these instructions, a detailed design drawing is created.

[1446] 4. Selection and Negotiation of Terms:

[1447] The server uses an AI-powered scheduling system to search a database of artisans, select the most suitable one, and contact them. For example, it might find an artisan who has previously produced similar garments and reconfirm the delivery time and cost requirements.

[1448] 5. Monitoring and management of real-world generation:

[1449] The server instructs the craftsman to report progress in real time, and progress reports are received as they occur. If a problem arises, alternative solutions are provided promptly.

[1450] 6. Confirmation of finished product and delivery:

[1451] The craftsman photographs the finished garment and sends the image to the server. The server then presents the image to the user and requests final confirmation. After the user confirms, the shipping process begins.

[1452] These steps allow users to efficiently bring their original designs to life and receive high-quality products.

[1453] Furthermore, an emotion engine that estimates the user's emotions may be incorporated. That is, the identification processing unit 290 may use the emotion identification model 59 to estimate the user's emotions and perform identification processing using the user's emotions.

[1454] This invention combines a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans), with an emotion engine that recognizes user emotions, to further enhance the user experience. This system includes the following key functions:

[1455] 1. Data entry and analysis

[1456] Users upload their original designs and ideas to the system in text, image, and video formats. This data is uploaded from the user's device.

[1457] The server receives the uploaded request data and begins analysis. For text data, natural language processing techniques are used to extract important keywords and requirements, while for image and video data, image recognition techniques are used to analyze shapes, colors, patterns, etc.

[1458] The server uses design AI to generate detailed blueprints based on the analysis results. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods.

[1459] 2. User sentiment analysis using an emotion engine

[1460] When analyzing request data, the server uses an emotion engine to analyze the user's emotions based on user input and feedback. This emotion data is used to generate blueprints and optimize feedback.

[1461] 3. Design drawing generation and feedback

[1462] The server presents the generated blueprint to the user and receives feedback while monitoring the user's emotional state. If the user expresses a specific emotion (e.g., satisfaction, dissatisfaction) towards the blueprint, the server modifies the blueprint accordingly.

[1463] 4. Selection and negotiation of real-world creators

[1464] The server uses a scheduling AI to select the most suitable real-world creator from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews.

[1465] The server contacts the selected real-world generator, negotiates working conditions (delivery date, cost, specifications), and confirms the work schedule once an agreement is reached.

[1466] 5. Monitoring and Management of Real-Generation

[1467] The server monitors the progress of the real-time creators in real time and receives progress reports. It also monitors the users' emotional states and provides emotional data to the real-time creators as needed to optimize their work.

[1468] In the event of a problem, the server immediately provides a recovery plan and gives appropriate instructions to the real-world creators. This ensures that work proceeds smoothly and high-quality products are produced.

[1469] 6. Confirmation of finished products and delivery

[1470] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The user then makes a final confirmation and reports any necessary corrections.

[1471] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[1472] Specific example

[1473] 1. Examples of custom-made clothing

[1474] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their clothing designs to the system. The server analyzes this data and extracts the necessary information. A design AI generates a detailed blueprint, presents it to the user while analyzing their emotions, and obtains their confirmation.

[1475] The AI ​​system selects the most suitable craftsman, negotiates terms, and then begins work. It monitors the progress in real time and communicates appropriately based on the user's feelings. The user is asked to give final confirmation of the finished garment, and after confirmation, it is promptly delivered.

[1476] 2. Examples of custom-made furniture

[1477] Users upload sketches, dimensions, and material details of their original furniture designs. The server analyzes this data and generates blueprints. It presents the blueprints while analyzing the user's emotions and incorporates their feedback.

[1478] The AI ​​selects the most suitable real-world producer, negotiates terms, and then assigns the work to the producer. It monitors the progress and resolves issues based on the user's emotions. The completed furniture is then requested to be confirmed by the user, and after confirmation, delivery is arranged.

[1479] This allows users to obtain high-quality, original products at low cost and in a short timeframe, while also providing a personalized experience powered by an emotional engine. Furthermore, artisans gain the opportunity to connect with a new customer base.

[1480] The following describes the processing flow.

[1481] Step 1:

[1482] Users upload their original designs and ideas to the system in text, image, and video formats. This data is then sent from the user's device to the server.

[1483] Step 2:

[1484] The server receives the uploaded request data and begins initial analysis. For text data, it uses natural language processing techniques to extract important keywords and requirements, while for image and video data, it uses image recognition techniques to analyze shapes, colors, patterns, etc.

[1485] Step 3:

[1486] The server uses an emotion engine to analyze user emotions from their input and feedback. Emotional data is extracted based on information obtained from the user's text, voice, images, and videos.

[1487] Step 4:

[1488] The server uses design AI based on the analysis results to generate detailed blueprints. These blueprints include specific instructions such as dimensions, materials, and manufacturing methods. The design AI also takes user sentiment data into consideration to generate blueprints that enhance user satisfaction.

[1489] Step 5:

[1490] The server presents the generated blueprint to the user. The user reviews the blueprint and provides feedback. At this stage, the emotion engine analyzes the user's emotional state in real time, reading emotions from the user's facial expressions and tone of voice.

[1491] Step 6:

[1492] The server revises its design based on user feedback and asks the user for final confirmation. This process can be repeated until the user is satisfied.

[1493] Step 7:

[1494] The server uses a scheduling AI to select the most suitable real-world creator from a database of artisans. Selection criteria include past performance, area of ​​expertise, reviews, and user sentiment data.

[1495] Step 8:

[1496] The server contacts the selected real-world generator and negotiates the terms of work (delivery date, cost, specifications). If an agreement is reached, the server finalizes the work schedule and requests the real-world generator to perform the work.

[1497] Step 9:

[1498] The server monitors the progress of the real-time generation process and receives progress reports. The emotion engine also monitors the user's emotional state in parallel and provides emotional data to the real-time generator as needed. This personalizes the work process and improves quality.

[1499] Step 10:

[1500] If a problem occurs during real-time generation, the server will immediately provide a recovery plan and give appropriate instructions to the craftsmen. For example, this may include providing additional resources or adjusting work procedures.

[1501] Step 11:

[1502] The server takes a picture of the completed real-world product and presents the image to the user for final confirmation. The emotion engine also analyzes the user's emotional state at the time of final confirmation and makes final adjustments if necessary.

[1503] Step 12:

[1504] After receiving final confirmation from the user, the server arranges for the prompt delivery of the finished product to the user. It works in cooperation with the delivery company to ensure the safe and timely delivery of the finished product to the user.

[1505] Step 13:

[1506] After receiving the product, users contribute to improving future services by providing feedback to the server. The emotion engine extracts emotion data from this feedback and uses it to improve the entire system.

[1507] Through the above processing steps, we can provide users with highly satisfying original products, and craftsmen can also gain opportunities to connect with a new customer base.

[1508] (Example 2)

[1509] Next, we will describe Example 2. In the following description, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1510] Traditional systems for translating original designs into actual products were time-consuming and costly, and struggled to accurately reflect user intentions and emotions. Furthermore, the process of negotiating terms with craftsmen, monitoring the progress of the real-world product creation, and resolving problems was cumbersome, resulting in a poor user experience.

[1511] The identification processing performed by the identification processing unit 290 of the data processing device 12 in Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for analyzing the request data using natural language processing or image recognition technology according to the format of the request data, means for generating a detailed design drawing using design AI based on the analyzed data, means for analyzing the user's emotional state and modifying the design drawing based on feedback, means for selecting the optimal real-world producer from a craftsman database using arrangement AI, means for negotiating terms with the selected real-world producer, means for monitoring the progress of real-world production in real time and providing emotional data as needed, means for immediately providing a recovery plan for problems that occur during real-world production, means for presenting the completed real-world product to the user and obtaining confirmation, and means for delivering the finished product to the user. This enables the rapid, low-cost, and high-quality realization of the user's designs and ideas into original products, and allows for a personalized experience that takes the user's emotions into consideration.

[1512] "Request data" refers to information provided by the user to the system, including original designs and ideas, and can be in text, image, or video format.

[1513] "Natural language processing" is a technique for extracting important keywords and requirements from text data, and involves using models such as BERT and GPT-3.

[1514] "Image recognition technology" is a technique that analyzes features such as color, shape, and pattern from image and video data, and involves using software such as the Google Cloud Vision API.

[1515] "Design AI" refers to an artificial intelligence system that automatically generates detailed design drawings based on user request data, and includes tools such as AutoCAD and Blender.

[1516] An "emotion engine" is a system that analyzes emotions from user input and feedback, using technologies such as IBM Watson Tone Analyzer.

[1517] "Arrangement AI" is an artificial intelligence system that selects the most suitable real-world creator from a database of skilled craftsmen.

[1518] A "real-world creator" is a craftsman or business that actually produces goods based on a user's request.

[1519] A "recovery plan" is a set of countermeasures to solve problems that occur during real-time generation.

[1520] This invention enhances the user experience by combining a comprehensive system for realizing users' original designs and ideas in collaboration with real creators (artisans) with an emotion engine that recognizes the user's emotions. This system includes the following main functions:

[1521] Data entry and analysis

[1522] Users upload their original designs and ideas to the system in text, image, and video formats. Data is sent from the user's device to the server. Specifically, a user uploads a sketch of a garment they designed from their device and sends the data by pressing the "Upload" button.

[1523] The server receives the uploaded request data and begins analysis. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image and video data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shapes, colors, patterns, etc. For example, it sends a sketch image of clothing to the Google Cloud Vision API to recognize its color code and shape.

[1524] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. Specifically, Blender is used to generate a 3D model of the clothing, and then dimension and material information is attached based on that model.

[1525] User sentiment analysis using an emotion engine

[1526] The server analyzes user emotions using an emotion engine (e.g., IBM Watson Tone Analyzer) when analyzing user input and feedback. This emotion data is used to generate blueprints and optimize feedback. When users provide text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, that emotion is analyzed.

[1527] Design drawing generation and feedback

[1528] The server presents the generated blueprint to the user and accepts feedback based on their emotional state. For example, if a user submits feedback such as "I want more red in this design," the server uses IBM Watson Tone Analyzer to analyze the emotion and revise the blueprint.

[1529] Selection of real-world creators and negotiation of terms

[1530] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. Specifically, the server queries the craftsman database and requests the appropriate craftsman.

[1531] The server contacts the selected real-world generators and negotiates working conditions (delivery date, cost, specifications). For example, it contacts craftsmen via email to reach an agreement on delivery date and cost.

[1532] Monitoring and management of real-world generation

[1533] The server monitors the progress of real-time creators and receives progress reports. It also monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives progress images from the creators periodically and shares the results of the user's emotional analysis with them.

[1534] The server will immediately provide a recovery plan and give appropriate instructions to the real-world generator in the event of a problem. For example, it will suggest alternative materials if a material shortage occurs.

[1535] Confirmation of finished products and delivery

[1536] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user then reviews the product and reports any necessary corrections. For example, the user might send an image of the finished garment and obtain final approval.

[1537] After the user's final confirmation, the server arranges for the finished product to be delivered to the user as quickly as possible. It works in cooperation with the delivery company to ensure the product is delivered safely and promptly to the user.

[1538] Specific example

[1539] Examples of custom-made clothing

[1540] Users upload sketches and detailed specifications (materials, colors, sizes, etc.) of their designed clothing to the system. The server analyzes this data and extracts the necessary information. A design AI generates detailed blueprints, presents them to the user while analyzing their emotions, and obtains their confirmation. A procurement AI selects the most suitable craftsman, negotiates terms, and begins work. The system monitors the progress in real time and communicates appropriately based on the user's emotions. The completed clothing is then asked for final confirmation from the user, and after confirmation, it is promptly delivered.

[1541] Example of a prompt

[1542] "I've uploaded an original sketch of a garment I designed. The material is cotton, the color is red, and the size is M. Please create a detailed blueprint for this design, analyze my feelings, and provide feedback."

[1543] The flow of the specific processing in Example 2 will be explained using Figure 13.

[1544] Step 1: Data Entry

[1545] Users can upload their original designs and ideas to the system in image, text, and video formats. Generally, users send data to the server using their device. When the "Upload" button is pressed, the selected design data is sent. Specifically, users can upload sketches of clothing or detailed product specifications (material, color, size, etc.) from their device.

[1546] Input: Original designs and ideas from users (images, text, videos)

[1547] Output: Design data sent to the server

[1548] Step 2: Data Analysis

[1549] The server receives the uploaded design data and analyzes it according to its format. For text data, it uses natural language processing techniques (e.g., BERT or GPT-3) to extract important keywords and requirements. For image data, it uses image recognition techniques (e.g., Google Cloud Vision API) to analyze shape, color, and patterns. For video data, it performs image recognition frame by frame.

[1550] Input: Design data sent to the server

[1551] Output: Analysis results (important keywords, shapes, colors, patterns, etc.)

[1552] Step 3: Generate blueprints

[1553] The server uses design AI (e.g., AutoCAD or Blender) to generate detailed design drawings based on the analyzed data. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. For example, Blender is used to generate a 3D model of clothing, and then dimensions and material information is added based on that model.

[1554] Input: Analysis results (important keywords, shapes, colors, patterns, etc.)

[1555] Output: Detailed design drawings (dimensions, materials, manufacturing methods, etc.)

[1556] Step 4: Sentiment Analysis

[1557] The server uses an emotion engine (for example, IBM Watson Tone Analyzer) to analyze emotions based on user input data and feedback. When a user provides text feedback expressing emotions such as "satisfied" or "dissatisfied" with a sketch, the server analyzes that emotion.

[1558] Input: User feedback (text format)

[1559] Output: Emotion analysis results

[1560] Step 5: Feedback and design revisions

[1561] The server presents the generated blueprint to the user and accepts feedback based on the sentiment analysis results. It then modifies the blueprint based on the user's feedback. For example, if a user submits a comment such as, "I want more red in this design," the server analyzes that feedback and adjusts the blueprint again.

[1562] Input: Sentiment analysis results, user feedback

[1563] Output: Revised blueprint

[1564] Step 6: Selecting a Real Generator

[1565] The server uses a scheduling AI to select the most suitable real-world craftsman from a database of craftsmen. Selection criteria include past performance, area of ​​expertise, and reviews. For example, the server queries the craftsman database and lists the most suitable craftsmen.

[1566] Input: Modified blueprint

[1567] Output: Selected real generators

[1568] Step 7: Negotiating Terms

[1569] The server contacts the selected real-world generator and negotiates the working conditions (delivery date, cost, specifications). Specifically, negotiations are conducted via email and telephone until an agreement is reached.

[1570] Input: Selected real generator

[1571] Output: Agreed working conditions (delivery date, cost, specifications)

[1572] Step 8: Monitoring the progress of real generation

[1573] The server monitors the progress of real-time creators and receives progress reports. It also simultaneously monitors the user's emotional state and provides emotional data to the real-time creators as needed to optimize their work. Specifically, it receives regular progress reports from the creators and shares the results of the user's emotional analysis.

[1574] Input: Agreed working conditions

[1575] Output: Progress report, sentiment analysis results

[1576] Step 9: Recovery in case of problems

[1577] If a problem occurs, the server will immediately provide a recovery plan and give appropriate instructions to the real-time generator. For example, if there is a shortage of materials, it will suggest alternative materials or adjust delivery dates.

[1578] Input: Progress report, sentiment analysis results

[1579] Output: Recovery Plan

[1580] Step 10: Check and ship the finished product.

[1581] The server photographs the completed real-world product and presents the image to the user for final confirmation. The user makes the final confirmation and reports any necessary corrections. After the user's final confirmation, the server arranges for the product to be delivered to the user as quickly as possible. Specifically, it arranges for a delivery company to ensure the product is delivered safely and promptly.

[1582] Input: Completed real product

[1583] Output: User's final confirmation, shipping arrangements

[1584] These processing steps enable users to quickly and cost-effectively translate their designs and ideas into high-quality original products, creating personalized experiences that take user emotions into consideration.

[1585] (Application Example 2)

[1586] Next, we will explain application example 2. In the following explanation, the data processing device 12 will be referred to as the "server" and the robot 414 as the "terminal".

[1587] This invention aims to solve the problem of providing high-quality, custom-made products that reflect the user's emotions in a short period of time and at low cost, particularly in the process of bringing original designs and ideas to life in collaboration with real-world creators (craftsmen). Conventional technologies lack a mechanism to adequately reflect the user's emotional feedback, and there is a lack of means to efficiently adjust the progress of real-world creation while monitoring it in real time. As a result, there have been problems in ensuring sufficient user satisfaction and quality for customized orders.

[1588] The specific processing performed by the specific processing unit 290 of the data processing device 12 in Application Example 2 is realized by the following means. In this invention, the server includes means for receiving and analyzing request data from the user, means for generating a design drawing based on the analyzed data, and emotion analysis means for analyzing the user's emotional state. This makes it possible to grasp the user's emotions in real time, generate a design drawing that reflects those emotions, and monitor the progress.

[1589] "Requested data" refers to information provided by users, including designs and ideas, and can take the form of text, images, videos, and other data.

[1590] "Analysis methods" refer to techniques for analyzing requested data and extracting important keywords, shapes, and patterns.

[1591] "Design drawing generation means" refers to a technology that creates specific design drawings based on analyzed data, generating instructions that include dimensions, materials, and processing methods.

[1592] "Sentiment analysis methods" refer to technologies that extract emotional data from user input and feedback and use it to generate and modify design blueprints.

[1593] "Verification methods" refer to technologies that present generated design drawings or finished products to users and receive their verification and feedback.

[1594] A "craftsman database" is a database that accumulates data about craftsmen, including information such as past achievements, areas of expertise, and reviews.

[1595] A "real-world creator" refers to a craftsman who works to bring a user's original design to life.

[1596] "Negotiation methods" refer to techniques for negotiating working conditions such as delivery date, cost, and specifications with selected real-world generators.

[1597] "Progress monitoring means" refers to technology that monitors the work progress of the real-time creator in real time and makes adjustments or recoveries as needed.

[1598] "Problem-solving methods" refer to technologies that identify problems that arise during real-world generation and provide appropriate solutions.

[1599] A "smartphone application interface" is an interface that allows users to input designs and feedback via their smartphones and check the progress of their work.

[1600] System Program Overview

[1601] The program for the system that implements this application example will be explained using the following method.

[1602] Overall flow

[1603] 1. Users upload designs and ideas as request data through a smartphone application interface.

[1604] 2. The server analyzes this request data and generates a design drawing.

[1605] 3. The server analyzes the user's emotional state using emotion analysis tools.

[1606] 4. Present the generated design to the user and receive their feedback.

[1607] 5. The server selects the most suitable real-world generator from the craftsman database and negotiates terms.

[1608] 6. The real-world creator begins work and monitors its progress.

[1609] 7. In the event of a problem, the server will provide a timely recovery solution.

[1610] 8. After the work is completed, present the finished product to the user and obtain their confirmation.

[1611] 9. After final confirmation, the finished product is delivered to the user.

[1612] Hardware and software to be used

[1613] Hardware: Smartphones, business servers, emotion measurement devices (cameras and voice recognition devices)

[1614] Software: Python-based emotion recognition engine (emotion_engine), design AI (design_ai), job handling AI (job_handling_ai)

[1615] Examples of data processing and calculations

[1616] Users upload their designs and ideas as text, images, or videos using a smartphone application interface. This data is analyzed on a server; text data is analyzed using natural language processing to extract key keywords, and image and video data is analyzed using image recognition technology to determine shape, color, and pattern. A design AI generates detailed blueprints including dimensions, materials, and manufacturing methods, and these blueprints are then fed back to the user based on sentiment analysis results from an emotion engine.

[1617] Specific example

[1618] scenario

[1619] Users open the jewelry shop app and upload images and text descriptions of their jewelry designs. The server analyzes this data and uses design AI to generate detailed blueprints. Simultaneously, an emotion engine analyzes the emotions the user is feeling at the time of upload and reflects them in the blueprints. The generated blueprints are presented to the user, who is asked to review and provide feedback via the smartphone interface. Based on the feedback, the server makes necessary improvements and finalizes the blueprints.

[1620] Example of a prompt

[1621] "Please upload your original jewelry design. What materials and manufacturing methods would you prefer for that design? Also, please tell us how you feel about creating this jewelry."

[1622] This system allows users to efficiently request customizable, bespoke products utilizing sentiment analysis and receive high-quality finished products. Furthermore, real-world creators can understand the user's specific requests and emotions as they work, resulting in a mutually satisfying process.

[1623] The flow of a specific process in Application Example 2 will be explained using Figure 14.

[1624] Step 1:

[1625] Users upload their designs and ideas as request data in text, image, or video format using a smartphone application interface. This sends the request data from the user to the server. The input is the user's design data, and the output is the transmission of data to the server. Specifically, the user opens the application, clicks the "Upload New Design" button, selects a file, and uploads it.

[1626] Step 2:

[1627] The server receives the uploaded request data and performs analysis using natural language processing techniques for text data and image recognition techniques for image and video data. The input is design data submitted by the user, and the output is the analysis results (such as important keywords and shape patterns). Specifically, the analysis algorithm running on the server processes the data and passes the analysis results to the next step.

[1628] Step 3:

[1629] The server uses design AI based on the analysis results to generate detailed design drawings. These drawings include specific instructions such as dimensions, materials, and manufacturing methods. The input is the analysis results, and the output is the detailed design drawing. Specifically, the design AI determines design parameters based on the analysis results, generates the design drawing, and saves it.

[1630] Step 4:

[1631] The server uses an emotion engine to analyze the user's emotional state and incorporate this into the design blueprint generation. Inputs are user request data and feedback data, while output is the emotion analysis result. Specifically, the emotion engine executes an emotion recognition algorithm, extracts the user's emotional data, and incorporates it into the blueprint.

[1632] Step 5:

[1633] The generated blueprint is presented to the user from the server, and they are asked to review it via a smartphone application. The input is the detailed blueprint, and the output is the user's feedback. Specifically, the blueprint is displayed in the application interface, and the user presses either the "Confirm" button or the "Request Revision" button.

[1634] Step 6:

[1635] Based on user feedback, the server modifies the design drawings as needed. The input is the user feedback, and the output is the modified design drawings. Specifically, the design AI generates and re-presents the design drawings based on the feedback.

[1636] Step 7:

[1637] The server selects the most suitable real-world craftsman from the craftsman database and negotiates terms. The input is a detailed blueprint, and the output is the work conditions agreed upon with the selected real-world craftsman. Specifically, the sourcing AI searches the database, selects the most suitable craftsman, and saves the results of the negotiation.

[1638] Step 8:

[1639] A selected real-time generator begins work, and the server monitors the progress in real time. Input is work start information, and output is progress data. Specifically, the generator sends progress reports to the server, which processes them.

[1640] Step 9:

[1641] If a problem occurs during the process, the server will analyze it and provide a timely recovery plan. The input is data about the problem that occurred, and the output is the recovery plan. Specifically, the server detects the problem, executes the recovery algorithm, and issues instructions.

[1642] Step 10:

[1643] After the work is completed, the finished product is presented to the user for confirmation. The input is information about the finished product, and the output is the user's final confirmation. Specifically, the server sends an image of the finished product to the user and requests confirmation.

[1644] Step 11:

[1645] After final confirmation, the server arranges for the finished product to be delivered to the user. Input is the user's delivery information, and output is a notification of delivery completion. Specifically, the server sends instructions to the delivery company and notifies the user of the delivery status.

[1646] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the controlled object 443 to output the result of the specific processing. The microphone 238 acquires audio indicating user input for the result of the specific processing. The control unit 46A transmits the audio data indicating user input acquired by the microphone 238 to the data processing unit 12. In the data processing unit 12, the specific processing unit 290 acquires the audio data.

[1647] Data generation model 58 is a type of so-called generative AI (Artificial Intelligence). One example of data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ), Gemini (Internet search) <url: https: gemini.google.com ?hl="ja">Examples of generative AI include the following. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 is input with prompts containing instructions, and with inference data such as audio data representing speech, text data representing text, and image data representing images. The data generation model 58 infers from the input inference data according to the instructions indicated by the prompts, and outputs the inference results in data formats such as audio data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[1648] In the above embodiment, an example was given in which the specific processing is performed by the data processing device 12, but the technology of this disclosure is not limited thereto, and the specific processing may also be performed by the robot 414.

[1649] Furthermore, the emotion identification model 59, acting as an emotion engine, may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to a specific mapping, which is an emotion map (see Figure 9). Similarly, the emotion identification model 59 may also determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[1650] Figure 9 shows an emotion map 400 in which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. The closer to the center of the concentric circles, the more primitive the emotions are located. Further out of the concentric circles, emotions representing states and actions arising from mental states are located. Emotion is a concept that includes feelings and mental states. On the left side of the concentric circles, emotions that are generally generated from reactions occurring in the brain are located. On the right side of the concentric circles, emotions that are generally induced by situational judgment are located. Above and below the concentric circles, emotions that are generally generated from reactions occurring in the brain and induced by situational judgment are located. In addition, the emotion of "pleasure" is located on the upper side of the concentric circles, and the emotion of "displeasure" is located on the lower side. Thus, in the emotion map 400, multiple emotions are mapped based on the structure in which emotions arise, and emotions that are likely to occur simultaneously are mapped close together.

[1651] These emotions are distributed at the 3 o'clock position on the Emotion Map 400, and usually fluctuate between feelings of security and anxiety. In the right half of the Emotion Map 400, situational awareness takes precedence over internal feelings, resulting in a calm impression.

[1652] The inside of the Emotion Map 400 represents inner thoughts, while the outside represents actions. Therefore, the further you go from the outside of the Emotion Map 400, the more visible (expressed in actions) your emotions become.

[1653] Here, human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. Similarly, in robots, cars, motorcycles, etc., emotions can be created based on various balances, such as posture and battery level. When these balances deviate from the ideal, it results in discomfort, and when they approach the ideal, it results in pleasure. The emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on a system for analyzing brain physiological signals of speech emotion recognition and emotion, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map contains emotions belonging to a region called "response," where sensation is dominant. The right half of the emotion map contains emotions belonging to a region called "situation," where situational awareness is dominant.

[1654] The emotion map defines two emotions that promote learning. One is the emotion around the middle of the negative "repentance" and "reflection" on the situation side. In other words, it is when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is the emotion around the positive "desire" on the reaction side. In other words, it is when the robot has positive feelings such as "I want more" or "I want to know more."

[1655] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​representing each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple training data sets, which are combinations of user input and emotion values ​​representing each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions located close together have similar values, as shown in the emotion map 900 in Figure 10. Figure 10 shows an example where multiple emotions such as "reassured," "calm," and "confident" have similar emotion values.

[1656] The above description primarily focuses on the functions of the data processing device 12 in relation to this disclosure. However, the system related to this disclosure is not necessarily implemented on a server. The system related to this disclosure may be implemented as a general information processing system. This disclosure may be implemented, for example, as a software program that runs on a personal computer or as an application that runs on a smartphone. The method related to this disclosure may be provided to users in SaaS (Software as a Service) format.

[1657] In the above embodiment, an example was given in which a specific process is performed by a single computer 22. However, the technology of this disclosure is not limited thereto, and a distributed processing of the specific process may be performed by multiple computers, including computer 22. For example, a data generation model 58 may be provided in an external device of the data processing device 12, and the external device may generate data according to the input data.

[1658] In the above embodiment, an example was given in which the specific processing program 56 is stored in the storage 32, but the technology of this disclosure is not limited thereto. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-temporary storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-temporary storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes specific processing according to the specific processing program 56.

[1659] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[1660] Furthermore, it is not necessary to store the entirety of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store the entirety of the specific processing program 56 in the storage 32; it is acceptable to store only a portion of the specific processing program 56.

[1661] The following types of processors can be used as hardware resources to perform specific processing. Examples of processors include a CPU, a general-purpose processor that functions as a hardware resource to perform specific processing by executing software, i.e., a program. Other examples of processors include dedicated electrical circuits, such as FPGAs (Field-Programmable Gate Arrays), PLDs (Programmable Logic Devices), or ASICs (Application Specific Integrated Circuits), which have circuit configurations specifically designed to perform specific processing. All of these processors have built-in or connected memory, and all of them perform specific processing by using memory.

[1662] The hardware resource that performs a specific process may consist of one of these various processors, or it may consist of a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Alternatively, the hardware resource that performs a specific process may consist of a single processor.

[1663] Examples of configurations using a single processor include, firstly, a configuration in which one or more CPUs and software are combined to form a single processor, and this processor functions as a hardware resource that performs a specific process. Secondly, there is a configuration using a processor that realizes the functions of the entire system, including multiple hardware resources that perform a specific process, on a single IC chip, as exemplified by SoCs (System-on-a-chip). In this way, a specific process is realized using one or more of the above types of processors as hardware resources.

[1664] Furthermore, the hardware structure of these various processors can more specifically utilize electrical circuits that combine circuit elements such as semiconductor devices. Also, the specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps can be deleted, new steps added, or the processing order rearranged, as long as it ...

Claims

1. A means of receiving and analyzing data requested by users, A means for generating design drawings based on analyzed data, A means of presenting the generated design drawings to the user and obtaining their confirmation, A method for selecting the most suitable real-world creator from a database of craftsmen, A means of negotiating terms with the selected real-world creators, A means of monitoring the progress of real generation, A means to solve problems that occur during real generation, A means of presenting the completed real-world product to the user and obtaining their confirmation, A system that includes means for delivering finished products to users.

2. The system according to claim 1, characterized in that the means for generating design drawings generates specific instructions including dimensions, materials, and processing methods.

3. The system according to claim 1, characterized in that the means for monitoring the progress of real generation monitors the craftsman's work process in real time and provides timely recovery measures if a problem occurs.

Images