System

Abstract

An object of a system according to an embodiment is to freely change a code based on a prompt.SOLUTION: A system includes a prompt analysis part, a code generation part, a code change part, and a user interface part. The prompt analysis unit analyzes the prompt. The code generation section generates a code based on the prompt analyzed by the prompt analysis section. The code modification unit modifies the code generated by the code generation unit in real time. The user interface unit confirms and executes the code generated by the code generation unit.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technology of the present disclosure relates to a system. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including an instruction sentence related to a description of the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2022-180282 Summary of the Invention [Problem to be solved by the invention]

[0004] Conventional technologies have had the problem that it is difficult for users to flexibly change the code when using pre-programmed applications.

[0005] The system according to the embodiment aims to enable the code to be freely changed based on prompts. [Means for solving the problem]

[0006] The system according to the embodiment includes a prompt analysis unit, a code generation unit, a code modification unit, and a user interface unit. The prompt analysis unit analyzes a prompt. The code generation unit generates code based on the prompt analyzed by the prompt analysis unit. The code modification unit modifies the code generated by the code generation unit in real time. The user interface unit checks and executes the code generated by the code generation unit. [Effects of the Invention]

[0007] The system according to the embodiment can allow the code to be freely changed based on the prompt. [Brief explanation of the drawings]

[0008] [Figure 1] 1 is a conceptual diagram showing an example of the configuration of a data processing system according to a first embodiment. [Figure 2] 1 is a conceptual diagram showing an example of main functions of a data processing device and a smart device according to a first embodiment. [Figure 3] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a second embodiment. [Figure 4] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and smart glasses according to a second embodiment. [Figure 5] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a third embodiment. [Figure 6] FIG. 11 is a conceptual diagram showing an example of main functions of a data processing device and a headset-type terminal according to a third embodiment. [Figure 7] FIG. 10 is a conceptual diagram showing an example of the configuration of a data processing system according to a fourth embodiment. [Figure 8] FIG. 10 is a conceptual diagram showing an example of main functions of a data processing device and a robot according to a fourth embodiment. [Figure 9] 1 shows an emotion map onto which multiple emotions are mapped. [Figure 10] 1 shows an emotion map onto which multiple emotions are mapped. DETAILED DESCRIPTION OF THE INVENTION

[0009] An example of an embodiment of a system according to the technology of the present disclosure will be described below with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a coded processor (hereinafter simply referred to as a "processor") may be a single arithmetic device or a combination of multiple arithmetic devices. Furthermore, the processor may be a single type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a coded RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by a processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a symbol is an interface including a communication processor, an antenna, etc. The communication I / F controls communication between multiple computers. Examples of communication standards applied to the communication I / F include wireless communication standards including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), and Bluetooth (registered trademark).

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed connected by "and / or."

[0016] [First embodiment] FIG. 1 shows an example of the configuration of a data processing system 10 according to the first embodiment.

[0017] 1, a data processing system 10 includes a data processing device 12 and a smart device 14. An example of the data processing device 12 is a server.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN (Wide Area Network) and / or a LAN (Local Area Network).

[0019] The smart device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (for example, a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 (see FIG. 2) acquires the data indicating the user input.

[0021] Output device 40 includes a display 40A and a speaker 40B, and presents data to a user by outputting the data in a form of expression that the user can perceive (e.g., audio and / or text). Display 40A displays visible information such as text and images in accordance with instructions from processor 46. Speaker 40B outputs audio in accordance with instructions from processor 46. Camera 42 is a compact digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 control the exchange of various information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the smart device 14.

[0024] 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0025] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0026] In the smart device 14, the specific processing is performed by the processor 46. The storage 50 stores a specific processing program 60. The specific processing program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the smart device 14 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0027] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device (e.g., a generation server) may have the data generation model 58. In this case, the data processing device 12 obtains a processing result (prediction result, etc.) using the data generation model 58 by communicating with the server device having the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device owned by a user (e.g., a mobile phone, a robot, a home appliance, etc.). Next, an example of processing by the data processing system 10 according to the first embodiment will be described.

[0028] (Example 1) A code generation system according to an embodiment of the present invention is a system that changes code freely based on prompts. This code generation system eliminates the need for a user to use a pre-programmed application and allows the system to generate and change code in real time based on user instructions. This allows the code generation system to generate and change code in real time based on user prompts.

[0029] A code generation system according to an embodiment includes a prompt analysis unit, a code generation unit, a code modification unit, and a user interface unit. The prompt analysis unit analyzes prompts. For example, the prompt analysis unit analyzes prompts using natural language processing technology and understands their content. The prompt analysis unit can also analyze prompts using a machine learning algorithm. The prompt analysis unit can also analyze prompts in text format, audio format, image format, and the like. The code generation unit generates code based on the prompt analyzed by the prompt analysis unit. For example, the code generation unit generates code using a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, and generates appropriate code based on a user's prompt. The code generation unit can also generate code based on a programming language or a generation algorithm used. The code modification unit modifies the code generated by the code generation unit in real time. For example, the code modification unit adds new functions to existing code based on user instructions. The code modification unit can also analyze existing code and make necessary modifications. The code modification unit can also modify code based on the timing and scope of the modifications. The user interface unit provides an interface for checking and executing the code generated by the code generation unit. For example, the user interface unit may provide an area for entering a prompt in a text box and displaying the generated code. The user interface unit may also provide a button for executing the generated code, allowing the user to easily check the code's operation. The user interface unit may also provide specific functions and display methods, such as a GUI, CLI, or voice interface. This allows the code generation system according to the embodiment to generate and modify code in real time based on user prompts. For example, a user can create an application that meets their needs by simply entering a simple prompt. Furthermore, even professional programmers can quickly generate and modify code, improving development efficiency.

[0030] The prompt analysis unit can generate more accurate code by referencing the user's past prompt history. For example, when the generation AI analyzes a prompt, the prompt analysis unit retrieves the user's past prompt history from a database and references similar prompts to generate more accurate code. For example, if there is a history of generating "code that displays a message when a button is clicked," the AI ​​generates optimal code based on that history. The prompt analysis unit also analyzes the user's past prompt history and extracts frequently used patterns and specific keywords. This allows the generation AI to more accurately understand the user's intent and generate appropriate code. For example, a user who frequently requests "code to connect to a database" can be provided with optimized connection code. The prompt analysis unit also learns the user's preferences and style based on the prompt history and reflects them in the generated code. For example, if a user prefers a specific coding style or library, the AI ​​generates code that matches that style. This allows the AI ​​to generate more accurate code by referencing the past prompt history.

[0031] The prompt analysis unit can ask additional questions in a dialogue format to gather more detailed information in order to gain a deeper understanding of the user's intention. For example, when the generation AI analyzes a prompt, the prompt analysis unit asks additional questions in a dialogue format to gain a deeper understanding of the user's intention. For example, it may ask, "What kind of message do you want to display?" to gather the user's specific requests. Furthermore, while the generation AI is analyzing the prompt, the prompt analysis unit presents options to the user to gather more detailed information. For example, it may ask, "When generating code to connect to a database, please select the type of database to use." Furthermore, when the generation AI analyzes the prompt, the prompt analysis unit sets up a feedback loop to confirm the user's intention. For example, it may ask, "Is this code correct?" and continue the dialogue until it matches the user's intention. In this way, by asking additional questions in a dialogue format, the user's intention can be understood more deeply and more detailed information can be gathered.

[0032] The prompt analysis unit supports voice input, allowing the user to give instructions by voice. For example, when the generation AI analyzes a prompt, the prompt analysis unit supports voice input, allowing the user to give instructions by voice. For example, a voice instruction such as "Generate code that displays a message when the button is clicked" is given. The prompt analysis unit also uses voice recognition technology to convert the user's voice instructions into text, and the generation AI analyzes the text and generates code. For example, a voice instruction such as "Generate code to connect to a database" is given. The prompt analysis unit also supports voice input, allowing the user to enter prompts without using their hands. For example, even if their hands are busy while programming, they can generate code by giving voice instructions. This means that by supporting voice input, the user can give instructions by voice.

[0033] The prompt analysis unit can visually display the results of prompt analysis, allowing the user to intuitively understand the content of the code to be generated. The prompt analysis unit, for example, visually displays the results of prompt analysis by the generation AI, allowing the user to intuitively understand the content of the code to be generated. For example, it illustrates the structure of the code. The prompt analysis unit also visually displays the results of prompt analysis, making it easier for the user to understand the flow of the code to be generated. For example, it displays a flowchart or sequence diagram. The prompt analysis unit also uses visual displays to allow the user to check each part of the code to be generated. For example, it displays a pop-up explanation corresponding to each line of code. In this way, by visually displaying the results of prompt analysis, the user can intuitively understand the content of the code to be generated.

[0034] When modifying code in real time, the code modification unit can visually display the differences in code before and after the change, allowing the user to easily understand the changes. For example, when the generation AI modifies code in real time, the code modification unit visually displays the differences in code before and after the change. For example, the code modification unit highlights the changed parts so that the user can understand the changes at a glance. The code modification unit also displays the code before and after the change side by side so that the user can compare the changes. For example, the code before the change is displayed on the left and the code after the change is displayed on the right. The code modification unit also visually displays the changes so that the user can easily understand the impact of the changes. For example, comments are added to the changed parts to explain the reason for the change. In this way, the differences in code before and after the change are visually displayed so that the user can easily understand the changes.

[0035] The code modification unit can automatically analyze the scope of impact of a change when changing code in real time and simultaneously update related parts of the code. For example, when the generation AI changes code in real time, the code modification unit automatically analyzes the scope of impact of the change and simultaneously updates related parts of the code. For example, if a function argument is changed, all locations that call the function are automatically updated. The code modification unit also analyzes the scope of impact of the change and automatically modifies related parts of the code. For example, if a variable name is changed, all locations that use that variable are automatically updated. The code modification unit also analyzes the scope of impact of the change and simultaneously updates related parts of the code, saving the user the trouble of having to make manual corrections. For example, if a class name is changed, all locations that use that class are automatically updated. This automatically analyzes the scope of impact of the change and simultaneously updates related parts of the code, reducing the user's effort.

[0036] The code modification unit can enable the user to specify changes by voice when modifying code in real time. For example, the code modification unit enables the user to specify changes by voice when the generation AI modifies code in real time. For example, the user can specify changes by voice, such as "Add a new argument to this function." The code modification unit also uses voice recognition technology to convert the user's voice instructions into text, and the generation AI analyzes the text and modifies the code. For example, the user can specify changes by voice, such as "Change this variable name." The code modification unit also supports voice input, allowing the user to specify changes to code without using their hands. For example, even if their hands are busy programming, they can modify the code by giving voice instructions. This allows the user to easily modify code by specifying changes by voice.

[0037] The code modification unit can instantly test the results of code changes made in real time in a simulation environment, allowing the user to confirm the impact of the changes. For example, after the generation AI changes the code in real time, the code modification unit instantly tests the results in a simulation environment, allowing the user to confirm the impact of the changes. For example, the modified code is executed and its operation is confirmed. The code modification unit also uses the simulation environment to test the results of the real-time changes, allowing the user to visually confirm the impact of the changes. For example, the operation of the modified code is displayed in a graph or chart. The code modification unit also tests the results of the real-time changes in a simulation environment, allowing the user to instantly confirm the impact of the changes. For example, it checks whether the modified code operates correctly. This allows the user to confirm the impact of the changes by immediately testing in the simulation environment.

[0038] The user interface unit displays detailed explanations for each part of the generated code, making it easier for the user to understand the meaning of the code. The user interface unit, for example, displays detailed explanations for each part of the generated code. For example, it displays an explanation corresponding to each line of code in a pop-up, making it easier for the user to understand the meaning of the code. The user interface unit also displays detailed explanations for each part of the generated code, making it easier for the user to understand the operation of the code. For example, it displays tooltips explaining the role of functions and the uses of variables. The user interface unit also displays detailed explanations for each part of the generated code, making it easier for even beginners to program to understand the content of the code. For example, it displays comments for each line of code. In this way, displaying detailed explanations for each part of the generated code makes it easier for the user to understand the meaning of the code.

[0039] The user interface unit can add a version control function for the generated code, allowing it to be compared with past versions. The user interface unit, for example, adds a version control function for the generated code, allowing it to be compared with past versions. For example, the user interface unit displays a change history and compares past versions with the current version. The user interface unit also uses the version control function to allow the user to revert to a past version of the generated code. For example, the user interface unit provides a function to select and restore a specific version. The user interface unit also adds a version control function for the generated code, visually displaying changes. For example, the user interface unit highlights the changed parts and compares them with past versions. In this way, adding the version control function for the generated code allows it to be compared with past versions.

[0040] The user interface unit can add a voice assistant function to enable the user to operate the interface by voice. The user interface unit can add a voice assistant function to enable the user to operate the interface by voice. For example, a voice command such as "Generate code." The user interface unit can also use voice recognition technology to convert the user's voice command into text, and the generation AI analyzes the text and executes the operation. For example, a voice command such as "Delete this function." The user interface unit can also add a voice assistant function to enable the user to operate the interface without using their hands. For example, even if their hands are busy while programming, they can issue voice commands to perform operations. In this way, adding a voice assistant function enables the user to operate the interface by voice.

[0041] The user interface unit can add a function to visually simulate the behavior of the generated code, allowing the user to intuitively understand the behavior of the code. The user interface unit, for example, adds a function to visually simulate the behavior of the generated code, allowing the user to intuitively understand the behavior of the code. For example, the execution results of the code are displayed in a graph or animation. The user interface unit also uses a simulation function to allow the user to visually confirm the behavior of the generated code. For example, the execution flow of the code is displayed in a flowchart. The user interface unit also adds a function to visually simulate the behavior of the generated code, making it easier for the user to understand the behavior of the code. For example, each step of the code is displayed in an animation. In this way, by visually simulating the behavior of the generated code, the user can intuitively understand the behavior of the code.

[0042] When optimizing code generated by the generation AI, the code optimization unit can refer to past optimization history and select the optimal optimization method. For example, when the generation AI optimizes code, the code optimization unit retrieves past optimization history from a database and selects the optimal optimization method. For example, it optimizes code based on optimization methods that have been successful in the past. The code optimization unit also analyzes past optimization history and extracts frequently used optimization methods. This allows the generation AI to select the optimal method and efficiently optimize the code. For example, it applies the optimal optimization method for a specific pattern. The code optimization unit also allows the generation AI to select the optimal method based on the optimization history and improve code performance. For example, it selects the optimal algorithm based on past data and optimizes the code. This allows the optimal optimization method to be selected by referring to past optimization history.

[0043] The code optimization unit allows the generation AI to explain the cause of an error in detail when performing an error check, making it easier for users to understand the error. For example, the code optimization unit displays an explanation such as, "This error occurs because the variable types do not match." The code optimization unit also provides detailed explanations of the results of the error check, making it easier for users to understand the cause of the error. For example, it highlights the location of the error and displays a detailed explanation of the error in a pop-up. The code optimization unit also provides detailed explanations of the cause of the error when the generation AI performs an error check, making it easier for even beginners to understand the error. For example, it provides a step-by-step explanation of the cause of the error and how to fix it. This detailed explanation makes it easier for users to understand the error.

[0044] The code optimization unit can visually display the results of code optimization and error checking, allowing the user to intuitively understand the results. For example, the code optimization unit visually displays the results of code optimization and error checking performed by the generation AI, allowing the user to intuitively understand the results. For example, it highlights optimized parts and error parts. The code optimization unit also visually displays the results of optimization and error checking, making it easier for the user to understand the results. For example, it displays the effects of optimization in graphs and charts. The code optimization unit also visually displays the results of optimization and error checking in the user interface, making it easier for even beginners to understand the results. For example, it illustrates how to correct error parts. In this way, by visually displaying the results of code optimization and error checking, the user can intuitively understand the results.

[0045] The code optimization unit can notify the user of the results of code optimization and error checking by voice, allowing the user to easily check the results. For example, the code optimization unit can notify the user of the results of code optimization and error checking performed by the generation AI by voice, allowing the user to easily check the results. For example, the code optimization unit can notify the user by voice, saying, "Optimization is complete." The code optimization unit can also use a voice notification function to allow the user to check the results of optimization and error checking without using their hands. For example, the code optimization unit can notify the user by voice, saying, "Errors were found." The code optimization unit can also add a voice notification function to allow the user to check the results of optimization and error checking while performing other tasks. For example, the code optimization unit can notify the user by voice, saying, "Code optimization is complete. No errors were found." This allows the user to easily check the results of code optimization and error checking by voice.

[0046] When the multilingual support unit supports multiple languages ​​for code generated by the generation AI, the multilingual support unit can automatically select the optimal language by referring to the user's past language selection history. For example, when the generation AI supports multiple languages, the multilingual support unit retrieves the user's past language selection history from a database and automatically selects the optimal language. For example, if the user has previously selected Python, the multilingual support unit preferentially selects Python. The multilingual support unit also analyzes the user's past language selection history and extracts frequently used languages. This allows the generation AI to automatically select a language that matches the user's preferences. For example, for a user who frequently uses JavaScript, the multilingual support unit preferentially selects JavaScript. The multilingual support unit also learns the user's preferences and style based on the language selection history and reflects them in the generated code. For example, for a user who prefers a specific programming language, the multilingual support unit generates code that matches that language. This allows the optimal language to be automatically selected by referring to the user's past language selection history.

[0047] The multilingual support unit can generate optimal code by taking into account the characteristics of each language when making the code generated by the generation AI multilingual. For example, when the generation AI performs multilingual support, the multilingual support unit takes into account the characteristics of each language to generate optimal code. For example, it takes into account the characteristics of Python to generate concise, easy-to-read code. The multilingual support unit also analyzes the characteristics of each language, and the generation AI generates optimal code. For example, it takes into account the asynchronous processing of JavaScript to generate appropriate asynchronous code. The multilingual support unit also takes into account the characteristics of each language when the generation AI performs multilingual support, generating code that is easy for users to understand. For example, it generates code that makes use of the object-oriented characteristics of Java. This allows the generation of optimal code by taking into account the characteristics of each language.

[0048] The multilingual support unit can visually display the results of multilingualization of code generated by the generation AI, allowing the user to intuitively understand the code in the language selected by the user. The multilingual support unit, for example, visually displays the results of multilingualization performed by the generation AI, allowing the user to intuitively understand the code in the language selected by the user. For example, it illustrates the structure of the code. Furthermore, by visually displaying the results of multilingualization, the multilingual support unit makes it easier for the user to understand the flow of the code in the language selected by the user. For example, it displays a flowchart or sequence diagram. Furthermore, the multilingual support unit uses visual displays to allow the user to check each part of the code in the language selected by the user. For example, it displays a pop-up with an explanation corresponding to each line of code. In this way, by visually displaying the results of multilingualization, the user can intuitively understand the code in the language selected by the user.

[0049] The multilingual support unit can notify the user by voice of the results of multilingualization of code generated by the generation AI, allowing the user to easily check the results. For example, the multilingual support unit can notify the user by voice of the results of multilingualization performed by the generation AI, allowing the user to easily check the results. For example, the multilingual support unit can notify the user by voice, saying, "Code has been generated in Python." The multilingual support unit can also use a voice notification function to allow the user to check the results of multilingualization without using their hands. For example, the multilingual support unit can notify the user by voice, saying, "Code has been generated in JavaScript." The multilingual support unit can also add a voice notification function to allow the user to check the results of multilingualization while performing other tasks. For example, the multilingual support unit can notify the user by voice, saying, "Code generation is complete. The selected language is Java." This allows the user to easily check the results by notifying the user by voice of the results of multilingualization.

[0050] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0051] The code generation system can further include a security check unit. The security check unit can detect security vulnerabilities in the generated code and suggest modifications. For example, it can detect code that poses a risk of SQL injection and suggest appropriate escaping processing. The security check unit can also check whether the generated code complies with the latest security guidelines and suggest modifications as necessary. For example, if an old encryption algorithm is used, it can suggest changing to a more secure algorithm. The security check unit can also automatically run security tests on the generated code and provide the test results to the user. This can increase the security of the generated code.

[0052] The code generation system may further include a performance optimization unit. The performance optimization unit can analyze the performance of the generated code and make optimization suggestions. For example, it can suggest changes to an algorithm to reduce the number of loops. The performance optimization unit can also measure the execution time of the generated code and identify bottlenecks. For example, if a particular function takes a long time to execute, it can suggest how to optimize that function. The performance optimization unit can also automatically run performance tests on the generated code and provide the test results to the user. This can improve the performance of the generated code.

[0053] The code generation system may further include a debugging support unit. The debugging support unit can automatically add debugging information to the generated code, making it easier for users to debug the code. For example, it can set breakpoints for each line of code and display variable values ​​during execution. The debugging support unit can also automatically generate unit tests for the generated code and provide the test results to the user. For example, it can generate a test case for each function and display the execution results. The debugging support unit can also automatically generate an error log for the generated code and provide information for identifying the cause of the error. This makes it easier for users to debug the generated code.

[0054] The code generation system may further include a code review support unit. The code review support unit may automatically perform a code review on the generated code and suggest improvements. For example, it may check the consistency of the coding style and suggest modifications in accordance with a style guide. The code review support unit may also automatically perform a quality evaluation on the generated code and provide the evaluation results to the user. For example, it may evaluate the readability and maintainability of the code and suggest improvements. The code review support unit may also automatically support pair programming on the generated code, enabling the user to review the code collaboratively with other programmers. This may improve the quality of the generated code.

[0055] The code generation system may further include a documentation generation unit. The documentation generation unit may automatically generate documentation for the generated code, making it easier for users to understand the contents of the code. For example, detailed descriptions may be added for each function, and usage examples may be provided. The documentation generation unit may also automatically generate API documentation for the generated code, making it easier for users to understand how to use the API. For example, descriptions of each API endpoint and examples of requests and responses may be provided. The documentation generation unit may also automatically generate tutorials for the generated code, making it easier for users to learn how to use the code. This may help users understand the generated code.

[0056] The processing flow of the first embodiment will be briefly explained below.

[0057] Step 1: The prompt analyzer analyzes the prompt. For example, the prompt analyzer uses natural language processing technology or machine learning algorithms to analyze the prompt and understand its content. The prompt analyzer can also analyze prompts in text format, audio format, image format, and other formats. Step 2: The code generation unit generates code based on the prompt analyzed by the prompt analysis unit. For example, the code generation unit generates code using a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, and generates appropriate code based on the user's prompt. The code generation unit may also generate code based on the programming language or generation algorithm used. Step 3: The code modification unit modifies the code generated by the code generation unit in real time. For example, the code modification unit adds new functions to existing code based on user instructions. The code modification unit can also analyze existing code and make necessary changes. The code modification unit can also modify the code based on the timing and scope of the changes. Step 4: The user interface unit provides an interface for checking and executing the code generated by the code generation unit. For example, the user interface unit may provide an area where a prompt can be entered in a text box and the generated code is displayed. The user interface unit may also provide a button for executing the generated code, allowing the user to easily check the code's operation. The user interface unit may also provide specific functions and display methods such as a GUI, CLI, or voice interface.

[0058] (Example 2) A code generation system according to an embodiment of the present invention is a system that changes code freely based on prompts. This code generation system eliminates the need for a user to use a pre-programmed application and allows the system to generate and change code in real time based on user instructions. This allows the code generation system to generate and change code in real time based on user prompts.

[0059] A code generation system according to an embodiment includes a prompt analysis unit, a code generation unit, a code modification unit, and a user interface unit. The prompt analysis unit analyzes prompts. For example, the prompt analysis unit analyzes prompts using natural language processing technology and understands their content. The prompt analysis unit can also analyze prompts using a machine learning algorithm. The prompt analysis unit can also analyze prompts in text format, audio format, image format, and the like. The code generation unit generates code based on the prompt analyzed by the prompt analysis unit. For example, the code generation unit generates code using a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, and generates appropriate code based on a user's prompt. The code generation unit can also generate code based on a programming language or a generation algorithm used. The code modification unit modifies the code generated by the code generation unit in real time. For example, the code modification unit adds new functions to existing code based on user instructions. The code modification unit can also analyze existing code and make necessary modifications. The code modification unit can also modify code based on the timing and scope of the modifications. The user interface unit provides an interface for checking and executing the code generated by the code generation unit. For example, the user interface unit may provide an area for entering a prompt in a text box and displaying the generated code. The user interface unit may also provide a button for executing the generated code, allowing the user to easily check the code's operation. The user interface unit may also provide specific functions and display methods, such as a GUI, CLI, or voice interface. This allows the code generation system according to the embodiment to generate and modify code in real time based on user prompts. For example, a user can create an application that meets their needs by simply entering a simple prompt. Furthermore, even professional programmers can quickly generate and modify code, improving development efficiency.

[0060] The prompt analysis unit can generate more accurate code by referencing the user's past prompt history. For example, when the generation AI analyzes a prompt, the prompt analysis unit retrieves the user's past prompt history from a database and references similar prompts to generate more accurate code. For example, if there is a history of generating "code that displays a message when a button is clicked," the AI ​​generates optimal code based on that history. The prompt analysis unit also analyzes the user's past prompt history and extracts frequently used patterns and specific keywords. This allows the generation AI to more accurately understand the user's intent and generate appropriate code. For example, a user who frequently requests "code to connect to a database" can be provided with optimized connection code. The prompt analysis unit also learns the user's preferences and style based on the prompt history and reflects them in the generated code. For example, if a user prefers a specific coding style or library, the AI ​​generates code that matches that style. This allows the AI ​​to generate more accurate code by referencing the past prompt history.

[0061] The prompt analysis unit can ask additional questions in a dialogue format to gather more detailed information in order to gain a deeper understanding of the user's intention. For example, when the generation AI analyzes a prompt, the prompt analysis unit asks additional questions in a dialogue format to gain a deeper understanding of the user's intention. For example, it may ask, "What kind of message do you want to display?" to gather the user's specific requests. Furthermore, while the generation AI is analyzing the prompt, the prompt analysis unit presents options to the user to gather more detailed information. For example, it may ask, "When generating code to connect to a database, please select the type of database to use." Furthermore, when the generation AI analyzes the prompt, the prompt analysis unit sets up a feedback loop to confirm the user's intention. For example, it may ask, "Is this code correct?" and continue the dialogue until it matches the user's intention. In this way, by asking additional questions in a dialogue format, the user's intention can be understood more deeply and more detailed information can be gathered.

[0062] The prompt analysis unit uses the emotion estimation function to analyze the user's emotional state and generate code that is simpler and easier to understand if the user is feeling stressed. For example, when the generation AI analyzes prompts, the prompt analysis unit analyzes the user's emotional state in real time and generates code that is simpler and easier to understand if the user is feeling stressed. For example, it provides code that simplifies complex algorithms. The prompt analysis unit also uses the emotion estimation function to enable the generation AI to enrich code comments and add easy-to-understand explanations if the user is feeling stressed. For example, it adds detailed comments to each line of code. The prompt analysis unit also analyzes the user's emotional state and, if the user is feeling stressed, the generation AI supports code generation in an interactive format. For example, it makes suggestions such as, "Shall I explain this part of the code in more detail?" This allows the generation AI to analyze the user's emotional state and generate code that is simpler and easier to understand if the user is feeling stressed.

[0063] The prompt analysis unit supports voice input, allowing the user to give instructions by voice. For example, when the generation AI analyzes a prompt, the prompt analysis unit supports voice input, allowing the user to give instructions by voice. For example, a voice instruction such as "Generate code that displays a message when the button is clicked" is given. The prompt analysis unit also uses voice recognition technology to convert the user's voice instructions into text, and the generation AI analyzes the text and generates code. For example, a voice instruction such as "Generate code to connect to a database" is given. The prompt analysis unit also supports voice input, allowing the user to enter prompts without using their hands. For example, even if their hands are busy while programming, they can generate code by giving voice instructions. This means that by supporting voice input, the user can give instructions by voice.

[0064] The prompt analysis unit can visually display the results of prompt analysis, allowing the user to intuitively understand the content of the code to be generated. The prompt analysis unit, for example, visually displays the results of prompt analysis by the generation AI, allowing the user to intuitively understand the content of the code to be generated. For example, it illustrates the structure of the code. The prompt analysis unit also visually displays the results of prompt analysis, making it easier for the user to understand the flow of the code to be generated. For example, it displays a flowchart or sequence diagram. The prompt analysis unit also uses visual displays to allow the user to check each part of the code to be generated. For example, it displays a pop-up explanation corresponding to each line of code. In this way, by visually displaying the results of prompt analysis, the user can intuitively understand the content of the code to be generated.

[0065] The prompt analysis unit uses the emotion estimation function to analyze the emotion a user has when entering a prompt in real time and provide feedback to elicit positive emotions. The prompt analysis unit, for example, uses the emotion estimation function to analyze the emotion a user has when entering a prompt in real time and provide feedback to elicit positive emotions. For example, it displays an encouraging message such as "That's a great idea!" The prompt analysis unit also analyzes the user's emotional state, and if negative emotions are detected, the generation AI provides positive feedback. For example, it displays a message such as "Let's try a little harder!" The prompt analysis unit also uses the emotion estimation function to provide an interface to elicit positive emotions when the user enters a prompt. For example, it displays appropriate feedback in real time in response to the user's input. This makes it possible to analyze the emotion a user has when entering a prompt in real time and provide feedback to elicit positive emotions.

[0066] When modifying code in real time, the code modification unit can visually display the differences in code before and after the change, allowing the user to easily understand the changes. For example, when the generation AI modifies code in real time, the code modification unit visually displays the differences in code before and after the change. For example, the code modification unit highlights the changed parts so that the user can understand the changes at a glance. The code modification unit also displays the code before and after the change side by side so that the user can compare the changes. For example, the code before the change is displayed on the left and the code after the change is displayed on the right. The code modification unit also visually displays the changes so that the user can easily understand the impact of the changes. For example, comments are added to the changed parts to explain the reason for the change. In this way, the differences in code before and after the change are visually displayed so that the user can easily understand the changes.

[0067] The code modification unit can automatically analyze the scope of impact of a change when changing code in real time and simultaneously update related parts of the code. For example, when the generation AI changes code in real time, the code modification unit automatically analyzes the scope of impact of the change and simultaneously updates related parts of the code. For example, if a function argument is changed, all locations that call the function are automatically updated. The code modification unit also analyzes the scope of impact of the change and automatically modifies related parts of the code. For example, if a variable name is changed, all locations that use that variable are automatically updated. The code modification unit also analyzes the scope of impact of the change and simultaneously updates related parts of the code, saving the user the trouble of having to make manual corrections. For example, if a class name is changed, all locations that use that class are automatically updated. This automatically analyzes the scope of impact of the change and simultaneously updates related parts of the code, reducing the user's effort.

[0068] The code modification unit uses the emotion estimation function to analyze the user's emotions when instructing a change, and if the user feels anxious, the code modification unit can provide a detailed explanation of the changes. For example, the code modification unit can use the emotion estimation function to analyze the user's emotions when instructing a change, and if the user feels anxious, the generation AI can provide a detailed explanation of the changes. For example, it can explain how the changes will affect the code's behavior. The code modification unit can also analyze the user's emotional state, and if the user feels anxious, the generation AI can provide a detailed document of the changes. For example, it can display a document that explains the changes and the scope of their impact. The code modification unit can also use the emotion estimation function to explain the changes in an interactive format if the user feels anxious when instructing a change. For example, it can provide a detailed explanation in response to a question such as, "Is this change safe?" This can help the user understand the changes if the user feels anxious by providing a detailed explanation of the changes.

[0069] The code modification unit can enable the user to specify changes by voice when modifying code in real time. For example, the code modification unit enables the user to specify changes by voice when the generation AI modifies code in real time. For example, the user can specify changes by voice, such as "Add a new argument to this function." The code modification unit also uses voice recognition technology to convert the user's voice instructions into text, and the generation AI analyzes the text and modifies the code. For example, the user can specify changes by voice, such as "Change this variable name." The code modification unit also supports voice input, allowing the user to specify changes to code without using their hands. For example, even if their hands are busy programming, they can modify the code by giving voice instructions. This allows the user to easily modify code by specifying changes by voice.

[0070] The code modification unit can instantly test the results of code changes made in real time in a simulation environment, allowing the user to confirm the impact of the changes. For example, after the generation AI changes the code in real time, the code modification unit instantly tests the results in a simulation environment, allowing the user to confirm the impact of the changes. For example, the modified code is executed and its operation is confirmed. The code modification unit also uses the simulation environment to test the results of the real-time changes, allowing the user to visually confirm the impact of the changes. For example, the operation of the modified code is displayed in a graph or chart. The code modification unit also tests the results of the real-time changes in a simulation environment, allowing the user to instantly confirm the impact of the changes. For example, it checks whether the modified code operates correctly. This allows the user to confirm the impact of the changes by immediately testing in the simulation environment.

[0071] The code modification unit can use the emotion estimation function to analyze the user's emotions in real time when making changes and provide feedback to elicit positive emotions. The code modification unit, for example, uses the emotion estimation function to analyze the user's emotions in real time when making changes and provide feedback to elicit positive emotions. For example, it can display an encouraging message such as, "What a great change!" The code modification unit also analyzes the user's emotional state, and if negative emotions are detected, the generation AI provides positive feedback. For example, it can display a message such as, "This change is going in a very good direction." The code modification unit also uses the emotion estimation function to provide an interface to elicit positive emotions when the user makes changes. For example, it can display appropriate feedback in real time in response to the user's input. This makes it possible to analyze the user's emotions in real time when making changes and provide feedback to elicit positive emotions.

[0072] The user interface unit displays detailed explanations for each part of the generated code, making it easier for the user to understand the meaning of the code. The user interface unit, for example, displays detailed explanations for each part of the generated code. For example, it displays an explanation corresponding to each line of code in a pop-up, making it easier for the user to understand the meaning of the code. The user interface unit also displays detailed explanations for each part of the generated code, making it easier for the user to understand the operation of the code. For example, it displays tooltips explaining the role of functions and the uses of variables. The user interface unit also displays detailed explanations for each part of the generated code, making it easier for even beginners to program to understand the content of the code. For example, it displays comments for each line of code. In this way, displaying detailed explanations for each part of the generated code makes it easier for the user to understand the meaning of the code.

[0073] The user interface unit can add a version control function for the generated code, allowing it to be compared with past versions. The user interface unit, for example, adds a version control function for the generated code, allowing it to be compared with past versions. For example, the user interface unit displays a change history and compares past versions with the current version. The user interface unit also uses the version control function to allow the user to revert to a past version of the generated code. For example, the user interface unit provides a function to select and restore a specific version. The user interface unit also adds a version control function for the generated code, visually displaying changes. For example, the user interface unit highlights the changed parts and compares them with past versions. In this way, adding the version control function for the generated code allows it to be compared with past versions.

[0074] The user interface unit uses the emotion estimation function to analyze the emotion of the user when operating the interface, and can simplify the interface if the user is feeling stressed. For example, the user interface unit uses the emotion estimation function to analyze the emotion of the user when operating the interface, and can simplify the interface if the user is feeling stressed. For example, unnecessary elements are hidden to simplify operation. The user interface unit also analyzes the user's emotional state, and if the user is feeling stressed, the generation AI changes the layout of the interface. For example, important information is placed prominently to make operation intuitive. The user interface unit also uses the emotion estimation function to provide support in an interactive format if the user is feeling stressed when operating the interface. For example, it asks questions such as, "Which part is difficult to understand?" As a result, if the user is feeling stressed, the interface is simplified to make operation easier.

[0075] The user interface unit can add a voice assistant function to enable the user to operate the interface by voice. The user interface unit can add a voice assistant function to enable the user to operate the interface by voice. For example, a voice command such as "Generate code." The user interface unit can also use voice recognition technology to convert the user's voice command into text, and the generation AI analyzes the text and executes the operation. For example, a voice command such as "Delete this function." The user interface unit can also add a voice assistant function to enable the user to operate the interface without using their hands. For example, even if their hands are busy while programming, they can issue voice commands to perform operations. In this way, adding a voice assistant function enables the user to operate the interface by voice.

[0076] The user interface unit can add a function to visually simulate the behavior of the generated code, allowing the user to intuitively understand the behavior of the code. The user interface unit, for example, adds a function to visually simulate the behavior of the generated code, allowing the user to intuitively understand the behavior of the code. For example, the execution results of the code are displayed in a graph or animation. The user interface unit also uses a simulation function to allow the user to visually confirm the behavior of the generated code. For example, the execution flow of the code is displayed in a flowchart. The user interface unit also adds a function to visually simulate the behavior of the generated code, making it easier for the user to understand the behavior of the code. For example, each step of the code is displayed in an animation. In this way, by visually simulating the behavior of the generated code, the user can intuitively understand the behavior of the code.

[0077] The user interface unit can use the emotion estimation function to analyze the emotions of the user when operating the interface in real time and provide feedback to elicit positive emotions. The user interface unit, for example, uses the emotion estimation function to analyze the emotions of the user when operating the interface in real time and provide feedback to elicit positive emotions. For example, it displays an encouraging message such as "Great operation!" The user interface unit also analyzes the user's emotional state, and if a negative emotion is detected, the generation AI provides positive feedback. For example, it displays a message such as "This operation is going in a very good direction." The user interface unit also uses the emotion estimation function to provide an interface to elicit positive emotions when the user operates the interface. For example, it displays appropriate feedback in real time in response to the user's input. This makes it possible to analyze the emotions of the user when operating the interface in real time and provide feedback to elicit positive emotions.

[0078] When optimizing code generated by the generation AI, the code optimization unit can refer to past optimization history and select the optimal optimization method. For example, when the generation AI optimizes code, the code optimization unit retrieves past optimization history from a database and selects the optimal optimization method. For example, it optimizes code based on optimization methods that have been successful in the past. The code optimization unit also analyzes past optimization history and extracts frequently used optimization methods. This allows the generation AI to select the optimal method and efficiently optimize the code. For example, it applies the optimal optimization method for a specific pattern. The code optimization unit also allows the generation AI to select the optimal method based on the optimization history and improve code performance. For example, it selects the optimal algorithm based on past data and optimizes the code. This allows the optimal optimization method to be selected by referring to past optimization history.

[0079] The code optimization unit allows the generation AI to explain the cause of an error in detail when performing an error check, making it easier for users to understand the error. For example, the code optimization unit displays an explanation such as, "This error occurs because the variable types do not match." The code optimization unit also provides detailed explanations of the results of the error check, making it easier for users to understand the cause of the error. For example, it highlights the location of the error and displays a detailed explanation of the error in a pop-up. The code optimization unit also provides detailed explanations of the cause of the error when the generation AI performs an error check, making it easier for even beginners to understand the error. For example, it provides a step-by-step explanation of the cause of the error and how to fix it. This detailed explanation makes it easier for users to understand the error.

[0080] The code optimization unit uses the emotion estimation function to analyze the user's emotions when performing error checking, and if the user feels anxious, it can provide detailed explanations on how to correct the error. For example, the code optimization unit uses the emotion estimation function to analyze the user's emotions when performing error checking, and if the user feels anxious, the generation AI provides detailed explanations on how to correct the error. For example, it displays an explanation such as, "To correct this error, you need to match the variable types." The code optimization unit also analyzes the user's emotional state, and if the user feels anxious, the generation AI provides detailed documentation on how to correct the error. For example, it displays a document that explains the location of the error and the correction procedure. The code optimization unit also uses the emotion estimation function to explain how to correct the error in an interactive format if the user feels anxious when performing error checking. For example, it provides detailed explanations in response to questions such as, "Which part do I need to change to correct this error?" This helps the user understand the problem by providing detailed explanations on how to correct the error if the user feels anxious.

[0081] The code optimization unit can visually display the results of code optimization and error checking, allowing the user to intuitively understand the results. For example, the code optimization unit visually displays the results of code optimization and error checking performed by the generation AI, allowing the user to intuitively understand the results. For example, it highlights optimized parts and error parts. The code optimization unit also visually displays the results of optimization and error checking, making it easier for the user to understand the results. For example, it displays the effects of optimization in graphs and charts. The code optimization unit also visually displays the results of optimization and error checking in the user interface, making it easier for even beginners to understand the results. For example, it illustrates how to correct error parts. In this way, by visually displaying the results of code optimization and error checking, the user can intuitively understand the results.

[0082] The code optimization unit can notify the user of the results of code optimization and error checking by voice, allowing the user to easily check the results. For example, the code optimization unit can notify the user of the results of code optimization and error checking performed by the generation AI by voice, allowing the user to easily check the results. For example, the code optimization unit can notify the user by voice, saying, "Optimization is complete." The code optimization unit can also use a voice notification function to allow the user to check the results of optimization and error checking without using their hands. For example, the code optimization unit can notify the user by voice, saying, "Errors were found." The code optimization unit can also add a voice notification function to allow the user to check the results of optimization and error checking while performing other tasks. For example, the code optimization unit can notify the user by voice, saying, "Code optimization is complete. No errors were found." This allows the user to easily check the results of code optimization and error checking by voice.

[0083] The code optimization unit can use the emotion estimation function to analyze the user's emotions in real time when performing error checking and provide feedback to elicit positive emotions. For example, the code optimization unit can use the emotion estimation function to analyze the user's emotions in real time when performing error checking and provide feedback to elicit positive emotions. For example, the code optimization unit can display an encouraging message such as, "It's great that you found an error!". The code optimization unit can also analyze the user's emotional state, and if negative emotions are detected, the generation AI can provide positive feedback. For example, the code optimization unit can display a message such as, "This error can be easily fixed." The code optimization unit can also use the emotion estimation function to provide an interface to elicit positive emotions when the user performs error checking. For example, the code optimization unit can display appropriate feedback in real time in response to the user's input. This allows the code optimization unit to analyze the user's emotions in real time when performing error checking and provide feedback to elicit positive emotions.

[0084] When the multilingual support unit supports multiple languages ​​for code generated by the generation AI, the multilingual support unit can automatically select the optimal language by referring to the user's past language selection history. For example, when the generation AI supports multiple languages, the multilingual support unit retrieves the user's past language selection history from a database and automatically selects the optimal language. For example, if the user has previously selected Python, the multilingual support unit preferentially selects Python. The multilingual support unit also analyzes the user's past language selection history and extracts frequently used languages. This allows the generation AI to automatically select a language that matches the user's preferences. For example, for a user who frequently uses JavaScript, the multilingual support unit preferentially selects JavaScript. The multilingual support unit also learns the user's preferences and style based on the language selection history and reflects them in the generated code. For example, for a user who prefers a specific programming language, the multilingual support unit generates code that matches that language. This allows the optimal language to be automatically selected by referring to the user's past language selection history.

[0085] The multilingual support unit can generate optimal code by taking into account the characteristics of each language when making the code generated by the generation AI multilingual. For example, when the generation AI performs multilingual support, the multilingual support unit takes into account the characteristics of each language to generate optimal code. For example, it takes into account the characteristics of Python to generate concise, easy-to-read code. The multilingual support unit also analyzes the characteristics of each language, and the generation AI generates optimal code. For example, it takes into account the asynchronous processing of JavaScript to generate appropriate asynchronous code. The multilingual support unit also takes into account the characteristics of each language when the generation AI performs multilingual support, generating code that is easy for users to understand. For example, it generates code that makes use of the object-oriented characteristics of Java. This allows the generation of optimal code by taking into account the characteristics of each language.

[0086] The multilingual support unit uses an emotion estimation function to analyze the emotion a user has when selecting a language, and can simplify language selection if the user is feeling stressed. The multilingual support unit, for example, uses the emotion estimation function to analyze the emotion a user has when selecting a language, and if the user is feeling stressed, the generation AI simplifies language selection. For example, the unit automatically selects the most appropriate language based on the user's past selection history. The multilingual support unit also analyzes the user's emotional state, and if the user is feeling stressed, the generation AI narrows down language selection candidates. For example, it displays only frequently used languages. The multilingual support unit also uses the emotion estimation function to enable the generation AI to provide support in a dialogue format if the user is feeling stressed when selecting a language. For example, it asks questions such as, "Which language do you want to use?" This simplifies language selection if the user is feeling stressed, making operation easier.

[0087] The multilingual support unit can visually display the results of multilingualization of code generated by the generation AI, allowing the user to intuitively understand the code in the language selected by the user. The multilingual support unit, for example, visually displays the results of multilingualization performed by the generation AI, allowing the user to intuitively understand the code in the language selected by the user. For example, it illustrates the structure of the code. Furthermore, by visually displaying the results of multilingualization, the multilingual support unit makes it easier for the user to understand the flow of the code in the language selected by the user. For example, it displays a flowchart or sequence diagram. Furthermore, the multilingual support unit uses visual displays to allow the user to check each part of the code in the language selected by the user. For example, it displays a pop-up with an explanation corresponding to each line of code. In this way, by visually displaying the results of multilingualization, the user can intuitively understand the code in the language selected by the user.

[0088] The multilingual support unit can notify the user by voice of the results of multilingualization of code generated by the generation AI, allowing the user to easily check the results. For example, the multilingual support unit can notify the user by voice of the results of multilingualization performed by the generation AI, allowing the user to easily check the results. For example, the multilingual support unit can notify the user by voice, saying, "Code has been generated in Python." The multilingual support unit can also use a voice notification function to allow the user to check the results of multilingualization without using their hands. For example, the multilingual support unit can notify the user by voice, saying, "Code has been generated in JavaScript." The multilingual support unit can also add a voice notification function to allow the user to check the results of multilingualization while performing other tasks. For example, the multilingual support unit can notify the user by voice, saying, "Code generation is complete. The selected language is Java." This allows the user to easily check the results by notifying the user by voice of the results of multilingualization.

[0089] The multilingual support unit can use the emotion estimation function to analyze the emotion a user has when selecting a language in real time and provide feedback to elicit positive emotions. For example, the multilingual support unit can use the emotion estimation function to analyze the emotion a user has when selecting a language in real time and provide feedback to elicit positive emotions. For example, the multilingual support unit can display an encouraging message such as, "Great choice!". The multilingual support unit can also analyze the user's emotional state, and if negative emotions are detected, the generation AI can provide positive feedback. For example, the message can be displayed such as, "This language is very easy to use." The multilingual support unit can also use the emotion estimation function to provide an interface to elicit positive emotions when the user selects a language. For example, the multilingual support unit can display appropriate feedback in real time in response to the user's input. This allows the multilingual support unit to analyze the emotion a user has when selecting a language in real time and provide feedback to elicit positive emotions.

[0090] The system according to the embodiment is not limited to the above-described example, and various modifications are possible, for example, as follows.

[0091] The code generation system can further include a security check unit. The security check unit can detect security vulnerabilities in the generated code and suggest modifications. For example, it can detect code that poses a risk of SQL injection and suggest appropriate escaping processing. The security check unit can also check whether the generated code complies with the latest security guidelines and suggest modifications as necessary. For example, if an old encryption algorithm is used, it can suggest changing to a more secure algorithm. The security check unit can also automatically run security tests on the generated code and provide the test results to the user. This can increase the security of the generated code.

[0092] The code generation system may further include a performance optimization unit. The performance optimization unit can analyze the performance of the generated code and make optimization suggestions. For example, it can suggest changes to an algorithm to reduce the number of loops. The performance optimization unit can also measure the execution time of the generated code and identify bottlenecks. For example, if a particular function takes a long time to execute, it can suggest how to optimize that function. The performance optimization unit can also automatically run performance tests on the generated code and provide the test results to the user. This can improve the performance of the generated code.

[0093] The code generation system may further include a debugging support unit. The debugging support unit can automatically add debugging information to the generated code, making it easier for users to debug the code. For example, it can set breakpoints for each line of code and display variable values ​​during execution. The debugging support unit can also automatically generate unit tests for the generated code and provide the test results to the user. For example, it can generate a test case for each function and display the execution results. The debugging support unit can also automatically generate an error log for the generated code and provide information for identifying the cause of the error. This makes it easier for users to debug the generated code.

[0094] The code generation system may further include a code review support unit. The code review support unit may automatically perform a code review on the generated code and suggest improvements. For example, it may check the consistency of the coding style and suggest modifications in accordance with a style guide. The code review support unit may also automatically perform a quality evaluation on the generated code and provide the evaluation results to the user. For example, it may evaluate the readability and maintainability of the code and suggest improvements. The code review support unit may also automatically support pair programming on the generated code, enabling the user to review the code collaboratively with other programmers. This may improve the quality of the generated code.

[0095] The code generation system may further include a documentation generation unit. The documentation generation unit may automatically generate documentation for the generated code, making it easier for users to understand the contents of the code. For example, detailed descriptions may be added for each function, and usage examples may be provided. The documentation generation unit may also automatically generate API documentation for the generated code, making it easier for users to understand how to use the API. For example, descriptions of each API endpoint and examples of requests and responses may be provided. The documentation generation unit may also automatically generate tutorials for the generated code, making it easier for users to learn how to use the code. This may help users understand the generated code.

[0096] The code generation system can further use an emotion estimation function to perform customization according to the user's emotions. For example, if the user is feeling stressed, the difficulty of the generated code can be lowered. For example, code with a simplified complex algorithm can be provided. Also, if the user has positive emotions, the difficulty of the generated code can be increased. For example, code with more advanced functions can be provided. Furthermore, the emotion estimation function can be used to provide feedback according to the user's emotions. For example, if the user is feeling anxious, an encouraging message can be displayed. This allows customization according to the user's emotions.

[0097] The code generation system can further use the emotion estimation function to customize the interface according to the user's emotions. For example, if the user is feeling stressed, the interface can be simplified by hiding unnecessary elements to simplify operation. On the other hand, if the user is feeling positive, the interface can be made more complex by displaying detailed setting options. The emotion estimation function can also be used to change the interface layout according to the user's emotions. For example, important information can be placed prominently to make operation intuitive. This makes it possible to customize the interface according to the user's emotions.

[0098] The code generation system can further use an emotion estimation function to provide learning support according to the user's emotions. For example, if the user is feeling stressed, the learning content can be simplified, for example, by providing learning materials that focus on basic concepts. Also, if the user has positive emotions, the learning content can be made more advanced, for example, by providing learning materials on applied topics. The emotion estimation function can also be used to adjust the learning pace according to the user's emotions. For example, if the user is tired, the learning pace can be slowed down. In this way, learning support can be provided according to the user's emotions.

[0099] The code generation system can also use an emotion estimation function to provide communication support that corresponds to the user's emotions. For example, if the user is feeling stressed, the generation AI can provide support in a dialogue format, asking questions such as, "Which part is difficult to understand?" If the user is feeling positive, the generation AI can provide a detailed explanation, such as, "This part of the code was written for this reason." The emotion estimation function can also be used to provide feedback that corresponds to the user's emotions. For example, if the user is feeling anxious, an encouraging message can be displayed. This allows communication support to be provided that corresponds to the user's emotions.

[0100] The code generation system can further use the emotion estimation function to customize error messages according to the user's emotions. For example, if the user is feeling stressed, the system can simplify the error message, for example, displaying a message such as "An error has occurred. Please check later for details." Also, if the user is feeling positive, the system can provide more detailed error messages, for example, displaying a detailed explanation such as "This error has occurred because the variable types do not match." The emotion estimation function can also be used to change the tone of the error message according to the user's emotions. For example, if the user is feeling anxious, the system can display a message in a gentler tone. This allows the system to customize error messages according to the user's emotions.

[0101] The processing flow of the second embodiment will be briefly explained below.

[0102] Step 1: The prompt analyzer analyzes the prompt. For example, the prompt analyzer uses natural language processing technology or machine learning algorithms to analyze the prompt and understand its content. The prompt analyzer can also analyze prompts in text format, audio format, image format, and other formats. Step 2: The code generation unit generates code based on the prompt analyzed by the prompt analysis unit. For example, the code generation unit generates code using a generation AI. The generation AI may be a text generation AI (e.g., LLM) or a multimodal generation AI, and generates appropriate code based on the user's prompt. The code generation unit may also generate code based on the programming language or generation algorithm used. Step 3: The code modification unit modifies the code generated by the code generation unit in real time. For example, the code modification unit adds new functions to existing code based on user instructions. The code modification unit can also analyze existing code and make necessary changes. The code modification unit can also modify the code based on the timing and scope of the changes. Step 4: The user interface unit provides an interface for checking and executing the code generated by the code generation unit. For example, the user interface unit may provide an area where a prompt can be entered in a text box and the generated code is displayed. The user interface unit may also provide a button for executing the generated code, allowing the user to easily check the code's operation. The user interface unit may also provide specific functions and display methods such as a GUI, CLI, or voice interface.

[0103] The specific processing unit 290 transmits the result of the specific processing to the smart device 14. In the smart device 14, the control unit 46A causes the output device 40 to output the result of the specific processing. The microphone 38B acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0104] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet Search<URL: https: / / openai.com / blog / chatgpt> Examples of generative AIs include the data generation model 58, such as a neural network model (e.g., a neural network model), and a neural network model (e.g., a neural network model). The data generation model 58 is obtained by performing deep learning on a neural network. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating speech, text data indicating text, and image data indicating an image is also input to the data generation model 58. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The specification processing unit 290 performs the above-mentioned specification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0105] Furthermore, the processing by the data processing system 10 described above is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart device 14, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart device 14. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information necessary for processing from the smart device 14 or an external device, and the smart device 14 acquires or collects information necessary for processing from the data processing device 12 or an external device.

[0106] [Second embodiment] FIG. 3 shows an example of the configuration of a data processing system 210 according to the second embodiment.

[0107] 3, the data processing system 210 includes a data processing device 12 and smart glasses 214. An example of the data processing device 12 is a server.

[0108] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0109] The smart glasses 214 include a computer 36, a microphone 238, a speaker 240, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, and the camera 42 are also connected to the bus 52.

[0110] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0111] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0112] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0113] Fig. 4 shows an example of the main functions of the data processing device 12 and the smart glasses 214. As shown in Fig. 4, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0114] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0115] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0116] In the smart glasses 214, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The smart glasses 214 also have a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0117] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0118] The specific processing unit 290 transmits the result of the specific processing to the smart glasses 214. In the smart glasses 214, the control unit 46A causes the speaker 240 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0119] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0120] The data processing system 210 according to the second embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 210 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the smart glasses 214, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the smart glasses 214. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the smart glasses 214 or an external device, etc., and the smart glasses 214 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0121] [Third embodiment] FIG. 5 shows an example of the configuration of a data processing system 310 according to the third embodiment.

[0122] 5, the data processing system 310 includes the data processing device 12 and a headset type terminal 314. An example of the data processing device 12 is a server.

[0123] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0124] The headset type terminal 314 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a display 343. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the display 343 are also connected to the bus 52.

[0125] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0126] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0127] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0128] Fig. 6 shows an example of the main functions of the data processing device 12 and the headset type terminal 314. As shown in Fig. 6, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0129] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0130] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0131] In the headset type terminal 314, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. Note that the headset type terminal 314 has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can also perform processing similar to that of the specific processing unit 290 using these models.

[0132] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0133] The specific processing unit 290 transmits the result of the specific processing to the headset type terminal 314. In the headset type terminal 314, the control unit 46A causes the speaker 240 and the display 343 to output the result of the specific processing. The microphone 238 acquires audio indicating a user input regarding the result of the specific processing. The control unit 46A transmits audio data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the audio data.

[0134] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0135] The data processing system 310 according to the third embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 310 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the headset type terminal 314, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the headset type terminal 314. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the headset type terminal 314 or an external device, etc., and the headset type terminal 314 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0136] [Fourth embodiment] FIG. 7 shows an example of the configuration of a data processing system 410 according to the fourth embodiment.

[0137] 7, the data processing system 410 includes a data processing device 12 and a robot 414. An example of the data processing device 12 is a server.

[0138] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, RAM 30, and storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a WAN and / or a LAN.

[0139] The robot 414 includes a computer 36, a microphone 238, a speaker 240, a camera 42, a communication I / F 44, and a control target 443. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The microphone 238, the speaker 240, the camera 42, and the control target 443 are also connected to the bus 52.

[0140] The microphone 238 receives instructions and the like from the user by receiving voice uttered by the user. The microphone 238 captures the voice uttered by the user, converts the captured voice into audio data, and outputs it to the processor 46. The speaker 240 outputs audio according to instructions from the processor 46.

[0141] Camera 42 is a small digital camera equipped with an optical system including a lens, aperture, and shutter, and an imaging element such as a CMOS image sensor or a CCD image sensor, and captures images of the user's surroundings (for example, an imaging range defined by an angle of view equivalent to the field of vision of a typical healthy person).

[0142] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for the exchange of various information between the processor 46 and the processor 28 via the network 54. The exchange of various information between the processor 46 and the processor 28 using the communication I / Fs 44 and 26 is carried out in a secure state.

[0143] The control object 443 includes a display device, LEDs in the eyes, and motors that drive the arms, hands, and feet. The posture and gestures of the robot 414 are controlled by controlling the motors of the arms, hands, and feet. Some of the emotions of the robot 414 can be expressed by controlling these motors. In addition, the facial expressions of the robot 414 can also be expressed by controlling the light emission state of the LEDs in the eyes of the robot 414.

[0144] Fig. 8 shows an example of the main functions of the data processing device 12 and the robot 414. As shown in Fig. 8, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32.

[0145] The processor 28 reads the specific processing program 56 from the storage 32 and executes the read specific processing program 56 on the RAM 30. The specific processing is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0146] The storage 32 stores a data generation model 58 and an emotion identification model 59. The data generation model 58 and the emotion identification model 59 are used by the identification processing unit 290. The identification processing unit 290 can estimate a user's emotion using the emotion identification model 59 and perform identification processing using the user's emotion. The emotion estimation function (emotion identification function) using the emotion identification model 59 performs various estimations and predictions regarding the user's emotion, including estimation and prediction of the user's emotion, but is not limited to these examples. Furthermore, the estimation and prediction of emotion also includes, for example, emotion analysis.

[0147] In the robot 414, the specific processing is performed by the processor 46. A specific processing program 60 is stored in the storage 50. The processor 46 reads the specific processing program 60 from the storage 50 and executes the read specific processing program 60 on the RAM 48. The specific processing is realized by the processor 46 operating as the control unit 46A in accordance with the specific processing program 60 executed on the RAM 48. The robot 414 also has a data generation model and an emotion identification model similar to the data generation model 58 and the emotion identification model 59, and can perform processing similar to that of the specific processing unit 290 using these models.

[0148] Note that a device other than the data processing device 12 may have the data generation model 58. For example, a server device may have the data generation model 58. In this case, the data processing device 12 communicates with the server device having the data generation model 58 to obtain a processing result (such as a prediction result) using the data generation model 58. Furthermore, the data processing device 12 may be a server device, or may be a terminal device (for example, a mobile phone, a robot, a home appliance, etc.) owned by a user.

[0149] The specific processing unit 290 transmits the result of the specific processing to the robot 414. In the robot 414, the control unit 46A causes the speaker 240 and the control target 443 to output the result of the specific processing. The microphone 238 acquires voice indicating a user input regarding the result of the specific processing. The control unit 46A transmits voice data indicating the user input acquired by the microphone 238 to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the voice data.

[0150] The data generation model 58 is a so-called generative AI. An example of the data generation model 58 is a generative AI such as ChatGPT. The data generation model 58 is obtained by performing deep learning on a neural network. The data generation model 58 receives a prompt containing an instruction, as well as inference data such as voice data representing speech, text data representing text, and image data representing an image. The data generation model 58 performs inference on the input inference data in accordance with the instruction indicated by the prompt and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The identification processing unit 290 performs the above-mentioned identification processing using the data generation model 58. The data generation model 58 may be a fine-tuned model so as to output an inference result from a prompt that does not include an instruction. In this case, the data generation model 58 can output an inference result from a prompt that does not include an instruction. The data processing device 12 and the like include multiple types of data generation models 58, and the data generation model 58 includes AIs other than the generative AI. The AI ​​other than the generative AI may be, for example, linear regression, logistic regression, decision tree, random forest, support vector machine (SVM), k-means clustering, convolutional neural network (CNN), recurrent neural network (RNN), generative adversarial network (GAN), or naive Bayes, and can perform various processes, but is not limited to these examples. The AI ​​may also be an AI agent. When the processes of each of the above-mentioned parts are performed by AI, the processes may be performed in part or entirely by AI, but are not limited to these examples. The processes performed by AI, including the generative AI, may be replaced with rule-based processes.

[0151] The data processing system 410 according to the fourth embodiment performs the same processing as the data processing system 10 according to the first embodiment. The processing by the data processing system 410 is executed by the specific processing unit 290 of the data processing device 12 or the control unit 46A of the robot 414, but may also be executed by the specific processing unit 290 of the data processing device 12 and the control unit 46A of the robot 414. Furthermore, the specific processing unit 290 of the data processing device 12 acquires or collects information required for processing from the robot 414 or an external device, etc., and the robot 414 acquires or collects information required for processing from the data processing device 12 or an external device, etc.

[0152] The emotion identification model 59 as an emotion engine may determine the user's emotion according to a specific mapping. Specifically, the emotion identification model 59 may determine the user's emotion according to an emotion map (see FIG. 9), which is a specific mapping. Similarly, the emotion identification model 59 may determine the robot's emotion, and the identification processing unit 290 may perform identification processing using the robot's emotion.

[0153] FIG. 9 illustrates an emotion map 400 on which multiple emotions are mapped. In the emotion map 400, emotions are arranged in concentric circles radiating from the center. Emotions closer to the center of the concentric circles are more primitive. Emotions representing states and behaviors arising from a state of mind are arranged on the outer edges of the concentric circles. The concept of emotion encompasses both emotions and mental states. Emotions generally generated from reactions occurring in the brain are arranged on the left side of the concentric circles. Emotions generally induced by situational judgment are arranged on the right side of the concentric circles. Emotions generally generated from reactions occurring in the brain and induced by situational judgment are arranged on the upper and lower sides of the concentric circles. Furthermore, the emotion of "pleasure" is arranged on the upper side of the concentric circles, and the emotion of "discomfort" is arranged on the lower side. In this way, in the emotion map 400, multiple emotions are mapped based on the structure by which emotions are generated, and emotions that tend to occur simultaneously are mapped close to each other.

[0154] These emotions are distributed in the 3 o'clock direction on emotion map 400, and typically fluctuate between relief and anxiety. In the right half of emotion map 400, situational awareness dominates over internal sensations, resulting in a sense of calm.

[0155] The inside of emotion map 400 represents what is going on in the mind, and the outside of emotion map 400 represents behavior, so the further you go outside emotion map 400, the more visible the emotions become (the more they are expressed in behavior).

[0156] Human emotions are based on various balances, such as posture and blood sugar levels. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. Emotions can also be created for robots, cars, and motorcycles, based on various balances, such as posture and remaining battery life. When these balances deviate from the ideal, a state of discomfort is expressed, and when they approach the ideal, a state of pleasure is expressed. An emotion map can be generated, for example, based on Dr. Mitsuyoshi's emotion map (Research on speech emotion recognition and brain physiological signal analysis systems for emotions, Tokushima University, doctoral dissertation: https: / / ci.nii.ac.jp / naid / 500000375379). The left half of the emotion map lists emotions belonging to the area called "reaction," where sensation is dominant. The right half of the emotion map lists emotions belonging to the area called "situation," where situational awareness is dominant.

[0157] The emotion map defines two emotions that promote learning. One is a negative emotion on the situation side, around the middle of "repentance" or "reflection." In other words, this occurs when the robot experiences negative emotions such as "I never want to feel this way again" or "I don't want to be scolded again." The other is a positive emotion on the response side, around "desire." In other words, this occurs when the robot experiences positive feelings such as "I want more" or "I want to know more."

[0158] The emotion identification model 59 inputs user input into a pre-trained neural network, obtains emotion values ​​indicating each emotion shown in the emotion map 400, and determines the user's emotion. This neural network is pre-trained based on multiple pieces of training data that are combinations of user input and emotion values ​​indicating each emotion shown in the emotion map 400. Furthermore, this neural network is trained so that emotions that are located close to each other have similar values, as in the emotion map 900 shown in FIG. 10. FIG. 10 shows an example in which multiple emotions, "relieved," "calm," and "reassuring," have similar emotion values.

[0159] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers including computer 22.

[0160] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable, computer-readable, non-transitory storage medium such as a USB (Universal Serial Bus) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0161] Alternatively, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 in response to a request from the data processing device 12.

[0162] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0163] The hardware resource for executing a specific process can be any of the following types of processors: A processor, for example, is a CPU, which is a general-purpose processor that functions as a hardware resource for executing a specific process by executing software, i.e., a program. A processor also includes a dedicated electrical circuit, such as an FPGA (Field-Programmable Gate Array), a PLD (Programmable Logic Device), or an ASIC (Application Specific Integrated Circuit), which is a processor with a circuit configuration designed specifically for executing a specific process. Each processor has built-in or connected memory, and each processor uses the memory to execute the specific process.

[0164] The hardware resource that executes the specific process may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0165] As an example of a system configured with a single processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a system that uses a processor that realizes the functions of an entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0166] Furthermore, the hardware structure of these various processors can be, more specifically, an electric circuit that combines circuit elements such as semiconductor devices. The specific processing described above is merely an example. Therefore, it goes without saying that unnecessary steps may be deleted, new steps may be added, or the processing order may be rearranged, without departing from the spirit of the invention.

[0167] In the above example, the first to fourth embodiments have been described separately, but some or all of these embodiments may be combined. The smart device 14, smart glasses 214, headset terminal 314, and robot 414 are merely examples, and they may be combined, or other devices may be used. In the above example, the first and second embodiments have been described separately, but they may be combined.

[0168] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0169] All publications, patent applications, and technical standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or technical standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0170] 10, 210, 310, 410 Data Processing Systems 12 Data Processing Device 14 Smart Devices 214 Smart Glasses 314 Headset-type terminal 414 Robot

Claims

1. a prompt parser for parsing a prompt; a code generation unit that generates code based on the prompt analyzed by the prompt analysis unit; a code modification unit that modifies the code generated by the code generation unit in real time; a user interface unit for checking and executing the code generated by the code generation unit; A system characterized by:

2. The prompt analysis unit Refer to the user's past prompt history to generate a more accurate code 2. The system of claim 1.

3. The prompt analysis unit Ask follow-up questions and gather more information interactively to better understand user intent 2. The system of claim 1.

4. The prompt analysis unit Analyzing the user's emotional state and generating the code that is easier to understand if the user is stressed 2. The system of claim 1.

5. The prompt analysis unit Supports voice input, allowing users to give voice commands 2. The system of claim 1.

6. The prompt analysis unit The prompt analysis results are visually displayed, allowing the user to intuitively understand the contents of the generated code.

2. The system of claim 1.

7. The prompt analysis unit Analyze users' emotions in real time as they type prompts and provide feedback to elicit positive emotions 2. The system of claim 1.

Images