Processor, program, and method for processing

Abstract

To provide a processor, a program, and a method for processing which allow software to be operated more easily.SOLUTION: The processor according to an embodiment includes an acquisition unit. The acquisition unit acquires instruction data output from a learned language model by inputting, into the learned language model, a prompt including a first letter string showing an operation that software is to perform and a second letter string instructing software to output instruction data for executing the operation by inputting the second letter string into the software.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a processing device, a program, and a processing method. [Background technology]

[0002] It takes time and effort to learn how to operate the software. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 10-240110 Summary of the Invention [Problem to be solved by the invention]

[0004] An object of the present invention is to provide a processing device, a program, and a processing method that enable software to be operated more easily than ever before. [Means for solving the problem]

[0005] A processing device according to an embodiment includes an acquisition unit that inputs a prompt, which includes a first character string indicating an operation to be performed by software and a second character string that, when input to the software, instructs the software to output instruction data for causing the software to perform the operation, to a trained language model, thereby acquiring the instruction data output from the language model. [Brief description of the drawings]

[0006] [Figure 1] FIG. 1 is a block diagram showing an example of a configuration of a main part of a software operation device according to a first embodiment and a second embodiment. [Diagram 2] 4 is a flowchart showing an example of processing according to the first embodiment performed by a processor in FIG. 1; [Diagram 3]10 is a flowchart showing an example of processing according to the second embodiment by a processor in FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0007] Hereinafter, software operation devices according to some embodiments will be described with reference to the drawings. Note that the scale of each part in each drawing used in the following description of the embodiments may be changed as appropriate. Also, for the sake of explanation, each drawing used in the following description of the embodiments may be shown with the configuration omitted. Also, in each drawing and in this specification, the same reference numerals indicate similar elements.

[0008] [First embodiment] 1 is a block diagram showing an example of the configuration of the main parts of a software operating device 100 according to the first embodiment. Note that each component of each device may be built-in or external.

[0009] Software operation device 100 is a device that outputs instruction data indicating the operation to be performed by software by inputting an operation to be performed by software in the form of a large language model (LLM). Details of the instruction data will be described later. Software operation device 100 includes, as an example, a processor 110, a read-only memory (ROM) 120, a random-access memory (RAM) 130, an auxiliary storage device 140, an input device 150, a display device 160, and a communication interface 170. A bus 180 and the like connect these components. Software operation device 100 is an example of a processing device.

[0010] The processor 110 is a central part of a computer that performs processing such as calculations and control necessary for the operation of the software operation device 100, and performs various calculations and processing. The processor 110 is, for example, a central processing unit (CPU), a micro processing unit (MPU), a system on a chip (SoC), a digital signal processor (DSP), a graphics processing unit (GPU), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field-programmable gate array (FPGA). Alternatively, the processor 110 is a combination of a plurality of these. The processor 110 may also be a combination of these with a hardware accelerator or the like. The processor 110 controls each part to realize various functions of the software operation device 100 based on programs such as firmware, system software, and application software stored in the ROM 120 or the auxiliary storage device 140. The processor 110 also executes processing described later based on the programs. A part or all of the programs may be incorporated in the circuit of the processor 110.

[0011] The ROM 120 and the RAM 130 are main storage devices of the computer with the processor 110 at its core. The ROM 120 is a non-volatile memory used exclusively for reading data. The ROM 120 stores, for example, firmware among the above programs. The ROM 120 also stores data used by the processor 110 when performing various processes.

[0012] The RAM 130 is a memory used for reading and writing data. The RAM 130 is used as a work area for storing data that is temporarily used when the processor 110 performs various processes. The RAM 130 is typically a volatile memory.

[0013] The auxiliary storage device 140 is an auxiliary storage device of a computer with the processor 110 at its core. The auxiliary storage device 140 is, for example, an EEPROM (electric erasable programmable read-only memory), a HDD (hard disk drive), or a flash memory. The auxiliary storage device 140 stores, for example, system software and application software among the above programs. The auxiliary storage device 140 also stores data used by the processor 110 in performing various processes, data generated by the processes in the processor 110, various setting values, and the like.

[0014] The programs stored in the auxiliary storage device 140 include one or more pieces of software (hereinafter referred to as "instruction software") that can be operated by instruction data. The instruction software is typically application software, but may be system software. The instruction software may be any type of software. For example, the instruction software may be office software such as document creation software, image editing software, video editing software, 3DCG (three-dimensional computer graphics) software, CAD (computer-aided design) software, file browsing software, simulation software, project management software, information management software, schedule management software, attendance management software, database software, email software, instant messenger, web browser, financial software, accounting software, educational software, programming software, integrated development environment, various editors, game development software, game software, OS (operating system), or other software. The instruction data is data that describes the content of the operation to be executed by the instruction software. When the instruction software reads the instruction data, it automatically executes the operation indicated by the instruction data. Note that a file including instruction data is referred to as an instruction file. An example of an instruction file is a file in which instruction data is written in a text format. An example of a file written in a text format is a batch file. Note that the data format of the instruction file may be other formats. The instruction data may also be called a macro. The instruction data is an example of instruction data that is input to software to cause the software to perform an operation.

[0015] The data stored in the auxiliary storage device 140 includes, for example, an LLM 141 and a software DB (database) 142.

[0016] The LLM 141 is a trained AI (artificial intelligence) model. The LLM 141 is a language model capable of natural language processing. The LLM 141 may be the same as a well-known LLM. Therefore, a detailed description of the LLM 141 will be omitted.

[0017] The software DB 142 is a database that stores information about each piece of instruction software. The information about the instruction software includes information indicating the specifications of instruction data for the instruction software. The information about the instruction software also includes, for example, information indicating what type of software the instruction software is and information indicating what functions it has. The software DB 142 is preferably a vector database.

[0018] The input device 150 accepts operations by an operator of the software operating apparatus 100 (hereinafter simply referred to as "operator"). The input device 150 is, for example, a keyboard, a keypad, a touchpad, a mouse, or a controller. The input device 150 may also be a device for voice input. The operator is an example of a user.

[0019] The display device 160 displays a screen for notifying an operator or the like of various pieces of information. The display device 160 is, for example, a display such as a liquid crystal display or an organic EL (electro-luminescence) display. A touch panel can also be used as the input device 150 and the display device 160. That is, a display panel provided in the touch panel can be used as the display device 160, and a pointing device provided in the touch panel by touch input can be used as the input device 150.

[0020] The communication interface 170 is an interface for the software operating device 100 to communicate via a network such as the Internet or a LAN (local area network).

[0021] The bus 180 includes a control bus, an address bus, a data bus, etc., and transmits signals exchanged among the various parts of the software operating device 100 .

[0022] The operation of the software operating device 100 according to the first embodiment will be described below with reference to Fig. 2 and other figures. Note that the contents of the processing in the following description of the operation are merely examples, and various processing that can obtain similar results can be used as appropriate. Fig. 2 is a flowchart showing an example of processing by the processor 110 of the software operating device 100. The processor 110 executes the processing of Fig. 2 based on a program stored in, for example, the ROM 120 or the auxiliary storage device 140.

[0023] In step ST11 of FIG. 2, processor 110 of software operating device 100 determines the instruction software to be operated (hereinafter referred to as "target software"). In other words, the target software is the instruction software for which instruction data is to be generated. The operator, for example, operates input device 150 to input information specifying the target software. Based on the input, processor 110 determines the instruction software specified by the input as the target software. Alternatively, processor 110 may determine predetermined instruction software as the target software.

[0024] In step ST12, the processor 110 generates an image corresponding to the input screen. Then, the processor 110 instructs the display device 160 to display the generated image. In response to the display instruction, the display device 160 displays the input screen.

[0025] The input screen is a screen for accepting input of instructions indicating what operation the target software is to perform. The input screen includes, as an example, an input field for inputting the contents of the instruction. The operator inputs the contents of the instruction into the input field using, for example, the input device 150. The operator inputs the contents of the instruction in, for example, a natural language. The contents of the instruction are not limited to those indicating a specific operation, but may also indicate a result to be obtained by the operation. The contents of the instruction may also instruct an operation by other methods. The contents of the instruction may also instruct a plurality of operations. The contents of the instruction may also be an instruction requiring a plurality of operations. Examples of the operations of the target software include, for example, editing of documents, images, videos, audio, 3DCG, CAD data, programs, or other data, data operation, data search, data creation, data deletion, data copy, data transmission / reception, data acquisition, execution of an action, operation of a game character, and change of software settings. The operations of the target software are not limited to those listed here.

[0026] The input screen also includes a confirmation button that the operator operates to confirm the content of an instruction, and the operator operates the confirmation button after inputting the content of an instruction in the input field.

[0027] Processor 110 may accept input of instructions by voice. In this case, processor 110 does not need to display an input screen.

[0028] In step ST13, processor 110 waits for the instruction content to be determined. That is, processor 110 waits for a predetermined operation, such as operating a decision button, to be performed with the instruction content input on the input screen. When the instruction content is input by voice, processor 110 determines that the instruction content has been determined, for example, when it determines that the vocalization indicating the instruction has ended. If processor 110 determines that the instruction content has been determined, it determines Yes in step ST13 and proceeds to step ST14.

[0029] In step ST14, the processor 110 inputs a prompt and the software DB 142 to the LLM 141. The prompt includes the instruction content input by the operator on the input screen and a first basic instruction sentence. As an example, the first basic instruction sentence includes sentences with the contents of the following (A1) to (A3).

[0030] (A1) A sentence indicating that software DB142 contains information about the target software and that the specifications of instruction data for the target software are indicated in software DB142. (A2) A sentence indicating that instruction data for the target software should be created in accordance with the specifications shown in software DB142. (A3) A sentence indicating that the instruction data is intended to cause the target software to perform the action indicated by the input instruction content.

[0031] The processor 110 executes the LLM 141 to which the prompt and the software DB 142 are input. The LLM 141 executed by the processor 110 generates an output for the input. That is, the LLM 141 generates and outputs a response to the instruction indicated by the prompt. Based on the input prompt, the LLM 141 analyzes the content indicated by the prompt and generates the response.

[0032] In step ST15, the processor 110 waits for the LLM 141 to output an answer. When the LLM 141 outputs an answer, the processor 110 acquires the answer. This answer is expected to include instruction data. The instruction data includes execution content information. The execution content information includes the content to be executed by the target software. The content is indicated by the instruction content entered on the input screen. The execution content information is described in, for example, a programming language, a data description language, or other computer language. Here, the subsequent processing will be described assuming that the answer content includes the instruction data. Note that if the answer content is not as expected, the processor 110 may perform, for example, a predetermined exception processing.

[0033] The instruction content in the prompt input in step ST14 is an example of a first character string indicating an operation to be performed by the software. The first basic instruction sentence is an example of a second character string that, when input to the software, instructs the software to output instruction data that causes the software to perform an operation.

[0034] As described above, processor 110 functions as an example of an acquisition unit by performing the processes of steps ST14 and ST15. The acquisition unit inputs a prompt including the first character string and the second character string into a trained language model, thereby acquiring instruction data output from the language model.

[0035] In step ST16, processor 110 generates an image corresponding to the result screen. Then, processor 110 instructs display device 160 to display this generated image. In response to the display instruction, display device 160 displays the result screen.

[0036] The result screen is a screen for notifying the operator of the answer content acquired in step ST15. The result screen is, for example, an input screen to which the answer content has been added.

[0037] In step ST17, the processor 110 acquires instruction data included in the response from the response acquired in step ST15. Then, the processor 110 generates an instruction file including the instruction data. Then, the processor 110 stores the instruction file in the auxiliary storage device 140 or the like.

[0038] In step ST18, processor 110 executes the instruction file generated in step ST17. That is, processor 110 executes the process indicated by the execution content information included in the instruction file. To this end, processor 110 instructs the target software to execute the process. Processor 110 issues this instruction, for example, by inputting the execution content information into the instruction software. Based on the instruction, the instruction software executes the process. After processing in step ST18, processor 110 returns to step ST13.

[0039] As described above, processor 110 functions as an example of an execution unit by performing the process of step ST18. The execution unit executes the instruction data to cause the software to perform an operation.

[0040] In addition, in the second or subsequent processing of step ST14, the processor 110 may input a prompt to the LLM 141 so as to continue the previous exchange. Here, the previous exchange refers to the prompt input to the LLM 141 in step ST14 up to the previous time, and the answer output in step ST15.

[0041] When carrying over past interactions, the processor 110 may input a prompt in which some of (A1) to (A3) are omitted or deleted to the LLM 141. The processor 110 may also include some or all of the past interactions in the prompt.

[0042] When continuing a past interaction, processor 110 may input a prompt including a sentence having the content of (A4) below.

[0043] (A4) A sentence indicating that the instruction data to be created this time will be executed after the instruction data created previously has been executed.

[0044] The software operating device 100 of the first embodiment inputs a prompt including a character string indicating an operation to be performed by the software and a first basic instruction sentence to the LLM 141. Then, the software operating device 100 of the first embodiment obtains instruction data for causing the software to perform the operation from the LLM 141. By using the instruction data, an operator can operate the software even if he or she does not know how to operate the software. In other words, the software operating device 100 of the embodiment makes it easier to operate software than ever before.

[0045] Furthermore, the software operating device 100 of the first embodiment operates the software via instruction data. If instruction data is not used, it is necessary to have the LLM 141 memorize the operation of the software. Therefore, the software operating device 100 of the first embodiment can operate the software more easily than if instruction data is not used.

[0046] Furthermore, the software operating device 100 of the first embodiment uses a sentence input by an operator as a character string indicating an operation to be performed by the software. Therefore, the operator can make the software perform the desired operation by inputting a natural language indicating the operation to be performed by the software. In other words, the software operating device 100 of the embodiment makes it easier to operate software than ever before.

[0047] Moreover, the software operating device 100 of the first embodiment executes instruction data to cause the software to perform the operation indicated by the instruction data. Therefore, by using the software operating device 100 of the embodiment, an operator can operate software more easily than in the past.

[0048] Furthermore, the software operating device 100 of the first embodiment can take over past interactions in inputting and outputting to the LLM 141. By taking over past interactions, the software operating device 100 of the first embodiment can create instruction data that takes the past interactions into consideration.

[0049] Second Embodiment The software operating device 100 according to the second embodiment does not specify the target software in advance.

[0050] The configuration of the software operating device 100 according to the second embodiment is the same as that of the first embodiment, so a description thereof will be omitted.

[0051] Hereinafter, the operation of the software operating device 100 according to the second embodiment will be described with reference to Fig. 3 etc. Fig. 3 is a flowchart showing an example of processing by the processor 110 of the software operating device 100. The processor 110 executes the processing of Fig. 3 based on a program stored in, for example, the ROM 120 or the auxiliary storage device 140.

[0052] In the second embodiment, the processor 110 proceeds to step ST12 after starting the process of FIG.

[0053] The input screen in the second embodiment is a screen for receiving input of instructions indicating what operation is desired to be performed by the instruction software. Here, it is not necessary to specify which instruction software is to be used. However, the instruction contents may include contents specifying what instruction software is to be used. The instruction contents may also include contents specifying which instruction software is to be used. The rest of the instruction contents are the same as those in the first embodiment.

[0054] In the second embodiment, if the processor 110 determines Yes in step ST13, the process proceeds to step ST21.

[0055] In step ST21, the processor 110 inputs a prompt and the software DB 142 to the LLM 141. The prompt includes the instruction content input by the operator on the input screen and a second basic instruction sentence. As an example, the second basic instruction sentence includes sentences such as the following (B1) to (B6).

[0056] (B1) A sentence indicating that software DB142 contains information about each instruction software, and that the specifications of the instruction data for each instruction software are shown in software DB142. (B2) A sentence indicating that the software DB142 is to be referred to, that an instruction software suitable for making the changes indicated by the input instruction content is to be selected, and that the user is to respond as to which instruction software has been selected. (B3) However, if the instruction software to be used is specified, the sentence should indicate that the specified software should be selected. (B4) A sentence indicating that instruction data for the selected instruction software should be created in accordance with the specifications shown in software DB142. (B5) A sentence indicating that the instruction data is intended to cause the selected instruction software to perform the action indicated by the input instruction content. (B6) A sentence indicating that the instruction data includes software designation information and execution content information. Note that the software designation information will be described later.

[0057] The second basic instruction sentence is an example of a second character string. The second basic instruction sentence is an example of a third character string that instructs the user to select software suitable for the operation. (B2) is an example of a third character string.

[0058] In step ST22, the processor 110 waits for the LLM 141 to output a response. When the LLM 141 outputs a response, the processor 110 acquires the response. This response is expected to include instruction data. The instruction data is expected to include, for example, software designation information and execution content information. The software designation information includes, for example, information indicating which instruction software is to be used. The software designation information includes, for example, a command instructing to start the instruction software. The execution content information of the second embodiment includes the content to be executed by the instruction software indicated by the software designation information. The content is indicated by the instruction content input on the input screen. The software designation information and the execution content information are described, for example, in a programming language, a data description language, or other computer language. Here, the following process will be described assuming that the response content includes instruction data, and the instruction data includes the software designation information and the execution content information. If the response content is not as expected, the processor 110 may perform, for example, a predetermined exception process. After processing step ST22, processor 110 proceeds to step ST16.

[0059] As described above, processor 110 functions as an example of an acquisition section by performing the processes of steps ST21 and ST22.

[0060] In the second embodiment, the execution of the instruction file includes the following processes (C1) to (C2).

[0061] (C1) The processor 110 starts up the instruction software indicated by the software designation information included in the instruction file.

[0062] (C2) Processor 110 executes the process indicated by the execution content information included in the instruction file. To do so, processor 110 instructs the instruction software opened in (C1) to execute the process. Based on the instruction, the instruction software executes the process.

[0063] In addition, in the second or subsequent processing of step ST21, the processor 110 may input a prompt to the LLM 141 so as to continue the previous exchange. Here, the previous exchange refers to the prompt input to the LLM 141 in step ST21 up to the previous time, and the answer output in step ST22.

[0064] When carrying over past interactions, the processor 110 may input a prompt in which some of (B1) to (B6) are omitted or deleted to the LLM 141. The processor 110 may also include some or all of the past interactions in the prompt.

[0065] When continuing a past interaction, processor 110 may input a prompt including a sentence having the content of (B7) below.

[0066] (B7) A sentence indicating that the instruction data to be created this time will be executed after the instruction data created previously has been executed.

[0067] When continuing a past interaction, processor 110 may input a prompt including a sentence having the content of (B8) below.

[0068] (B8) A sentence indicating that the same instruction software will be used as last time.

[0069] The software operating device 100 of the second embodiment provides the same effects as the first embodiment.

[0070] Moreover, the software operation device 100 of the second embodiment inputs a prompt including a third character string instructing the LLM 141 to select software suitable for the operation to be performed, and a second character string instructing the LLM 141 to output instruction data for the selected software. Then, the software operation device 100 of the second embodiment obtains a response indicating which software has been selected from the LLM 141. Furthermore, the software operation device 100 of the second embodiment obtains instruction data for the software selected by the LLM 141. Therefore, even if the operator does not know which software can be used to perform the operation by using the software operation device 100 of the second embodiment, the software operation device 100 of the second embodiment selects software suitable for the operation. Furthermore, the operator can use the software operation device 100 of the second embodiment to cause the software selected by the software operation device 100 to perform the operation.

[0071] The above first and second embodiments may be modified as follows.

[0072] In the above embodiment, software operation device 100 uses software DB 142 as external data available to LLM 141. However, software operation device 100 may use an LLM that has learned software DB 142 instead of LLM 141. In this case, processor 110 of software operation device 100, for example, additionally learns software DB 142 in LLM 141.

[0073] In the above embodiment, the software operating device 100 uses the software DB 142 as external data available to the LLM 141. However, the software operating device 100 may input the software DB 142 to the LLM 141 as a prompt.

[0074] The instruction data in the second embodiment may not include software designation information. In this case, the processor 110 starts the instruction software without relying on the instruction data. For example, the answer content acquired in step ST22 of FIG. 3 may include a description indicating which instruction software to use in a portion other than the instruction data. The processor 110 identifies the instruction software to be used based on this description, and starts the instruction software. Alternatively, the processor 110 may start the instruction software designated by the operator. In this case, the operator starts the instruction software indicated in the answer content displayed in step ST16.

[0075] The second basic instruction sentence may include a sentence instructing the operator to create appropriate instruction software if there is no instruction software suitable for executing the instruction content input by the operator on the input screen. If the prompt includes this sentence, the LLM 141 creates instruction software suitable for the instruction content. This sentence is an example of a fourth character string instructing the operator to create software suitable for an operation if there is no software suitable for the operation. In this way, even if there is no instruction software suitable for executing the operation indicated by the instruction content, the software operating device 100 of the embodiment can perform the operation by using the created software.

[0076] The instruction file may be executed from the instruction software when the instruction software is running. In this case, the operator can operate the instruction software to read the instruction file and cause the instruction software to execute the instruction file.

[0077] In the above embodiment, processor 110 automatically executes the instruction file. However, processor 110 does not have to automatically execute the instruction file. In this case, processor 110 returns to step ST13 after processing step ST17. The operator can instruct software operating device 100 to execute the instruction file by performing an operation to open the saved instruction file. Based on the instruction, processor 110 of software operating device 100 executes the instruction file.

[0078] Instead of software DB 142, software operation device 100 may use information in which data similar to software DB 142 is stored in a format other than a database.

[0079] The software operation device of the embodiment may be composed of a plurality of devices. The software operation device of the embodiment may be realized by using cloud computing. As an example, the software operation device includes a terminal device and a server device. The server device is a server in cloud computing. The server device executes LLM141. The terminal device transmits a prompt to the server device through a communication network such as the Internet or a LAN (local area network). The processor of the server device uses the prompt to execute the processes of steps ST14 and ST15 in FIG. 2 or steps ST21 and ST22 in FIG. 3. Then, the server device transmits the acquired answer to the terminal device through the communication network. The terminal device uses the answer to execute the processes of steps ST16 and after in FIG. 2 or steps ST16 and after in FIG. 3.

[0080] The processor 110 may implement part or all of the processes implemented by the programs in the above-described embodiments by using a hardware circuit configuration.

[0081] The program for implementing the process of the embodiment is transferred in a state in which it is stored in, for example, a non-transitory computer-readable storage medium in the device. However, the device may be transferred in a state in which the program is not stored therein. The program may then be transferred separately and written to the device. In this case, the program may be transferred by, for example, recording the program in a removable non-transitory computer-readable storage medium, or by downloading the program via a network such as the Internet or a LAN (local area network).

[0082] Although the embodiment of the present invention has been described above, it is merely an example and does not limit the scope of the present invention. The embodiment of the present invention can be implemented in various forms without departing from the gist of the present invention. [Explanation of symbols]

[0083] 100 Software operation device 110 Processor 120 ROM 130 RAM 140 Auxiliary storage 141 LLM 142 Software DB 150 Input Devices 160 Display Devices 170 Communication Interface 180 Bus

Claims

1. A processing device comprising an acquisition unit that acquires the instruction data output from a trained language model by inputting a prompt including a first string indicating an operation that the software is to perform and a second string that, when input to the software, instructs the software to output instruction data that causes the software to perform the operation.

2. The processing device of claim 1 , wherein the first string includes a sentence entered by a user.

3. the prompt includes a third character string instructing the user to select appropriate software for the operation; The processing device of claim 1 , wherein the second string includes a string instructing the processing device to output the instruction data for a selected piece of software.

4. 4. The processing device of claim 3, wherein the prompt includes a fourth string instructing the processing device to create software suitable for the operation if the software does not exist.

5. The processing device according to claim 1 , further comprising an execution unit that executes the instruction data to cause the software to perform the operation.

6. A processor included in the processing device, A program that functions as an acquisition unit that acquires the instruction data output from a trained language model by inputting a prompt into the trained language model, the instruction data including a first string indicating an operation that the software is to perform, and a second string that, when input to the software, instructs the software to output instruction data that will cause the software to perform the operation.

7. A processing method comprising: inputting a prompt including a first string indicating an operation to be performed by software and a second string that, when input to the software, instructs the software to output instruction data to cause the software to perform the operation, into a trained language model, thereby obtaining the instruction data to be output from the language model.

Images