Data processing device, data processing method and program

Abstract

To provide a data processing device, a data processing method, and a program that can efficiently select an appropriate application even when a plurality of applications subject to selection exists.SOLUTION: An embodiment of a data processing device includes: a selection unit for selecting a group according to an instruction content from among a plurality of groups classified based on functional similarities of a plurality of applications; a processing unit for performing a process according to the instruction content, based on a specific process using a data generation model that generates data according to an input content; and an output unit for outputting a result of the specific process. The processing unit obtains the result of the specific process using the output of the data generation model when the input content includes data including an application definition related to the selected group and the instruction content, and the specific process is a process of specifying an application that executes a process according to the instruction content from among the applications belonging to the group.SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The technology disclosed herein relates to a data processing device, a data processing method, and a program. [Background technology]

[0002] In recent years, large-scale language models (LLMs) have been attracting attention. For example, Patent Document 1 discloses a large-scale language model called TaslMatrix.AI. The technology described in Patent Document 1 selects an API (Application Programming Interface) from among multiple APIs according to a user's instruction, and executes processing according to the user's instruction using the selected API. [Prior art documents] [Patent documents]

[0003] [Non-Patent Document 1] “TaskMatrix.AI: Completing Tasks by Connecting Foundation Models with Millions of APIs” Yaobo Liang_, Chenfei Wu_, Ting Song_, Wenshan Wu_, Yan Xia, Yu Liu, Yang Ou,Shuai Lu, Lei Ji, Shaoguang Mao, Yun Wang, Linjun Shou, Ming Gong, Nan Duany Microsoft{yalia, chewu, tsong, wenswu, yanxia, yluiu, yang.ou, shuailu, leiji,shaoguang.mao, wangyun, lisho, migon, nanduan}@microsoft.com, arXiv:2303.16434v1 [cs.AI] 29 Mar 2023 Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the conventional technology, when there are multiple applications (APIs) to select from, there is room for improvement in efficiently selecting an appropriate application. [Means for solving the problem]

[0005] A first aspect of the technology disclosed herein is a data processing device that includes an input unit that accepts user input indicating instruction content, a selection unit that selects a group corresponding to the instruction content from a plurality of groups into which a plurality of applications are classified based on functional similarity, a processing unit that performs processing corresponding to the instruction content based on a identification process using a data generation model that generates data corresponding to the input content, and an output unit that outputs a result of the identification process, wherein the processing unit obtains a result of the identification process using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is used as the input content, the identification process is a process of designating an application that performs processing corresponding to the instruction content from applications belonging to the group, and the output unit outputs information representing the designated application as a result of the identification process.

[0006] A second aspect of the technology disclosed herein is a data processing method including a receiving step in which a processor receives a user input indicating an instruction content, a selection step in which a group corresponding to the instruction content is selected from a plurality of groups into which a plurality of applications are classified based on functional similarity, a processing step in which processing corresponding to the instruction content is performed based on a identification process using a data generation model that generates data corresponding to the input content, and an output step in which a result of the identification process is output, wherein the processing step obtains a result of the identification process using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is used as the input content, the identification process is a process of designating an application that performs processing corresponding to the instruction content from applications belonging to the group, and the output step is a process of outputting information representing the designated application as a result of the identification process.

[0007] A third aspect of the technology disclosed herein is a program for causing a computer to execute a process including a receiving step of receiving a user input indicating an instruction content, a selection step of selecting a group corresponding to the instruction content from a plurality of groups into which a plurality of applications are classified based on functional similarity, a processing step of performing processing corresponding to the instruction content based on a identification process using a data generation model that generates data corresponding to the input content, and an output step of outputting a result of the identification process, wherein the processing step obtains a result of the identification process using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is used as the input content, the identification process is a process of designating an application that performs processing corresponding to the instruction content from applications belonging to the group, and the output step is a process of outputting information representing the designated application as a result of the identification process. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2]FIG. 2 is a conceptual diagram showing an example of main functions of a data processing device and a smart device. [Diagram 3] The outline of the specific process is shown below. [Figure 4] 2 shows a schematic functional configuration of a specific processing unit of the data processing device. [Diagram 5] 13 illustrates an example of an operational flow of a specific process performed by a data processing device. [Figure 6] 11 shows an example of instruction contents and presented information. [Figure 7] 1 is an example of a prompt generated by a data processing device. [Figure 8] 13 illustrates an example of an operational flow of a reception and output process by a terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a program according to the technology of the present disclosure will be described with reference to the accompanying drawings.

[0010] First, the terms used in the following description will be explained.

[0011] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0012] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0013] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0014] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0015] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0016] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0017] As shown in Fig. 1, a data processing system 10 includes a data processing device 12 and a terminal device 14. An example of the data processing device 12 is a server. An example of the terminal device 14 is a personal computer such as a laptop or desktop computer, or a smart device such as a smartphone or smart glasses. In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure.

[0018] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0019] The terminal device 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0020] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicating the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicating the user input.

[0021] The output device 40 includes a display 40A and a speaker 40B, and presents data to the person 20 by outputting the data in a form of expression that the person 20 can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a Complementary Metal-Oxide-Semiconductor (CMOS) image sensor or a Charge Coupled Device (CCD) image sensor.

[0022] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0023] FIG. 2 shows an example of the main functions of the data processing device 12 and the terminal device 14.

[0024] The data processing device 12 of this embodiment is a server device that provides a large-scale language model. As shown in FIG. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific process program 56 is stored in the storage 32. The specific process program 56 is an example of a "program" according to the technology of the present disclosure. The processor 28 reads the specific process program 56 from the storage 32 and executes the read specific process program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific process unit 290 according to the specific process program 56 executed on the RAM 30. By operating in this manner, the specific process unit 290 functions as a large-scale language model using the data generation model 58.

[0025] The storage 32 stores a data generation model 58. The data generation model 58 is used by a specific processing unit 290. The storage 32 also stores application definition information 60, the details of which will be described later.

[0026] In the terminal device 14, the reception output processing is performed by the processor 46. A reception output program 62 is stored in the storage 50. The reception output program 62 is used in conjunction with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 62 from the storage 50, and executes the read reception output program 62 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 62 executed on the RAM 48.

[0027] Furthermore, the storage 50 stores a plurality of applications that can be used in the terminal device 14 as an application group 63. In this embodiment, the term "application" is used as a general term including various types of software, APIs, and VFMs (Visual Foundation Models).

[0028] Next, the processing of the specific processing unit 290 when the data processing device 12 functions as a large-scale language model using the data generation model 58 will be described.

[0029] Fig. 3 is a diagram showing a schematic diagram of how the identification processing unit 290 functions as a large-scale language model. As shown in Fig. 3, a plurality of applications that are the application group 63 installed in the terminal device 14 are classified by the identification processing unit 290 into a plurality of application groups AG (n applications: AG_1 to AG_n in Fig. 3) based on similarity in function. The application group AG in this embodiment is an example of a "group" according to the technology of the present disclosure. The condition for functional similarity may be, for example, having the same type of function. Specifically, for example, it may be a group of applications for browsing Web (World Wide Web) sites, a group of applications for editing images, etc.

[0030] The user 310 inputs instruction contents 320, which indicate the contents of the processing to be instructed, to the terminal device 14, and the instruction contents 320 are transmitted from the terminal device 14 to the data processing device 12. The identification processing unit 290 selects an application group AG according to the instruction contents 320 from among the multiple application groups AG included in the application group 63. The identification processing unit 290 then generates a prompt P, which includes application definition information 60, which is a definition of each application included in the selected application group AG (AG_1 in FIG. 3), the instruction contents 320, and an instruction sentence for specifying an application, and inputs the generated prompt P to the data generation model 58. Specifically, the prompt P is a prompt for designating an application name AN, which indicates the name of an application suitable for executing the instruction contents 320, from among the multiple applications included in the application group AG (AG_1 in FIG. 3), with reference to the application definition information 60. Here, the application definition information 60 is information indicating matters necessary for selecting an application based on a function. The application definition information 60 explains how to use an application. As an example, the definition of an application in this embodiment includes the name, usage, input / output, and usage examples of the application, but the items included in the definition of an application are not limited to those exemplified in this embodiment. For example, different items may be included depending on the function, type, etc. of the application.

[0031] In this embodiment, the application definition information 60 is provided by a provider of the application, such as an application developer, in association with the application. The application definition information 60 may use the information provided in association with the application as is, or may be generated by the specific processing unit 290 from the information provided in association with the application. The application definition information 60 is not limited to being provided by the application developer, and may be provided by other means. Specifically, for example, the application definition information 60 may be provided by a system administrator. The application definition information 60 may also be provided by feedback from a user who has used each application via this system.

[0032] In response to the input prompt P, the data generation model 58 outputs an application name AN ​​that indicates an application suitable for executing the process corresponding to the instruction content 320.

[0033] The terminal device 14 executes processing according to the instruction content 320 using an application corresponding to the application name AN ​​from among a plurality of applications stored in the application group 63.

[0034] In this manner, according to the data processing device 12 of this embodiment, the specific processing unit 290 functions as a large-scale language model using the data generation model 58, so that processing according to the instruction content 320 of the user 310 is executed by an appropriate application, and the execution result is provided to the user 310.

[0035] In the data processing device 12 of this embodiment, an application appropriate for executing processing according to the instruction content 320 is specified by the data generation model 58 from among the application groups AG selected from among a plurality of application groups AG classified based on functional similarities. Therefore, according to the data processing device 12 of this embodiment, even when there are a plurality of applications to be selected, it is possible to efficiently select an appropriate application.

[0036] As shown in FIG. 4, the specific processing unit 290 includes an input unit 292, a selection unit 293, a processing unit 294, and an output unit 296.

[0037] The input unit 292 acquires a user input received by the terminal device 14. Specifically, the user input is an instruction content 320 that indicates the content of the process instructed by the user. The user input may be a text input or a voice input, and the input method is not limited. For example, user inputs such as "Please draw a picture of a cat," "Please change the background to a night sky," "Please create materials for a presentation," and "Please create the text of an email" are sent from the terminal device 14 to the data processing device 12.

[0038] The selection unit 293 acquires the application definition information 60 of each of the applications included in the application group 63 of the terminal device 14 from the terminal device 14, and stores the application definition information 60 in the storage 32. Furthermore, the selection unit 293 classifies the applications included in the application group 63 into a plurality of application groups AG based on the similarity of functions based on the application definition information 60. Note that the method by which the selection unit 293 classifies the applications into the application groups AG based on the application definition information 60 is not limited. For example, for each item included in the application definition information 60, a numerical value representing similarity may be derived based on the degree of word agreement or the like, and applications having a threshold representing similarity equal to or greater than a predetermined value may be grouped together into one application group AG. Furthermore, for example, the application definition information 60 of each of the applications and a prompt P including an instruction for classifying the applications into a plurality of application groups AG based on the similarity of the definition information may be generated, and input to the data generation model 58, so that the data generation model 58 classifies the application groups AG. Furthermore, for example, when the application definition information 60 includes information regarding the use of the application, the selection unit 293 may classify the applications into the application groups AG based on the information. Furthermore, for example, with regard to an API, the selection unit 293 may classify the application groups AG based on a document that is an API specification that describes the definition of the input and output.

[0039] The selection unit 293 selects an app group AG corresponding to the instruction content 320 from the multiple app groups AG. Note that the method by which the selection unit 293 selects an app group AG corresponding to the calibration image 430 from the multiple app groups AG is not particularly limited. For example, a tag indicating the content of the included application may be assigned to the app group AG, and the selection unit 293 may select the app group AG most related to the instruction content 320 based on the tag assigned to the app group AG. Also, as the tag in this case, for example, a word having a high degree of match among words included in the app definition information 60 of each application may be used.

[0040] Unlike the present embodiment, the application definition information 60 does not have to be stored in the storage 32 of the data processing device 12. In this case, the selection unit 293 appropriately acquires the application definition information 60 from each of a plurality of applications included in the application group 63 of the terminal device 14.

[0041] The processing unit 294 performs specific processing using the data generation model 58 .

[0042] Specifically, when the input unit 292 receives an instruction content 320 from the terminal device 14, the processing unit 294 generates a prompt P including an instruction sentence for designating an application suitable for executing processing according to the instruction content 320 from among the applications included in the application group AG selected by the selection unit 293, and inputs the prompt P to the data generation model 58. For example, when the instruction content 320 of "Please change the background to a night sky" is received, the processing unit 294 generates a prompt P (see also FIG. 7) saying "Please tell me the best application for changing the background of a picture in XX format to a night sky. The definition of each application is..." and inputs the prompt P to the data generation model 58. Then, the processing unit 294 acquires the application name AN ​​output from the data generation model 58.

[0043] As a result of the identification process, the output unit 296 transmits the application name AN ​​specified by the data generation model 58 and an execution instruction for executing the instruction content 320 by the application corresponding to the application name AN ​​to the terminal device 14. In the terminal device 14, the control unit 46A executes the instruction content 320 by the application corresponding to the application name AN.

[0044] The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is ChatGPT (Internet search<URL: https: / / openai.com / blog / chatgpt> ) and other generation AIs. The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 performs inference on the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data and text data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization.

[0045] Next, the operation of the data processing system 10 will be described.

[0046] First, an example of the flow of specific processing performed by the data processing device 12 will be described with reference to FIG.

[0047] In the identification process shown in FIG. 5, first, in step S100, the processing unit 294 acquires the instruction content 320 received by the input unit 292.

[0048] User 310 inputs instruction content 320 in terminal device 14. Fig. 6 shows an example of instruction content 320 displayed on display 40A of terminal device 14. In the example shown in Fig. 6, instruction content 320 includes "Please change the background of this image to a night scene", an image, and the file name of the image, "Image / absdepanda.png".

[0049] In the next step S102, the selection unit 293 acquires the application definition information 60 from the terminal device 14 and stores it in the storage 32. Note that if the application definition information 60 has been stored in advance in the storage 32 before executing the identification process shown in FIG. 5, the process of this step can be omitted.

[0050] In the next step S104, the selection unit 293 classifies the multiple applications included in the application set 63 into multiple application groups AG based on the similarity of the functions of each application, as described above. Note that if the classification into application groups AG has been performed before executing the identification process shown in Fig. 5 and information that can identify which applications are included in each application group AG has been obtained in advance, the process of this step can be omitted.

[0051] In the next step S106, the selection unit 293 selects the application group AG according to the instruction content 320 acquired in step S100, as described above.

[0052] In the next step S108, the processing unit 294 generates a prompt P including an instruction statement for designating an application suitable for executing the instruction content 320 from the applications included in the application group AG selected in step S106, as described above.

[0053] In the next step S110, the processing unit 294 acquires the application name AN ​​by using the data generation model 58 as described above. Specifically, the processing unit 294 inputs the prompt P generated in the above step S106 to the data generation model 58, and acquires the application name AN ​​output from the data generation model 58. FIG. 7 shows an example of the prompt P input to the data generation model 58 and an example of the output from the data generation model 58 according to the prompt P. The prompt P shown in FIG. 7 is an example of the prompt P generated by the processing unit 294 in response to the instruction content 320 shown in FIG. 6 described above. Specifically, the prompt P is a prompt generated when the instruction content 320 is an instruction to change the background of a certain image to a night sky, and the application group AG selected by the selection unit 293 includes four applications. More specifically, the processing unit 294 generates a prompt P including the data format of the image included in the instruction content 320, an instruction extracted from the instruction content 320, and the application definition information 60 included in the selected application group AG.

[0054] In the next step S112, the output unit 296 outputs, as described above, the application name AN ​​acquired in step S110 above and an execution instruction for executing the instruction content 320 by an application corresponding to the application name AN ​​to the terminal device 14 via the communication I / F 26.

[0055] In the next step S114, the input unit 292 acquires the execution result of the process according to the instruction content 320 from the terminal device 14 via the communication I / F 26. For example, in the example shown in Fig. 6, the image 400, which is the execution result of the process according to the instruction content 320, is acquired.

[0056] In the next step S116, the output unit 296 generates presentation information for presenting the execution result acquired in the above step S114 to the terminal device 14. For example, the output unit 296 generates information including the execution result and the application name AN ​​of the used application (see FIG. 6, presentation information 410).

[0057] In the next step S118, the output unit 296 outputs the presentation information generated in step S116 to the terminal device 14 via the communication I / F 26.

[0058] In the next step S120, it is determined whether or not to end the identification process shown in Fig. 5. Until a predetermined end condition is satisfied, the determination in step S120 is a negative determination, the process returns to step S100, and the processes in steps S100 to S118 are repeated. In this embodiment, an example of the end condition is when an end instruction is received from the terminal device 14. Therefore, the processes in steps S100 to S118 are repeated until an end instruction is received from the terminal device 14. On the other hand, if the predetermined end condition is satisfied, the determination in step S120 is a positive determination, and the identification process shown in Fig. 5 is terminated.

[0059] Next, an example of the flow of the reception and output process executed by the terminal device 14 will be described with reference to FIG.

[0060] In the reception and output process shown in FIG. 8, first, in step S200, the control unit 46A transmits the instruction content 320 input by the user 310 to the communication I / F 44.

[0061] In the next step S202, the control unit 46A receives, via the communication I / F 44, the application name AN ​​and the process execution instruction output by the data processing device 12 in the process of step S112 of the above-mentioned identification process (see FIG. 5).

[0062] In the next step S204, control unit 46A selects an application corresponding to application name AN ​​from among the multiple applications included in application group 63, and executes instruction content 320 using the selected application. In the example shown in FIG. 6, image 400 is obtained as a result of executing instruction content 320.

[0063] In the next step S206, the control unit 46A transmits the execution result obtained by the process of step S204 to the data processing device 12 via the communication I / F 44. In the example shown in FIG.

[0064] In the next step S208, the control unit 46A receives, via the communication I / F 44, the presentation information 410 output by the data processing device 12 in the process of step S118 of the above-mentioned identification process (see FIG. 5).

[0065] In the next step S210, the control unit 46A causes the display 40A to display the presentation information 410 as illustrated in FIG.

[0066] In the next step S212, the control unit 46A determines whether or not to end the reception output process shown in Fig. 8. Until an instruction to end the process is input by the user 310, the determination in step S212 is negative, the process returns to step S200, and the processes in steps S200 to S210 are repeated. On the other hand, when an instruction to end the process is input by the user 310, the determination in step S212 is positive, and the process proceeds to step S214.

[0067] In step S214, the control unit 46A transmits an end instruction to the data processing device 12 via the communication I / F 44. When the process of step S214 ends, the acceptance and output process shown in FIG. 8 ends.

[0068] As described above, the data processing device 12 includes an input unit 292 that accepts a user input indicating the instruction content 320, and a selection unit 293 that selects an application group AG corresponding to the instruction content 320 from a plurality of application groups AG obtained by classifying a plurality of applications based on functional similarities. The data processing device 12 also includes a processing unit 294 that performs processing corresponding to the instruction content 320 based on a specification process using a data generation model 58 that generates data corresponding to the input content, and an output unit 296 that outputs a result of the specification process.

[0069] The processing unit 294 acquires the result of the identification process by using the output of the data generation model 58 when the input contents are application definition information 60, which is a definition of the application related to the selected application group AG, and a prompt P including the instruction content 320. The identification process is a process of specifying an application that executes a process according to the instruction content 320 from the applications belonging to the application group AG. The output unit 296 outputs information indicating the specified application as a result of the identification process.

[0070] In this manner, in the data processing device 12, a plurality of applications are classified into a plurality of app groups AG based on the similarity of their functions, and the selection unit 293 selects an application for processing the instruction content 320 from the selected app group AG. Therefore, even when there are a plurality of applications to select from, it is possible to efficiently select an appropriate application.

[0071] As described above, the condition for functional similarity may be that the apps have the same type of functions. Even apps of the same type may have different functions and operating speeds, and the optimal app may differ depending on the work to be performed by the app. Specifically, when there is an image editing app A that is multifunctional but requires many resources to operate, and an image editing app B that has limited functions but does not require many resources, if the process can be handled by the image editing app B, the image editing app B may be used to complete the process faster. In such a case, by further managing apps included in a group of apps of the same type as groups according to the same type of functions, it becomes possible to select the optimal app in response to a user's instruction.

[0072] If the application group 63 does not include an application suitable for executing processing according to the instruction content 320, for example, if there is no application capable of executing processing according to the instruction content 320, the output unit of the data processing device 12 may output information indicating that the instruction content 320 cannot be executed to the terminal device 14. For example, if the instruction content 320 is content related to image processing and the application group 63 does not include an application that executes image processing, the output unit may output information indicating that the instruction content 320 cannot be executed to the terminal device 14.

[0073] Note that the method of determining whether or not an application suitable for executing a process according to the instruction content 320 exists in the data processing device 12 is not limited. For example, when the selection unit 293 cannot select the application group AG, it may determine that an application suitable for executing a process according to the instruction content 320 does not exist. In addition, in the above embodiment, the data generation model 58 always specifies an application suitable for executing the instruction content 320, but the data generation model 58 may determine whether or not an application suitable for executing a process according to the instruction content 320 exists.

[0074] Furthermore, as described above, when it is determined that there is no application appropriate or executable for executing the process according to the instruction content 320, the output unit 296 of the data processing device 12 may output, for example, information recommending the installation of an application that can appropriately execute the process according to the instruction content 320 to the terminal device 14. For example, for applications that can be installed in the terminal device 14 via the network 54, the selection unit 293 may select an application appropriate for executing the process according to the instruction content 320 by referring to the usage method of the application definition information 60 or the like, and the output unit 296 may output information recommending the installation of the selected application to the terminal device 14.

[0075] Further, the data processing device 12 may have programming capabilities. In this case, as described above, when it is determined that there is no application suitable or executable for executing the process according to the instruction content 320, an application suitable for executing the process according to the instruction content 320 may be created, and the created application may be executed. In this case, the created application may be included in the application group 63 to enable reuse. The created application may also be made available in other devices. When the data processing device 12 creates an application in this way, the application definition information 60 of the created application is also created.

[0076] In addition, in the above embodiment, a form has been described in which the data generation model 58 specifies one application, but if the processing corresponding to the instruction content 320 can be executed by combining multiple applications, the form may also be such that the data generation model 58 can specify the multiple applications to be combined.

[0077] As described above, the selection unit 293 may classify the applications into the app groups AG before executing the identification process. In this case, it is preferable that the selection unit 293 obtains information that can identify which applications are included in each app group AG in advance. Since the applications included in the app group 63 of the terminal device 14 change, when the selection unit 293 classifies the applications into the app groups AG before executing the identification process, it is preferable to classify the applications into the app groups AG at a predetermined timing. For example, the selection unit 293 may periodically monitor the applications included in the app group 63 of the terminal device 14, and when a new application is installed, the selection unit 293 may classify the newly installed application into one of the app groups AG. Alternatively, in this case, all applications included in the app group 63 may be newly classified into the app groups AG.

[0078] Furthermore, the application group AG may have a hierarchical structure in which subgroups exist.

[0079] As described above, the application definition information 60 is provided by a provider that provides the application together with the application, but a so-called generation AI or the like may be used to generate this application definition information 60. For example, the generation AI may ask the provider about items included in the application definition information 60, and generate and output the application definition information 60 according to the contents of the information asked. In this case, the generation AI or the like may also automatically check the consistency between the application definition information 60 and the actual application, that is, without the manual effort of a developer or the like.

[0080] As described above, since an application is selected based on the application definition information 60, information regarding the use of the application may be fed back to the creator of the application definition information 60 (the provider of the application in the above embodiment). By providing feedback in this manner, the content of the application definition information 60 can be made more appropriate.

[0081] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0082] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0083] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0084] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0085] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0086] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0087] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0088] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0089] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0090] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, or standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0091] 10 Data Processing System 12 Data Processing Device 14 Terminal Equipment

Claims

1. an input unit that accepts user input indicating an instruction content; a selection unit that selects a group according to the instruction from a plurality of groups into which a plurality of applications are classified based on similarity in function; A processing unit that performs processing according to the instruction content based on a specific process using a data generation model that generates data according to the input content; an output unit that outputs a result of the specific processing, the processing unit acquires a result of the identification process by using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is used as the input content; and the identifying process is a process of designating an application that executes a process according to the instruction content from among applications that belong to the group; The output unit outputs information indicating a specified application as a result of the identification process. Data processing device.

2. The definition includes the name of the application, the method of use, inputs and outputs, and examples of use.

2. A data processing apparatus according to claim 1.

3. The selection unit classifies, when a new application is installed, the newly installed application into one of the plurality of groups.

2. A data processing apparatus according to claim 1.

4. When a new application is installed, the selection unit performs the classification for all the installed applications.

2. A data processing apparatus according to claim 1.

5. If the plurality of applications does not include an application capable of executing a process corresponding to the instruction content, The output unit outputs information indicating that the instruction content cannot be executed.

2. A data processing apparatus according to claim 1.

6. If the plurality of applications does not include an application capable of executing a process corresponding to the instruction content, The output unit outputs information recommending the installation of an application capable of executing a process according to the instruction content.

2. A data processing apparatus according to claim 1.

7. The processor: a receiving step of receiving a user input indicating an instruction content; a selection step of selecting a group according to the instruction from a plurality of groups into which a plurality of applications are classified based on similarity in function; A processing step of performing processing according to the instruction content based on a specific process using a data generation model that generates data according to the input content; and an output step of outputting a result of the specific processing, the processing step, acquiring a result of the identification process using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is input as the input content; the identifying process is a process of designating an application that executes a process according to the instruction content from among applications that belong to the group; The output step outputs information representing a designated application as a result of the identification process. Data processing methods.

8. a receiving step of receiving a user input indicating an instruction content; a selection step of selecting a group according to the instruction from a plurality of groups into which a plurality of applications are classified based on similarity in function; A processing step of performing processing according to the instruction content based on a specific process using a data generation model that generates data according to the input content; and an output step of outputting a result of the specific processing, the processing step, acquiring a result of the identification process using an output of the data generation model when data including a definition of an application related to the selected group and the instruction content is input as the input content; the identifying process is a process of designating an application that executes a process according to the instruction content from among applications that belong to the group; The output step outputs information representing a designated application as a result of the identification process. A program for causing a computer to execute a process including the above.

Images