Data processing device, data processing method, and data processing program

Abstract

To enable appropriate response to a user input.SOLUTION: A data processing device disclosed herein is configured to: acquire a specific video including a video image showing predetermined work content and audio related to the work content as input data; input a prompt including the acquired specific video and an instruction for generating explanatory information explaining the work content shown in the specific video to a generative model to generate explanatory information corresponding to the prompt; and output the generated explanatory information to an electronic device as a result of specific processing.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The technology disclosed herein relates to a data processing device, a data processing method, and a data processing program. [Background technology]

[0002] Patent document 1 discloses a persona chatbot control method performed by at least one processor, the method including the steps of receiving a user utterance, adding the user utterance to a prompt including a description and related instruction sentence regarding the chatbot character, encoding the prompt, and inputting the encoded prompt into a language model to generate a chatbot utterance in response to the user utterance. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2022-180282 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the prior art leaves room for improvement in responding appropriately to user input. [Means for solving the problem]

[0005] A first aspect of the data processing device includes an acquisition unit that acquires input data indicating user input, a processing unit that performs specific processing using a generative model that generates information corresponding to the input data, and an output unit that outputs a result of the specific processing to an electronic device, wherein the acquisition unit acquires a specific video as the input data, the specific video including a video showing a specified work content and audio related to the work content, the processing unit inputs the specific video acquired by the acquisition unit and a prompt including an instruction to generate explanatory information explaining the work content shown in the specific video to the generative model to generate explanatory information corresponding to the prompt, and the output unit outputs the explanatory information generated by the processing unit to the electronic device as a result of the specific processing.

[0006] A data processing device of a second aspect is the data processing device of the first aspect, wherein the prompt includes an instruction to express an emphasized portion shown in the specific video in an emphasized manner in the explanation information.

[0007] A data processing device of a third aspect is a data processing device of the first or second aspect, wherein the acquisition unit acquires, as the input data, attribute information indicating attributes of a user who is to view the explanatory information, and the processing unit inputs the specific video and the attribute information acquired by the acquisition unit and a prompt including instructions to generate the explanatory information into the generative model, thereby generating explanatory information corresponding to the prompt.

[0008] A data processing device of a fourth aspect is a data processing device of any one of the first to third aspects, wherein the processing unit fine-tunes the generative model using multiple organization data related to a specified organization held by the specified organization as learning data.

[0009] A data processing device of a fifth aspect is a data processing device of any of the first to fourth aspects, wherein the acquisition unit acquires past explanatory information generated by the processing unit as the input data, and the processing unit inputs a prompt including the specific video and the past explanatory information acquired by the acquisition unit and an instruction to update the past explanatory information into the generative model, thereby generating explanatory information corresponding to the prompt.

[0010] A data processing method of the sixth aspect involves a computer executing a process in which input data indicating a user input is acquired, a specific process is performed using a generative model that generates information corresponding to the input data, a result of the specific process is output to an electronic device, a specific video including a video showing a specified work content and audio related to the work content is acquired as the input data, a prompt including the acquired specific video and an instruction to generate explanatory information explaining the work content shown in the specific video is input to the generative model, explanatory information corresponding to the prompt is generated, and the generated explanatory information is output to the electronic device as a result of the specific process.

[0011] A data processing program of the seventh aspect causes a computer to execute the following processes: acquire input data indicating user input, perform a specific processing using a generative model that generates information corresponding to the input data, output a result of the specific processing to an electronic device, acquire as the input data a specific video including video showing a specified work content and audio related to the work content, input the acquired specific video and a prompt including instructions to generate explanatory information explaining the work content shown in the specific video to the generative model, generate explanatory information corresponding to the prompt, and output the generated explanatory information to the electronic device as a result of the specific processing. [Brief description of the drawings]

[0012] [Figure 1] FIG. 1 is a conceptual diagram illustrating an example of a configuration of a data processing system. [Diagram 2] 2 is a conceptual diagram showing an example of main functions of a data processing device and a user terminal. FIG. [Diagram 3] 2 shows a schematic functional configuration of a specific processing unit of the data processing device. [Figure 4] 13 illustrates an example of an operational flow of a specific process performed by a data processing device. [Diagram 5] FIG. 11 is a first diagram showing an example of a display on a display of a user terminal. [Figure 6] FIG. 2 is a second diagram showing an example of a display on a user terminal. [Figure 7] FIG. 3 is a third diagram showing an example of a display on a user terminal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0013] Hereinafter, an example of an embodiment of a data processing device, a data processing method, and a data processing program according to the technique of the present disclosure will be described with reference to the accompanying drawings.

[0014] First, the terms used in the following description will be explained.

[0015] In the following embodiments, a signed processor (hereinafter simply referred to as a "processor") may be one arithmetic device or a combination of multiple arithmetic devices. Furthermore, a processor may be one type of arithmetic device or a combination of multiple types of arithmetic devices. Examples of arithmetic devices include a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), a GPGPU (General-Purpose computing on Graphics Processing Units), an APU (Accelerated Processing Unit), or a TPU (Tensor Processing Unit).

[0016] In the following embodiments, a signed RAM (Random Access Memory) is a memory in which information is temporarily stored and is used as a working memory by the processor.

[0017] In the following embodiments, the coded storage is one or more non-volatile storage devices that store various programs, various parameters, etc. Examples of non-volatile storage devices include flash memory (SSD (Solid State Drive)), magnetic disks (e.g., hard disks), and magnetic tapes.

[0018] In the following embodiments, a communication I / F (Interface) with a code is an interface including a communication processor and an antenna. The communication I / F controls communication between multiple computers. An example of a communication standard applied to the communication I / F is a wireless communication standard including 5G (5th Generation Mobile Communication System), Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.

[0019] In the following embodiments, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. In addition, in this specification, the same idea as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0020] FIG. 1 shows an example of the configuration of a data processing system 10 according to the embodiment.

[0021] As shown in Fig. 1, the data processing system 10 includes a data processing device 12 and a user terminal 14. An example of the data processing device 12 is a server. An example of the user terminal 14 is a PC (Personal Computer). In this embodiment, the data processing device 12 is an example of a "data processing device" according to the technology of the present disclosure, and the user terminal 14 is an example of an "electronic device" according to the technology of the present disclosure.

[0022] The data processing device 12 includes a computer 22, a database 24, and a communication I / F 26. The computer 22 is an example of a "computer" according to the technology of the present disclosure. The computer 22 includes a processor 28, a RAM 30, and a storage 32. The processor 28, the RAM 30, and the storage 32 are connected to a bus 34. The database 24 and the communication I / F 26 are also connected to the bus 34. The communication I / F 26 is connected to a network 54. Examples of the network 54 include a wide area network (WAN) and / or a local area network (LAN).

[0023] The user terminal 14 includes a computer 36, a reception device 38, an output device 40, a camera 42, and a communication I / F 44. The computer 36 includes a processor 46, a RAM 48, and a storage 50. The processor 46, the RAM 48, and the storage 50 are connected to a bus 52. The reception device 38, the output device 40, and the camera 42 are also connected to the bus 52.

[0024] The reception device 38 includes a touch panel 38A and a microphone 38B, and receives user input. The touch panel 38A detects contact with a pointer (e.g., a pen or a finger) to receive user input by the touch of the pointer. The microphone 38B detects the user's voice to receive user input by voice. The control unit 46A transmits data indicative of the user input received by the touch panel 38A and the microphone 38B to the data processing device 12. In the data processing device 12, the specific processing unit 290 acquires the data indicative of the user input.

[0025] The output device 40 includes a display 40A and a speaker 40B, and presents data to a person by outputting the data in a form of expression that the person can perceive (e.g., voice and / or text). The display 40A displays visible information such as text and images according to instructions from the processor 46. The speaker 40B outputs voice according to instructions from the processor 46. The camera 42 is a compact digital camera equipped with an optical system including a lens, an aperture, and a shutter, and an imaging element such as a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor or a CCD (Charge Coupled Device) image sensor.

[0026] The communication I / F 44 is connected to a network 54. The communication I / Fs 44 and 26 are responsible for transmitting and receiving various types of information between the processor 46 and the processor 28 via the network 54.

[0027] FIG. 2 shows an example of main functions of the data processing device 12 and the user terminal 14.

[0028] As shown in Fig. 2, in the data processing device 12, a specific process is performed by the processor 28. A specific processing program 56 is stored in the storage 32. The specific processing program 56 is an example of a "data processing program" according to the technology of the present disclosure. The processor 28 reads the specific processing program 56 from the storage 32, and executes the read specific processing program 56 on the RAM 30. The specific process is realized by the processor 28 operating as a specific processing unit 290 in accordance with the specific processing program 56 executed on the RAM 30.

[0029] The storage 32 stores a data generation model 58. The data generation model 58 is used by the specific processing unit 290.

[0030] In the user terminal 14, the reception output processing is performed by the processor 46. A reception output program 60 is stored in the storage 50. The reception output program 60 is used together with the specific processing program 56 by the data processing system 10. The processor 46 reads out the reception output program 60 from the storage 50, and executes the read reception output program 60 on the RAM 48. The reception output processing is realized by the processor 46 operating as the control unit 46A in accordance with the reception output program 60 executed on the RAM 48.

[0031] Next, a description will be given of the processing of the specific processing unit 290 when the data processing device 12 performs specific processing for generating information corresponding to input data indicating a user input. FIG.

[0032] As shown in FIG. 3, the specific processing unit 290 includes an acquisition unit 292, a processing unit 294, and an output unit 296.

[0033] The acquisition unit 292 acquires input data indicating a user input received by the user terminal 14. Specifically, the acquisition unit 292 acquires input data based on a text input and a voice input of the user received by the user terminal 14. The acquisition unit 292 also acquires a specific video including a video showing a specific work content and a voice related to the work content transmitted from the user terminal 14 as input data. The video included in the specific video is, for example, a recorded display content of the display 40A of the user terminal 14 during a demonstration of executing a specific function on a computer. The voice included in the specific video is, for example, the voice of the user explaining the work content performed using the user terminal 14 during a demonstration of executing a specific function on a computer.

[0034] The processing unit 294 performs a specific process using the data generation model 58. The data generation model 58 is a so-called generative AI (Artificial Intelligence). An example of the data generation model 58 is a generative AI such as ChatGPT. ChatGPT is, for example, an Internet search engine.<URL: https: / / openai.com / blog / chatgpt> and internet searches<URL: https: / / aismiley.co.jp / ai_news / chatgpt-tsukattemita / > Since this is publicly known as disclosed in, for example, US Pat. No. 6,399,343, detailed description thereof will be omitted.

[0035] The data generation model 58 is obtained by making a neural network perform deep learning. A prompt including an instruction is input to the data generation model 58, and inference data such as voice data indicating a voice, text data indicating a text, and image data indicating an image is input. The data generation model 58 infers the input inference data according to the instruction indicated by the prompt, and outputs the inference result in a data format such as voice data, text data, and image data. Here, inference refers to, for example, analysis, classification, prediction, and / or summarization. The data generation model 58 is an example of a "generation model" according to the technology of the present disclosure.

[0036] The output unit 296 outputs the result of the specific processing to the user terminal 14. In the user terminal 14, the control unit 46A causes the output device 40 to output the result of the specific processing.

[0037] Next, an example of the flow of the specific processing will be described with reference to Fig. 4. Note that the flow of the specific processing shown in Fig. 4 is an example of the "data processing method" according to the technology of the present disclosure.

[0038] Fig. 4 shows an example of an operation flow relating to the operation of the specific processing unit 290 performing specific processing to generate explanatory information explaining the work content shown in the specific video. The explanatory information includes an operation manual showing the operation procedure for executing a specific function on a computer and learning materials for learning about the specific function. The operation flow shown in Fig. 4 is performed by the specific processing unit 290 reading out the specific processing program 56 from the storage 32, expanding it in the RAM 30, and executing it. The operation flow shown in Fig. 4 is automatically executed repeatedly, for example, at regular intervals.

[0039] In step S10 shown in Fig. 4, the identification processing unit 290 judges whether or not a predetermined trigger condition is satisfied. Here, if the identification processing unit 290 judges that the trigger condition is satisfied (step S10: YES), the process proceeds to step S11. On the other hand, if the identification processing unit 290 judges that the trigger condition is not satisfied (step S10: NO), the process ends the operation flow shown in Fig. 4. As an example, the identification processing unit 290 judges that the trigger condition is satisfied when it acquires input data indicating a user input that a creation start button 76 (see Fig. 5) or an update start button 79 (see Fig. 7), which will be described later, has been operated.

[0040] In step S11, the identification processing unit 290 acquires input data. For example, the identification processing unit 290 acquires a specific moving image as the input data. Then, the identification processing unit 290 proceeds to step S12.

[0041] In step S12, the specific processing unit 290 generates a prompt to be input to the data generation model 58 based on the input data acquired in step S11. For example, the specific processing unit 290 generates a prompt including a specific video acquired as input data, an instruction to generate explanatory information explaining the work content shown in the specific video, and an instruction to emphasize and express the emphasized part shown in the specific video in the explanatory information. The emphasized part is a part emphasized by the user using intonation such as a difference in pitch and volume in the voice included in the specific video. The data generation model 58 infers the emphasized part in the voice included in the specific video by performing a known voice recognition process on the specific video input as the prompt. Here, the specific processing unit 290 generates a prompt such as "Please generate a work manual as explanatory information explaining the work content shown in the following specific video. Also, please emphasize and express the emphasized part shown in the specific video in the generated work manual. Specific video: yyyxxxzzz111.wav". Then, the specific processing unit 290 proceeds to step S13.

[0042] In step S13, the identification processing unit 290 inputs the prompt generated in step S12 to the data generation model 58, and acquires the result of the identification process based on the output of the data generation model 58. Here, the identification processing unit 290 acquires explanatory information (e.g., a work manual) corresponding to the prompt as the result of the identification process. Then, the identification processing unit 290 proceeds to step S14.

[0043] In step S14, the identification processing unit 290 outputs the result of the identification processing acquired in step S13 to the user terminal 14. This makes it possible to view the explanatory information on the user terminal 14. Note that as long as the explanatory information is possible to view on the user terminal 14, the output format of the result of the identification processing by the identification processing unit 290 is not limited. Then, the identification processing unit 290 ends the operation flow shown in FIG.

[0044] Next, a display example of the display 40A of the user terminal 14 will be described with reference to Fig. 5 to Fig. 7. Specifically, the display examples shown in Fig. 5 to Fig. 7 show the display contents of the display 40A while a specific application (hereinafter, "specific app") for generating and displaying explanation information is being executed.

[0045] Fig. 5 is a first diagram showing a display example of the display 40A of the user terminal 14. Specifically, Fig. 5 shows a generation screen for generating explanatory information in a specific application. As shown in Fig. 5, a selection button 70, a selection button 72, video information 74, and a creation start button 76 are displayed on the display 40A.

[0046] The selection button 70 is a button for selecting the generation of new explanatory information in the specific process shown in Fig. 4. The selection button 72 is a button for selecting the update of previously generated explanatory information in the specific process shown in Fig. 4. Only one of the selection button 70 and the selection button 72 can be selected. Fig. 5 shows an example in which the selection button 70 is selected by user input, and the inside of the frame of the selected selection button 70 is shown in black.

[0047] Video information 74 is a portion showing a specific video uploaded to a specific application. As an example, the specific video uploaded to a specific application in Fig. 5 is "video 74A". Note that the method of uploading the specific video is not particularly limited, and any known method may be used as appropriate.

[0048] The creation start button 76 is a button for starting the generation of explanatory information based on the specific video indicated in the video information 74. The creation start button 76 is operable only when the specific video has been uploaded to the specific application. Here, when the creation start button 76 is operated, the specification process shown in Fig. 4 proceeds to step S11, and the specific video indicated in the video information 74 (video 74A) is acquired as input data.

[0049] Fig. 6 is a second diagram showing a display example of the display 40A of the user terminal 14. Specifically, Fig. 6 shows an example of the explanatory information (work manual) generated in the specific process shown in Fig. 4 after the creation start button 76 is operated in the display example shown in Fig. 5.

[0050] As shown in FIG. 6, text information 80, a description image 82, a description image 84, and a description image 86 are displayed on the display 40A.

[0051] The text information 80 indicates the contents of the operation manual in text. As an example, in FIG. 6, "ToDo List Application User Manual 1. Adding a New ToDo 1-1. 1-2. " is displayed as the text information 80. As an example, the operation manual is a user manual on how to use a ToDo list application for executing a task management function as a predetermined function on a computer. Although details of the operation procedure are omitted, as described above, the display 40A displays the operation procedure for executing the task management function on a computer as the text information 80.

[0052] Explanatory image 82, explanatory image 84, and explanatory image 86 are images in which a portion of a specific video is captured in accordance with the corresponding portion of the sentence shown in text information 80 to aid in understanding of the sentence. Specifically, explanatory image 82 and explanatory image 84 are images in which a portion of a specific video is captured in order to aid in understanding of the portion of the sentence shown in text information 80 that reads "1. Add new ToDo 1-1." In addition, explanatory image 86 is an image in which a portion of a specific video is captured in order to aid in understanding of the portion of the sentence shown in text information 80 that reads "1. Add new ToDo 1-2."

[0053] Here, in the explanatory image 84, a highlight display 90 is applied to indicate a portion to be emphasized in the above-mentioned operation manual. As an example, the highlight display 90 indicates an emphasized portion in the audio included in the specific moving image inferred by the data generation model 58 by a dashed frame in the explanatory image 84.

[0054] Fig. 7 is a third diagram showing a display example of the display 40A of the user terminal 14. Specifically, Fig. 7 shows an update screen for updating previously generated explanation information in a specific application. As shown in Fig. 7, a selection button 70, a selection button 72, video information 74, past information 78, and an update start button 79 are displayed on the display 40A.

[0055] As described above, the video information 74 is a portion showing a specific video uploaded to a specific application. As an example, the specific video uploaded to a specific application in FIG. 7 is "video 74B." Specifically, video 74B is a video that includes an updated portion or a changed portion of the work content shown in video 74A shown in FIG.

[0056] The past information 78 is a portion showing past explanatory information uploaded to a specific application. As an example, the past explanatory information uploaded to a specific application in Fig. 7 is a "work manual 78A". Specifically, the work manual 78A is explanatory information (work manual) generated by the data generation model 58 using the moving image 74A shown in Fig. 5 as a specific moving image.

[0057] The update start button 79 is a button for starting an update of past explanatory information based on a specific video indicated in the video information 74 and past explanatory information indicated in the past information 78. The update start button 79 is operable only when a specific video and past explanatory information have been uploaded to a specific application. Here, when the update start button 79 is operated, the specification process shown in FIG. 4 proceeds to step S11, and the specific video (video 74B) indicated in the video information 74 and past explanatory information (work manual 78A) indicated in the past information 78 are obtained as input data. The flow of the specification process from step S12 onwards will be described below.

[0058] In step S12 shown in FIG. 4, the specific processing unit 290 generates a prompt to be input to the data generation model 58 based on the input data acquired in step S11. For example, the specific processing unit 290 generates a prompt including the specific video and past explanatory information acquired as input data, and an instruction to update the past explanatory information. Here, the specific processing unit 290 generates a prompt such as "Please update the following past explanatory information based on the work content shown in the following specific video. Specific video: yyyxxxzzz222.wav Past explanatory information: Work manual A.pdf". Then, the specific processing unit 290 proceeds to step S13.

[0059] In step S13, the identification processing unit 290 inputs the prompt generated in step S12 to the data generation model 58, and acquires the result of the identification process based on the output of the data generation model 58. Here, the identification processing unit 290 acquires the updated work manual corresponding to the prompt as the result of the identification process. Then, the identification processing unit 290 proceeds to step S14.

[0060] In step S14, the identification processing unit 290 outputs the result of the identification processing acquired in step S13 to the user terminal 14. This makes it possible to view the updated work manual on the user terminal 14. Then, the identification processing unit 290 ends the operation flow shown in FIG.

[0061] As described above, the specific processing unit 290 acquires, as input data, a specific video including a video showing a predetermined work content and a sound related to the work content. In addition, the specific processing unit 290 inputs the acquired specific video and a prompt including an instruction to generate explanatory information explaining the work content shown in the specific video to the data generation model 58, and generates explanatory information corresponding to the prompt. Then, the specific processing unit 290 outputs the generated explanatory information to the user terminal 14 as a result of the identification process. As a result, the data processing device 12 according to this embodiment can appropriately respond to a user input. Specifically, the data processing device 12 can appropriately generate explanatory information explaining the work content shown in the specific video based on the specific video acquired as input data showing the user input.

[0062] Moreover, the above-mentioned prompt input to the data generation model 58 includes an instruction to emphasize the emphasized part shown in the specific video in the explanatory information. As a result, according to the data processing device 12 of the present embodiment, the part emphasized by the user in the specific video can be appropriately emphasized and expressed in the explanatory information by using intonation such as pitch difference and volume difference.

[0063] Furthermore, the specific processing unit 290 acquires the generated past explanatory information as input data. Then, the specific processing unit 290 inputs the acquired specific video and past explanatory information, and a prompt including an instruction to update the past explanatory information, to the data generation model 58, and generates explanatory information corresponding to the prompt. As a result, according to the data processing device 12 of the present embodiment, it is possible to generate explanatory information in which the past explanatory information is appropriately updated, based on the specific video and past explanatory information acquired as input data.

[0064] Here, the specific processing unit 290 may acquire, as input data, attribute information indicating the attributes of the user who is to view the explanatory information. The attribute information is information indicating age, sex, educational background, work history, job title, family structure, and the like. For example, the specific processing unit 290 acquires attribute information input by the user who is to view the explanatory information. In this case, the specific processing unit 290 inputs the acquired specific video and attribute information, and a prompt including an instruction to generate explanatory information, to the data generation model 58, and generates explanatory information corresponding to the prompt. As a result, according to the data processing device 12 of this embodiment, explanatory information tailored to the attributes of the user who is to view the explanatory information can be generated, and the understanding of the user who views the explanatory information can be improved.

[0065] Furthermore, the specific processing unit 290 may fine-tune the data generation model 58 using a plurality of organization data related to the predetermined organization held by the predetermined organization as learning data. The predetermined organization includes a private corporation and a public corporation. The organization data is data that is not disclosed to the outside of the predetermined organization via the Internet or the like. As a result, according to the data processing device 12 of this embodiment, it is possible to generate not only explanatory information that explains general work contents, but also explanatory information that explains unique work contents of the predetermined organization.

[0066] Here, in the above embodiment, the image included in the specific video is a recording of the display contents of the display 40A of the user terminal 14 during a demonstration of executing a specific function on a computer, but is not limited to this. For example, the image may be an image captured by a camera installed in a specific facility. Similarly, the sound included in the specific video is not limited to the voice of a user explaining the work content performed using the user terminal 14 during a demonstration of executing a specific function on a computer. For example, the sound may be a sound made by a user appearing in the image captured by a camera installed in a specific facility.

[0067] The data processing system 10 according to the present disclosure has been described above mainly with respect to the functions of the data processing device 12, but the data processing system 10 according to the present disclosure is not necessarily implemented in a server. The data processing system 10 according to the present disclosure may be implemented as a general information processing system. The present disclosure may be implemented, for example, as a software program that runs on a personal computer, or an application that runs on a smartphone or the like. The method according to the present disclosure may be provided to a user in the form of SaaS (Software as a Service).

[0068] In the above embodiment, an example was given in which a specific process is performed by one computer 22, but the technology disclosed herein is not limited to this, and distributed processing of the specific process may be performed by multiple computers, including computer 22.

[0069] In the above embodiment, an example in which the specific processing program 56 is stored in the storage 32 has been described, but the technology of the present disclosure is not limited to this. For example, the specific processing program 56 may be stored in a portable computer-readable non-transitory storage medium such as a Universal Serial Bus (USB) memory. The specific processing program 56 stored in the non-transitory storage medium is installed in the computer 22 of the data processing device 12. The processor 28 executes the specific processing in accordance with the specific processing program 56.

[0070] In addition, the specific processing program 56 may be stored in a storage device such as a server connected to the data processing device 12 via the network 54, and the specific processing program 56 may be downloaded and installed on the computer 22 upon request from the data processing device 12.

[0071] It is not necessary to store all of the specific processing program 56 in a storage device such as a server connected to the data processing device 12 via the network 54, or to store all of the specific processing program 56 in the storage 32; only a portion of the specific processing program 56 may be stored.

[0072] As the hardware resource for executing the specific process, various processors as shown below can be used. An example of the processor is a CPU, which is a general-purpose processor that functions as a hardware resource for executing the specific process by executing software, i.e., a program. Another example of the processor is a dedicated electric circuit, which is a processor having a circuit configuration designed exclusively for executing the specific process, such as a Field-Programmable Gate Array (FPGA), a Programmable Logic Device (PLD), or an Application Specific Integrated Circuit (ASIC). Each processor has a built-in or connected memory, and each processor executes the specific process by using the memory.

[0073] The hardware resource that executes the specific process may be one of these various processors, or may be a combination of two or more processors of the same or different types (for example, a combination of multiple FPGAs, or a combination of a CPU and an FPGA). Also, the hardware resource that executes the specific process may be a single processor.

[0074] As an example of a configuration using one processor, first, one processor is configured by combining one or more CPUs and software, and this processor functions as a hardware resource that executes a specific process. Second, there is a configuration using a processor that realizes the functions of the entire system including multiple hardware resources that execute a specific process on a single IC chip, as typified by SoC (System-on-a-chip). In this way, a specific process is realized using one or more of the above-mentioned various processors as hardware resources.

[0075] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit that combines circuit elements such as semiconductor elements. The specific processes described above are merely examples. It goes without saying that unnecessary steps may be deleted, new steps may be added, or the order of processes may be changed without departing from the spirit of the invention.

[0076] The above description and illustrations are detailed descriptions of the parts related to the technology of the present disclosure, and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, function, action, and effect is an example of the configuration, function, action, and effect of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above description and illustrations, within the scope of the gist of the technology of the present disclosure. In addition, in order to avoid confusion and to facilitate understanding of the parts related to the technology of the present disclosure, the above description and illustrations omit explanations of technical common sense that do not require explanation in order to enable the implementation of the technology of the present disclosure.

[0077] All publications, patent applications, and standards mentioned in this specification are herein incorporated by reference to the same extent as if each individual publication, patent application, and standard was specifically and individually indicated to be incorporated by reference. [Explanation of symbols]

[0078] 10 Data Processing System 12 Data Processing Device 14 User terminal 290 Specific Processing Department 292 Acquisition Department 294 Processing section 296 Output section

Claims

1. an acquisition unit for acquiring input data indicative of a user input; A processing unit that performs a specific process using a generative model that generates information according to the input data; an output unit that outputs a result of the specific processing to an electronic device; The acquisition unit is As the input data, a specific video including a video showing a predetermined work content and a sound related to the work content is acquired, The processing unit includes: inputting the specific video acquired by the acquisition unit and a prompt including an instruction to generate explanatory information explaining the work content shown in the specific video into the generative model, thereby generating explanatory information corresponding to the prompt; The output unit is outputting the explanation information generated by the processing unit to the electronic device as a result of the identification process; Data processing device.

2. The prompt includes an instruction to emphasize the highlighted portion shown in the specific video in the description information.

2. A data processing apparatus according to claim 1.

3. The acquisition unit is acquiring, as the input data, attribute information indicating attributes of a user who is to view the explanatory information; The processing unit includes: inputting the specific video and the attribute information acquired by the acquisition unit, and a prompt including an instruction to generate the explanatory information, into the generation model, thereby generating explanatory information corresponding to the prompt; 2. A data processing apparatus according to claim 1.

4. The processing unit includes: Fine-tuning the generative model using a plurality of organization data related to the predetermined organization held by the predetermined organization as training data; 2. A data processing apparatus according to claim 1.

5. The acquisition unit is As the input data, past explanation information generated by the processing unit is acquired, The processing unit includes: inputting the specific video and the past explanatory information acquired by the acquisition unit, and a prompt including an instruction to update the past explanatory information, into the generation model, thereby generating explanatory information corresponding to the prompt; 2. A data processing apparatus according to claim 1.

6. Obtaining input data indicative of user input; performing a determination process using a generative model that generates information according to the input data; outputting a result of the specific processing to an electronic device; As the input data, a specific video including a video showing a predetermined work content and a sound related to the work content is acquired, inputting the acquired specific video and a prompt including an instruction to generate explanatory information explaining the work content shown in the specific video into the generative model, thereby generating explanatory information corresponding to the prompt; outputting the generated explanatory information to the electronic device as a result of the identification process; A data processing method in which processing is performed by a computer.

7. Obtaining input data indicative of user input; performing a determination process using a generative model that generates information according to the input data; outputting a result of the specific processing to an electronic device; As the input data, a specific video including a video showing a predetermined work content and a sound related to the work content is acquired, inputting the acquired specific video and a prompt including an instruction to generate explanatory information explaining the work content shown in the specific video into the generative model, thereby generating explanatory information corresponding to the prompt; outputting the generated explanatory information to the electronic device as a result of the identification process; A data processing program for causing a computer to execute processing.

Images