Program, information processing system and information processing method

Abstract

Provides information on operational processes, including tacit knowledge. [Solution] A program for causing one or more computers to execute the following steps: an action type receiving step for receiving an action type; a tentative process information obtaining step for obtaining tentative process information indicating a tentative action process by inputting a prompt including an instruction to generate an action process corresponding to the action type into a generation model; a tentative process information presenting step for presenting the tentative process information to an actor; a video obtaining step for obtaining an action video of the actor to whom the tentative process information was presented performing the action process; a process information obtaining step for obtaining process information by inputting a prompt including the action video and an instruction to generate process information indicating the action process into a generation model; and a process information output step for outputting the process information.

Description

[Technical Field]

[0001] The present disclosure relates to a program, an information processing system, and an information processing method. [Background technology]

[0002] Manuals describing motion processes are created in various fields, such as motion processes in manufacturing, experimental procedures in research and development, and series of motions in sports. Patent Document 1 discloses a video manual creation device that generates video manuals. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 7023427 Summary of the Invention [Problem to be solved by the invention]

[0004] In recent years, generative AI, particularly large-scale language models (LLMs), has acquired the ability to directly understand video and verbalize its content. Using such generative AI, it is possible to generate manuals from video. However, there are problems, such as the experimental manuals generated by generative AI only containing the word "stir," leaving the reader unable to understand detailed conditions such as rotation speed and temperature. Furthermore, there is the problem that it is difficult to transfer the so-called tacit knowledge of professional athletes or experts in a certain skill to successors or others.

[0005] The objective of the present disclosure is to provide information on operational processes, including tacit knowledge, based on video. [Means for solving the problem]

[0006] In order to solve the above-mentioned problems, one aspect of the program of the present disclosure is a program for causing one or more computers to execute the following steps: an action type receiving step for receiving an action type; a tentative process information acquisition step for acquiring tentative process information indicating a tentative action process by inputting a prompt including an instruction to generate an action process corresponding to the action type into a generative model; a tentative process information presentation step for presenting the tentative process information to an operator; a video acquisition step for acquiring an action video of the operator, to whom the tentative process information has been presented, performing the action process; a process information acquisition step for acquiring the process information by inputting a prompt including the action video and an instruction to generate process information indicating the action process into a generative model; and a process information output step for outputting the process information. [Effects of the Invention]

[0007] According to the present disclosure, it is possible to provide information on operational steps including tacit knowledge based on video. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram illustrating a system configuration. [Figure 2] 4 is a flowchart showing a process information generation process according to the first embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of tentative process information. [Figure 4] FIG. 10 is a diagram illustrating an example of process information. [Figure 5] 10 is a flowchart showing a process information generation process according to a second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] (First embodiment) 1 is an overall configuration diagram of a system 1 according to a first embodiment. The system 1 generates and outputs process information indicating a sequence of movements based on a video of a movement, such as an experimental procedure or a movement in a sport, which is a series of movements.

[0010] The system 1 includes an information processing device 10, a user device 20, a generative model server device 30, and a camera 40. The information processing device 10, the user device 20, the generative model server device 30, and the camera 40 are connected to each other so as to be able to communicate with each other via a communication network N such as the Internet.

[0011] The information processing device 10 is a computer or the like, and obtains process information from an operational video using a generative model 300 stored in a generative model server device 30. The information processing device 10 is configured by a computer or the like, and includes a control unit 100, a memory unit 110, and a communication unit 120.

[0012] The control unit 100 includes a processor such as a CPU (Central Processing Unit) and controls the operation of the information processing device 10. The storage unit 110 includes, for example, a HDD (Hard Disk Drive), RAM (Random Access Memory), ROM (Read Only Memory), and SSD (Solid State Drive). Furthermore, the storage unit 110 is not limited to being built into the information processing device 10, and may be a storage medium (for example, a USB memory) that is detachably attached to the information processing device 10. In this embodiment, the storage unit 110 stores various programs executed by the control unit 100 and various data.

[0013] The communication unit 120 includes a communication interface for communicating with an external device wirelessly or via a wired connection. The control unit 100 transmits and receives data to and from the generative model server device 30 and the user device 20 via the communication unit 120.

[0014] The generative model server device 30 is a computer or the like. The generative model server device 30 stores a generative model 300 that receives a question (prompt) as input and outputs an answer. The generative model 300 is a learning model generated by machine learning, and generates an answer in response to the prompt. The generative model may be a Generative Pre-trained Transformer or a language model based thereon. Examples of the generative model 300 include ChatGPT, GPT-3, GPT-4 by OpenAI, and GPT-J by EleutherAI.

[0015] The generative model server device 30 is used for large-scale natural language processing tasks, such as automatically performing machine translation, sentence generation, document classification, question answering, etc. The generative model 300 learns from a huge amount of data and then makes predictions for new input. The generative model server device 30 hosts the generative model 300, invokes the generative model 300 in response to a request, and returns the results. When the generative model server device 30 according to this embodiment receives a prompt from the information processing device 10, it performs computational processing to generate an answer sentence. The generated answer sentence is then transmitted to the information processing device 10.

[0016] The user device 20 is an information processing device used by an actor performing a series of actions and related parties. The user device 20 may be a portable device such as a smartphone, PC, or tablet. The user device 20 includes a control unit 200, a memory unit 210, a communication unit 220, an operation unit 230, and a display unit 240. The control unit 200, the memory unit 210, and the communication unit 220 are similar to the control unit 100, the memory unit 110, and the communication unit 120 of the information processing device 10, respectively. The operation unit 230 accepts user operations. Examples of the operation unit 230 include a keyboard and a mouse. The display unit 240 is, for example, a monitor, and displays various screens. The operation unit 230 and the display unit 240 may be integrated into one unit as a touch panel.

[0017] Next, a description will be given of the configuration of the control unit 100 of the information processing device 10. The control unit 100 executes a program stored in the storage unit 110 to function as an action type reception unit 101, an action video acquisition unit 102, an interaction unit 103, and an output processing unit 104. Note that, in the following, the processes described as being performed by the action type reception unit 101, the action video acquisition unit 102, the interaction unit 103, and the output processing unit 104 are processes performed by the control unit 100 executing the program.

[0018] The action type receiving unit 101 receives information indicating the type of action to be the target of the action process to be created. Examples of action types include experimental procedures, work processes such as metal processing, and a series of actions in sports. The action type is input in the user device 20, for example, in response to a user operation on the operation unit 230. The action type receiving unit 101 receives the action type from the user device 20 via the communication unit 120.

[0019] The motion video acquisition unit 102 acquires a motion video in which a motion corresponding to the motion type is captured. The motion video is captured by the camera 40. The motion video acquisition unit 102 acquires the motion video from the camera 40 via the communication unit 120. The motion video may be only video, or may also include audio.

[0020] The dialogue unit 103 transmits a prompt to the generation model server device 30 via the communication unit 120, thereby obtaining an answer obtained by the generation model 300 from the generation model server device 30. The dialogue unit 103 transmits, for example, a prompt input in response to a user operation on the user device 20. The dialogue unit 103 then obtains an answer corresponding to the prompt. The output processing unit 104 controls the output of information. For example, the output processing unit 104 outputs process information indicating an operation process to the user device 20 via the communication unit 120. The processing of each functional unit will be described in detail later.

[0021] 2 is a flowchart showing the process information generation process by the information processing device 10. First, in step S100, the action type receiving unit 101 receives an action type. Here, a case will be described in which the operation procedure of "organic synthesis reaction using t-BuLi (200 ml)" is acquired as the action type.

[0022] Next, in step S102, the dialogue unit 103 transmits a prompt including an instruction to generate an action process corresponding to the action type received in step S100 to the generative model server device 30. In the generative model server device 30, the prompt is input to the generative model 300. Then, process information indicating the action process is output from the generative model 300. Hereinafter, the process information created in step S102 will be referred to as provisional process information.

[0023] FIG. 3 is a diagram showing an example of tentative process information. The tentative process information indicates an operational process generated by the generative model 300. However, although there are techniques (tacit knowledge) such as tips and know-how of so-called experts with a certain degree of experience in carrying out an "organic synthesis reaction using t-BuLi (200 ml)," information related to such tacit knowledge is not reflected in the operational process generated by the generative model 300. It is preferable that the techniques of such experts are also reflected in the procedural manual for the "organic synthesis reaction using t-BuLi (200 ml)." Therefore, the information processing device 10 uses this tentative process information as a starting point to generate process information indicating an operational process that even an inexperienced user can refer to to properly carry out the "organic synthesis reaction using t-BuLi (200 ml)."

[0024] After the processing of step S102, in step S104, the dialogue unit 103 acquires tentative process information from the generative model server device 30. Next, in step S106, the output processing unit 104 presents the tentative process information acquired in step S104 to the performer. Specifically, the output processing unit 104 transmits the tentative process information to the user device 20 via the communication unit 120. In the user device 20, the display unit 240 displays the tentative process information. This allows the performer to confirm the experimental procedure indicated in the tentative process information. Furthermore, by comparing the work content of the experiment that the performer is about to perform with the experimental procedure indicated in the tentative process information, the performer can understand which work corresponds to the skills of an expert, or so-called tacit knowledge, and can understand which parts should be focused on being photographed as the skills of an expert.

[0025] Thereafter, in step S108, the camera 40 captures the actor performing a series of experimental tasks. The camera 40 starts capturing the footage as appropriate, controlled by the actor or the like. At this time, it is preferable for the actor to provide a warning for any part that corresponds to tacit knowledge. Next, in step S110, the motion video acquisition unit 102 of the information processing device 10 acquires the motion video captured by the camera 40. Here, the motion video is a video showing the actor performing a series of experimental tasks. The motion video captured in this manner shows the motion process performed by the actor who has checked the tentative process information and grasped which parts constitute tacit knowledge. Therefore, by using this motion video, it is possible to generate process information that includes tacit knowledge.

[0026] Next, in step S112, the dialogue unit 103 transmits a prompt to the generation model server device 30. The prompt includes the following information: Operation video ·Temporary process information - Instructions for generating process information based on operation video When generating process information, instructions to refer to temporary process information and extract different parts In the generative model server device 30, a prompt is input to the generative model 300. Then, the generative model 300 outputs process information indicating an operational process.

[0027] Fig. 4 is a diagram showing an example of process information. The process information shown in Fig. 4 was generated based on a video of an experimental task performed by an operator who confirmed the provisional process information shown in Fig. 3. In the process information, items indicated by a star shape are contents that are shown in more detail than those in the provisional process information, i.e., tacit knowledge. In this way, the process information generated from the video of an experimental task performed by an operator who confirmed the provisional process information shows more detailed contents than those in the provisional process information, i.e., contents equivalent to the skills of an expert.

[0028] 2, after the processing of step S112, in step S114, the dialogue unit 103 acquires process information from the generative model server device 30. Next, in step S116, the output processing unit 104 causes the display unit 240 of the user device 20 to display the process information acquired in step S112. This completes the process information generation processing.

[0029] As described above, in the system 1 of this embodiment, the information processing device 10 first generates tentative process information using a generative model and presents it to the performer. Then, the information processing device 10 obtains a motion video by filming a series of actions performed by the performer after checking the tentative process information, and acquires process information based on the motion video. This makes it possible to provide information on the motion process, including tacit knowledge, based on the video.

[0030] As a first variant of the first embodiment, the prompt sent in step S110 only needs to include an operational video and a generation instruction to generate process information based on the operational video, and does not need to include provisional process information.

[0031] As a second modification of the first embodiment, in the process information, content that differs from the content included in the provisional process information may be displayed in a different display mode from the content that is the same as the content included in the provisional process information. For example, the different content and the same content may be displayed in different colors.

[0032] As a third modified example of the first embodiment, the tentative process information may be created by a user such as an operator and input to the user device 20. In this case, the information processing device 10 acquires the tentative process information from the user device 20.

[0033] As a fourth modification of the first embodiment, the information processing device 10 may use the process information obtained in step S114 as provisional process information and repeat steps S106 to S114, thereby obtaining more appropriate process information.

[0034] As a fifth modification of the first embodiment, the information processing device 10 may repeat steps S102 to S114 using the process information obtained in step S114 as provisional process information. In this case, the prompt transmitted in step S102 includes a generation instruction to generate provisional process information based on the process information obtained in S114.

[0035] As a sixth modified example of the first embodiment, in step S112, the dialogue unit 103 may acquire the process information through a two-stage process. That is, first, the dialogue unit 103 acquires a prompt for analysis by transmitting, together with the operation video, a prompt including an instruction to generate a prompt for analyzing the operation video to the generative model server device 30. Next, the dialogue unit 103 acquires the process information by transmitting the operation video and the acquired prompt for analysis to the generative model server device 30.

[0036] As a seventh modification of the first embodiment, the prompt sent in step S112 may further include, as a comparison motion video, a motion video by a performer for whom provisional process information has not been presented. This makes it possible to obtain process information that refers to the difference between the comparison motion video and the motion video.

[0037] (Second embodiment) Next, the system 1 of the second embodiment will be described, focusing mainly on the differences from the system 1 of the first embodiment.

[0038] FIG. 5 is a flowchart showing a process information generation process by the information processing device 10 of the second embodiment. First, in step S200, the action type receiving unit 101 receives an action type. This process is the same as the process of step S100 described with reference to FIG. 2 in the first embodiment. Next, in step S202, the dialogue unit 103 transmits a prompt including an instruction to generate a shooting plan according to the action type to the generative model server device 30. In the generative model server device 30, the prompt is input to the generative model 300, and the shooting plan is output from the generative model 300. As a result, for example, a shooting plan for shooting a video of the operating procedure of "organic synthesis reaction using t-BuLi (200 ml)" is output. The shooting plan describes the operating procedure, as well as points to note and content to be captured. Here, examples of content to be captured include reagent labels, zoomed images of the hand, etc.

[0039] Next, in step S204, the dialogue unit 103 acquires a shooting plan from the generative model server device 30. Next, in step S206, the output processing unit 104 presents the shooting plan acquired in step S204 to the performer. Specifically, the output processing unit 104 transmits the shooting plan to the user device 20 via the communication unit 120. In the user device 20, the display unit 240 displays the shooting plan. This allows the performer to check the shooting plan. The shooting plan shows a series of actions to be shot. Furthermore, the shooting plan includes points to note when performing the actions and what points should be included in the video when shooting. However, it is thought that the tacit knowledge possessed by the performer himself, which cannot be presented from the generative model, is not revealed. By checking the shooting plan, the performer can understand what points correspond to tacit knowledge.

[0040] Thereafter, in step S208, the camera 40 captures the actor performing a series of experimental tasks. The camera 40 starts capturing the footage as appropriate, controlled by the actor or the like. At this time, it is preferable for the actor to provide a warning about any part that corresponds to tacit knowledge. Next, in step S210, the action video capture unit 102 of the information processing device 10 captures the action video captured by the camera 40.

[0041] Next, in step S212, the dialogue unit 103 transmits a prompt to the generative model server device 30. The prompt includes the following information: Operation video Motion planning - Instructions for generating process information based on operation video - When generating process information, instructions to refer to the operation plan and extract different parts In the generative model server device 30, a prompt is input to the generative model 300. Then, the generative model 300 outputs process information indicating an operational process.

[0042] Next, in step S214, the dialogue unit 103 acquires process information from the generative model server device 30. Next, in step S216, the output processing unit 104 displays the process information acquired in step S212 on the display unit 240 of the user device 20. This completes the process information generation process. The processes of steps S214 and S216 are the same as the processes of steps S114 and S116 in FIG. 2, respectively.

[0043] As described above, in the system 1 of this embodiment, the information processing device 10 first generates a shooting plan using a generative model and presents it to the performer. Then, the information processing device 10 obtains a motion video by shooting a series of actions performed by the performer who has confirmed the shooting plan, and acquires process information obtained based on the motion video. This makes it possible to provide information on the motion process, including tacit knowledge, based on the video.

[0044] As a first variant of the second embodiment, in the process information displayed in the second embodiment, content relating to operations that were not included in the imaging plan may be displayed in a display format different from content relating to operations that were included in the imaging plan.

[0045] As a second modification of the second embodiment, in the second embodiment, the prompt sent in step S212 may further include, as a comparison motion video, a motion video of an actor to whom an imaging plan has not been presented. This makes it possible to obtain process information that refers to the difference between the comparison motion video and the motion video.

[0046] As a third modified example of the second embodiment, similarly to the sixth modified example of the first embodiment, in step S212, the dialogue unit 103 may acquire the process information through two-stage processing. That is, first, the dialogue unit 103 acquires a prompt for analysis by transmitting, together with the operation video, a prompt including an instruction to generate a prompt for analyzing the operation video to the generative model server device 30. Next, the dialogue unit 103 acquires the process information by transmitting the operation video and the acquired prompt for analysis to the generative model server device 30.

[0047] In a first modification of the first and second embodiments, the output processing unit 104 only needs to output the process information to another device, and the output destination is not limited to the user device 20.

[0048] As a second modification of the first and second embodiments, the process information may include a reason for performing each operation process. In this case, the prompt including the instruction to generate the process information further includes an instruction to present the reason.

[0049] As a third modification of the first and second embodiments, the information processing device 10 may divide a step included in a target action into multiple steps and output step information for each step. For example, assume that the action is divided into a first step, a second step, and a third step. In this case, the information processing device 10 first outputs the step information for the first step. Next, when outputting the step information for the second step, the information processing device 10 includes a generation instruction to generate the step information by referring to the step information for the first step in the prompt for generating the step information. This allows for the acquisition of step information that takes into account the step information for the previous step.

[0050] As a third variant of the first and second embodiments, the information processing device 10 may use process information about an operation that is the same as or similar to the target operation as reference information, and may include a generation instruction for generating process information based on the reference information in the prompt for generating process information.

[0051] The above-described embodiment is an example for carrying out the present invention, and various other embodiments are possible. For example, various modifications and changes are possible within the scope of the gist of the present invention as defined in the claims, such as applying one modified example to another modified example. For example, some of the components of the above-described embodiment may be omitted, and the order of processing may be changed or omitted.

[0052] (Addendum) The matters described in the above embodiments will be supplemented below.

[0053] (Appendix 1) On one or more computers, an operation type receiving step of receiving an operation type; a tentative process information acquisition step of acquiring tentative process information indicating a tentative operation process by inputting a prompt including an instruction to generate an operation process corresponding to the operation type into a generative model; a tentative process information presenting step of presenting the tentative process information to an operator; a video acquisition step of acquiring a video of the operator performing the operation process to which the tentative process information has been presented; a process information acquisition step of acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; a process information output step of outputting the process information; A program to execute.

[0054] (Appendix 2) 2. The program of claim 1, wherein the prompt further includes the tentative process information.

[0055] (Appendix 3) In the process information output step, the process information is displayed on a display unit, The program according to claim 2, wherein content of the process information that differs from content included in the tentative process information is displayed in a display format that differs from content that is the same as the content included in the tentative process information.

[0056] (Appendix 4) On one or more computers, an operation type receiving step of receiving an operation type; a shooting plan acquisition step of acquiring a shooting plan for the motion process by inputting a prompt including an instruction to generate a shooting plan for a video showing a motion process corresponding to the motion type into a generative model; an imaging plan presenting step of presenting the imaging plan to an operator; a moving image acquisition step of acquiring a moving image of the performer performing the movement step to which the imaging plan is presented; a process information acquisition step of acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; a process information output step of outputting the process information; A program to execute.

[0057] (Appendix 5) 5. The program of claim 4, wherein the prompt further includes the imaging plan.

[0058] (Appendix 6) 5. The program according to claim 1, wherein the process information includes information associating the operation process with a reason for performing the operation process.

[0059] (Appendix 7) The program according to claim 1 or 4, wherein the prompt further includes a comparative action video in which the action step is performed by an actor to whom the tentative process information has not been presented.

[0060] (Appendix 8) one or more processors; the one or more processors: an operation type receiving step of receiving an operation type; a prompt including an instruction to generate an action process corresponding to the action type is input to a generative model, thereby acquiring tentative process information indicating a tentative action process; presenting the tentative process information to an operator; acquiring a motion video in which the performer, to whom the tentative process information has been presented, performs the motion process; acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; An information processing system that outputs the process information.

[0061] (Appendix 9) one or more processors; the one or more processors: an operation type receiving step of receiving an operation type; a prompt including an instruction to generate a shooting plan for a video showing an action step corresponding to the action type is input to a generative model, thereby acquiring a shooting plan for the action step; presenting the imaging plan to an operator; Acquire a motion video in which the performer to whom the shooting plan is presented performs a motion step; acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; An information processing system that outputs the process information.

[0062] (Appendix 10) 1. An information processing method executed by one or more processors, comprising: an operation type receiving step in which the one or more processors receive an operation type; a tentative process information acquisition step in which the one or more processors input a prompt including an instruction to generate an action process corresponding to the action type into a generative model, thereby acquiring tentative process information indicating a tentative action process; a tentative process information presenting step in which the one or more processors present the tentative process information to an operator; a video acquisition step in which the one or more processors acquire a motion video in which the performer, to whom the tentative process information is presented, performs the motion process; a process information acquisition step in which the one or more processors input a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model, thereby acquiring the process information; a process information output step in which the one or more processors output the process information; An information processing method for executing the above.

[0063] (Appendix 11) 1. An information processing method executed by one or more processors, comprising: an operation type receiving step in which the one or more processors receive an operation type; a shooting plan acquisition step in which the one or more processors input a prompt including an instruction to generate a shooting plan for a video showing an action step corresponding to the action type into a generative model, thereby acquiring a shooting plan for the action step; an imaging plan presenting step in which the one or more processors present the imaging plan to an operator; a moving image acquisition step in which the one or more processors acquire a moving image in which the performer to whom the imaging plan is presented performs a moving image; a process information acquisition step in which the one or more processors input a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model, thereby acquiring the process information; a process information output step in which the one or more processors output the process information; An information processing method for executing the above. [Explanation of symbols]

[0064] 1 System 10. Information processing equipment 20 User Device 30. Generative model server device 40 Camera 100 control section 101 Operation type reception section 102 Operation video acquisition unit 103 Dialogue Section 104 Output Processing Unit 110 Storage section 120 Communications Department 200 control section 210 Storage section 220 Communications Department 230 Operation section 240 Display section 300 Generative Models

Claims

1. On one or more computers, an operation type receiving step of receiving an operation type; a tentative process information acquisition step of acquiring tentative process information indicating a tentative operation process by inputting a prompt including an instruction to generate an operation process corresponding to the operation type into a generative model; a tentative process information presenting step of presenting the tentative process information to an operator; a video acquisition step of acquiring a video of the operator performing the operation process to which the tentative process information has been presented; a process information acquisition step of acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; a process information output step of outputting the process information; and a program for executing the process information output step.

2. The program described in Claim 1, wherein the prompt input to the generation model in the process information acquisition step further includes the provisional process information.

3. In the process information output step, the process information is displayed on a display unit, The program according to claim 2 , wherein the process information that is different from the tentative process information is displayed in a display mode that is different from the display mode of the same content as the tentative process information.

4. On one or more computers, an operation type acquisition step of acquiring an operation type; a shooting plan acquisition step of acquiring a shooting plan for the motion process by inputting a prompt including an instruction to generate a shooting plan for a video showing a motion process corresponding to the motion type into a generative model; an imaging plan presenting step of presenting the imaging plan to an operator; a moving image acquisition step of acquiring a moving image of the performer performing the movement step to which the imaging plan is presented; a process information acquisition step of acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; a process information output step of outputting the process information; and a program for executing the process information output step.

5. A program as described in Claim 4, wherein the prompt input into the generation model in the process information acquisition step further includes the shooting plan.

6. The program according to claim 1 , wherein the process information includes information associating the operation process with a reason for performing the operation process.

7. The program according to claim 1 , wherein the prompt further includes a comparative motion video in which the motion step is performed by an actor to whom the tentative process information has not been presented.

8. one or more processors; the one or more processors: Accepts the operation type, a prompt including an instruction to generate an action process corresponding to the action type is input to a generative model, thereby acquiring tentative process information indicating a tentative action process; presenting the tentative process information to an operator; acquiring a motion video in which the performer, to whom the tentative process information has been presented, performs the motion process; acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; An information processing system that outputs the process information.

9. one or more processors; the one or more processors: Accepts the operation type, a prompt including an instruction to generate a shooting plan for a video showing an action step corresponding to the action type is input to a generative model, thereby acquiring a shooting plan for the action step; presenting the imaging plan to an operator; Acquire a motion video in which the performer to whom the shooting plan is presented performs a motion step; acquiring the process information by inputting a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model; An information processing system that outputs the process information.

10. 1. An information processing method executed by one or more processors, comprising: an action type receiving step in which the one or more processors receive an action type; and a tentative process information acquiring step in which the one or more processors acquire tentative process information indicating a tentative action process by inputting a prompt including an instruction to generate an action process corresponding to the action type into a generative model. a tentative process information presenting step in which the one or more processors present the tentative process information to an operator; a video acquisition step in which the one or more processors acquire a motion video in which the performer, to whom the tentative process information is presented, performs the motion process; a process information acquisition step in which the one or more processors input a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model, thereby acquiring the process information; a process information output step in which the one or more processors output the process information.

11. 1. An information processing method executed by one or more processors, comprising: an operation type receiving step in which the one or more processors receive an operation type; a shooting plan acquisition step in which the one or more processors input a prompt including an instruction to generate a shooting plan for a video showing an action step corresponding to the action type into a generative model, thereby acquiring a shooting plan for the action step; an imaging plan presenting step in which the one or more processors present the imaging plan to an operator; a video acquisition step in which the one or more processors acquire a motion video in which the performer, to whom the imaging plan has been presented, performs a motion step; a process information acquisition step in which the one or more processors input a prompt including the operation video and an instruction to generate process information indicating the operation process into a generative model, thereby acquiring the process information; a process information output step in which the one or more processors output the process information.

Images