Information processing system, information processing method, and program

Abstract

This technology relates to an information processing system, an information processing method, and a program that enable efficient video production. An information processing system according to one aspect of the present technology generates and displays information outlining video content to be produced on the basis of design information that includes at least one of requirements and constraints, and generates and displays suggestion information regarding the content of the video content in response to prompts entered by a user. In addition, the information processing system generates and displays a storyboard in which cut images showing a screen composition of each cut of the video content, reflecting the content of the suggestion information selected by the user, are arranged in an order the images were captured. The technology can be applied to servers that provide cloud services.

Description

Information Processing System, Information Processing Method, and Program

[0001] The present technology particularly relates to an information processing system, an information processing method, and a program that enable efficient video production.

[0002] Video production is mainly carried out in the flow of pre-production including planning, composition, and scenario creation, production including shooting, recording, and material production, and post-production including editing. There is a web service that supports a series of operations by video creators.

[0003] “Creators' Cloud”, [searched on January 9, 2025], Internet <URL https: / / creatorscloud.sony.net / ja-jp / >

[0004] As an operation of pre-production, a storyboard is created. To create a storyboard, various creations such as considering a scenario or the content of video content are required. It is convenient for video creators to have support for creation each time.

[0005] The present technology has been made in view of such a situation and enables efficient video production. [[ID=??]]

[0006] Note that the description of the problem does not prevent the existence of other problems.

[0007] An information processing system according to one aspect of the present technology generates and displays information indicating an outline of video content to be produced based on design information including at least any one of requirements and constraints, generates and displays proposal information regarding the content of the video content in response to a prompt input by a user, and generates and displays a storyboard in which cut images indicating the screen composition of each cut of the video content reflecting the content of the proposal information selected by the user are arranged in shooting order. The information processing system includes an information processing unit.

[0008] It seems there is a typo in the original text where the tag is missing its closing tag in the English translation. I've left it as it is in the translation to match the original structure. Also, the tag [[ID=??]] is likely a mistake in the original, and I've left it as is in the translation. If you have any further questions or need clarification, feel free to ask.In one aspect of this technology, information outlining the video content to be produced is generated and displayed based on design information that includes at least one of the requirements and constraints, and suggested information regarding the content of the video content is generated and displayed in response to prompts entered by the user. A storyboard is generated and displayed in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the suggested information selected by the user, are arranged in the order of shooting.

[0009] This figure shows an example configuration of an information processing system according to one embodiment of this technology. This figure shows the process of video production using a video production support tool. This figure shows the flow of video production using a video production support tool. This is a flowchart explaining the process in PLAN TO SHOOT. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This figure shows an enlarged view of the display area. This figure shows an enlarged view of the selection area. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This is a flowchart explaining the process in SHOOT TO PLAN. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This is a flowchart explaining the process in SHOOT TO REFINE. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This is a flowchart explaining the process in REFINE TO SHOOT. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This figure shows an example of a tool screen. This is a block diagram showing an example of a hardware configuration of a video production support server. This block diagram shows an example of a functional configuration of a video production support server. This figure shows another example of a tool screen during PLAN TO SHOOT. This figure shows an enlarged view of the design information area.

[0010] The following describes the configurations for implementing this technology. The explanation will proceed in the following order: 1. Overview of the technology 2. PLAN TO SHOOT 3. SHOOT TO PLAN 4. SHOOT TO REFINE 5. REFINE TO SHOOT 6. Configuration of the video production support server 7. UI variations 8. Other functions of the video production support tool 9. Others

[0011] <<Overview of this technology>> This technology supports video production by video creators. Video content produced by video creators includes various types of content using video, such as dramas, movies, anime, documentaries, advertisements, and music videos. Video creators can be professional or amateur creators.

[0012] Support for video production is provided, for example, through tools offered by servers on the internet. Video creators access these servers using their own computers, such as PCs, and then use the tools provided through a screen displayed by an application such as a browser to create their videos.

[0013] Sometimes there's a gap between planning and filming, such as "we can't film it exactly as we envisioned it." Similarly, there can be a gap between filming and editing, such as "the footage doesn't turn out as we expected during filming." Editing (improvement) in this context refers to the process of refining the footage used as source material in post-production. After the initial filming, editing is performed to refine the footage, and then another filming session takes place. The final resulting footage is then used as source material in post-production.

[0014] This technology provides a video production support tool that seamlessly bridges the gaps between planning and filming, and between filming and editing. Video production using this support tool proceeds in a way that allows video creators to collaborate with AI (Artificial Intelligence) agents that act like teammates.

[0015] Traditionally, bridging the aforementioned gaps required gaining experience, honing skills, and collaborating with other creators. However, by utilizing the video production support tools provided by this technology, such actions become unnecessary. Video creators can proceed with video production more efficiently.

[0016] <Video Production System> Figure 1 shows an example of the configuration of an information processing system according to one embodiment of this technology.

[0017] An information processing system 1 according to one embodiment of this technology is configured by connecting a video production support server 11 and a user terminal 12 via a network such as the Internet. In the example in Figure 1, only one user terminal 12 is shown, but in reality, multiple user terminals 12 are connected to the video production support server 11.

[0018] The video production support server 11 is a server on a network. The video production support server 11 is configured by one computer or multiple computers. Video production support tools are provided when a predetermined program is executed on the video production support server 11.

[0019] The user terminal 12 is a computer such as a PC used by a video creator. Other computers such as smartphones and tablet devices may also be used for video production. By operating the user terminal 12 and accessing the video production support server 11, video production support tools are provided to the video creator.

[0020] Figure 2 shows the process of creating a video using a video production support tool.

[0021] As shown in Figure 2, the video production support tool is equipped with the Teammate AI function. Teammate AI is an AI-powered agent. The Teammate AI function is realized by a machine learning model prepared on the video production support server 11.

[0022] The video production support server 11 is equipped with machine learning models, such as neural networks, generated by machine learning. The machine learning models that realize the functions of Teammate AI consist of inference models, generation models, recognition models, classification models, analysis models, and prediction models that obtain outputs according to inputs using various algorithms. On the video production support server 11, various AI processes such as inference processing, generation processing, recognition processing, classification processing, analysis processing, and prediction processing are performed using the machine learning models, and the functions of Teammate AI are realized.

[0023] Not all functions of Teammate AI are implemented by machine learning models prepared on the video production support server 11; at least some functions may be implemented using machine learning models prepared on an external server. The functions of Teammate AI may be provided to video creators by linking the machine learning models prepared on the video production support server 11 with the machine learning models prepared on the external server.

[0024] For example, if a video creator requests references to be used as material for a storyboard from the Teammate AI, as shown in speech bubble #1, the Teammate AI will prepare the requested references and present them to the video creator, as shown in speech bubble #2. Prompts, which are requests or instructions to the Teammate AI, are entered using, for example, text data. Prompts may also be entered using voice.

[0025] The prompts may be entered by having the user select from options presented as candidates by the teammate AI, or the prompts may be automatically entered by interpreting the video creator's actions. In addition to text data, various forms of data such as image data and audio data are entered by the video creator. Image data includes at least one of the following: still images or moving images (video).

[0026] In addition to providing references, Teammate AI provides various types of information used in planning video content, such as scenario information, budget information, schedule information, and storyboards. As will be described later, Teammate AI also provides information used for shooting and editing.

[0027] Figure 3 shows the flow of video production using video production support tools.

[0028] The illustration of a person on the left in Figure 3 represents the planning stage (PLAN), the illustration of a camera in the center represents the shooting stage (SHOOT), and the illustration of film on the right represents the editing stage (REFINE). As shown by arrows #11 to #14, the video production phase using video production support tools includes the following phases: 1. PLAN TO SHOOT 2. SHOOT TO PLAN 3. SHOOT TO REFINE 4. REFINE TO SHOOT

[0029] The PLAN TO SHOOT and SHOOT TO PLAN phases shown on the left side of Figure 3 are the preparation phases before the actual imaging. The PLAN TO SHOOT and SHOOT TO PLAN processes may be performed once each in that order, or they may be performed alternately and repeatedly.

[0030] The SHOOT TO REFINE and REFINE TO SHOOT phases shown on the right side of Figure 3 are the phases during the actual imaging. The SHOOT TO REFINE and REFINE TO SHOOT operations may be performed once each in that order, or they may be performed alternately and repeatedly. After the actual imaging has been performed once, it is also possible to return to the PLAN TO SHOOT and SHOOT TO PLAN phases that preceded the actual imaging.

[0031] The video production support tool includes functions for each phase. These phase-specific functions can be used at any time. As described later, video creators can perform tasks in each phase using a whiteboard provided on the screen, which has no restrictions on the input area or content. Furthermore, video creators can co-create video content with teammate AI and other video creators.

[0032] The following describes the work involved in each phase using video production support tools.

[0033] <<PLAN TO SHOOT>> PLAN TO SHOOT is the planning phase before filming. The video creator will primarily perform the following tasks using the functions of Teammate AI: - Organizing requirements and constraints - Gathering references - Conceptualizing the overall scenario - Drawing sketches - Creating storyboards

[0034] The teammate AI in PLAN TO SHOOT primarily includes the following functions: • Project summary generation function • Reference generation function • Idea generation support function • Storyboard generation function

[0035] In addition to these functions, Teammate AI possesses at least some of the various functions that can be realized through AI processing, such as general conversation (chat) functions, search functions, image recognition functions, speech recognition functions, translation functions, image generation functions, speech synthesis functions, natural language processing functions, image analysis functions, and speech analysis functions.

[0036] <Processing Flow During PLAN TO SHOOT> The processing of the video production support server 11 during PLAN TO SHOOT will be explained by referring to the flowchart in Figure 4. Examples of screens presented to the video creator during each process will be described later. The processing of each step shown in Figure 4 will be executed in any order, or in parallel with the processing of other steps. The same applies to the other flowcharts described later.

[0037] In step S1, the video production support server 11 imports the design information. The design information, which serves as preliminary documentation, includes, for example, the following: • Theme • Objective • Target audience attributes • Format (live-action, animation, CG, etc.) • Deadline • Filming location • Equipment (filming equipment, recording equipment, editing equipment, equipment for material production) • Budget

[0038] Theme, purpose, target audience attributes, and format are requirements for the video content. Deadline, filming location, equipment, and budget are constraints. Other information may be included as requirements for the video content, or as constraints. Only requirements for the video content may be included in the design information, or only constraints for the video content may be included. The design information includes at least one of either requirements or constraints for the video content.

[0039] In step S2, the video production support server 11 generates a project summary based on the imported design information. The project summary is information that summarizes the outline of the video content to be produced. The project summary is generated by inputting the design information into a machine learning model that realizes Teammate AI. At least some of the information contained in the design information is used as input to the machine learning model. The project summary generation function is realized by a machine learning model generated by machine learning using the design information and the project summary.

[0040] In step S3, the video production support server 11 generates references in response to design information and prompts entered by the video creator. The references are information that serves as suggestions regarding the content of the video. The references are generated by inputting the design information and prompts into the machine learning model that implements the Teammate AI. The reference generation function is realized by a machine learning model generated through machine learning using the design information, the information entered as prompts, and the reference information.

[0041] In step S4, the video production support server 11 presents a UI for brainstorming ideas in a dialogue format with the teammate AI. By inputting prompts entered by the video creator into the machine learning model that implements the teammate AI, information that will serve as ideas for the video content is generated and presented to the video creator. The video creator looks at the presented information, adopts their preferred ideas, and decides on the content of the video. The content decided by the video creator is acquired by the video production support server 11.

[0042] In step S5, the video production support server 11 generates a storyboard based on design information, project summary, references, and content decided by the video creator. The storyboard is a blueprint of the video content, visualizing the flow and structure of the video content. The storyboard is constructed by arranging cut images, which are images showing the screen composition of each cut that makes up a scene, in the order they are shot. Any images, such as sketch-style images, can be used as cut images.

[0043] A scene is a segment of video footage at any given time. A scene can consist of any unit of video footage, such as a series of videos of a specific scene or a series of videos of a specific location. Typically, a single video content consists of multiple scenes.

[0044] A cut (shot) is a continuous video segment obtained by one shooting from the start to the end of shooting. Usually, one scene is composed of multiple cuts.

[0045] By inputting each piece of information into the machine learning model that realizes the teammate AI, a storyboard is generated. As inputs to the machine learning model, at least some of the information including design information, project summary, reference, and content determined by the video creator is used.

[0046] For example, based on design information, project summary, reference, and content determined by the video creator, the content of each scene is generated, and by dividing each scene, the content of each cut that constitutes each scene is generated. As information indicating the content of each scene and each cut, for example, the following information is generated. ・Cut number, scene number ・Time of cut, scene ・Information on the orientation of the subject ・Information on the moving direction of the subject ・Composition (screen size, camera position, height, angle) ・Camera work ・Dialogue ・Information on lighting ・Title ・Lens used ・Settings of camera / lens

[0047] Based on the content of each cut, a cut image showing the screen composition of each cut is generated and used for presenting the storyboard. By inputting design information, project summary, reference, and content determined by the video creator into the machine learning model that realizes the teammate AI, a storyboard is generated. The storyboard generation function is realized by a machine learning model generated by machine learning using information such as design information, project summary, reference, and content determined by the video creator, and the storyboard.

[0048] <UI at the time of PLAN TO SHOOT> ・Import screen of design information Fig. 5 is a diagram showing an example of a tool screen used for importing design information. The tool screen in Fig. 5 is a screen displayed on the display of the user terminal 12 in the process of step S1 in Fig. 4.

[0049] A whiteboard area 101 is formed across almost the entire tool screen. The whiteboard area 101 is used not only for displaying various types of information, but also for inputting information by video creators. The display range of the whiteboard area 101 can be switched by operating input devices such as a keyboard or mouse.

[0050] Input areas 111 and 112 are displayed approximately in the center of the whiteboard area 101. The narrow strip-shaped input area 111 is used for inputting prompts. When a prompt is entered, the response information is displayed in a designated location on the whiteboard area 101.

[0051] The rectangular input area 112 is used for inputting design information. For example, the import of design information is started by dragging and dropping a design information file prepared on the user terminal 12 into the input area 112. The contents of the imported design information are analyzed by the video production support server 11 and used to generate a project summary.

[0052] • Project Summary Display Screen Figure 6 shows an example of a tool screen for displaying a project summary. The tool screen in Figure 6 is the screen displayed on the user terminal 12's display when the process in step S2 of Figure 4 is performed. In Figure 6, the same reference numerals are used for components that are the same as those described above. Duplicate explanations are omitted as appropriate. The same applies to other figures described later.

[0053] The display area 121, located approximately in the center of the whiteboard area 101, is the display area for the project summary. The project summary, generated based on design information and other data, is displayed in the display area 121. The project summary is presented using not only text data but also images.

[0054] Figure 7 is a magnified view of the display area 121.

[0055] As shown in Figure 7, the project summary includes information such as the project overview, objectives, target audience attributes, and the tone and style of the video content. Video creators can edit each piece of information generated by the video production support server 11 as needed.

[0056] Returning to the explanation of Figure 6, the selection area 122 is displayed to the right of the display area 121. The selection area 122 is used to select the operations to be performed after reviewing the contents of the project summary. The display area 121 and the selection area 122 are displayed connected by an arrow pointing from the display area 121 to the selection area 122.

[0057] Figure 8 is a magnified view of the selected region 122.

[0058] As shown in Figure 8, an area for inputting prompt data is provided above the selection area 122, and prompt candidates are displayed below it. Prompt candidates include idea generation, storyboard generation, reference collection (generation), shooting location review, and 3D simulation. After reviewing the contents of the project summary, the video creator can input the following requests to the teammate AI.

[0059] - Reference display screen Figure 9 shows an example of a tool screen for displaying references. The tool screen in Figure 9 is the screen that is displayed on the user terminal 12's display when reference collection is requested, such as by using the display of the selection area 122, and the process of step S3 in Figure 4 is performed.

[0060] The display area 123, located approximately in the center of the whiteboard area 101, is the reference display area. References generated according to the video creator's requests are displayed in the display area 123. In the example in Figure 9, the display area 123 is displayed to the right of the display area 121 instead of the display of the selection area 122.

[0061] References #21 to #24 displayed within the display area 121 are information generated by the video production support server 11. For example, each time a prompt is entered, a reference corresponding to that prompt is generated and displayed. Reference #21 is information that shows the scenario of the video content. An image and text showing the content of the draft scenario generated based on the setting information are displayed as reference #21.

[0062] Reference #22 consists of four images generated in response to a video creator's request for a collection of elevator scenes. A comment indicating that an elevator scene reference has been generated is displayed, as shown in popup #31.

[0063] Reference #23 is information that suggests the content of a scene, generated by inputting an illustration. For example, if a video creator inputs an illustration they have drawn themselves and requests the generation of scene content, the video production support server 11 generates information suggesting the content of the scene and presents it as Reference #23.

[0064] Reference #24 is information that suggests relationships in each scene, generated by inputting information about the characters. In this way, various types of information are generated by the video production support server 11 and presented to the video creator. The video creator will then proceed with their project planning, referring to the contents of the reference.

[0065] A chat area 124 is displayed at the right edge of the whiteboard area 101. The video creator can use the chat area 124 to evaluate the reference content and request further suggestions.

[0066] Figure 10 shows another example of a tool screen that displays a reference. The tool screen in Figure 10 is the screen that is displayed on the user terminal 12's display when the process in step S4 of Figure 4 is performed.

[0067] In the example in Figure 10, a comment for reference #21 is generated and displayed in popup #32. The video creator can update the reference by adding information to or requesting information from the comment. In the example in Figure 25, reference #25 is added and displayed.

[0068] In this way, video creators can review ideas presented by the AI ​​teammate and proceed with their projects by interacting with the AI ​​teammate. For example, a storyboard is generated that reflects the references selected by the video creator, such as by giving them high ratings.

[0069] - Storyboard display screen Figure 11 shows an example of a tool screen for displaying a storyboard. The tool screen in Figure 11 is the screen that is displayed on the user terminal 12's display when storyboard generation is requested using the display of the selection area 122, etc., and the process of step S5 in Figure 4 is performed.

[0070] The display area 125 to the right of the whiteboard area 101 is the storyboard display area. The storyboard generated according to the video creator's request is displayed in the display area 125. In the example in Figure 11, the display area 125 is displayed to the right of the display area 123.

[0071] The arrangement of images displayed in each row within the display area 125 represents, for example, a scene. A storyboard is constructed by arranging cut images that show the screen composition of the cuts that make up each scene in the order they were shot. The order of the scenes is indicated by arrows displayed between each scene. In the example in Figure 11, a comment suggesting that a shot list be created based on the storyboard is displayed in pop-up #41.

[0072] The shot list is a list of footage shot during the SHOOT TO PLAN phase. Footage shot during SHOOT TO PLAN is used to further improve the storyboard. The shooting conducted during SHOOT TO PLAN is sometimes referred to as a test shoot. The comment in pop-up #41 is a message instructing you to conduct a test shoot based on the storyboard.

[0073] As described above, PLAN TO SHOOT involves planning tasks such as organizing requirements and constraints, gathering references, conceptualizing the overall scenario, drawing sketches, and creating storyboards. Video creators can efficiently advance the planning of video content by evaluating information presented by the Teammate AI and communicating their intentions to the Teammate AI.

[0074] Multiple video creators can share the same screen and collaborate on the planning process. During the planning process, each video creator can communicate with each other via text, voice, or other means on the aforementioned tool screen.

[0075] <<SHOOT TO PLAN>> SHOOT TO PLAN is the phase in which location scouting is conducted based on the storyboard. The video creator mainly performs the following tasks using the functions of Teammate AI: - Test shooting - Preview of test footage - Refine the storyboard

[0076] In location scouting using video production support tools, various checks are performed, including verifying the filming location, background, ambient light and lighting, sound, and travel time. In addition, preliminary filming is conducted using a camera. Previews of the preliminary footage are then conducted, for example, at the filming location.

[0077] The teammate AI in SHOOT TO PLAN primarily includes the following functions: • Review function for preliminary footage • Function to suggest improvements

[0078] <Processing flow during SHOOT TO PLAN> The processing of the video production support server 11 during SHOOT TO PLAN will be explained with reference to the flowchart in Figure 12. For example, the processing of the video production support server 11 is realized by communicating with the user terminal 12 that has been taken to the location scouting site.

[0079] In step S11, the video production support server 11 displays the storyboard generated by PLAN TO SHOOT. The video creator performs preliminary shooting of each shot while viewing the storyboard displayed on the user terminal 12's display. For example, the creator selects which shot to shoot on the storyboard, and then performs preliminary shooting of the selected shot.

[0080] In step S12, the video production support server 11 imports the temporary footage. For example, footage shot during temporary shooting and stored in the camera's internal storage is transmitted (uploaded) to the video production support server 11 using the camera's communication function or the communication function of an external device such as a user terminal 12 connected to the camera. The video production support server 11 receives the uploaded temporary footage, stores it in its storage, and manages it in association with scene numbers, cut numbers, etc. If multiple temporary takes are shot for a single cut, the multiple temporary footages are managed in association with a single cut, etc.

[0081] In step S13, the video production support server 11 replaces the cut images on the storyboard with temporary footage. Instead of the cut image that was previously displayed, a thumbnail of the temporary footage for the corresponding cut is displayed. The uploading of temporary footage and the replacement of cut images are performed immediately after the temporary shooting, each time the video production support server 11 acquires temporary footage for each cut, so that the progress of the temporary shooting and the storyboard are synchronized. By selecting any cut image, playback of the temporary footage for the corresponding cut begins.

[0082] In step S14, the video production support server 11 generates and displays evaluations and improvement suggestions for the temporary footage of each take. For example, evaluation criteria are pre-set by the video creator. By inputting the temporary footage into the machine learning model that implements the Teammate AI, evaluations and improvement suggestions for the temporary footage are generated. Information on the cuts in which the temporary footage was shot is also used as input to the machine learning model as appropriate. The function of generating evaluations and improvement suggestions for temporary footage is realized by a machine learning model generated by machine learning using information on the footage, evaluation criteria, evaluations, and improvement suggestions.

[0083] In step S15, the video production support server 11 refines the storyboard in response to input from the video creator. For example, if the video creator reviews the evaluation and improvement suggestions of the trial footage and makes input to adopt the improvement suggestions or modify the content of the cuts, the storyboard is modified to reflect the input from the video creator.

[0084] <UI during SHOOT TO PLAN> - Replacement with temporary footage Figure 13 shows an example of a tool screen that displays a storyboard. The tool screen in Figure 13 is the screen that is displayed on the user terminal 12's display when the process in step S13 of Figure 12 is performed.

[0085] The video area 201 at the left edge of the whiteboard area 101 is the shot list display area. Each image displayed in two columns within the video area 201 is a thumbnail of the temporary footage shot and uploaded to the video production support server 11. If multiple temporary takes have been performed for the same shot, a thumbnail of the temporary footage for each take will be displayed.

[0086] In the example shown in Figure 13, images P1 to P3 displayed in the upper row of the whiteboard area 101 are cut images from Scene 02, and images P11 to P15 displayed in the lower row are cut images from Scene 03. Below each cut image, information such as a description of the cut is displayed as appropriate.

[0087] Images P11 and P12 of images P11-P15 in Scene 03 have already been replaced with thumbnails of the temporary footage. The presence of frame #51 on image P13 indicates that it is being replaced with the thumbnail of the temporary footage shown in frame #52 on the shot list. For example, if the temporary filming of Scene 03 continues and all the temporary filming of Scene 03 is completed, then the five images P11-P15 on the Scene 03 storyboard will have been replaced with thumbnails of the temporary footage.

[0088] Because thumbnails of footage obtained through test shoots are displayed as cut images in the storyboard, video creators can more easily visualize whether the resulting video content will match their plan.

[0089] Figure 14 shows an example of a tool screen that displays the evaluation and improvement suggestions for the preliminary footage. The tool screen in Figure 14 is the screen that is displayed on the user terminal 12's display when the process in step S14 of Figure 12 is performed.

[0090] At the top of the tool screen, the Evaluation tab 211 and the Improvement Suggestions tab 212 are displayed. The Evaluation tab 211 is selected when displaying the evaluation of the preliminary footage, and the Improvement Suggestions tab 212 is selected when displaying suggestions for improving the preliminary footage. In the example in Figure 14, the Improvement Suggestions tab 212 is selected.

[0091] Below the Evaluation tab 211 and the Improvement Suggestion tab 212, a large rectangular area called the Display Area 221 is provided. The Display Area 221 is the display area for the selected temporary video footage. The temporary video footage selected using the thumbnail displayed in the Display Area 222 located at the bottom of the tool screen is enlarged and displayed in the Display Area 221. Below the Display Area 221, buttons used for playback operations of the temporary video footage, such as play start / pause and fast forward / rewind, are displayed.

[0092] While playing the test footage, if the evaluation tab 211 is selected, the evaluation of the test footage will be displayed in the display area 221. If the improvement suggestion tab 212 is selected, improvement suggestions for the test footage will be displayed in the display area 221. Video creators can view the test footage and check the evaluations and improvement suggestions from Teammate AI for each shot.

[0093] In the example in Figure 14, frame #61 is displayed around the face of the person in the selected test footage, and pop-up #62 displays comments suggesting improvements to how the face is captured. Instead of text comments, images such as icons or audio could be used to present the information suggesting improvements.

[0094] By reviewing the evaluation and improvement suggestions for the test footage, video creators can replace the test footage for each shot in the storyboard with other footage, or rearrange the order of the shots. If there is no test footage that matches the improvement suggestions, the video creator can also reshoot the test footage. The storyboard is refined (revised) based on the video creator's input into the tool screen.

[0095] The display area 222 displays thumbnails of each shot that make up a particular scene. It is possible to switch the content displayed in the display area 222, for example, by displaying thumbnails of multiple take test footage of a particular shot.

[0096] The right side of the tool screen is provided with a display area 231 and a detail area 232. The display area 231 is the display area for temporary footage of different takes of the selected cut. Below the display area 231, the detail area 232 displays information related to the evaluation of the selected temporary footage. In the example in Figure 14, information such as evaluations of camera work, shot size, and character movement are displayed.

[0097] Below the detail area 232, a setting button 233 and a selection button 234 are displayed. The setting button 233 is used when setting evaluation criteria. The selection button 234 is used when selecting the currently selected temporary footage to be reflected in the storyboard.

[0098] Figure 15 shows an example of another tool screen that displays a storyboard. The tool screen in Figure 15 is the screen that is displayed on the user terminal 12's display when the process in step S15 of Figure 12 is performed.

[0099] In the example in Figure 15, the refined storyboard is displayed in the whiteboard area 101. The three thumbnails of Scene 03, enclosed in frame #71, are thumbnails of the temporary footage that has been modified by rearranging or other means. A comment suggesting rearranging the order of the cuts is displayed in pop-up #72.

[0100] As described above, SHOOT TO PLAN involves preliminary filming and storyboard refinement. Video creators can improve their storyboards by reviewing and revising information such as improvement suggestions provided by Teammate AI. They can also refine the project content while scouting locations, feeling as if they are working together with Teammate AI.

[0101] <<SHOOT TO REFINE>> SHOOT TO REFINE is the phase in which the actual filming is carried out based on the storyboard. The video creator mainly performs the following tasks using the functions of Teammate AI: - Actual filming - Selection of the best take - Preview of the actual filming footage

[0102] The Teammate AI in SHOOT TO REFINE primarily includes the following functions: • Shooting schedule generation function • Review function for the actual shooting footage

[0103] <Processing flow during SHOOT TO REFINE> Refer to the flowchart in Figure 16 to explain the processing of the video production support server 11 during SHOOT TO REFINE.

[0104] In step S21, the video production support server 11 generates a shooting schedule based on the storyboard. The shooting schedule information is generated by inputting the storyboard information into a machine learning model that implements the Teammate AI. The shooting schedule generation function is implemented by a machine learning model generated by machine learning using the storyboard and shooting schedule information. Information input by the video creator in the form of dialogue with the Teammate AI is also used in the generation of the shooting schedule as appropriate. The shooting schedule includes the following information for each scene: ・Scene number ・Shooting date and time ・Shooting location ・Actors ・Shooting schedule ・List of costumes and props ・List of equipment

[0105] The timing of the call schedule generation can be set to any time before the actual shoot, such as during the SHOOT TO PLAN process. After the call schedule is generated, the actual shoot is carried out according to the call schedule. Subsequent processing is achieved, for example, by communicating with the user terminal 12 brought to the shooting location.

[0106] In step S22, the video production support server 11 receives the main footage. For example, footage shot during the main shoot and stored in the camera's internal storage is uploaded to the video production support server 11 using the camera's communication function or the communication function of an external device such as a user terminal 12 connected to the camera. The video production support server 11 receives the uploaded main footage, stores it in its storage, and manages it by associating it with scene numbers, cut numbers, etc. If multiple takes are performed for a single cut, the multiple main footages are managed by associating them with a single cut, etc.

[0107] In step S23, the video production support server 11 replaces the cut images on the storyboard with the actual footage shot. Instead of the cut images that were previously displayed, a thumbnail of the actual footage shot for the corresponding cut is displayed. The uploading of the actual footage shots and the replacement of the cut images are performed immediately after the actual shooting, each time the video production support server 11 acquires the actual footage shots for each cut, thereby synchronizing the progress of the actual shooting with the storyboard.

[0108] In step S24, the video production support server 11 generates and displays an evaluation and improvement suggestion for the actual footage of each take. For example, the evaluation criteria are pre-set by the video creator. By inputting the actual footage into the machine learning model that implements the Teammate AI, an evaluation and improvement suggestion for the actual footage is generated. Information on the cuts in which the actual footage was shot is also used as input to the machine learning model as appropriate. The function of generating the evaluation and improvement suggestion for the actual footage is realized by a machine learning model generated through machine learning using the footage, evaluation criteria, evaluation, and improvement suggestion. The video production support server 11 selects the best take according to the input of the video creator who has confirmed the evaluation of the actual footage.

[0109] The best take of the final footage may be automatically selected by the teammate AI. In this case, for example, the final footage of the take with the highest rating would be selected from among multiple takes of the final footage.

[0110] When the best take from multiple takes of the main footage is selected, the storyboard display is updated. The thumbnail of the selected best take is displayed as the corresponding cut image in the storyboard.

[0111] If the selection of the best take is complete, in step S25, the video production support server 11 performs a preview playback of the filmed video. Preview playback can be performed in any unit, such as scene by scene, cut by cut, or all scenes.

[0112] <UI during SHOOT TO REFINE> - Figure 17, which shows the replacement using the actual footage, is an example of a tool screen that displays a storyboard. The tool screen in Figure 17 is the screen that is displayed on the user terminal 12's display when the process in step S23 of Figure 16 is performed.

[0113] In the example shown in Figure 17, images P101 to P103, displayed in the upper row of the whiteboard area 101, are cut images from Scene 02, while images P111 to P115, displayed in the lower row, are cut images from Scene 03. Below each cut image, information such as a description of the cut is displayed as appropriate.

[0114] Image P111 in Scene 03, among images P111-P115, is an image that has already been replaced with a thumbnail from the actual filmed footage. The presence of frame #101 in image P112 indicates that it is being replaced with a thumbnail from the actual filmed footage. This kind of storyboard update is performed after each actual filming of a shot.

[0115] Since the thumbnails of the footage obtained during the shoot are displayed as cut images in the storyboard, video creators can more easily visualize whether the video content is turning out as planned.

[0116] Figure 18 shows an example of a tool screen that displays the evaluation and improvement suggestions for the recorded video. The tool screen in Figure 18 is the screen that is displayed on the user terminal 12's display when the process in step S24 of Figure 16 is performed.

[0117] The tool screen in Figure 18 is basically the same as the tool screen described with reference to Figure 14. In the example in Figure 18, the evaluation tab 211 is selected. During playback of the recorded video, the evaluation of each cut of the recorded video is displayed in the display area 221 and the detail area 232. The video creator may also be allowed to input their own evaluations.

[0118] In the display area 222 located at the bottom of the tool screen, thumbnails of multiple takes of the selected cut are displayed side by side. By selecting a thumbnail displayed in the display area 222, the video creator can check the evaluation of different takes of the cut.

[0119] Figure 19 shows another example of a tool screen that displays an evaluation of the recorded video and suggestions for improvement.

[0120] The display area 251 on the right side of the tool screen in Figure 19 is a display area for thumbnails of different versions of the main footage. For a given shot, different versions of the main footage may be shot with variations in timing or composition. Multiple versions of the main footage of the selected shot are displayed vertically.

[0121] The video creator can review the initial footage and evaluate and suggest improvements for each shot. After reviewing the evaluation and suggestions for improvement, the video creator can reshoot the scene or select the best take. The best take selected in this way is then used as material for post-production.

[0122] <<REFINE TO SHOOT>> REFINE TO SHOOT is the phase in which the shooting plan for the main shoot is improved. The video creator will primarily perform the following tasks using the functions of Teammate AI: - Modify the shooting plan

[0123] The Teammate AI in REFINE TO SHOOT primarily includes the following functions: • Generation of 3D spatial data of the shooting location • Camera angle suggestion function

[0124] The 3D spatial data of the filming location is data of a 3D model that represents the filming location as a virtual space. 3D models of various objects present on the filming location, including the actors, are placed as objects in the virtual space.

[0125] <Processing flow during REFINE TO SHOOT> The processing of the video production support server 11 during REFINE TO SHOOT will be explained by referring to the flowchart in Figure 20. For example, the processing of the video production support server 11 is realized by communicating with the user terminal 12 that has been brought to the actual shooting site.

[0126] In step S31, the video production support server 11 generates 3D spatial data of the filming location based on the actual filmed video. By inputting the actual filmed video of each shot into the machine learning model that realizes Teammate AI, 3D spatial data of the filming location for that shot is generated. The function of generating 3D spatial data of the filming location is realized by a machine learning model generated by machine learning using 2D video and a 3D model.

[0127] In step S32, the video production support server 11 simulates the camera angle using 3D spatial data. The video production support server 11 sets an arbitrary position in the virtual space represented by the 3D spatial data as the camera position and generates a simulated video based on the 3D spatial data, showing what it would look like if filmed from that camera position. The generation of the simulated video may also be performed using a machine learning model.

[0128] In step S33, the video production support server 11 suggests camera angles by displaying the simulation video generated in step S32. For example, based on information about the shooting location for each shot, multiple possible camera positions are estimated, and multiple simulation videos with different camera angles are displayed as proposed camera positions for the actual shoot.

[0129] In step S34, the video production support server 11 modifies the shooting plan based on information such as the camera angles selected by the video creator. The shooting schedule and storyboard are revised to reflect the changes in the shooting plan. For example, the camera angle information included in the shooting schedule and storyboard is revised.

[0130] <UI during REFINE TO SHOOT> - Screen displaying the evaluation and improvement suggestions for the shot footage Figure 21 shows an example of a tool screen that displays the evaluation and improvement suggestions for the shot footage. The tool screen in Figure 21 is the same screen as the tool screen explained with reference to Figures 14, 18, etc. If the 3D button 261 is operated while checking the evaluation of the shot footage, 3D spatial data representing the shooting location of the selected cut is generated and the camera angle is simulated.

[0131] Figure 22 shows an example of a tool screen for displaying 3D spatial data. The tool screen in Figure 22 is the screen displayed on the user terminal 12's display when the process in step S33 of Figure 20 is performed.

[0132] In the example shown in Figure 22, the simulated image is displayed in the display area 221, with the camera position being the horizontal position of object #111. Object #111 is a 3D model of the performer that appears in the actual footage of the selected shot. Objects such as buildings are placed behind object #111.

[0133] Figures 23 and 24 show examples of camera angles. As shown in Figures 23 and 24, camera angles are suggested using simulated images with different camera positions. Video creators can select their preferred camera angle from the suggested angles or modify the suggested camera angles to choose the camera angle to use for actual shooting.

[0134] Video creators can refine their shooting plans by reviewing the footage they've already shot. After refining the shooting plan, the main shoot is repeated according to the revised shooting schedule and storyboard. By repeating the REFINE TO SHOOT and SHOOT TO REFINE processes, it becomes possible to obtain high-quality footage for post-production.

[0135] <<Configuration of the Video Production Support Server>> Figure 25 is a block diagram showing an example of the hardware configuration of the video production support server 11. The video production support server 11 is configured using computers having the configuration shown in Figure 25. The video production support server 11 can be configured using one computer or multiple computers. The user terminal 12 also has the same configuration as shown in Figure 25.

[0136] The processing circuit 501, ROM (Read Only Memory) 502, and RAM (Random Access Memory) 503 are interconnected by a bus 504.

[0137] An input / output interface 505 is connected to the bus 504. An input unit 506, an output unit 507, a storage unit 508, a communication unit 509, and a drive 510 are connected to the input / output interface 505.

[0138] The input unit 506 consists of physical or virtual operating means such as a keyboard, mouse, or touch panel. The output unit 507 consists of a display or speaker. The storage unit 508 consists of a hard disk, non-volatile or volatile memory, etc., and stores various types of information (including programs). The communication unit 509 is a network interface, etc., and communicates with the outside world via wired or wireless means. The drive 510 drives removable media 511 such as a magnetic disk, optical disk, magneto-optical disk, or semiconductor memory.

[0139] The processing circuit 501 includes a processor such as a CPU (Central Processing Unit) or a DSP (Digital Signal Processor). The processing circuit 501 loads a program stored in the memory unit 508 into the RAM 503 via the input / output interface 505 and the bus 504, and performs a series of processes by executing the program.

[0140] Figure 26 is a block diagram showing an example of the functional configuration of the video production support server 11.

[0141] In the video production support server 11, the information processing unit 521 is realized by the execution of a predetermined program by the processing circuit 501. The information processing system 1, which includes the video production support server 11, is provided with the information processing unit 521. The information processing system 1 is a system equipped with the information processing unit 521.

[0142] The information processing unit 521 consists of an AI processing unit 531, a display control unit 532, an information acquisition unit 533, an information management unit 534, an image acquisition unit 535, and an image management unit 536.

[0143] The AI ​​processing unit 531 performs various AI processes. The AI ​​processing unit 531 is equipped with a machine learning model that realizes the Teammate AI. For example, the AI ​​processing unit 531 inputs information corresponding to the video creator's input into the machine learning model that realizes the Teammate AI, and generates information to present to the video creator based on the output of the machine learning model.

[0144] When the functions of the teammate AI are implemented using an external server, the AI ​​processing unit 531 communicates with the external server and, in cooperation with the external server, generates information to present to the video creator. For example, the AI ​​processing unit 531 sends information that will be input to the machine learning model to the external server and receives the output of the machine learning model sent from the external server. Based on the information sent from the external server, the AI ​​processing unit 531 generates information to present to the video creator.

[0145] In this way, various AI processes are performed by the AI ​​processing unit 531. For example, the processes in steps S2 to S5 in Figure 4, which are processes when PLAN TO SHOOT, the processes in steps S14 and S15 in Figure 12, which are processes when SHOOT TO PLAN, the processes in steps S21 and S24 in Figure 16, which are processes when SHOOT TO REFINE, and the process in step S31 in Figure 20, which is processes when REFINE TO SHOOT, are all processes performed by the AI ​​processing unit 531.

[0146] The display control unit 532 communicates with the user terminal 12 by controlling the communication unit 509 and displays various screens of the video production support tool on the user terminal 12's display. For example, the processes in steps S11 and S13 in Figure 12, which are processes during SHOOT TO PLAN, the processes in steps S23 and S25 in Figure 16, which are processes during SHOOT TO REFINE, and the processes in steps S32 and S33 in Figure 20, which are processes during REFINE TO SHOOT, are processes performed by the display control unit 532. In addition, the process of displaying each of the above-mentioned screens on the user terminal 12's display according to the processing of other processing units is also a process performed by the display control unit 532.

[0147] The information acquisition unit 533 communicates with the user terminal 12 by controlling the communication unit 509 and acquires information transmitted from the user terminal 12. Information entered by the video creator, such as design information, is acquired by the information acquisition unit 533. Information indicating the content of the video creator's operations on the tool screen is also acquired by the information acquisition unit 533. For example, the process in step S1 of Figure 4, which is the process during PLAN TO SHOOT, is performed by the information acquisition unit 533.

[0148] The Information Management Unit 534 manages various types of information, including information acquired by the Information Acquisition Unit 533 and information related to video content under production. Design information, storyboard information, shooting schedule information, etc., are managed by the Information Management Unit 534 in association with the ID of the video creator, etc. The processing in step S34 of Figure 20, which is the process during REFINE TO SHOOT, is performed by the Information Management Unit 534.

[0149] The video acquisition unit 535 communicates with the user terminal 12 by controlling the communication unit 509 and acquires video transmitted from the user terminal 12. The preliminary video and the final video are acquired by the information acquisition unit 533. For example, the process in step S12 of Figure 12, which is the process when SHOOT TO PLAN, and the process in step S22 of Figure 16, which is the process when SHOOT TO REFINE, are processes performed by the video acquisition unit 535.

[0150] The video management unit 536 manages various types of video, including video acquired by the video acquisition unit 535, video content in production, and video acquired as reference. Temporary footage, final footage, etc., are managed by the video management unit 536 in association with the video creator's ID, etc.

[0151] <<UI Variations>> Figure 27 shows another example of the tool screen when using PLAN TO SHOOT.

[0152] A whiteboard area 101 is provided across almost the entire tool screen, and a display area 601 is provided at the bottom of the tool screen. In the example in Figure 27, the storyboard is displayed in the display area 601. Images P201 to P210 placed in the display area 601 are cut images of the selected scene. The display area 601 has the same function as, for example, the display area 222 in Figure 14.

[0153] In the whiteboard area 101, the same images as P201 to P205 are placed in their respective positions, and the order of the cuts is indicated by arrows connecting the images. Each image displays associated information such as the shooting location and references. In addition, based on the input of the video creator, a suggestion comment from the teammate AI is displayed in pop-up #201.

[0154] In the example shown in Figure 27, a design information area 602 is provided on the left side of the tool screen. The design information area 602 is the display area for the design information described above. The contents of the design information imported into the video production support server 11 are displayed in the design information area 602.

[0155] Figure 28 is a magnified view of the design information area 602.

[0156] As shown in Figure 28, the design information includes the following: • Purpose • Target audience attributes • Subject of filming • Video length • Budget • Schedule • Filming location • Equipment

[0157] By continuously displaying design information on the tool screen, video creators can proceed with their planning using the whiteboard area 101 while confirming the requirements and constraints of the video content. Furthermore, they can add or modify the design information while working on their planning using the whiteboard area 101.

[0158] It is possible to have a tool screen with this configuration displayed during PLAN TO SHOOT. For example, the whiteboard area 101 is used to brainstorm ideas using the teammate AI, and the display in the design information area 602 is used to organize requirements and constraints. The display in the display area 601 is used to check the overall flow of the video content. When information is added to one area or information in one area is modified, the corresponding information in other areas is updated in sync.

[0159] <<Other Functions of the Video Production Support Tool>> In addition to the functions described above, various other functions can be incorporated into the video production support tool. At least some of the following functions will be incorporated into the video production support tool. The functions described below are mainly used in the PLAN TO SHOOT phase.

[0160] <Teammate AI Functions> ・Video Interpreter Function Teammate AI is equipped with a video interpreter function. The video interpreter function takes non-video type information as input and outputs information that shows what the video would look like if the content of the input information were expressed as video. The information generated by the video interpreter function is presented to video creators. For example, information such as magazines, novels, anime, and music can be used as input for the video interpreter function.

[0161] The output may be a video representing the content of the input information, or it may be a different type of information other than video, such as still images, text, or audio. When training the machine learning model that implements Teammate AI, machine learning is performed using various types of information other than video, as well as video information representing the content of the information.

[0162] This will enable video creators from all fields, including magazine editors, novelists, animators, and musicians, to create videos based on non-video content. By using the video interpreter function during the planning stage, video creators can gain ideas about what their work would look like when turned into a video.

[0163] - Teammate AI characteristic switching function: Teammate AI is equipped with a function to switch the characteristics of its AI processing. This function is used when generating arbitrary information, such as collecting references, brainstorming ideas, and generating storyboards. For example, there are characteristics for divergence, which are used when information is divergent, and characteristics for convergence, which are used when information is converged.

[0164] When characteristics for divergence are set, for example, information is generated that does not take into account the project requirements and constraints included in the design information, such as budget, shooting location, and time, based on the input of a video creator. On the other hand, when characteristics for convergence are set, for example, information is generated that takes into account the project requirements and constraints included in the design information, and is feasible within the scope of those requirements and constraints, based on the input of a video creator. The generated information is displayed on the tool screen and presented to the user.

[0165] By switching the characteristics of AI processing, information with varying degrees of feasibility can be generated. Video creators can switch the AI ​​processing characteristics of their teammate AI according to their own needs.

[0166] <Functionality of the Creator-Specific Map> The video production support tool includes a creator-specific map function. The creator-specific map is used by video creators at various stages, such as during the planning stage, to check information about the filming location.

[0167] The creator-dedicated map site may be managed by the video production support server 11, or it may be managed by an external server. When a video creator performs a predetermined operation, access to the site is made, and the creator-dedicated map is displayed on the tool screen. Similarly, when a site review is requested using the display of the selection area 122 (Figure 8), the creator-dedicated map is also displayed on the tool screen.

[0168] When a video creator selects a location on the creator-dedicated map, useful information will be displayed if they choose that location as their filming site. For example, detailed information about the buildings at the selected location, information on how to obtain filming permits, information for shooting good footage (shooting angles, noise levels, equipment settings, etc.), and information on how sunlight enters the location will be displayed.

[0169] By using a map specifically designed for creators, video creators can efficiently select filming locations.

[0170] The creator-dedicated map also functions as a community site for video creators to exchange information. Video creators can register information about filming locations. Information registered by multiple video creators is linked to locations on the creator-dedicated map and managed on the video production support server 11. By registering information about filming locations they have used, multiple video creators can share information about filming locations.

[0171] A service to handle the procedures for obtaining permission for filming may be provided and made available from the tool screen. Information registered in the creator-only map may be presented by the teammate AI. In this case, the information registered in the creator-only map will be used to train the machine learning model that implements the teammate AI.

[0172] <Video Reverse Engineering Function> Teammate AI will be equipped with a video reverse engineering function. This function takes a reference video as input and outputs information about how the reference video was shot. The information generated by the video reverse engineering function will be presented to the video creator.

[0173] For example, information such as the type of camera and lens, camera and lens settings, shooting time, and lighting settings will be presented. Teammate AI's AI processing will analyze the type of camera and lens used to shoot the reference video, the camera and lens settings used to shoot the reference video, the shooting time of the reference video, and the lighting settings at the time of shooting the reference video.

[0174] For example, if a video creator performs an action based on information presented by Teammate AI, the shooting method of the reference video will be reflected in the storyboard, etc. During the training of the machine learning model that realizes Teammate AI, learning is performed using various videos equivalent to reference videos and information on how they were shot.

[0175] By utilizing the reverse engineering function for video, video creators can easily check how reference videos are shot, how to use similar techniques, and incorporate these into their own video content. Furthermore, video creators can identify the characteristics of trending videos and incorporate them into their own content.

[0176] By presenting various analyzed information, it becomes possible to improve the skills of video creators. When analyzing reference videos, personal information contained in the reference videos may be removed before being used for analysis.

[0177] <Reconstruction Video Generation Function> The teammate AI will be equipped with a reconstruction video generation function. This function takes information such as storyboards and design information as input and outputs a video that expresses the content of the input information. The video generated by the reconstruction video generation function will be presented to the video creator.

[0178] During the training of the machine learning model that will realize the teammate AI, learning will be performed using information such as storyboards and design information, as well as various videos equivalent to reconstructed footage.

[0179] By using the function to generate reconstructed footage before test shoots or the actual shoot, video creators can obtain an image of the shoot in advance. In so-called virtual productions that use a large display wall as a background, video creators can obtain an image of the background video to be displayed on the display wall.

[0180] It is also possible to use thumbnails of the recreated footage as cut images to make up the storyboard. Because more detailed storyboards are prepared, video creators can communicate smoothly with other video creators who are members of the production team.

[0181] <Production Member Matching Function> The Teammate AI is equipped with a function to match production members necessary for video production. The video production support server 11 manages various information about video creators, such as their abilities and past works. For example, when a storyboard is generated, information on video creators who have the necessary abilities to produce the video according to the storyboard, or video creators who have produced similar works in the past, is presented as production member information.

[0182] For example, information such as the names and contact information of the production members is provided. When training the machine learning model that realizes the teammate AI, the learning is performed using the storyboard information and the production member information. The production member matching function may be provided by a site managed by the video production support server 11.

[0183] There are limitations to producing videos alone, and there is a need to entrust work to reliable video creators or creators with skills that the creator lacks. By using a matching function for production members, video creators can easily find the people they need for their video production. Freelance creators can also get work by registering their information on the video production support server 11.

[0184] <Project Management Function> Teammate AI will be equipped with a project management function. This function generates confirmation items for each stage of the entire video production process and presents them to the video creator as tasks. For example, information on confirmation items for each stage related to schedule management, budget management, equipment management, etc., will be generated and presented using a predetermined template.

[0185] By utilizing the project management function, video creators can more easily manage their tasks. Furthermore, having each task presented by the teammate AI helps prevent tasks from being forgotten.

[0186] The information regarding the items to be confirmed will be shared among the other production members. Not only will they be able to view it, but each production member will also be able to modify the information regarding the items to be confirmed.

[0187] <<Other>> <About the Program> The series of processes described above can be executed by hardware or by software. When the series of processes are executed by software, the programs that make up the software are installed on the computer. Here, the computer includes computers built into dedicated hardware, and general-purpose personal computers that can perform various functions by installing various programs, for example, the configuration shown in Figure 25.

[0188] The program executed by the computer (processing circuit 501 (Figure 25)) can be provided by recording it on a removable medium 511, such as a package medium. The program can also be provided via wired or wireless transmission media, such as a local area network, the internet, or digital satellite broadcasting.

[0189] In a computer, a program can be installed in the storage unit 508 via the input / output interface 505 by inserting the removable media 511 into the drive 510. Alternatively, a program can be received by the communication unit 509 from another device, such as a server, via a wired or wireless transmission medium, and installed in the storage unit 508. Furthermore, programs can be pre-installed in the ROM 502 or the storage unit 508.

[0190] The programs executed by the computer may be programs that are processed chronologically in the order described herein, or they may be programs that are processed in parallel or at necessary times, such as when a call is made.

[0191] The processes that a computer performs according to a program do not necessarily have to follow the order described in the flowchart. In other words, the processes that a computer performs according to a program include processes that are executed in parallel or individually (e.g., parallel processing and object-based processing).

[0192] The program may be processed by a single computer (processor), or it may be processed in a distributed manner by multiple computers. Furthermore, the program may be transferred to a remote computer and executed there.

[0193] In this specification, a system means one component or a collection of multiple components (devices, modules (parts), etc.). Therefore, one or more components of a computer, for example, only the processor, or a combination of a processor and memory, constitute a system. Regarding a collection of multiple components, it is not necessary whether all components reside in the same enclosure. Therefore, multiple devices housed in separate enclosures and connected via a network, or a single device containing multiple modules within a single enclosure, are all systems. Furthermore, for example, an entire computer, or a combination of a computer and other devices such as a server (not shown), also constitutes a system.

[0194] The information processing system 1 can be configured with a video production support server 11 and a user terminal 12, or it can be configured with only the video production support server 11. Furthermore, the information processing system 1 can be configured with the processing circuit 501 of the video production support server 11.

[0195] The embodiments of this technology are not limited to those described above, and various modifications are possible without departing from the spirit of this technology.

[0196] Furthermore, the effects described herein are merely illustrative and not limiting, and other effects may also occur.

[0197] <Examples of configuration combinations> This technology can also be configured as follows:

[0198] (1) An information processing system comprising an information processing unit that generates and displays information indicating an outline of video content to be produced based on design information including at least one of requirements and constraints; generates and displays proposed information regarding the content of the video content in response to a prompt entered by the user; and generates and displays a storyboard in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the proposed information selected by the user, are arranged in the order of shooting. (2) The information processing system according to (1), wherein the information processing unit generates and displays the proposed information each time the prompt is entered; and generates and displays the storyboard of the video content, reflecting the content of the proposed information selected by the user from among the displayed proposed information. (3) The information processing system according to (1) or (2), wherein the information processing unit acquires a trial video obtained by trial shooting performed in accordance with the storyboard before the actual shooting, manages it in association with each cut; plays back the trial video; and modifies the storyboard in response to the user's input. (4) The information processing system according to (3), wherein the information processing unit, upon acquiring the preliminary footage, displays a thumbnail of the preliminary footage of the corresponding cut as the cut image in place of the cut image on the storyboard. (5) The information processing system according to (3) or (4), wherein the information processing unit generates and displays information that constitutes an improvement plan for how to shoot the corresponding cut based on the preliminary footage, and modifies the storyboard in response to the user's input regarding the improvement plan. (6) The information processing system according to any one of (3) to (5), wherein the information processing unit generates and displays a shooting schedule for the actual shoot based on the storyboard. (7) The information processing system according to (6), wherein the information processing unit acquires the actual footage obtained from the actual shoot carried out according to the shooting schedule, manages it in association with each cut, and displays a thumbnail of the actual footage of the corresponding cut as the cut image in place of the cut image on the storyboard.(8) The information processing system according to (7), wherein the information processing unit displays a thumbnail of one of the main footages selected from multiple takes of the same shot as the cut image. (9) The information processing system according to (8), wherein the information processing unit selects the best take of the main footage from multiple takes of the same shot based on evaluation criteria set by the user. (10) The information processing system according to (9), wherein the information processing unit generates a 3D model of the shooting location where the main footage was shot based on the main footage, and generates a simulation video based on the 3D model where the camera position is at an arbitrary location on the shooting location. (11) The information processing system according to (10), wherein the information processing unit displays the simulation video as information indicating a proposed camera position for the main footage. (12) The information processing system according to any one of (1) to (11), wherein the information processing unit generates and displays information indicating a video that represents the content of input information, based on input information other than video input by the user. (13) The information processing unit generates and displays the proposed information, which has a different degree of feasibility, according to the settings made by the user, according to any of (1) to (12). (14) The information processing unit generates and displays a map, which has registered information about the shooting location by multiple other users, according to any of (1) to (13). (15) The information processing unit generates and displays information about how the reference video was shot, according to any of (1) to (14), after analyzing a reference video input by the user. (16) The information processing unit generates and displays a reconstructed video that represents the content of the information input by the user, according to any of (1) to (15). (17) The information processing unit generates and displays a candidate for a member to produce the video content, based on information about multiple other users, according to any of (1) to (16).(18) An information processing method comprising: generating and displaying information indicating an outline of video content to be produced based on design information including at least one of requirements and constraints; generating and displaying proposed information regarding the content of the video content in response to a prompt entered by a user; and generating and displaying a storyboard in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the proposed information selected by the user, are arranged in the order of shooting. (19) A program for causing a computer to perform a process comprising: generating and displaying information indicating an outline of video content to be produced based on design information including at least one of requirements and constraints; generating and displaying proposed information regarding the content of the video content in response to a prompt entered by a user; and generating and displaying a storyboard in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the proposed information selected by the user, are arranged in the order of shooting.

[0199] 1 Information processing system, 11 Video production support server, 12 User terminal, 521 Information processing unit, 531 AI processing unit, 532 Display control unit, 533 Information acquisition unit, 534 Information management unit, 535 Video acquisition unit, 536 Video management unit

Claims

1. An information processing system comprising an information processing unit that generates and displays information outlining the video content to be produced based on design information including at least one of the requirements and constraints; generates and displays suggested information regarding the content of the video content in response to prompts entered by the user; and generates and displays a storyboard in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the suggested information selected by the user, are arranged in the order of shooting.

2. The information processing unit generates and displays the suggested information each time the prompt is input, and generates and displays the storyboard of the video content that reflects the content of the suggested information selected by the user from among the displayed suggested information, according to claim 1.

3. The information processing unit acquires preliminary footage obtained by preliminary shooting performed in accordance with the storyboard before the actual shooting, manages it in association with each cut, plays back the preliminary footage, and modifies the storyboard in accordance with the user's input, as described in claim 1.

4. The information processing system according to claim 3, wherein, in response to acquiring the provisionally filmed video, the information processing unit displays a thumbnail of the provisionally filmed video of the corresponding cut as the cut image in place of the cut image of the storyboard.

5. The information processing system according to claim 3, wherein the information processing unit generates and displays information that represents a proposed improvement in how to shoot a corresponding cut based on the provisional footage, and modifies the storyboard in response to the user's input regarding the proposed improvement.

6. The information processing system according to claim 3, wherein the information processing unit generates and displays the shooting schedule for the actual filming based on the storyboard.

7. The information processing unit acquires the actual footage obtained by the actual shooting carried out in accordance with the shooting schedule and manages it in association with each cut, and displays a thumbnail of the actual footage of the corresponding cut as the cut image in place of the cut image of the storyboard, according to claim 6.

8. The information processing system according to claim 7, wherein the information processing unit displays a thumbnail of one take of the main footage selected from multiple takes of the same cut as the cut image.

9. The information processing system according to claim 8, wherein the information processing unit selects the best take of the main footage from among multiple takes of the same shot, based on evaluation criteria set by the user.

10. The information processing system according to claim 9, wherein the information processing unit generates a 3D model of the shooting site where the actual shooting was performed based on the actual shooting video, and generates a simulated video based on the 3D model where an arbitrary position in the shooting site is the camera position.

11. The information processing system according to claim 10, wherein the information processing unit displays the simulation image as information indicating a proposed camera position for the actual shooting.

12. The information processing system according to claim 1, wherein the information processing unit generates and displays information showing an image representing the content of the input information based on input information other than video input by the user.

13. The information processing system according to claim 1, wherein the information processing unit generates and displays the proposed information, which has a different degree of feasibility, according to the settings made by the user.

14. The information processing system according to claim 1, wherein the information processing unit displays a map in which information about the shooting location has been registered by multiple other users.

15. The information processing system according to claim 1, wherein the information processing unit analyzes the reference video input by the user, generates and displays information regarding how the reference video was filmed.

16. The information processing system according to claim 1, wherein the information processing unit generates and displays a reconstructed video representing the content of the information input by the user.

17. The information processing system according to claim 1, wherein the information processing unit selects and displays candidates for members who produce the video content based on information about a plurality of other users.

18. An information processing method comprising: generating and displaying information indicating an outline of video content to be produced based on design information including at least one of requirements and constraints; generating and displaying suggested information regarding the content of the video content in response to a prompt entered by a user; and generating and displaying a storyboard in which cut images showing the screen configuration of each cut of the video content, which reflects the content of the suggested information selected by the user, are arranged in the order of shooting.

19. A program for causing a computer to perform the following processes: generating and displaying information outlining the video content to be produced based on design information including at least one of the requirements and constraints; generating and displaying suggested information regarding the content of the video content in response to prompts entered by the user; and generating and displaying a storyboard in which cut images showing the screen configuration of each cut of the video content, reflecting the content of the suggested information selected by the user, are arranged in the order of shooting.

Images