Imaging assistance device and operation method and operation program for imaging assistance device

Abstract

This imaging assistance device is for assisting image capturing on the basis of scenario information, and comprises a processor. The processor acquires imaging status information related to the scenario information, and corrects the scenario information on the basis of the imaging status information.

Description

Photographing support device, operation method and operation program for photographing support device

[0001] The technology of the present disclosure relates to a photography support device, an operation method for a photography support device, and an operation program.

[0002] Japanese Patent Application Laid-Open Publication No. 2023-075024 describes an imaging device that includes an imaging unit that captures an image of a subject and generates image data, an output unit that outputs information to a user, and a control unit that controls the output unit based on recording data in which information about a first video is recorded and the image data generated by the imaging unit, and the control unit outputs to the output unit shooting auxiliary information that assists the user in aligning the second video with the first video while the imaging unit is shooting the second video.

[0003] One embodiment of the technique of the present disclosure provides a photography support device, an operation method for a photography support device, and an operation program that can flexibly respond to actual photography environments and the like compared to conventional devices.

[0004] The photography support device relating to the technology disclosed herein is a photography support device that supports image photography based on scenario information, and is equipped with a processor.The processor acquires photography status information that is related to the scenario information, and modifies the scenario information based on the photography status information.

[0005] The shooting status information preferably includes shooting environment information.

[0006] The filming environment information preferably includes information about the weather, time of day, season, filming location, filming equipment, performers, and staff.

[0007] The shooting status information preferably includes video of the scenes that have been shot, and may also include video that has been input in association with the scenario information.

[0008] The processor preferably has a function of editing the footage of the shot scene based on the scenario information when there is an inconsistency between the footage of the shot scene and the scenario information.

[0009] The scenario information preferably includes information indicating the degree of need for protection for each item.

[0010] The processor preferably has auxiliary functions to assist in generating the scenario information.

[0011] The auxiliary function is preferably a function that adds recommended conditions regarding the shooting method based on the scenario information.

[0012] The processor preferably executes control to output the scenario information to the display unit.

[0013] The operating method of a photography support device relating to the technology disclosed herein is a method of operating a photography support device that supports image photography based on scenario information, obtains photography status information that is related to the scenario information, and modifies the scenario information based on the photography status information.

[0014] The operating program of the photography support device relating to the technology disclosed herein is an operating program of the photography support device that supports image photography based on scenario information, and causes a processor to execute processing including obtaining photography status information that is related to the scenario information, and modifying the scenario information based on the photography status information.

[0015] The technology disclosed herein allows for more flexible response to the actual shooting environment and the like than conventional techniques.

[0016] 1 is a diagram showing an overview of a photography support device; FIG. 2 is a diagram showing an example of a hardware configuration constituting a photography support device or the like; FIG. 3 is a diagram showing an overview of processing by the photography support device; FIG. 4 is a diagram showing an example of a menu screen; FIG. 5 is a diagram showing an example of scenario information; FIG. 6 is a diagram showing an example of the configuration of video data; FIG. 7 is a diagram showing an example of scenario correction processing; FIG. 8 is a flowchart showing variations of scenario information, illustrating the processing procedure of photography support processing centered on scenario correction processing; FIG. 9 is a diagram showing an example of scenario correction processing based on scenes that have already been filmed; FIG. 10 is a diagram showing an example of processing to edit video data based on scenes that have already been filmed; FIG. 11 is a diagram showing another example of processing to edit video data based on scenes that have already been filmed; FIG. 12 is a diagram showing an example of an auxiliary function that assists in generating scenario information.

[0017] As shown in FIG. 1 , the photography support device 10 is a device that supports the shooting of image content. When the user 11 shoots image content using the photography device 20, the photography support device 10 presents scenario information 30 to the user 11. The image content is, for example, a video, and the following description will use a video as an example. The scenario information 30 is the scenario for the video and includes at least one of a story including a plot summary, a storyboard, information about the scene to be shot, and recommended conditions regarding the shooting method, and is information that defines the shooting content of the video. As shown in FIG. 1 , the scene information includes information about the scene settings, such as the shooting location, weather, and time of day, as well as an overview of the scene. Recommended conditions regarding the shooting method include camerawork, angle of view, exposure, and shooting equipment. Information about the shooting equipment is, for example, information about a zoom lens and a wide-angle lens. Such recommended conditions regarding the shooting method can be said to define the shooting content of the video in the sense that they have the effect of adding effects to the video or changing the atmosphere of the video. The scenario information 30 in FIG. 1 exemplifies the overview, location, weather, and time of day of the scene to be shot.

[0018] FIG. 1 shows an example in which a user 11 is a cameraman who shoots a video of an actor 12 performing. The user 11 shoots using a camera device 20. One example of the camera device 20 is a smartphone with a camera function. One example of the photography support device 10 is a portable notebook PC (Personal Computer). The photography support device 10 provides scenario information 30 to the user 11. The user 11 shoots a video with the camera device 20 while checking the scenario information 30. For example, the user 11 checks the location, weather, time of day, etc. based on the scenario information 30, and shoots an outline of a scene such as "a date in the park." The shot video data 36 is sent from the camera device 20 to the photography support device 10.

[0019] 2, the computer constituting the photography support device 10 includes a display unit 21, an input unit 22, a processor 23, a memory 24, a communication I / F (Interface) 25, a storage 27, and a camera 28. These components are interconnected via a bus line 26.

[0020] The display unit 21 is a display that displays scenario information 30. The input unit 22 is a keyboard, mouse, etc. for inputting operation instructions. If the display unit 21 is configured as a touch panel display, the touch panel display also constitutes the input unit 22. The storage 27 is a data storage device such as a flash memory, a hard disk drive, or a solid state drive. The memory 24 is a work memory for the processor 23 to execute processing, and is configured by a RAM (Random Access Memory) etc.

[0021] The storage 27 stores control programs such as an operating system, various application programs, and various data associated with these programs. The processor 23 is, for example, a CPU (Central Processing Unit). The processor 23 loads the programs stored in the storage 27 into the memory 24 and executes processing in accordance with the programs, thereby providing overall control of each part of the computer. The communication I / F 25 is a network interface for communication via a communication network and performs transmission control. The camera 28 is a built-in digital camera.

[0022] The smartphone constituting the photographing device 20 also has the same hardware configuration as the photographing support device 10 and includes the elements shown in FIG. 2. Note that while the camera 28 is an essential element of the photographing device 20, the photographing support device 10 does not necessarily have to include the camera 28. In the following, to distinguish between the hardware configuration of the photographing support device 10 and the hardware configuration of the photographing device 20, the reference numerals of the hardware configuration of the photographing support device 10 will be denoted by a subreference numeral "A," and the reference numerals of the hardware configuration of the photographing device 20 will be denoted by a subreference numeral "B." For example, the processor 23 will be referred to as processor 23A in the hardware configuration of the photographing support device 10 and as processor 23B in the hardware configuration of the photographing device 20.

[0023] 3, a program 41 is stored in the storage 27A of the photographing support device 10. The program 41 is an example of an operation program for the photographing support device according to the technique of the present disclosure. The processor 23A executes the program 41 to function as a processing unit that executes photographing support processing.

[0024] The photography support processing includes a scenario presentation processing, a scenario correction processing, and a scenario creation support processing. The scenario presentation processing is a processing for presenting scenario information 30, and specifically, executes control for outputting the scenario information 30 to the display unit 21A. The scenario information 30 is registered in advance in the photography support device 10 by a user such as the user 11. The registered scenario information 30 is stored in the storage 27A. The processor 23A reads the scenario information 30 from the storage 27A and outputs it to the display unit 21A.

[0025] As will be described later, the scenario correction process is a process of correcting the scenario information 30 based on the shooting status information. The shooting status information is information related to the scenario information 30, and the shooting status information includes, for example, shooting environment information. The shooting environment information is information about the environment when shooting is performed using the shooting device 20, such as the location, weather, and time of day of shooting. The scenario information 30 also specifies the location, weather, and time of day, so the shooting environment is related to the scenario information 30 in the sense that they mutually influence each other. For example, if the scenario information 30 specifies the weather as "sunny" but the weather at the actual location where shooting is performed is "rainy," shooting may be possible by correcting the scenario information 30 to one that sets the weather to "rainy." In this way, the scenario correction process is a process of correcting the scenario information 30 based on the shooting environment information.

[0026] The scenario correction process, for example, uses a machine learning model incorporating generative AI (Artificial Intelligence) technology. Generative AI can generate new content such as text and images based on input data such as text. In the scenario correction process, for example, based on the input shooting environment information and the original scenario information 30, correction is made to new scenario information 30 that is compatible with the shooting environment information. The corrected scenario information 30 is also displayed on the display unit 21A.

[0027] The scenario creation support process is a process that supports the user 11 or the like in creating, editing, and registering scenario information 30 .

[0028] The storage 27B of the photographing device 20 stores an application program (AP) 42 for cooperation with the photographing support device 10. By executing the cooperation AP 42, the processor 23B of the photographing device 20 can cooperate with the processor 23A that executes the photographing support process. Specifically, the photographing device 20 acquires photographing environment information via the communication I / F 25 and captures video using the camera 28B. The photographing environment information acquired while the cooperation AP 42 is running and the captured video data 36 are handled, for example, under the management of the processor 23B that executes the cooperation AP 42 and the processor 23A that executes the photographing support process. For example, the storage folder for the video data 36 and the accompanying information attached to the video data 36 are specified by the cooperation AP 42 and the photographing support device 10. Furthermore, cooperation between the cooperation AP 42 and the photographing support device 10 also enables communication of control information for the photographing support process between the photographing support device 10 and the photographing device 20.

[0029] The image capture environment information is acquired by the image capture device 20, for example, by the image capture device 20 accessing a website that distributes weather information. The location can be acquired using the surrounding GPS (Global Positioning System) function. Information on the time zone and season can be acquired, for example, from a clock program and a calendar program attached to the operating system. While this example illustrates an example in which the image capture device 20 acquires the image capture environment information, if the image capture support device 10 is also located in the same place as the image capture device 20, the image capture support device 10 may also acquire the image capture environment information.

[0030] 4 shows an example of a menu screen 46 that displays a menu of executable processes in the photography support device 10. The menu screen 46 functions as an operation screen configured with a GUI (Graphical User Interface). The menu screen 46 is provided with a scenario presentation button 46A, a scenario modification button 46B, and a scenario creation support button 46C.

[0031] The scenario presentation button 46A is an operation unit that instructs the scenario presentation process, i.e., the display of scenario information 30 on the display unit 21A. When the scenario presentation button 46A is operated, for example, a selection screen for scenario information 30 is displayed. Then, the scenario information 30 selected on the selection screen is displayed.

[0032] The scenario modification button 46B is an operation section for instructing the execution of a scenario modification process. When the scenario modification button 46B is operated, a selection screen for scenario information 30 is displayed. Then, on the selection screen, the user 11 selects the scenario information 30 that they wish to modify and the scene to be modified contained in the scenario information 30, and if the selected scene can be modified, the scenario modification process is executed.

[0033] The scenario creation support button 46C is an operation unit that instructs the execution of the scenario creation support process. When the scenario creation support button 46C is operated, in the case of new creation, a new scenario editing screen is displayed and editing is accepted. On the editing screen, it is also possible to edit scenario information 30 that has already been created. In addition, registration of created scenario information 30 is also executed in the scenario creation support process.

[0034] Fig. 5 shows an example of scenario information 30. The example shown in Fig. 5 is scenario information for a video depicting the beginning of a relationship between a bride and groom to be screened at a wedding. In this example, it is assumed that the actors 12 shown in Fig. 1 are the bride and groom, and the friends of the bride and groom are users 11 who shoot the video based on the scenario information 30.

[0035] The scenario information 30 is titled "The Bride and Groom's First Love" and is made up of three scenes with titles: "Meeting," "Confession," and "Proposal." Each scene includes an outline, scene setting items, and a storyboard. The outline is text information that shows the story including the outline of the scene. The setting items are items that represent the situation of the scene, such as the location, weather, time of day, and season. The setting content and importance are recorded for each setting item. The storyboard is an illustration that shows the image of the scene. The storyboard also includes information on the general angle of view.

[0036] For example, in the scene 1 "Encounter," the summary is "At Meiji Shrine, the groom ran into a gazebo due to a sudden rain and met the bride." The setting content for each setting item includes information such as "Meiji Shrine" and "temple / shrine" as the location, "rain" as the weather, "evening" as the time period, and "summer" as the season.

[0037] The importance level is information indicating the degree of need for protection for each setting item, with higher importance levels indicating greater need for protection. The need for protection indicates the need to protect the setting content from modification, with a high need meaning that the setting content is less likely to be subject to modification in the scenario modification process. The importance level is set, for example, in three levels: “high,” “medium,” and “low.” “High” means that the content must be protected and cannot be modified. “Medium” means that the content should be protected without modification as much as possible, and “low” means that modification is permitted. Thus, the lower the importance level, the greater the tolerance for modification. In the example of Scene 1, with regard to the location setting content, “Meiji Shrine” is “low,” while “Temple / Shrines” is “medium.” This means that the location does not have to be “Meiji Shrine,” but a “Temple / Shrines” is preferable. Furthermore, the importance of the weather being “rainy” is “high,” while the other settings are “low.”

[0038] The storyboard includes an illustration of the groom rushing into the gazebo where the bride was sheltering from the rain, followed by an illustration of the bride and groom sheltering side by side in the gazebo.

[0039] In scene 2, "Confession," the summary is that "a while after the two began dating, the groom confessed his love to the bride while they were out at the park." The setting details for each setting item include information such as the location "park," the weather "sunny," the time of day "evening," and the season "summer." The storyboard includes an illustration of the couple sitting on a bench and an illustration of the groom confessing his love to the bride while riding a boat on the pond in the park. The importance of all setting details items is "low."

[0040] In scene 3, "Proposal," the summary is that "the groom proposed to the bride on the same rainy day in the gazebo at Meiji Shrine, the same place where they met." The settings for each setting item are the same as in scene 1, and the importance of each setting is also the same as in scene 1.

[0041] Each scene is provided with timing buttons (indicated by "S" and "E," respectively) that indicate the timing for starting and ending shooting. The timing buttons are operated to start and end shooting of the scene. For example, the processor 23A recognizes the video transmitted from the image capture device 20 while these two timing buttons are being operated as the video of that scene.

[0042] By setting the scene boundaries in this way, the video data 36 is divided and recorded for each scene. For example, as shown in Fig. 6, the video data 36 in the storage 27A is made up of video data 36_1, video data 36_2, and video data 36_3 for three scenes 1 to 3. The video data 36 is also stored in association with the corresponding scenario information 30.

[0043] 7 shows a specific example of scenario modification processing. The example shown in FIG. 7 shows an example in which the processor 23A performs scenario modification processing on the "confession" of scene 2 of the original scenario information 30 based on the shooting environment information. In the scenario information 30 shown in FIG. 5, the weather being "rainy" is the catalyst for the bride and groom meeting, and is therefore a very important element. Therefore, in scenes 1 and 3, the importance of the weather being "rainy" is set to "high," which means that it cannot be modified.

[0044] Based on this information in the scenario information 30, the user 11 recognizes the importance of weather for scenes 1 and 3, and therefore decides to wait for a rainy day to shoot scenes 1 and 3. Meanwhile, since the weather for scene 2 is "sunny," according to the original scenario information 30, the shooting should be done on a "sunny" day. However, if the shooting date is on a different day, this may not be possible due to the schedules of the user 11 and the actor 12. In such a case, the user 11 would like to shoot scene 2 on the same day as scenes 1 and 3. For scene 2, the importance of "sunny" weather is set to "low," which means that weather adjustments are permitted. In such a case, the user 11 instructs the photography support device 10 to execute a scenario adjustment process for scene 2.

[0045] When scenes 1 and 3 are shot, the weather at the shooting locations is "rainy," so when scene 2 is shot on the same day at the same locations, "rainy" is input as the shooting environment information. Processor 23A executes the scenario modification process for scene 2 based on the shooting environment information of "rainy."

[0046] The original scenario information 30 includes a title such as "How the Bride and Groom Met," titles for each scene such as "Meeting," "Confession," and "Proposal," and text summarizing each scene. In addition to such an overview of the entire video, text summarizing scene 2, which is the target of correction, is also recorded. Processor 23A uses a machine learning model incorporating generative AI technology to interpret the meaning of the plot and story of scenario information 30 from the text of the original scenario information 30, and corrects the content of scene 2 using the constraint that the weather is "rain."

[0047] As shown in Figure 7, in the original scene 2, the groom confesses his love in a "park" under "sunny" weather, but in the revised version of scene 2, the scene has been revised to one in a "cafe" under "rainy" weather. In this way, processor 23A modifies the location setting in the scenario information 30 for scene 2 to match the weather. Furthermore, processor 23A also modifies the storyboard to an illustration depicting the bride and groom sitting facing each other in a "cafe" in accordance with the revised setting, with "rain" falling outside the window in accordance with the filming environment information of "rain." In Figure 7, the modified parts from the original in the revised version are indicated by hatching.

[0048] The operation of the above configuration will be described with reference to the flowchart shown in Fig. 8. As an example, the user 11 creates scenario information 30 in advance and registers it in the photography support device 10. Then, since it is important that the weather in scenes 1 and 3 of the scenario information 30 be "rainy," the user 11 waits for a rainy day to perform photography. The user 11 travels to Meiji Shrine with the bride and groom, who will be performers 12, sets up the photography support device 10 at the location where photography will be performed, and begins photography with the photography device 20.

[0049] Then, after shooting scenes 1 and 3 in "rainy" weather, if the user 11 also wants to shoot scene 2 in "sunny" weather, the user instructs the photography support device 10 to execute a scenario correction process. When the user instructs the photography support device 10 to execute a scenario correction process for the selected scene 2 on the menu screen 46 (step ST1000), the photography support device 10 starts the scenario correction process for scene 2.

[0050] In step ST1100, the processor 23A of the photographing support device 10 acquires photographing environment information as photographing status information. As the photographing environment information, "rain" which is the weather at the current location where the user 11 is present is input.

[0051] In step ST1200, the processor 23A considers whether or not the original scenario information 30 can be modified based on the input shooting environment information. First, the weather in scene 2 of the original scenario information 30 is "sunny," while the weather in the shooting environment information is "rainy." The processor 23A compares the original scenario information 30 with the shooting environment information and determines that an inconsistency exists regarding the weather. Then, the importance of weather in scene 2 is "low," and weather is an item for which modification is permitted. Therefore, the processor 23A determines that modification is possible (Y in step ST1200) and proceeds to step ST1300. If modification is not possible here (N in step ST1200), the processor 23A proceeds to step ST1700 and displays a message indicating that modification is not possible.

[0052] In step ST1300, the processor 23A executes a scenario modification process using the machine learning model of the generation AI, and generates modified scenario information 30 that matches the weather of the actual shooting environment, which is "rain," as shown in Figure 7.

[0053] In step ST1400, the processor 23A displays the revised scenario on the display unit 21A, thereby proposing the revised scenario to the user 11. Then, the processor 23A requests the user 11 for approval.

[0054] In step ST1500, if the user 11 accepts the revised scenario (Y in step ST1500), the revision contents of the scenario information 30 are confirmed and the revised scenario is presented. On the other hand, if the revised scenario is not accepted, the process proceeds to step ST1700.

[0055] If the user 11 accepts the revised scenario, the video is shot according to the revised scenario. The revised scenario includes modifications based on the actual shooting environment, but still follows the outline of the original scenes, and does not disrupt the overall story of the video, which is the beginning of the bride and groom's relationship.

[0056] As described above, the photography support device 10 according to the technology of the present disclosure is a photography support device that supports image photography based on scenario information 30, and includes a processor 23A. The processor 23A acquires photography status information that is related to the scenario information 30 and modifies the scenario information 30 based on the photography status information. As shown as an example in the above embodiment, even when the original scenario information 30 differs from the actual photography environment, the scenario information 30 is modified in accordance with the actual photography environment. This allows for more flexible response to the actual photography environment, etc., than in the past.

[0057] The shooting status information also includes shooting environment information. Since shooting environment information, such as the weather at the shooting location, is often impossible to change manually, a function for modifying scenario information in accordance with shooting environment information is highly convenient for users. The shooting environment information includes information on the weather, time of day, season, shooting location, shooting equipment, performers, and staff. In the above example, an example was given in which the scenario information 30 is modified in accordance with the weather, but other examples are also conceivable in which the scenario information 30 is modified based on information other than the weather.

[0058] For example, regarding the time period, if a scene in the original scenario information 30 was a "morning" scene, it may be changed to a "night" scene if the scene is to be shot at "night." Similarly, regarding the season, it may be changed from a "summer" scene to a "winter" scene. Similarly, regarding the shooting location, it may be changed from a "Meiji Shrine" scene to a "Hikawa Shrine" scene, or from a "shrine" scene to a "temple" scene.

[0059] As described above, the scenario information 30 may also include information about the shooting equipment. In this case, a scene designated to be shot with shooting equipment with a focal length of more than 500 mm may be modified to a scene that can be shot with shooting equipment with a focal length of 100 mm or less. Regarding the performers 12, a scene designated with multiple male performers 12 may be modified to a scene with multiple female performers. Regarding the staff, for example, when multiple cameramen are designated to shoot the same scene from multiple directions, the scene may be modified to a scene that can be shot with a single cameraman. Since such shooting environment information often changes in actual situations, being able to respond to such changes in the situation is extremely convenient.

[0060] Furthermore, the scenario information 30 includes information indicating the degree of protection required for each item to be set, as shown by the importance level as an example. This makes it possible to protect items in the scenario information 30 that should not be changed. By explicitly indicating the need for such protection, it is possible to more appropriately reflect in the revised version the content that was considered important in the original scenario information 30.

[0061] In the above example, the importance is set in three levels, "high," "medium," and "low," but it may also be set in two levels, such as "modifiable" and "unmodifiable," or it may be an evaluation value based on a numerical value, etc. The evaluation value may also be a continuous value or a discrete value.

[0062] The processor 23A also executes control to output the scenario information 30 to the display unit 21A. Having such a function is convenient when a terminal equipped with the display unit 21A, such as a tablet terminal in addition to a notebook PC, functions as the photographing support device 10. The photographing support device 10 may also be configured as a server that provides photographing support services to a plurality of client terminals. In this case, the processor 23A presents the scenario information 30 by distributing the scenario information 30 to each client terminal instead of or in addition to outputting it to the display unit 21A.

[0063] The photographing support device 10 according to the technique of the present disclosure can be modified in various ways other than the above embodiment.

[0064] "Variation 1" As shown in FIG. 9, variations in the content of modifications to a scenario based on shooting status information including shooting environment information can include modifying the location, which is a scene setting, or modifying the shooting order of multiple scenes. An example of modifying a location is to suggest a change of location to match the weather in the original scenario information. For example, if the weather in the scenario information is "rainy," but the weather at the current shooting location is "cloudy," the processor 23A suggests a location where it is actually "raining." More specifically, if the weather in Tokyo is "cloudy" and the weather in Kamakura, Kanagawa, is "rainy," the processor 23A suggests changing the shooting location from "Meiji Shrine," which is "cloudy," to "Tsuruoka Hachimangu Shrine," which is "rainy."

[0065] Furthermore, when modifying the shooting order of multiple scenes, the content is to propose changing the shooting order of the original scenes. Specifically, when the weather at the current location is "rainy," processor 23A proposes changing the scene to be shot next after scene 1, which has "rainy" weather, from scene 2, which has "sunny" weather, to scene 3, which has "rainy" weather.

[0066] "Variation 2" In the above embodiment, the shooting environment information was used as an example of shooting status information, but the shooting status information also includes footage of scenes that have already been shot. For example, the scenario information 30 may be modified based on the video data 36 of scene 1, including subsequent scenes. The footage of scenes that have already been shot can be said to be shooting status information that is related to the scenario information 30, in the sense that it is footage that has been shot based on the scenario information 30.

[0067] In the modified example shown in Fig. 10, the original scenario information 30 specifies "rain" as the weather for scenes 1 and 3, similar to the scenario information 30 shown in Fig. 5. In this case, consider a case where scene 1 is shot, but the shot video data 36 for scene 1 does not show any rain. There are various possible reasons why the rain may not be visible, such as when the weather is cloudy when the shot is taken, when there is a sudden change in the weather, such as the rain suddenly stopping, or when the user 11 fails to set the exposure properly.

[0068] In such a case, the processor 23A analyzes the video data 36, ​​which is the video of the already-filmed scene 1, and determines whether rain is present. The processor 23A then compares the video data 36 with the original scenario information 30 to determine that there is an inconsistency between the weather in the scenario information 30 and the weather in the video data 36. In the scenario information 30 of FIG. 10, the weather in scene 3, which follows scene 1, is also "rainy," and the importance of the weather being "rainy" is "low," making it possible to correct the weather. Based on this information, the processor 23A determines that the weather in scene 3 can be corrected, and corrects scene 3 from a "rainy" scene to a "cloudy" scene. In this way, the processor 23A may correct the scenario information 30 based on the video of the already-filmed scene as filming status information.

[0069] Note that Modification Example 2 is an example in which video of a scene that has already been filmed is used as an example of shooting status information, and the scenario information 30 is corrected based on the video of the scene that has already been filmed. Alternatively, video input in association with the scenario information 30 may be used as shooting status information. In other words, the processor 23A may correct the scenario information 30 based on video input in association with the scenario information 30, in addition to or instead of the video of the scene that has already been filmed.

[0070] The images input in relation to the scenario information 30 are, for example, images that are specified in advance in the scenario information 30 as images related to the scenario information 30. More specifically, there is an image of a scene from a movie that is desired to be used as a motif for a certain scene included in the scenario information 30. It may also be the entire movie that was used as reference throughout the entire scenario information 30. These images are used as reference for the revision policy, for example, when the processor 23A executes a scenario revision process, such as revising the scenario information 30 in a direction that brings it closer to these images.

[0071] "Variation 3" Variation 3 shown in FIG. 11 is an example of editing video of a filmed scene when there is an inconsistency between the video of the filmed scene and the scenario information 30, as opposed to variation 2. As in variation 2, the video data 36 of the filmed scene 1 does not show rain, even though the weather in the scenario information 30 is "rain." In this case, the processor 23A performs video editing on the video data 36 of the filmed scene 1 to match the weather in the scenario information and insert "rain." This eliminates the inconsistency between the scenario information 30 and the video of the filmed scene. In this way, the processor 23A may have a function to edit the video of the filmed scene based on the scenario information 30 when there is an inconsistency between the video of the filmed scene and the scenario information. This reduces the effort required to reshoot scenes that have already been filmed.

[0072] As another specific example of Modification 3, as shown in Fig. 12, when a scene is specified in the scenario information 30 to have camerawork that zooms in on the faces of the bride and groom, but the scene is shot without zooming in, the footage of the shot scene can be edited using the editing function. In editing, the areas of the faces of the two people can be extracted from the image, and the extracted areas can be enlarged to add a zoom-in effect.

[0073] Variation 4 Variation 4 shown in FIG. 13 is an example in which the processor 23A has an auxiliary function that assists in generating the scenario information 30. As an example, the auxiliary function is a function that adds recommended conditions regarding the shooting method based on the scenario information 30. In the example shown in FIG. 13, for Scene 2 of the original "Confession," the storyboard only contains an illustration of two people on a boat zoomed out. Based on this original scenario information 30, the processor 23A adds the scenario information 30. In the modified version of the example shown in FIG. 13, an illustration of the faces of the two people on the boat zoomed in is added, and the angle of view is indicated by the illustration. In addition, information instructing the camera work "zoom in" is recorded. The processor 23A interprets the meaning of the text "The groom confesses" recorded in the summary, and adds camera work zooming in on the faces of the two people and a storyboard showing the image and angle of view as a dramatic effect that matches that meaning. The camera work "zoom in" and the angle of view are examples of recommended conditions regarding the shooting method.

[0074] Although zoom-in is used as an example of camera work, other camera works include zoom-out, pan / tilt, fix, etc. The processor 23A may use image recognition technology to interpret the meaning of illustrations shown as storyboards in addition to the meaning of text recorded in the scenario information 30, and may add camera works that can achieve appropriate dramatic effects from among these camera works. The processor 23A may also interpret the meaning of the scenario information 30 to add appropriate shooting conditions such as the angle of view and exposure in addition to the camera work.

[0075] In this way, the processor 23A may have an auxiliary function of assisting in the generation of the scenario information 30 by adding new information based on the scenario information 30. This improves convenience for the user 11. Furthermore, by adding recommended conditions regarding the shooting method, it is possible to present specific ways of shooting, further improving convenience for the user 11.

[0076] In this example, camera work and angle of view are used as examples of recommended conditions for the shooting method, but other shooting conditions such as exposure and shooting equipment may also be added. Regarding exposure, for example, the processor 23A may add a specific exposure value based on weather information, etc. Furthermore, the processor 23A may add specific shooting equipment, such as a 200 mm zoom lens, based on the angle of view of a storyboard, etc.

[0077] The auxiliary function may be, for example, a function to generate and add a storyboard from text information representing the summary. The processor 23A may also determine the importance of the location, weather, and the like from the text information representing the summary and set the determined importance. For example, if the processor 23A determines that the weather is not important based on the text information representing the summary, the processor 23A sets the importance of the weather to "low" and sets the weather as a modifiable element.

[0078] Furthermore, in the above embodiment, a moving image is used as an example of image content, but the image content may be other than a moving image, such as a photo movie. A photo movie is image content that displays multiple still images in chronological order, and includes a slide show.

[0079] As described above, the photography support device 10 may be configured as a server that provides a service of distributing scenario information 30 to multiple client terminals via a communication network. The photography support device 10 and the photography device 20 may be implemented as a single smartphone. The photography device 20 may not be a smartphone, but may be a video camera, which is a dedicated photography device. The video camera may be a dedicated video camera or a digital still camera with a video shooting function.

[0080] The above description can be used to understand the technologies described in the following supplementary items. [Supplementary Item 1] A photography support device that supports image shooting based on scenario information, the photography support device comprising a processor, the processor acquiring shooting status information associated with the scenario information, and correcting the scenario information based on the shooting status information. [Supplementary Item 2] The photography support device described in Supplementary Item 1, in which the shooting status information includes shooting environment information. [Supplementary Item 3] The photography environment information includes information on the weather, time of day, season, shooting location, shooting equipment, performers, and staff. The photography support device described in Supplementary Item 2. [Supplementary Item 4] The photography support device described in any one of Supplementary Items 1 to 3, in which the shooting status information includes video of scenes that have already been shot. [Supplementary Item 5] The photography support device described in any one of Supplementary Items 1 to 4, in which the shooting status information includes video input in association with the scenario information. [Supplementary Item 6] The photographing support device according to any one of Supplementary Items 1 to 5, wherein the processor has a function of editing the video of the photographed scene based on the scenario information when there is an inconsistency between the video of the photographed scene and the scenario information. [Supplementary Item 7] The photographing support device according to any one of Supplementary Items 1 to 6, wherein the scenario information includes information indicating the degree of need for protection for each item. [Supplementary Item 8] The photographing support device according to any one of Supplementary Items 1 to 7, wherein the processor has an auxiliary function of assisting in the generation of scenario information. [Supplementary Item 9] The photographing support device according to Supplementary Item 8, wherein the auxiliary function is a function of adding recommended conditions for the photographing method based on the scenario information. [Supplementary Item 10] The photographing support device according to any one of Supplementary Items 1 to 9, wherein the processor executes control to output the scenario information to a display unit. [Supplementary Item 11] A method for operating a photography support device that supports image photography based on scenario information, the method comprising: acquiring photography status information that is related to the scenario information; and correcting the scenario information based on the photography status information.[Supplementary Item 12] An operating program for a photography support device that supports image photography based on scenario information, the operating program for the photography support device causing a processor to execute processing including: acquiring photography status information that is related to the scenario information; and modifying the scenario information based on the photography status information.

[0081] In the above embodiment, the hardware structure of the processing unit that executes the photography support process can be any of the following various processors: The various processors include a CPU, which is a general-purpose processor that executes software to function as various processing units, as well as a programmable logic device (PLD), which is a processor whose circuit configuration can be changed after manufacture, such as an FPGA (Field Programmable Gate Array), and / or a dedicated electrical circuit, such as an ASIC (Application Specific Integrated Circuit), which is a processor having a circuit configuration designed specifically for executing specific processing.

[0082] A single processing unit may be configured with one of these various processors, or may be configured with a combination of two or more processors of the same or different types (e.g., a combination of multiple FPGAs and / or a combination of a CPU and an FPGA).Furthermore, multiple processing units may be configured with a single processor.

[0083] Examples of configuring multiple processing units with a single processor include, first, a form in which one processor is configured with a combination of one or more CPUs and software, as typified by computers such as client and server, and this processor functions as multiple processing units. Second, a form in which a processor is used to realize the functions of the entire system including multiple processing units with a single IC (Integrated Circuit) chip, as typified by systems on chips (SoCs). In this way, various processing units are configured using one or more of the above-mentioned various processors as a hardware structure.

[0084] Furthermore, more specifically, the hardware structure of these various processors can be an electric circuit (circuitry) that combines circuit elements such as semiconductor elements.

[0085] The technology of the present disclosure can be appropriately combined with the various embodiments and / or various modified examples described above. Furthermore, it is not limited to the above embodiments, and various configurations can be adopted without departing from the spirit of the present disclosure. Furthermore, the technology of the present disclosure extends not only to programs but also to storage media that non-temporarily store programs. Furthermore, the technology of the present disclosure also extends to program products.

[0086] The above-described description and illustrations are a detailed explanation of the parts related to the technology of the present disclosure and are merely an example of the technology of the present disclosure. For example, the above description of the configuration, functions, actions, and effects is an explanation of an example of the configuration, functions, actions, and effects of the parts related to the technology of the present disclosure. Therefore, it goes without saying that unnecessary parts may be deleted, new elements may be added, or replacements may be made to the above-described description and illustrations within the scope of the gist of the technology of the present disclosure. Furthermore, to avoid confusion and facilitate understanding of the parts related to the technology of the present disclosure, the above-described description and illustrations omit explanations of common technical knowledge that do not require particular explanation to enable the implementation of the technology of the present disclosure.

[0087] In this specification, "A and / or B" is synonymous with "at least one of A and B." In other words, "A and / or B" means that it may be only A, only B, or a combination of A and B. Furthermore, in this specification, the same concept as "A and / or B" is also applied when three or more things are expressed by connecting them with "and / or."

[0088] The disclosure of Japanese Patent Application No. 2023-170649, filed on September 29, 2023, is incorporated herein by reference in its entirety. In addition, all documents, patent applications, and technical standards described herein are incorporated herein by reference to the same extent as if each individual document, patent application, and technical standard was specifically and individually indicated to be incorporated by reference.

Claims

1. A photography support device that supports image shooting based on scenario information, the photography support device comprising a processor, the processor acquiring shooting status information that is related to the scenario information, and modifying the scenario information based on the shooting status information.

2. The photography support device according to claim 1, wherein the photography status information includes photography environment information.

3. The photography support device according to claim 2, wherein the photography environment information includes information on weather, time period, season, photography location, photography equipment, performers, and staff.

4. The photography support device according to claim 1, wherein the photography status information includes video of a scene that has already been photographed.

5. The photography support device according to claim 1, wherein the photography status information includes video input in association with the scenario information.

6. The photographing support device of claim 1, wherein the processor has a function of editing the image of the photographed scene based on the scenario information when there is an inconsistency between the image of the photographed scene and the scenario information.

7. The photography support device according to claim 1, wherein the scenario information includes information indicating the degree of need for protection for each item.

8. The photography support device according to claim 1, wherein the processor has an auxiliary function for assisting in the generation of the scenario information.

9. The photography support device according to claim 8, wherein the auxiliary function is a function of adding recommended conditions regarding a photography method based on the scenario information.

10. The photography support device according to claim 1, wherein the processor executes control to output the scenario information to a display unit.

11. A method for operating a photography support device that supports image shooting based on scenario information, comprising: acquiring shooting status information that is related to the scenario information; and modifying the scenario information based on the shooting status information.

12. An operating program of a photography support device that supports image shooting based on scenario information, the operating program causing a processor to execute processes including: acquiring shooting status information that is related to the scenario information; and modifying the scenario information based on the shooting status information.

Images