Reproduction information generating apparatus

Abstract

To provide a reproduction information generation device for generating reproduction information by retrieving a plurality of cuts to be used on the basis of theme information.SOLUTION: The video editing device 50A includes a first information setting unit (retrieval information setting unit 512) that sets attribution information of the plurality of image materials as first information based on the theme inputted by the first inputting unit (communication unit 530), a second information setting unit (retrieval information setting unit 512) that sets analysis information of the plurality of image materials as second information based on the attribution information and the theme, and a retrieving unit 513 that retrieves a plurality of cuts to be used for a video from the plurality of image materials using the first information and the second information set based on the theme information.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to a playback information generating device. [Background technology]

[0002] There is known a technique for generating a new video file from a plurality of uploaded video files, thereby enabling video content of the same type to be viewed consecutively (see Patent Document 1). Conventional technology involves continuously playing existing videos, which is not necessarily appealing to viewers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. WO2019 / 130585 Summary of the Invention

[0004] A playback information generating device according to a first aspect of the present invention is a playback information generating device that generates information necessary for playing a video based on a plurality of image materials, and comprises: a first input unit that inputs information on the theme of the video to be played; a first information setting unit that sets attribute information of the plurality of image materials as first information based on the theme input by the first input unit; a second information setting unit that sets analysis information of the plurality of image materials as second information based on the attribute information and the theme; and a search unit that searches for a plurality of cuts to be used in the video from the plurality of image materials using the first information and the second information that are respectively set based on the theme information. [Brief explanation of the drawings]

[0005] [Figure 1] 1 is a schematic diagram illustrating a video editing system according to an embodiment of the invention; [Figure 2]FIG. 2(a) is a diagram illustrating the configuration of the main parts of a smartphone as an example of a terminal, and FIG. 2(b) is a diagram illustrating the configuration of the main parts of a video editing device. [Figure 3] 10A and 10B are diagrams illustrating an example of extracting cuts and generating a playlist. [Figure 4] 10A and 10B are diagrams illustrating an example of extracting cuts and generating a playlist. [Figure 5] 10A and 10B are diagrams illustrating an example of extracting cuts and generating a playlist. [Figure 6] 10 is a flowchart showing the flow of processing performed on a user's smartphone. [Figure 7] 10 is a flowchart showing the flow of processing performed by the video editing device on the cloud side. [Figure 8] Fig. 8(a) is a diagram showing an example of a notification screen notifying completion of preparation for video playback, and Fig. 8(b) is a diagram showing an example of a video playback screen. [Figure 9] 10 is a flowchart showing the flow of processing performed on a user's smartphone. [Figure 10] 10 is a flowchart showing the flow of processing performed by the video editing device on the cloud side. [Figure 11] Fig. 11(a) is a diagram illustrating an example of an automatic editing setting screen, and Fig. 11(b) is a diagram illustrating an example of a notification screen that notifies the user that preparations for playing a moving image have been completed. [Figure 12] FIG. 10 is a diagram illustrating program provision. DETAILED DESCRIPTION OF THE INVENTION

[0006] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. <System Overview> 1 is a schematic diagram illustrating a video editing system 1 including a video editing device 50A according to an embodiment of the invention, a terminal, etc. The video editing device 50A is configured by, for example, a server called a cloud 50. The number of servers that make up the video editing device 50A is not limited to one as shown in the figure, and may be increased. Video editing device 50A is connected to a terminal or the like used by a user via a wired or wireless network. Fig. 1 illustrates a wireless connection. The terminal or the like is, for example, a smartphone 200, an overhead display 100, a projector 300, a display (not shown), or the like, which allows the user to watch videos.

[0007] Note that the connection between the video editing device 50A and the terminal via a communication network is just an example, and the connection may be configured as a local network that is not connected to the Internet, for example.

[0008] For example, based on a request from a pre-registered user's terminal or the like, the video editing device 50A generates a moving image with audio using image material (called a cut) extracted from image material stored in the video editing device 50A and image material stored in another server (not shown) connected to the video editing device 50A via a communication network or the like. The image material includes moving images and still images. Furthermore, the image material may be owned by multiple people. In other words, there may be multiple image material owned by different owners (copyright holders, etc.), and these image material may be stored in the video editing device 50A or another server or the like. The other server or the like is a general term for a server, computer, data storage device, etc. It is also possible to generate a moving image without audio.

[0009] A user views the moving images generated by video editing device 50A using a terminal, etc. For example, video editing device 50A plays back the moving images and distributes them to smartphone 200, overhead display 100, projector 300, etc. used by the user.

[0010] Streaming playback and download playback are known as techniques for playing video data when a user watches a video on a terminal or the like. In the case of streaming playback, the data of the moving images played back by video editing device 50A is acquired fragmentarily by the user's terminal, etc. in the form of temporary files called caches. The acquired caches are data temporarily stored in the memory of the user's terminal, etc., and are appropriately deleted when playback of the video on the user's terminal, etc. is completed, so that they do not remain as video data on the user's terminal, etc. after playback. On the other hand, in the case of download playback, the user's terminal etc. acquires the moving image data file provided from video editing device 50A and stores it in the memory of the terminal etc. The user's terminal etc. reads the data from the file stored in the memory as needed to display the moving image, so the file as moving image data remains in the memory of the terminal even after playback is completed. This embodiment is compatible with both of the above-mentioned playback methods.

[0011] Regarding cuts extracted from image materials stored in video editing device 50A and other servers, etc., rights such as licenses for generating videos using those cuts, playing and distributing the generated videos, etc., shall be separately negotiated between the copyright holder, etc. and the user. Rights regarding the use of music, etc. used as background music (BGM) during playback shall also be separately negotiated between the copyright holder, etc. and the user.

[0012] FIG. 2(b) is a diagram illustrating an example of the configuration of the main parts of a video editing device 50A, and FIG. 2(a) is a diagram illustrating an example of the configuration of the main parts of a smartphone 200 as an example of a terminal or the like. <Smartphone Overview> The smartphone 200 includes a display / input unit 210, a control unit 220, a data storage unit 230, a communication unit 240, a speaker 250, and a photographing unit 260. The control unit 220 includes a schedule information management unit 221, a relationship information management unit 222, and a metadata assignment unit 223.

[0013] The display / input unit 210 is configured by a touch panel display that combines a display function for displaying images, icons, etc., with an operation unit function for accepting touch operations by the user on the display surface. The display / input unit 210 displays icons associated with applications (application programs) or data, the user's calendar information (schedule information), the user's friendship information, and images (still images and videos), and also accepts operations from the user on the smartphone 200.

[0014] The control unit 220 is configured with a CPU, a ROM, a RAM, etc., and controls the operation of the smartphone 200 based on a control program. The control unit 220 executes an application associated with a touched icon, etc. Furthermore, if a moving image content is associated with a touched icon, etc., the control unit 220 causes the smartphone 200 to function as a moving image playback device by, for example, starting an application for playing the content.

[0015] The data storage unit 230 is configured with a non-volatile memory or the like. The data storage unit 230 stores data such as the user's schedule information, the user's friendship information, image material captured by the image capture unit 260, and contact information such as a phone book. The recorded data is updated as necessary. The data storage unit 230 can also store files of moving image data provided by the video editing device 50A during the download playback. The control unit 220 controls the recording (saving) and reading of data in the data storage unit 230.

[0016] The communication unit 240 performs voice communication that causes the smartphone 200 to function as a mobile phone, and data communication for transmitting and inputting information between the smartphone 200 and other servers, etc. Furthermore, when a short-range wireless communication device such as an external speaker or headphones is used, the communication unit 240 also performs wireless communication with this device.

[0017] The speaker 250 plays audio based on audio data associated with a moving image, for example. Note that audio based on the audio data can also be played from an external speaker or headphones (not shown) equipped with a wireless communication function, instead of being played from the speaker 250. When an external speaker or headphones are used, the audio data is wirelessly transmitted to the external speaker or headphones via the communication unit 240, for example.

[0018] The photographing unit 260 has a photographing lens and an image sensor, and captures one or more images. The photographing unit 260 can capture both moving images and still images. The image material captured by the photographing unit 260 is stored in the data storage unit 230. The image material stored in the data storage unit 230 can be uploaded to the video editing device 50A via the communication unit 240 and saved in the storage unit 520 of the video editing device 50A.

[0019] <Video editing equipment overview> Video editing device 50A includes a control unit 510, a storage unit 520, and a communication unit 530. Control unit 510 includes a template selection unit 511, a search information setting unit 512, a search unit 513, an extraction unit 514, a framing determination unit 515, a playlist generation unit 516, and a metadata attachment unit 517.

[0020] The control unit 510 is configured with a CPU, a ROM, a RAM, etc., and controls the operation of each unit of the video editing device 50A based on a control program. The storage unit 520 stores, for example, user registration information, image materials uploaded by the user or a third party, templates (described later), video data generated by the control unit 510, and playlists (described later). The communication unit 530 performs data communication for transmitting and inputting information between the user's terminal and other servers.

[0021] Hereinafter, as an example, a detailed description will be given of an operation process in which video editing device 50A edits a video image in response to a request from a user terminal. The video generated in response to a request from a user terminal or the like may be a digest video edited using existing video and still image materials so that it plays back in a shorter time than the original image materials, or a video edited so that it plays back in a time equal to or longer than the original image materials. In the embodiment, an example of generating a digest video will be described. A user requests video editing device 50A to generate a video by communicating the theme of the video to be generated from a terminal, etc. The theme may be the name of an event, such as "wedding," "XX trip," "XX party," or "funeral."

[0022] <Template> The storage unit 520 stores a plurality of general-purpose templates showing outline scenarios for videos for each anticipated theme. When the communication unit 530 receives a theme and a video generation request transmitted from a user's terminal or the like, the template selection unit 511 selects a template associated with the received theme from the plurality of general-purpose templates. The template has a preset total playback time for playing and displaying the video. The total playback time is the playback time from the first frame to the last frame of the video. The total playback time can be increased or decreased by a change request transmitted from the user's terminal or the like. This allows the playback time of the resulting video to be lengthened or shortened.

[0023] The templates will be explained further. The theme of the video is "wedding ceremony," and the template corresponding to the video to be screened at the wedding reception hall on the day of the wedding displays a scenario introducing the backgrounds of the bride and groom to the guests of the wedding reception. Also, the theme of the video is "wedding ceremony," and the template corresponding to the video to be distributed to the guests of the wedding reception at a later date displays a scenario looking back at scenes of the venue, the bride and groom, and the guests, etc., filmed on the day of the wedding ceremony.

[0024] Furthermore, the template corresponding to the video with the theme "XX Trip" shows a scenario including the purpose of the trip, an introduction to the places visited, and scenes showing the companions and scenery photographed during the trip. The template for the video with the theme "△△ Meeting" shows a scenario including the purpose of the meeting, an introduction to the venue, and scenes of participants and scenery photographed during the meeting. The template for videos with the theme "funeral" shows a scenario for remembering the deceased by introducing episodes from the deceased's life and scenes showing the deceased's favorite scenery.

[0025] The template is a general-purpose template that assumes various events in an average person's life, and is adjusted by control unit 510 to suit the lifestyle of the user who requested the video generation. For example, control unit 510 adjusts the general-purpose template to suit the user's event schedule based on the degree of matching between the general-purpose template and the user's event schedule recorded as calendar information in smartphone 200 or the like of the user who requested the video generation.

[0026] The control unit 510 adopts the generic template as is when the ratio of specific events (called the event ratio) among the events listed in the event schedule of the user who requested the video generation matches or is similar to the ratio of events appearing in the generic template. Furthermore, when the ratio of specific events among the events in the event schedule of the user who requested the video generation is lower than the ratio of events appearing in the generic template, the control unit 510 adjusts the generic template by removing events in categories in which the user does not participate, based on information about the user's past event schedule. For example, if the user has never participated in a golf tournament, golf-related scenes are excluded from the template. Conversely, if the event ratio of the event schedule of the user who requested video generation is higher than the event ratio appearing in the general-purpose template, the control unit 510 will use information on the user's past event schedule to create an adjusted template that has a higher event ratio of categories that the user frequently participates in than the general-purpose template. For example, if the user frequently participates in golf tournaments, the template will include many golf-related scenes. It should be noted that various events are added to and deleted from a user's event schedule as the user grows and ages, and therefore the event schedule can be considered to reflect the user's lifestyle.

[0027] The control unit 510 determines rules for adjusting the template, such as deleting specific events from the general-purpose template (reducing the number of events listed in the template) or increasing the event ratio of specific events among the events listed in the general-purpose template, for each user based on the degree of match or similarity between the event schedule of the user who requested video generation and the general-purpose template, and will review the above rules as appropriate if the degree of matching changes as the user who requested video generation grows, etc.

[0028] <Image material> The search unit 513 searches for image materials to be used in generating a moving image. In this embodiment, the search unit 513 searches for image materials within a search range (described later) based on the template adjusted as described above to suit the lifestyle of the user who requested the generation of the moving image. The image materials to be searched are image materials related to the events described in the template. In this embodiment, if there are few events described in the template, the number of search targets will be fewer, and if there are many events described in the template, the number of search targets will increase. Therefore, adjusting the template to suit the lifestyle of the user who requested the generation of the moving image also leads to the selection of image materials to be searched for generating a moving image.

[0029] <Search range> The search information setting unit 512 determines the search range for searching for image material based on the theme of the moving image communicated by the user who requested the moving image creation and the friendships of the user who requested the moving image creation. The user's friendship information includes, for example, data recorded as contact information in the smartphone 200 or the like of the user who requested the moving image creation, call records, email records, and SNS (Social Networking Service) usage records of the smartphone 200 or the like. The search information setting unit 512 uses this friendship information to identify the user's family, friends, and acquaintances.

[0030] For example, videos created with the theme "wedding" are often viewed by wedding reception attendees, primarily relatives and friends, including the family of the user who requested the video creation. Generally, when viewing videos, viewers are more interested in videos featuring familiar people than in videos featuring people they do not know. Therefore, the search information setting unit 512 sets the search range for the theme "wedding" to image material captured by relatives and friends of the main characters of the wedding (the bride and groom) and image material featuring such relatives and friends, among the image material stored in the storage unit 520 and other servers (not shown) connected to the video editing device 50A via a network (hereinafter referred to as the storage unit 520, etc.). Determining the search range based on the theme and friendships of the video makes it easier to collect image material that will attract and pique the viewer's interest.

[0031] Furthermore, for example, videos generated for a theme such as "XXX Trip" are often viewed primarily by the user who requested the video generation and their travel companions. Therefore, the search information setting unit 512 sets the search range for the theme "XXX Trip" to image materials stored in the storage unit 520, etc., taken by the main traveler (representative, group leader, etc.) and their travel companions, particularly image materials that include travel companions whose frequency of interaction with the main traveler is higher than a predetermined standard. Whether the frequency of interaction is higher than the standard is determined based on, for example, the number of call records, the number of email records, and SNS usage records. In this way, determining the search range based on the theme and friendships of the video makes it easier to collect image materials that will attract and heighten the viewer's interest.

[0032] Furthermore, for example, videos generated for a theme of "△△ Party" are often viewed primarily by the user who requested the video generation and the participants of the party. Therefore, the search information setting unit 512 sets the search range for the theme of "△△ Party" to image materials stored in the storage unit 520, etc., taken by the main character (organizer) of the party and the participants, and in particular, image materials that include participants who have interacted with the main character more frequently than a predetermined standard. The frequency of interaction is determined in the same way as in the case of the theme of "XX Trip" described above. In this way, by defining the search range based on the theme of the video and the friendships, it becomes easier to collect image materials that will attract and increase the viewer's interest.

[0033] <Relationship information> The search information setting unit 512 identifies, based on friendship information recorded on the smartphone 200 or the like of the user who requested video generation, people who have a friendship with the user and who have the same event (such as a "wedding," "XX trip," or "△△ party") recorded in the event schedule of their own smartphone or the like, as event participants. If the event is a "wedding," the event participants are equivalent to the attendees of the wedding. If the event is a "XX trip," the event participants are equivalent to the people accompanying the XX trip. If the event is a "△△ party," the event participants are equivalent to the attendees of the △△ party.

[0034] <metadata> The search unit 513 searches for image material that includes people identified by the search information setting unit 512, such as family members, friends, and acquaintances, based on the metadata, within the search range set by the search information setting unit 512. The metadata is image information that is assigned to each image material, and is recorded in the data file of the image material or in an attached file. Since metadata is well known, a detailed description of the metadata itself will be omitted. However, the metadata can include information such as who (or what person) is depicted in the image material, when and where what event was filmed, and by whom. In the embodiment, when the image material is a moving image, information is recorded in chronological order corresponding to the frames of the moving image. Therefore, it is possible to know who (or what person) is depicted in each frame by analyzing the metadata instead of playing and analyzing the image material. By using such metadata, the search unit 513 can more efficiently search for image material taken by the family, friends, or acquaintances of the user who requested video generation, or for image material depicting the user's family, friends, or acquaintances, compared to when metadata is not used.

[0035] Examples of information that may be included in the metadata include shooting information, video analysis information, and information related to the photographer. Shooting information is information about the time of shooting of the image material, and includes information such as the shooting date and time, aperture, shutter speed, ISO (International Organization for Standardization) sensitivity, information about WB (white balance) adjustment, information about VR (Vibration reduction) correction, gimbal information (information about shake reduction processing), and information about zoom lens correction, recorded contents in EXIF ​​(Exchangeable image file format). The shooting information includes location information from a GPS (Global Positioning System) and information at the time of shooting, such as environmental information (for example, temperature). The above-mentioned photographing information may be called attribute information of the image material. The video analysis information includes subject detection information, face detection information, object recognition information, individual subject recognition information, movement of the main subject, distance to the main subject, size of the main subject, featured subject, highlight scenes of the video, framing area of ​​the video, subject interest, etc. Therefore, information on the subjects and the like appearing in the image material can be found simply by reading the metadata without reading the image data. The video analysis information may be called image material analysis information (or image analysis information). The photographer-related information includes basic information for recognizing the photographer etc. from the image material, copyright information, information on the availability range of the image material, etc. The photographer-related information may be included in the attribute information of the image material. In the embodiment, when an image material is photographed by smartphone 200, metadata is assigned to the image material by metadata assignment unit 223. It is not necessary to assign all of the shooting information, video analysis information, and photographer-related information as metadata at the time of shooting; for example, video analysis information may be added by analyzing the photographed video later, such as during charging or while the phone function is on standby. If metadata is not attached to the image material uploaded to the video editing device 50A, or if some of the metadata information is missing, the metadata attachment unit 517 analyzes the image and can attach or add information to the image material as metadata, such as what (who) is depicted in the image material, when and where what event was filmed, and by whom.

[0036] <Cut> The extraction unit 514 extracts cuts in accordance with the template adjusted as described above. For example, the extraction unit 514 selects image material containing cuts that match the template from the image material searched by the search unit 513 as candidates for adoption, and extracts cuts from the image material of the candidates for adoption. When extracting cuts from the image material of the candidates for adoption, the extraction unit 514 uses information indicating the user's friendships. If there is ample time for the total playback of the video to be generated, it is possible to extract a wide range of cuts from the image materials of the candidates for use, but if there is not enough time for the total playback of the video to be generated, it is difficult to extract cuts from all of the image materials of the candidates for use. Therefore, the extraction unit 514 adjusts the extraction rate of cuts from the image materials using information on the frequency of interaction with the user who requested the video generation. For example, in the case of image material that includes friends or acquaintances, the adoption rate of image material that includes people who have a higher frequency of interaction with the user who requested the video creation than a predetermined standard can be increased, and the extraction rate of cuts from the adopted image material can be increased. Conversely, individuals who interact less frequently with the user who requested the creation of the video than the above standard may have a lower adoption rate of image material and a lower extraction rate of cuts from the adopted image material than individuals who interact more frequently. Extracting cuts based on the friendships of the user who requested the video generation will increase interest and concern in the video among people who frequently interact with the user.

[0037] <Framing, etc.> The framing determination unit 515 determines the display order, display speed (or display time), and framing during playback and display of the cuts extracted by the extraction unit 514. Framing is, for example, an electronic zoom setting, and indicates which area of ​​the extracted cut is to be displayed and at what zoom magnification. The framing determination unit 515 uses information indicating the user's social relationships when determining the display order, display speed (or display time per frame), and framing during playback and display of the cuts. For example, in the case of image material that includes friends or acquaintances, cuts that include people who have interacted with the user who requested the video creation more frequently than a predetermined standard may be displayed preferentially, or may be displayed for a longer period of time than other people, or framing may be performed to electronically zoom in and display the person in a larger proportion of the screen. Conversely, for a person whose frequency of interaction with the user who requested the video generation is lower than the above standard, the display priority of the extracted cuts may be lowered compared to a person with a higher frequency of interaction, the display time may be shorter than for other people, and the framing process of electronically zooming in on the faces of friends and acquaintances may be omitted.

[0038] <Music material> The search unit 513 further searches for music and sound effects to be used as background music, which are described in the template adjusted as described above. The template describes music and other information predetermined for each event. If the template contains fewer events, the number of music pieces will be fewer, and if the template contains more events, the number of music pieces will be greater. It is also possible to search for music specified by the user, regardless of the music specified in the template. The search range for music, sound effects, etc. may exceed the search range for image materials. In other words, music, etc. may be searched for broadly, without being limited to the search range for image materials.

[0039] <Playlist> Video editing device 50A typically generates a moving image by editing multiple cuts extracted from image material. In an embodiment, instead of or in addition to generating a moving image, a playlist for playing back the moving image can be created. A playlist is a list describing the necessary information for playing back and displaying a moving image, i.e., the display order, display speed, framing, background music, and location of the data for the cuts and background music extracted by extraction unit 514 and determined by framing determination unit 515. If a playlist exists, the moving image can be played back based on the playlist. In other words, the moving image can be played back and displayed even if the generated moving image data itself does not exist. The location of the data is, for example, a uniform resource locator (URL) where the data is stored. In the embodiment, the playlist generation unit 516 generates a playlist for sequentially playing back the cuts and background music extracted by the extraction unit 514 and subjected to framing processing, etc., by the framing determination unit 515, in accordance with the selected template.

[0040] (1) Examples of the Theme "Wedding" With reference to Figures 3 to 5, which explain examples of cut extraction and playlist generation, examples of playlist generation ((Example 1-1) to (Example 1-5)) when the theme of a video is "wedding" will be described. In (Example 1-1), a playlist of videos to be screened at User X's wedding reception is generated. In (Example 1-2), a playlist is generated by adding image material to the video in (Example 1). In (Example 1-3), a playlist of videos to be screened at the wedding reception of User X's child S is generated. In (Example 1-4), a playlist is generated by adding image material to the video in (Example 1-3). In (Example 1-5), a playlist of videos to be screened at User X's funeral is generated.

[0041] <Playlist Generation Example> (Example 1-1) Videos to be Screened at User X's Wedding Reception An example of the procedure for generating a playlist will be described with reference to FIG. 3. In Example 1-1, in response to a request from User X, the playlist generation unit 516 generates a playlist 660A of videos to be screened at User X's wedding reception. User X is the groom and is marrying Bride Y. The videos to be screened at the reception introduce User X's background, etc. to the guests of the reception, and also introduce Bride Y's background, etc. to the guests of the reception. These videos may also include how the couple met, etc. Below, the procedure for creating a playlist of videos introducing User X's background, etc. will be described as an example. The procedure for creating a playlist of videos introducing Bride Y's background, etc. will not be described as it is the same as for User X.

[0042] 3, two horizontal lines 21 and 22 represent the time axis. Line 21 is the time axis for user X's parent P and covers the time from user X's birth to user X's wedding. After user X's wedding, it continues to line 21 in FIG. 4. 3 is the time axis for user X, and includes the time from the birth of user X to the marriage of user X. After the marriage of user X, it continues to the line 22 in FIG.

[0043] Image material 601 is image material taken by parent P of user X at the time of user X's birth, entrance ceremony, childhood, coming-of-age ceremony, etc. Image material 602 is image material taken by parent P of user X at user X's wedding. For ease of explanation, image materials 601 and 602 are arranged along line 21 according to the years when they were taken.

[0044] Image material 603 is image material taken by user X himself during his adolescence, coming-of-age ceremony, and thereafter. Image material 604 is image material taken by user X on the day of his wedding and during the preparation period. The image material taken by user X himself from the time of his birth until he entered school is not included in image material 603 because X was not able to take any photographs of his childhood. For ease of explanation, image materials 603 and 604 are arranged along line 22 according to the years in which they were taken.

[0045] <Decide the search range> Upon receiving a request from user X to create a moving image introducing the upbringing of user X on the theme of "wedding," video editing device 50A identifies user X's parents, who are family members, based on information recorded on user X's smartphone 200 or the like. Search information setting unit 512 sets image material 601 uploaded by parent P of user X as the search range among image material stored in video editing device 50A or the like. Search information setting unit 512 further sets image material 603 uploaded by user X among image material stored in video editing device 50A, etc. as the search range. That is, in this embodiment, image material 601 and 603 are the search range. Furthermore, the video editing device 50A may identify relatives of the user X based on information recorded in the smartphone 200 of the user X. The search information setting unit 512 may include, in the search range, image material uploaded by the identified relatives among the image material stored in the video editing device 50A.

[0046] The search unit 513 searches for image material as follows, using information indicating the friendships of the user X, which is acquired based on information recorded in the smartphone 200 or the like of the user X who has requested the generation of the moving image.

[0047] <Find stock images related to the event> The template associated with the theme "wedding" includes a scene in the scenario that introduces the background of the bride and groom. The search unit 513 analyzes the images of the image materials 601 and 603 that are within the search range, and searches for video and still image materials related to the event described in the template based on the details described in the template. The image analysis is performed by referring to the metadata attached to the image material. If no metadata is attached to the image material or if some information in the metadata is missing, the search unit 513 causes the metadata attachment unit 517 to analyze the image, and searches for video and still image materials related to the event described in the template based on the analysis results. The results of the analysis by the metadata adding unit 517 are recorded (added or added) as metadata to be added to each image material.

[0048] <Search for image materials that feature the user> The search unit 513 further searches for image material in which the user X appears, while referring to the metadata of the image material 601 and 603 within the search range. The extraction unit 514 selects image materials related to events with high importance as candidates for adoption from the image materials searched by the search unit 513 based on the importance of the events entered in the event schedule of the user X's smartphone 200, etc., and extracts cuts from the image materials of the candidates for adoption.

[0049] 3, the image materials selected by extraction unit 514 from image materials 601 and 603 are illustrated as group 650. Group 650 is a group of image materials for ease of explanation, and does not need to be stored together in the same area of ​​memory or the like; it is sufficient if the image materials to be included in group 650 are associated with each other.

[0050] <Extract a cut> The extraction unit 514 extracts cuts that match the items described in the template from the image materials of the group 650. For example, if the template contains cuts from User X's kindergarten days, elementary school days, and other cuts, these cuts are extracted. The extraction unit 514 includes cuts that show people who are friends with User X in the extracted cuts.

[0051] The extraction unit 514 preferentially extracts from the image materials of the group 650 shots that show people with high interaction frequencies, based on the frequency of interaction determined from the number of call records, the number of email records, and SNS usage records. Furthermore, the cuts are extracted so that the total time of the extracted cuts does not exceed the total playback time defined in the template, for example.

[0052] <Search for songs etc.> Furthermore, the search unit 513 searches the video editing device 50A and the like for data such as music to be written in the template.

[0053] <Generate a playlist> The framing determination unit 515 determines the display order, display speed, and framing during playback display of cuts extracted from the image materials included in the group 650 based on the items described in the template so that they are displayed in the display order, display speed, and framing described in the template, and also determines the playback order, playback speed, and playback volume of the music and sound effects used as background music based on the items described in the template. Furthermore, when determining the display order, display speed, etc. of the above cuts, the framing determination unit 515 may prioritize cuts that show people with a high frequency of interaction over cuts that show people with a low frequency of interaction.

[0054] The playlist generation unit 516 generates a playlist 650A that describes the display order, display speed, framing, playback order of background music, playback speed, playback volume, etc. of the determined cuts. The playlist 650A is associated with the image material that makes up the group 650 and temporarily saved in a predetermined area of ​​the video editing device 50A. The group 650 can be considered as a collection of the image material that makes up the group 650 and the playlist 650A.

[0055] <Present to user> Video editing device 50A transmits playlist 650A to, for example, user X's smartphone 200 or the like, and obtains user X's confirmation. User X watches the videos played for preview on smartphone 200 or the like based on playlist 650A, and if user X desires to make changes (modifications), transmits specific modifications to video editing device 50A. For example, user X may transmit modifications to video editing device 50A, such as reducing the number of childhood shots and increasing the number of boyhood shots, or replacing boyhood shots with shots based on different image material. Furthermore, if the user X judges that no changes are necessary after viewing the video images reproduced for preview based on the playlist 650A, the user X transmits a signal indicating this to the video editing device 50A.

[0056] When a correction instruction is received from user X, playlist generation unit 516 corrects playlist 650A based on the request from user X. Video editing device 50A transmits corrected playlist 660A to user X's smartphone 200 or the like, and obtains user X's confirmation again. Video editing device 50A receives a signal from user X indicating that corrected playlist 660A has been confirmed, and sets confirmed playlist 660A as playlist 660A to be played on the day of the wedding ceremony. When video editing device 50A receives a signal from user X indicating that playlist 650A has been confirmed without any modifications, video editing device 50A uses confirmed playlist 650A as playlist 660A to be played on the day of the wedding ceremony.

[0057] Playlist 660A is stored in a predetermined area of ​​storage unit 520 in association with image materials of group 660 that are ultimately adopted from among the image materials included in group 650. Group 660 can be considered as a collection of image materials and playlist 660A that make up group 660.

[0058] Information indicating the save location of playlist 660A is transmitted to user X's smartphone 200 or the like. User X inputs the save location of playlist 660A into the playback device that will play the videos on the day of the wedding. As a result, the playback device plays back the videos based on playlist 660A on the day of the wedding. When distributing playlist 660A to a playback device that will play the playlist on the day of the wedding, video editing device 50A can send only playlist 660A, or it can send playlist 660A and the associated image material 660 (in other words, the image material that was ultimately adopted from the image material included in group 650).

[0059] (Example 1-2) Adding Image Material to Video Shown at the Wedding Reception Next, an example of the procedure for adding video material to a playlist will be described. In Example 1-2, playlist generation unit 516 adds image material shot on the day of the wedding to video playlist 660A generated in Example 1-1. Playlist 680A for later distribution (main character) and playlist 690A for later distribution (attendees) to which the image material has been added are distributed to attendees of the wedding reception after the wedding, for example. The playlist generation unit 516 generates a playlist 680A for later distribution to the main characters, the bride and groom, and a playlist 690A for later distribution to attendees such as relatives and friends. The video editing device 50A again uses the information indicating the friendship relationships of user X used in Example 1-1.

[0060] <Decide the search range> When search information setting unit 512 of video editing device 50A receives a request from user X to create a moving image to be distributed to attendees at the wedding reception after the wedding, search information setting unit 512 searches image materials 602 and 604, and image material 605, among image materials stored in video editing device 50A, etc. As described above, image material 602 is an image material taken and stored by user X's parent P at user X's wedding. Image material 604 is an image material taken and stored by user X on the day of user X's wedding and during the preparation period. Furthermore, image material 605 is an image material taken and stored by attendees at the wedding reception.

[0061] <Search for image materials that include users and attendees> The search unit 513 searches for video footage and still image footage in which the user X and the bride Y appear, while referring to the metadata of the image materials 602, 604, and 605 in the search range. The search unit 513 also searches for video footage and still image footage in which the attendees appear, while referring to the metadata of the image materials 602, 604, and 605 in the search range.

[0062] The extraction unit 514 selects image materials that feature people who frequently interact with user X or bride Y as candidates for use from the image materials searched by the search unit 513 based on the friendship relationship information recorded on user X's smartphone 200, etc., and extracts cuts from the image materials of the candidates for use.

[0063] In FIG. 3, image materials that have already been grouped as group 650 by video editing device 50A and image materials that have been selected by extraction unit 514 from image materials 602, 604, and 605 are shown together as group 670. Of group 670, image materials that were included in group 650 do not necessarily need to be included in group 670. Also, group 670 is a collection of image materials for the sake of clarity, and does not need to be collected and stored in the same area of ​​memory or the like; it is sufficient if the image materials to be included in group 670 are associated with each other.

[0064] <Extract a cut> The video editing device 50A extracts cuts that match the details described in the template from the image materials of the group 670. For example, if the template includes cuts showing scenes on the day of the wedding that the participants were not watching (the participants were not present) and cuts showing the excitement of the wedding reception, these cuts are extracted. The cuts extracted by the extraction unit 514 are mainly cuts showing people who have a friendship with the user X.

[0065] The extraction unit 514 preferentially extracts from the image materials of the group 670 shots that show people with a high frequency of interaction, based on the frequency of interaction determined from the number of call records, the number of email records, and SNS usage records. Furthermore, the cuts are extracted so that the total time of the extracted cuts does not exceed the total playback time defined in the template, for example.

[0066] <Search for songs etc.> Furthermore, the search unit 513 searches the video editing device 50A and the like for data such as music to be written in the template.

[0067] <Add to playlist> The framing determination unit 515 determines the display order, display speed, and framing during playback of the cuts extracted from the image materials of group 670 based on the items described in the template, and also determines the playback order, playback speed, and playback volume of the music and sound effects used as background music based on the items described in the template. Furthermore, when determining the display order, display speed, etc. of the above cuts, the framing determination unit 515 prioritizes cuts that show people with a high frequency of interaction over cuts that show people with a low frequency of interaction.

[0068] The playlist generation unit 516 generates a playlist 670A that describes the display order of the determined cuts, the display speed, the playback order of framing and background music, the playback speed, and the volume during playback. The playlist 670A is associated with the image material that makes up the group 670 and temporarily saved in a predetermined area of ​​the video editing device 50A. The group 670 can be considered as a collection of the image material that makes up the group 670 and the playlist 670A.

[0069] <Present to user> Video editing device 50A transmits playlist 670A to, for example, user X's smartphone 200 or the like, and obtains user X's confirmation. User X watches videos that are played for preview on smartphone 200 or the like based on playlist 670A, and if user X desires to make changes (modifications), transmits specific modifications to video editing device 50A. For example, user X may transmit modifications to video editing device 50A, such as reducing the number of shots of the main character and increasing the number of shots of the participants, or replacing the shots of the participants with shots based on different image material. Furthermore, if user X judges that no changes are necessary after viewing the video images reproduced for preview based on playlist 670A, user X transmits a signal indicating this to video editing device 50A.

[0070] When a correction instruction is received from user X, playlist generation unit 516 corrects playlist 670A based on the request from user X. Video editing device 50A transmits corrected playlist 670A to user X's smartphone 200 or the like, and obtains user X's confirmation again. Upon receiving a signal from user X indicating that corrected playlist 670A has been confirmed, video editing device 50A sets confirmed playlist 670A as playlist 680A or 690A for later distribution. When video editing device 50A receives a signal from user X indicating that playlist 670A has been confirmed without any modifications, it sets confirmed playlist 670A as playlist 680A or 690A for later distribution.

[0071] Playlists 680A and 690A are stored in a predetermined area of ​​storage unit 520 in association with the image material of group 680 or image material of group 690 that was ultimately adopted from among the image materials included in group 670, respectively.

[0072] Information indicating the save location of playlist 680A is sent to the smartphones of the bride and groom, and information indicating the save location of playlist 690A is sent to the smartphones of attendees such as relatives and friends. The bride and groom or attendees input the save location of playlist 680A or 690A into a playback device that will play the playlist after the wedding ceremony. When distributing playlist 680A to a playback device for playback after the wedding, video editing device 50A can send only playlist 680A, or it can send image material 680 associated with playlist 680A (in other words, the image material ultimately adopted from among the image materials included in group 670). Similarly, when distributing playlist 690A to a playback device for playback after the wedding, video editing device 50A can send only playlist 690A, or it can send playlist 690A and the associated image material 690 (in other words, the image material that was selected from the image material included in group 670).

[0073] (Example 1-3) Videos to be Screened at the Wedding Reception of User X's Child S With reference to FIG. 4, an example of the procedure for generating a playlist will be described. In Example 1-3, in response to a request from User X's child S, the playlist generation unit 516 generates a playlist 710A of videos to be screened at the wedding reception of User X's child S. User X's child S is assumed to be the groom and to be marrying Bride Z. The videos to be screened at the reception are videos introducing the background of User X's child S to the guests of the reception, and videos introducing the background of Bride Z to the guests of the reception. Each of the two videos also includes information about how the two met. Below, the procedure for creating a playlist of videos introducing the background of User X's child S will be described as an example. The procedure for creating a playlist of videos introducing the background of Bride Z will not be described here because it is the same as for User X's child S.

[0074] In Fig. 4, two horizontal lines 21, 22, and 23 represent the time axis. Line 21, which continues from Fig. 3, is the time axis for parent P of user X and covers the period from the birth of grandchild (child S of user X) to the death of parent P of user X. The line 22 continuing from FIG. 3 is the time axis for user X and includes the time from the birth of user X's child S to the marriage of user X's child S. After the marriage of user X's child S, it continues to line 22 in FIG. The line 23 is the time axis for the child S of the user X, and includes the time from the birth of the child S of the user X to the marriage of the child S of the user X.

[0075] Image material 611 is image material taken by parent P of user X at the time of birth, kindergarten entrance ceremony, childhood, etc. of child S of user X. Image material 612 is image material taken by user X at the time of birth, kindergarten entrance ceremony, childhood, adolescence, etc. of child S of user X. Image material 613 is image material taken by user X at the wedding of child S of user X. For ease of explanation, the image materials 611, 612 and 613 are arranged along line 21 or 22 according to the years in which they were taken.

[0076] Image material 614 is image material taken by user X's child S himself during his adolescence, coming-of-age ceremony, and thereafter. Image material 615 is image material taken by user X's child S on the day of his wedding and during the preparation period. The image material taken by user X's child S himself from the time of his birth until he started school is not included in image material 614 because X's child S himself was not able to take any photographs of his childhood. For ease of explanation, image materials 614 and 615 are arranged along line 23 according to the years in which they were taken.

[0077] <Decide the search range> The video editing device 50A receives a request from the child S of the user X to create a video introducing the upbringing of the child S of the user X on the theme of "wedding." The video editing device 50A identifies the parent (user X) of the child S of the user X and the parent P of the user X based on information recorded on the smartphone 200 of the child S of the user X. The search information setting unit 512 sets the search range to image material 611 uploaded by the parent P of the user X among the image material stored in the video editing device 50A. The search information setting unit 512 sets the search range to image material 612 uploaded by the user X among the image material stored in the video editing device 50A. The search information setting unit 512 further sets the search range to image material 614 uploaded by the child S of the user X among the image material stored in the video editing device 50A or the like. That is, in this embodiment, the image materials 611, 612 and 614 are the search range. The video editing device 50A may identify relatives such as a relative of the child S of the user X based on information recorded in the smartphone 200 of the user X. The search information setting unit 512 may include, in the search range, image material uploaded by the identified relatives among the image material stored in the video editing device 50A.

[0078] The search unit 513 searches for image material as follows, using information indicating the friendships of the child S of the user X, which is acquired based on information recorded in the smartphone 200 of the user X who requested the generation of the video.

[0079] <Find stock images related to the event> As described above, the template associated with the theme "wedding" includes a scene in the outline scenario that introduces the background of the bride and groom. The search unit 513 analyzes the images of the image materials 611, 612, and 614 within the search range, and searches for video and still image materials related to the event described in the template based on the details described in the template. The image analysis is performed by referring to the metadata attached to the image material. If no metadata is attached to the image material or if some information in the metadata is missing, the search unit 513 causes the metadata attachment unit 517 to analyze the image, and searches for video and still image materials related to the event described in the template based on the analysis results. The results of the analysis by the metadata adding unit 517 are recorded (added or added) as metadata to be added to each image material.

[0080] <Search for stock images of User X's child S> The search unit 513 further searches for image material in which the child S of the user X appears, while referring to the metadata of the image material 611, 612, and 614 within the search range. The extraction unit 514 selects image materials related to events with high importance as candidates for adoption from the image materials searched by the search unit 513 based on the importance of the events entered in the event schedule of the smartphone 200 or the like of the child S of the user X, and extracts cuts from the image materials of the candidates for adoption.

[0081] 4, the image materials selected by extraction unit 514 from image materials 611, 612, and 614 are illustrated as a group 700. Group 700 is a group of image materials for ease of explanation, and does not need to be stored together in the same area of ​​memory or the like; it is sufficient if the image materials to be included in group 700 are associated with each other.

[0082] <Extract a cut> The extraction unit 514 extracts cuts that match the items described in the template from the image materials of the group 700. For example, if the template contains cuts of user X's child S from kindergarten, elementary school, and other cuts, these cuts are extracted. The extraction unit 514 includes cuts that show people who are friends with user X's child S in the extracted cuts.

[0083] The extraction unit 514 preferentially extracts from the image materials of the group 700 cuts that show people with high interaction frequencies, based on the frequency of interaction determined from the number of call records, the number of email records, and SNS usage records. Furthermore, the cuts are extracted so that the total time of the extracted cuts does not exceed the total playback time defined in the template, for example.

[0084] <Search for songs etc.> Furthermore, the search unit 513 searches the video editing device 50A and the like for data such as music to be written in the template.

[0085] <Generate a playlist> The framing determination unit 515 determines the display order, display speed, and framing during playback of cuts extracted from the image materials included in the group 700 based on the items described in the template, and also determines the playback order, playback speed, and playback volume of the music and sound effects used as background music based on the items described in the template. Furthermore, when determining the display order, display speed, etc. of the above cuts, the framing determination unit 515 prioritizes cuts that show people with a high frequency of interaction over cuts that show people with a low frequency of interaction.

[0086] The playlist generation unit 516 generates a playlist 700A that describes the display order of the determined cuts, the display speed, the playback order of framing and background music, the playback speed, and the volume during playback. The playlist 700A is associated with the image material that makes up the group 700 and temporarily saved in a predetermined area of ​​the video editing device 50A. The group 700 can be considered as a collection of the image material that makes up the group 700 and the playlist 700A.

[0087] <Present to child S of user X> The video editing device 50A transmits the playlist 700A to, for example, the smartphone 200 of the child S of the user X, and obtains confirmation from the child S of the user X. The child S of the user X watches the videos played for preview on the smartphone 200, etc., based on the playlist 700A, and if he or she wishes to make changes (modifications), he or she transmits specific details of the modifications to the video editing device 50A. For example, the child S of the user X may wish to reduce the number of childhood shots and increase the number of boyhood shots, or to replace boyhood shots with shots based on different image material, and transmits the details of the modifications to the video editing device 50A. Furthermore, if the child S of the user X views the video images played back for preview based on the playlist 700A and determines that no changes are necessary, the child S transmits a signal indicating this to the video editing device 50A.

[0088] When a correction instruction is received from child S of user X, the playlist generation unit 516 corrects the playlist 700A based on the request from child S of user X. The video editing device 50A transmits the corrected playlist 710A to the smartphone 200 or the like of child S of user X, and obtains confirmation from child S of user X again. After receiving the corrected playlist 710A from the child S of the user X, the video editing device 50A, upon receiving a confirmation signal from the child S of the user X, sets the confirmed playlist 710A as the playlist 710A to be played on the day of the wedding ceremony. When video editing device 50A receives a signal from child S of user X indicating that playlist 700A has been confirmed without any modifications, video editing device 50A designates confirmed playlist 700A as playlist 710A to be played on the day of the wedding ceremony.

[0089] Playlist 710A is stored in a predetermined area of ​​storage unit 520 in association with the image material of group 710 that was ultimately adopted from among the image materials included in group 700. Group 710 can be considered as a collection of the image materials and playlist 710A that make up group 710.

[0090] Information indicating the save location of playlist 710A is transmitted to the smartphone 200 or the like of child S of user X. Child S of user X inputs the save location of playlist 710A into the playback device that will play the playlist on the day of the wedding. When distributing playlist 710A to a playback device that will play the playlist on the day of the wedding, video editing device 50A can send only playlist 710A, or it can send playlist 710A and the associated image material 710 (in other words, the image material that was ultimately adopted from among the image materials included in group 700).

[0091] (Example 1-4) Adding Image Material to Video Shown at the Wedding Reception Next, an example of the procedure for adding image information to a playlist will be described. In Example 1-4, the playlist generation unit 516 adds image material shot on the day of the wedding to the playlist 700A of videos generated in Example 1-3. The playlist for later distribution (main character) and the playlist for later distribution (attendees) to which the image material has been added are distributed to attendees of the wedding reception after the wedding, for example. The playlist generation unit 516 generates a playlist (main characters) for later distribution to the main characters, the bride and groom, and a playlist (attendees) for later distribution to attendees such as relatives and friends. The video editing device 50A again uses information indicating the friendships of the child S of the user X used in Example 1-3.

[0092] <Decide the search range> When search information setting unit 512 of video editing device 50A receives a request from child S of user X to create a moving image to be distributed to attendees at the wedding reception after the wedding, search information setting unit 512 searches image materials 613 and 615 and image material 616 among image materials stored in video editing device 50A, etc. As described above, image material 613 is an image material taken by user X at the wedding of child S of user X. Image material 615 is an image material taken by child S of user X himself on the day of the wedding and during the preparation period. Furthermore, image material 616 is an image material taken by attendees at the wedding reception.

[0093] <User X's child S searches for stock images of attendees> The search unit 513 searches for moving image materials and still image materials in which the child S and bride Z of the user X appear, while referring to the metadata of the image materials 613, 615, and 616 in the search range. The search unit 513 also searches for moving image materials and still image materials in which the attendees appear, while referring to the metadata of the image materials 613, 615, and 616 in the search range.

[0094] The extraction unit 514 selects image materials that show people who frequently interact with the child S of the user X or the bride Z as candidates for adoption from the image materials searched by the search unit 513 based on the friendship relationship information recorded on the smartphone 200 of the child S of the user X, etc., and extracts cuts from the image materials of the candidates for adoption.

[0095] 4, image materials that have already been grouped as group 700 by video editing device 50A and image materials that extraction unit 514 has adopted from image materials 613, 615, and 616 are illustrated as group 720. Group 720 is a grouping of image materials for ease of explanation, and does not need to be stored together in the same area of ​​memory or the like, as long as the image materials to be included in group 720 are associated with each other.

[0096] <Extract a cut> The video editing device 50A extracts cuts that match the details described in the template from the image materials of the group 720. For example, if the template includes cuts showing scenes on the day of the wedding that the participants were not watching (the participants were not present) and cuts showing the excitement of the wedding reception, these cuts are extracted. The extraction unit 514 extracts cuts that mainly show people who are friends with the child S of the user X.

[0097] The extraction unit 514 preferentially extracts from the image materials of the group 720 cuts that show people with high interaction frequencies, based on the frequency of interaction determined from the number of call records, the number of email records, and SNS usage records. Furthermore, the cuts are extracted so that the total time of the extracted cuts does not exceed the total playback time defined in the template, for example.

[0098] <Search for songs etc.> Furthermore, the search unit 513 searches the video editing device 50A and the like for data such as music to be written in the template.

[0099] <Add to playlist> The framing determination unit 515 determines the display order, display speed, and framing during playback of the cuts extracted from the image materials of group 720 based on the items described in the template, and also determines the playback order, playback speed, and playback volume of the music and sound effects used as background music based on the items described in the template. Furthermore, when determining the display order, display speed, etc. of the above cuts, the framing determination unit 515 prioritizes cuts that show people with a high frequency of interaction over cuts that show people with a low frequency of interaction.

[0100] The playlist generation unit 516 generates a playlist 720A that describes the display order and display speed of the determined cuts, the playback order and playback speed of framing and background music, etc., and the volume during playback. The playlist 720A is associated with the image material that makes up the group 720 and temporarily saved in a predetermined area of ​​the video editing device 50A. The group 720 can be considered as a collection of the image material that makes up the group 720 and the playlist 720A.

[0101] <Present to child S of user X> The video editing device 50A transmits the playlist 720A to, for example, the smartphone 200 of the child S of the user X, and obtains confirmation from the child S of the user X. The child S of the user X watches the videos played for preview on the smartphone 200, etc., based on the playlist 720A, and if he or she wishes to make changes (modifications), he or she transmits specific details of the modifications to the video editing device 50A. For example, the child S of the user X may wish to reduce the number of shots of the main character and increase the number of shots of the participants, or to replace the shots of the participants with shots based on different image material, and transmits the details of the modifications to the video editing device 50A. Furthermore, if the child S of the user X judges that no changes are necessary after viewing the video images played back for preview based on the playlist 720A, the child S transmits a signal indicating this to the video editing device 50A.

[0102] When a correction instruction is received from child S of user X, the playlist generation unit 516 corrects the playlist 720A based on the request from child S of user X. The video editing device 50A transmits the corrected playlist 720A to the smartphone 200 or the like of child S of user X, and obtains confirmation from child S of user X again. When the video editing device 50A receives a signal from the child S of the user X indicating that the corrected playlist 720A has been confirmed, the video editing device 50A sets the confirmed playlist 720A as a playlist for later distribution (main character) or a playlist for later distribution (attendees). Furthermore, when the video editing device 50A receives a signal from the child S of the user X indicating that the playlist 720A has been confirmed without any modifications, the confirmed playlist 720A is set as a playlist for later distribution (main character) or a playlist for later distribution (attendees).

[0103] The playlist for later distribution (main character) and the playlist for later distribution (attendees) are each associated with the image material ultimately adopted from among the image materials included in group 720 and stored in a specified area of ​​storage unit 520.

[0104] Information indicating the save location of the playlist for later distribution (main) is sent to the smartphone of the bride and groom, and information indicating the save location of the playlist for later distribution (guests) is sent to the smartphone of the attendees, such as relatives and friends. The bride and groom or attendees input the save location of each playlist into the playback device that will play the playlists after the wedding ceremony. When distributing the playlist to a playback device that will play the video after the wedding, video editing device 50A can send only the playlist, or it can send the image materials associated with the playlist (in other words, the image materials ultimately adopted from the image materials included in group 720).

[0105] (Example 1-5) Videos to be Shown at User X's Funeral An example of the procedure for generating a playlist will be described with reference to Fig. 5. In Example 1-5, the playlist generation unit 516 generates a playlist 750A of videos to be shown at the funeral of user X at the request of user X while he or she is still alive, or at the request of user X's child S or the like after his or her death. The following describes an example of the procedure for creating a playlist of videos that introduce what user X looked like while he or she was alive.

[0106] 5, a horizontal line 22 represents the time axis. The line 22, which continues from FIG. 4, is the time axis for user X and includes the period from the wedding of user X's child S to the death of user X. The image material 621 is image material taken by the user X in his prime, sixtieth birthday, old age, etc. For ease of explanation, the image material 621 is arranged along a line 22 according to the years when the images were taken. Group 670 is a collection of image materials and playlist 670A that make up group 670, which were described with reference to Fig. 3. Group 720 is a collection of image materials and playlist 720A that make up group 720, which were described with reference to Fig. 4.

[0107] <Decide the search range> The following is an example in which user X requests video editing device 50A to create a video (playlist) to be screened at a funeral. Upon receiving a request from user X to create a video introducing user X's life and other aspects with the theme of "funeral," video editing device 50A identifies user X's family members, such as user X's child S, based on information stored in user X's smartphone 200 or the like. Search information setting unit 512 searches for image material 621 uploaded by user X among image material stored in video editing device 50A or the like. Furthermore, search information setting unit 512 searches for image material included in group 670 and group 720 described above among image material stored in video editing device 50A or the like. Search information setting unit 512 may also search for image material 611, 612, and 613 not included in group 670 and group 720. Group 670 is a collection of image materials and playlist 670A that make up group 670, which were described with reference to Fig. 3. Group 720 is a collection of image materials and playlist 720A that make up group 720, which were described with reference to Fig. 4. That is, in this embodiment, the image material 621 and the image materials that make up the groups 670 and 720 are the search range.

[0108] <Find stock images related to the event> The template associated with the theme "funeral" includes scenes in its outline scenario that introduce turning points in user X's life. The search unit 513 analyzes the image material 621 within the search range and the images of the image materials that make up groups 670 and 720, and searches for video and still image materials related to the event described in the template based on the details described in the template. The image analysis is performed by referring to the metadata attached to the image material. If no metadata is attached to the image material or if some information in the metadata is missing, the search unit 513 causes the metadata attachment unit 517 to analyze the image, and searches for video and still image materials related to the event described in the template based on the analysis results. The results of the analysis by the metadata adding unit 517 are recorded (added or added) as metadata to be added to each image material.

[0109] <Search for stock images of User X> The extraction unit 514 selects image materials related to events with high importance as candidates for adoption from the image materials searched by the search unit 513 based on the importance of the events entered in the event schedule of the user X's smartphone 200, etc., and extracts cuts from the image materials of the candidates for adoption.

[0110] 5, image material 621 and the image material selected by extraction unit 514 from among the image material constituting groups 670 and 720 are illustrated collectively as group 750. Group 750 is a grouping of image material for ease of explanation, and does not need to be stored together in the same area of ​​memory or the like, as long as the image material to be included in group 750 is associated with one another.

[0111] <Extract a cut> The extraction unit 514 extracts cuts that match the details described in the template from the image materials of the group 750. For example, if the template includes cuts of user X in his youth, cuts from his wedding, and other cuts of him with his family, friends, and coworkers, the extraction unit 514 extracts these cuts. The extraction unit 514 includes cuts of user X and people who are friends with user X in the cuts to be extracted.

[0112] The extraction unit 514 preferentially extracts from the image materials of the group 750 shots that show people with high interaction frequencies, based on the frequency of interaction determined from the number of call records, the number of email records, and SNS usage records. Furthermore, the cuts are extracted so that the total time of the extracted cuts does not exceed the total playback time defined in the template, for example.

[0113] <Search for songs etc.> Furthermore, the search unit 513 searches for data such as music described in the template from the video editing device 50A, etc. The search unit 513 can also search for music that is specified by the user and that is to the user's taste, regardless of the music defined in the template.

[0114] <Generate a playlist> The framing determination unit 515 determines the display order, display speed, and framing during playback of cuts extracted from the image materials included in group 750 based on the items described in the template, and also determines the playback order, playback speed, and playback volume of the music and sound effects used as background music based on the items described in the template. Furthermore, when determining the display order, display speed, etc. of the above cuts, the framing determination unit 515 prioritizes cuts that show people with a high frequency of interaction over cuts that show people with a low frequency of interaction.

[0115] The playlist generation unit 516 generates a playlist 750A that describes the display order, display speed, framing, playback order of the BGM, etc., playback speed, playback volume, etc. of the determined cuts. The playlist 750A is associated with the image material that makes up the group 750 and temporarily saved in a predetermined area of ​​the video editing device 50A. The group 750 can be considered as a collection of the image material that makes up the group 750 and the playlist 750A.

[0116] <Present to User X> Video editing device 50A transmits playlist 750A to, for example, user X's smartphone 200 or the like, and obtains user X's confirmation. User X watches videos that are played for preview on smartphone 200 or the like based on playlist 750A, and if user X desires to make changes (modifications), transmits specific modifications to video editing device 50A. For example, user X may transmit modifications to video editing device 50A, such as adding more shots with family or shots of club activities. Furthermore, if the user X judges that no changes are necessary after viewing the video images reproduced for preview based on the playlist 750A, the user X transmits a signal indicating this to the video editing device 50A.

[0117] When a correction instruction is received from user X, playlist generation unit 516 corrects playlist 750A based on the request from user X. Video editing device 50A transmits corrected playlist 750A to user X's smartphone 200 or the like, and obtains user X's confirmation again. Video editing device 50A receives a signal from user X indicating that corrected playlist 750A has been confirmed, and sets confirmed playlist 750A as playlist 760A to be played on the day of the funeral. Upon receiving a signal from user X indicating that playlist 750A has been confirmed without any modifications, video editing device 50A uses confirmed playlist 750A as playlist 760A to be played on the day of the funeral.

[0118] Playlist 760A is stored in a predetermined area of ​​storage unit 520 in association with image material of group 760 that was ultimately adopted from among the image materials included in group 750. Group 760 can be considered as a collection of image materials and playlist 760A that make up group 760.

[0119] Information indicating the save location of playlist 760A is transmitted to user X's smartphone 200, etc. The chief mourner, who was informed of the save location by user X while he was alive, inputs the save location of playlist 760A into the playback device that will play it on the day of the funeral. When distributing playlist 760A to a playback device that will play the playlist on the day of the funeral, video editing device 50A can send only playlist 760A, or can send playlist 760A and the associated image material 760 (in other words, the image material that was ultimately adopted from among the image materials included in group 750).

[0120] <Explanation of the flowchart> In the above-described video providing system 1, the flow of processing performed by the user's smartphone 200 and video editing device 50A when the theme is "wedding" will be described with reference to the flowcharts shown in FIGS. FIG. 6 shows the flow of processing performed by the smartphone 200 on the user side, and FIG. 7 shows the flow of processing performed by the video editing device 50A on the cloud 50 side. When video editing device 50A receives a request for video generation from user X's smartphone 200 specifying the theme "wedding," video providing system 1 automatically generates videos and / or playlists. For example, based on the calendar information recorded in smartphone 200, video and / or playlists are created up to 10 days before the planned wedding date.

[0121] (Smartphone) When the user X starts an application for requesting generation of a moving image, the control unit 220 executes the process shown in Fig. 6. Note that it is assumed that communication between the smartphone 200 and the cloud 50 is possible as needed, such as in steps S216, S218, S220, and S234, which will be described later.

[0122] 6, the control unit 220 causes the display / input unit 210 to display, for example, a group of icons (not shown) corresponding to a plurality of pre-prepared video themes, and when the user touches the icon indicating the video theme "wedding", the process proceeds to step S204. The video theme is input by the user X.

[0123] In step S204, the schedule information management unit 221 of the control unit 220 starts checking the user's event schedule recorded as the user X's calendar information. In step S206, the schedule information management unit 221 of the control unit 220 confirms the wedding date of the user X, and the process proceeds to step S208. In step S208, the relationship information management unit 222 of the control unit 220 refers to user X's relationship information (data recorded in the data storage unit 230 as contact information, user X's call records, email records, SNS usage records, etc.), checks user X's relationships with family, relatives, friends, etc., and the guests at the wedding reception, and proceeds to step S210.

[0124] In step S210, the control unit 220 determines whether or not invitee information is present. If the control unit 220 is able to confirm the invitee information in step S208, it makes a positive decision in step S210 and proceeds to step S216. If the control unit 220 is unable to confirm the invitee information in step S208, it makes a negative decision in step S220 and proceeds to step S212.

[0125] In step S212, the control unit 220 causes the display / input unit 210 to display a message requesting user X to input invitee information, and the process proceeds to step S214. In step S214, the control unit 220 accepts input of invitee information from the display / input unit 210, and the process proceeds to step S216. The operation of inputting the invitee information is performed by user X.

[0126] In step S216, control unit 220 transmits information to video editing device 50A via communication unit 240, and the process proceeds to step S220. The information transmitted from smartphone 200 to video editing device 50A includes the theme of the video, social relationship information such as user X's family, relatives, and friends, and guest information for the wedding reception. In addition, if the video to be generated is a video generated by adding image material to a video created in the past (corresponding to (Example 1-2) or (Example 1-4) above), it includes information about the video created in the past.

[0127] In step S220, control unit 220 acquires playlist 650A from video editing device 50A via communication unit 240. Note that control unit 220 can also acquire a video (a video based on playlist 650A) instead of or together with playlist 650A. Video editing device 50A according to the embodiment starts generating a video (or playlist 650A) after receiving information (S216) transmitted from smartphone 200. Therefore, control unit 220 waits for a notification of completion of generation from video editing device 50A, for example, and then acquires the generated video (or playlist 650A).

[0128] In step S222, control unit 220 acquires the cuts described in playlist 650A, in other words, the cuts necessary to play back the video based on playlist 650A, from video editing device 50A or another server, and proceeds to step S224. Note that if a video to replace playlist 650A is acquired, acquisition of the cuts may be omitted.

[0129] In step S224, the control unit 220 makes preparations for playing back the video (or video based on the playlist) on the display / input unit 210, and the process proceeds to step S226. 8(a) is a diagram showing an example of a notification screen displayed on the display / input unit 210 to notify completion of preparations for playback of a video (or a video based on a playlist). Along with a message 81, an operation button 82 for instructing playback for confirmation, in other words, "preview," is displayed. The control unit 220 causes the display / input unit 210 to display the message 81, and the process proceeds to step S228.

[0130] In step S228, the control unit 220 determines whether or not a playback instruction has been issued. If the operation button 82 displayed on the display / input unit 210 has been operated, the control unit 220 makes an affirmative decision in step S228 and proceeds to step S230. If the operation button 82 has not been operated, the control unit 220 makes a negative decision in step S228 and waits for an operation for a playback instruction. The operation for a playback instruction is performed by user X.

[0131] In step S230, the control unit 220 starts playing back a video image for confirmation, and the process proceeds to step S232. 8(b) is a diagram showing an example of a video playback screen displayed on the display / input unit 210. Video is played back and displayed in window 85. A list of file names indicating the group of shots used to generate the video is displayed in window 89. A user X viewing the previewed moving image can pause the playback by operating a known pause icon, or can fast-rewind or fast-forward by operating a fast-rewind icon or fast-forward icon. Of the list displayed in window 89, the file name corresponding to the frame image displayed in window 85 is highlighted by cursor 90. Instead of displaying a list of file names in the window 89, thumbnail images may be displayed side by side. User X can operate the setting change button 86 to input an instruction to change the display speed and framing of the frame image displayed in the window 85, for example. Furthermore, the user X can operate the re-edit button 87 to input instructions to change the order of the cuts that make up the moving image, to delete a cut, to add another cut, etc. When the re-edit button 87 is operated, another screen (not shown) may be displayed on the display / input unit 210 to accept instructions from the user X to change the order of the cuts, to delete a cut, or to add another cut, etc. When the user X has finished inputting the content instructing the video to be modified, the user X operates the save button 88. Note that the user X also operates the save button 88 when, after checking the video, he or she determines that no changes are necessary.

[0132] 6, the control unit 220 determines whether the result is "OK" (no modification of the video (or playlist)). If the save button 88 is operated without changing the settings or re-editing, or if the save button 88 is operated after changing the settings or re-editing, the control unit 220 makes an affirmative decision in step S232 and proceeds to step S234. If the setting change button 86 or the re-edit button 87 is operated, the control unit 220 makes a negative decision in step S232 and proceeds to step S218. The input operations related to setting change and re-editing and the operation of the save button 88 are performed by user X.

[0133] In step S218, the control unit 220 transmits a correction instruction to the video editing device 50A via the communication unit 240, and the process proceeds to step S220. The transmitted correction instruction includes information indicating a change in the order of the cuts constituting the video, an instruction to delete a cut, an instruction to add another cut, etc. The operation of transmitting the correction instruction is performed by the user X.

[0134] In step S234, the control unit 220 transmits "OK" to the video editing device 50A via the communication unit 240, and the process in Fig. 6 ends. "OK" is a signal indicating that the user X has confirmed that the video does not need to be modified. The operation to transmit "OK" is performed by the user X.

[0135] (Video editing equipment) Control unit 510 of video editing device 50A repeatedly executes the process shown in FIG. 7, control unit 510 receives information (S216 (FIG. 6)) transmitted from smartphone 200 of user X, and proceeds to step S504. FIG. 7 shows an example in which the theme of the video included in the received information is "wedding ceremony."

[0136] In step S504, the template selection unit 511 of the control unit 510 selects the "wedding" template, and the process proceeds to step S506. In step S506, the search unit 513 searches for a cut in accordance with the template. Based on the template, the information search setting unit 512 sets some of the attribute information indicating the time, location, venue, photographer, etc. of the image material as first information for search (which may also be called first search conditions). Increasing the number of first information items reduces the number of image material items that are hit by the search. By setting the first information, it is possible to set the search range to image material photographed at a specific time, location, or by a specific photographer. Furthermore, the information search setting unit 512 sets, based on the template, analysis information indicating the main subject, scene, framing area, etc., captured in the image material as second information for the search (which may also be called second search conditions). Increasing the number of second information also reduces the number of image material hits in the search. By setting the second information, it is possible to set the search range to image material that captures a specific subject, image material captured in a specific framing, etc. As described above, the information search and setting section 512 can function as a first information setting section that sets the first information and a second information setting section that sets the second information. In the embodiment, the search range in step S506 is image material taken by family members or relatives and image material that shows family members or relatives based on the first information and second information set in the search information setting unit 512. In step S508, the search unit 513 searches for a cut based on the template. The search range in step S508 is image material taken by friends, acquaintances, etc., based on the first information and second information set by the search information setting unit 512, and image material in which friends, acquaintances, etc. appear.

[0137] In step S510, the extraction unit 514 selects image material that includes a cut that conforms to the template from the image material searched by the search unit 513 as an image material candidate for adoption, and extracts cuts from the image material candidate for adoption.

[0138] In step S512, control unit 510 determines whether the adopted cut has been licensed. If the use has been licensed, control unit 510 makes an affirmative decision in step S512 and proceeds to step S514. If the use has not been licensed, control unit 510 makes a negative decision in step S512 and returns to step S506. After returning to step S506, control unit 510 searches for and extracts licensed cuts again, for example, instead of the cuts for which license has not been obtained. Note that if a candidate cut is selected in advance from image material for which license has been granted and a cut is extracted, the processing of step S512 may be omitted, and step S510 may be followed by step S514.

[0139] In step S514, the playlist generation unit 516 of the control unit 510 generates a playlist 650A that describes the display order of cuts, the display speed, the playback order of framing and background music, the playback speed, and the volume during playback, and then the process proceeds to step S516.

[0140] In step S516, control unit 510 associates playlist 650A with the image materials that make up group 650, temporarily stores the playlist in a predetermined area of ​​storage unit 520, and proceeds to step S518. In step S518, control unit 510 transmits playlist 650A to smartphone 200 of user X, and proceeds to step S520. Note that control unit 510 can also transmit a video (a video based on playlist 650A) in place of or together with playlist 650A, in response to a request from smartphone 200.

[0141] In step S520, control unit 510 transmits the group of cuts described in playlist 650A to smartphone 200, and proceeds to step S522. Note that if a video image replacing playlist 650A is transmitted in step S518, or if playback is performed on smartphone 200 based on playlist 650A, transmission of the group of cuts may be omitted. In step S522, the control unit 510 determines whether or not “OK” has been received. As described above, “OK” is a confirmation signal indicating that the user X has determined that the moving image does not need to be modified. If control unit 510 has received "OK", it makes an affirmative decision in step S522 and proceeds to step S524. If control unit 510 has not received "OK", it makes a negative decision in step S522 and proceeds to step S526. In step S526, control unit 510 receives a correction instruction (S218 (FIG. 6)) transmitted from smartphone 200, and returns to step S506. After returning to step S506, control unit 510 repeats the processes from step S506 onwards.

[0142] In step S524, control unit 510 associates playlist 650A or playlist 660A obtained after playlist 650A has been modified with the image material of group 660 that was ultimately adopted from among the image materials included in group 650, stores the associated image material in a predetermined area of ​​storage unit 520, and terminates the processing of FIG. 7.

[0143] (2) Examples of the theme "○○ Travel" <Playlist creation example> An example of generating a playlist when the theme of the video is "XX Travel" will be described. In the case of the theme "XX Trip," although illustration is omitted, the explanation will be given assuming a time axis similar to the time axis illustrated in FIG. 3 for each of the user who requested the creation of the video and the companions who traveled with the user. The length of the time axis corresponds to the travel period of the "XX Trip." The search targets can then be image materials taken by the user who requested the creation of the video and the companions during the "XX Trip."

[0144] <Explanation of the flowchart> In the above-described video providing system 1, the flow of other processes performed by the user's smartphone 200 and video editing device 50A when the theme is "XX Travel" will be described with reference to the flowcharts shown in FIGS. FIG. 9 shows the flow of processing performed by the smartphone 200 on the user side, and FIG. 10 shows the flow of processing performed by the video editing device 50A on the cloud 50 side. Video providing system 1 automatically generates a video and / or a playlist when video editing device 50A receives a request for generating a video specifying the theme "travel" from user smartphone 200. For example, based on the records of schedule information and location information recorded in user smartphone 200 or the like, video and / or a playlist are created within a predetermined period of time after receiving the request.

[0145] (Smartphone) When a user starts an application for requesting generation of a moving image, the control unit 220 executes the process shown in Fig. 9. Note that it is assumed that communication between the smartphone 200 and the cloud 50 is possible as needed, such as in steps S266, S268, S270, and S284, which will be described later.

[0146] In step S252 of Fig. 9, the control unit 220 requests the user to input a video theme and necessary information. For example, a group of icons (not shown) corresponding to a plurality of pre-prepared video themes are displayed on the display / input unit 210, and when the user touches an icon indicating the video theme "XX Trip," the automatic editing setting screen shown in Fig. 11(a) is displayed on the display / input unit 210. Fig. 11(a) is a diagram illustrating an example of the automatic editing setting screen displayed on the display / input unit 210. An input field 91 for the purpose of the trip, an input field 92 for the itinerary, and an input field 93 for the travel locations are displayed, along with an operation button 94 for instructing "start" to generate a video. The user inputs the video theme, the purpose of the trip, the itinerary, and the travel locations. When a touch operation is performed on the button 94, the control unit 220 proceeds to step S254.

[0147] In step S254, the schedule information management unit 221 of the control unit 220 checks the schedule corresponding to the theme "XX Travel" among the user's event schedule recorded as the user's calendar information, thereby grasping the travel dates and itinerary. In step S256, the relationship information management unit 222 of the control unit 220 refers to the user's relationship information (data recorded in the data storage unit 230 as contact information, the user's call records, email records, SNS usage records, etc.), confirms the user's relationships with family, relatives, friends, etc., and proceeds to step S258. In step S258, the schedule information management unit 221 of the control unit 220 confirms the itinerary of the trip based on the information obtained in S254, and the process proceeds to step S260.

[0148] In step S260, the control unit 220 determines whether or not travel companion information is available. If the control unit 220 is able to confirm the travel companion information in steps S254 to S258, it makes an affirmative decision in step S260 and proceeds to step S266. If the control unit 220 is unable to confirm the travel companion information in steps S254 to S258, it makes a negative decision in step S260 and proceeds to step S262.

[0149] In step S262, the control unit 220 displays a message on the display / input unit 210 requesting the user to input accompanying person information, and the process proceeds to step S264. In step S264, the control unit 220 accepts input of accompanying person information from the display / input unit 210, and the process proceeds to step S266. The operation of inputting accompanying person information is performed by the user.

[0150] In step S266, control unit 220 transmits information to video editing device 50A via communication unit 240, and proceeds to step S270. The information transmitted from smartphone 200 to video editing device 50A includes the theme of the video, the purpose of the trip, the travel schedule (itinerary), the location (travel destination), the user's friendship information, and travel companion information.

[0151] In step S270, control unit 220 acquires a playlist from video editing device 50A via communication unit 240. Note that control unit 220 can also acquire a video (a video based on a playlist) instead of or together with the playlist. Video editing device 50A according to the embodiment starts generating a video (or a playlist) after receiving information (S266) transmitted from smartphone 200. Therefore, control unit 220 waits for a notification of completion of generation from video editing device 50A, for example, and then acquires the generated video (or playlist).

[0152] In step S272, control unit 220 acquires the cuts described in the playlist, in other words, the cuts necessary to play back the video based on the playlist, from video editing device 50A or another server, etc., and proceeds to step S274. Note that if a video that replaces the playlist has been acquired, acquiring the cuts may be omitted.

[0153] In step S274, the control unit 220 makes preparations for playing back the video (or video based on the playlist) on the display / input unit 210, and the process proceeds to step S276. 11(b) is a diagram showing an example of a notification screen displayed on the display / input unit 210 to notify completion of preparations for playback of a video (or a video based on a playlist). Along with a message 81, an operation button 82 for instructing playback for confirmation, in other words, "preview," is displayed. The control unit 220 causes the display / input unit 210 to display the message 81, and the process proceeds to step S278.

[0154] In step S278, the control unit 220 determines whether or not a playback instruction has been issued. If the operation button 82 displayed on the display / input unit 210 has been operated, the control unit 220 makes an affirmative decision in step S278 and proceeds to step S280. If the operation button 82 has not been operated, the control unit 220 makes a negative decision in step S278 and waits for an operation to issue a playback instruction. An operation related to a playback instruction is performed by the user.

[0155] In step S280, the control unit 220 starts playing back a video for confirmation, and the process proceeds to step S282. As explained with reference to Figure 8(b), a user viewing a preview video can pause playback by operating a known pause icon, or fast-rewind or fast-forward by operating a fast-forward icon or a fast-rewind icon. Furthermore, the user can operate the setting change button 86 to input an instruction to change the display speed and framing of the frame image displayed in the window 85, for example. Furthermore, the user can operate the re-edit button 87 to change the order of the cuts that make up the moving image, to instruct the deletion of a cut, or to instruct the addition of another cut. When the user has finished inputting the instructions for modifying the moving image, the user operates the save button 88. Note that the user also operates the save button 88 if, after checking the moving image, the user determines that no changes are necessary.

[0156] 9, the control unit 220 determines whether the result is "OK" (no modification of the video (or playlist)). If the save button 88 is operated without changing the settings or re-editing, or if the save button 88 is operated after changing the settings or re-editing, the control unit 220 makes an affirmative decision in step S282 and proceeds to step S284. If the setting change button 86 or the re-edit button 87 is operated, the control unit 220 makes a negative decision in step S282 and proceeds to step S268. The input operations related to setting change and re-editing and the operation of the save button 88 are performed by the user.

[0157] In step S268, control unit 220 transmits a correction instruction to video editing device 50A via communication unit 240, and the process proceeds to step S270. The transmitted correction instruction includes information indicating an instruction to rearrange the order of cuts constituting the video, an instruction to delete a cut, an instruction to add another cut, etc. The operation to transmit the correction instruction is performed by the user.

[0158] In step S284, control unit 220 transmits "OK" to video editing device 50A via communication unit 240, and ends the process in Fig. 9. "OK" is a signal indicating that the user has confirmed that no correction to the video is necessary. The operation to transmit "OK" is performed by the user.

[0159] (Video editing equipment) Control unit 510 of video editing device 50A repeatedly executes the process shown in FIG. 10, control unit 510 receives information (S266 (FIG. 9)) transmitted from user smartphone 200, and proceeds to step S554. FIG. 10 shows an example in which the theme of the moving image included in the received information is "XX Trip."

[0160] In step S554, the template selection unit 511 of the control unit 510 selects the template "XX Trip", and the process proceeds to step S556. In step S556, the search unit 513 searches for a cut in accordance with the template. The information search setting unit 512, functioning as a first information setting unit, sets some of the attribute information of the image material as first information for search (which may be called first search conditions) based on the template. Also, the information search setting unit 512, functioning as a second information setting unit, sets some of the analysis information of the image material as second information for search (which may be called second search conditions) based on the template. The search range in step S556 is image material taken by the user or companions during the trip based on the first information and second information set in the search information setting unit 512, and image material showing the user and companions during the trip. In step S558, the search unit 513 searches for cuts based on the template. For example, if the template contains a scenario introducing a travel destination, image material that is widely available online may be included in the search range. The weather at the user's travel destination may not necessarily be good on the day the user visits. Therefore, the information search setting unit 512, functioning as the first information setting unit, does not include information identifying the photographer and the shooting date in the first information. As a result, when it comes to image material introducing the user's travel destination, image material that has been photographed by a third party other than the user or their companions on a day other than the day the user visited and that has been posted online can be included in the search range. In the embodiment, the search range of step S558 includes image material that shows scenery of tourist spots and other locations that are the user's travel destinations, based on the first information and second information set in the search information setting unit 512, and that is, for example, image material that was photographed by a third party on a day with good weather, even though the photograph date is different from the user's travel date.

[0161] In step S560, the extraction unit 514 selects image material that includes a cut that conforms to the template from the image material searched by the search unit 513 as an image material candidate for adoption, and extracts cuts from the image material candidate for adoption.

[0162] In step S562, control unit 510 determines whether or not the adopted cut has been licensed. If licensed, control unit 510 makes an affirmative decision in step S562 and proceeds to step S564. If license has not been obtained, control unit 510 makes a negative decision in step S562 and returns to step S556. After returning to step S556, control unit 510 may, for example, search for and extract licensed cuts instead of cuts for which license has not been obtained. Note that when selecting candidates for adoption from image material for which license has been obtained in advance and extracting cuts, the processing of step S562 may be omitted, and control may proceed to step S564 after step S560.

[0163] In step S564, the playlist generation unit 516 of the control unit 510 generates a playlist that describes the display order of cuts, the display speed, the playback order of framing and background music, the playback speed, and the volume during playback, and then the process proceeds to step S566.

[0164] In step S566, control unit 510 associates the playlist with the image materials that make up the group (plurality of adopted image materials), temporarily stores the playlist in a predetermined area of ​​storage unit 520, and proceeds to step S568. In step S568, control unit 510 transmits the playlist to user smartphone 200, and proceeds to step S570. Note that control unit 510 can also transmit a video (a video based on the playlist) instead of or together with the playlist, in response to a request from smartphone 200.

[0165] In step S570, control unit 510 transmits the group of shots described in the playlist to smartphone 200, and proceeds to step S572. Note that if a video image replacing the playlist has been transmitted in step S568, or if playback is performed on smartphone 200 based on the playlist, transmission of the group of shots may be omitted. In step S572, control unit 510 determines whether or not “OK” has been received. As described above, “OK” is a signal indicating that the user has confirmed that correction of the moving image is not necessary. If control unit 510 has received "OK", it makes an affirmative decision in step S572 and proceeds to step S574. If control unit 510 has not received "OK", it makes a negative decision in step S572 and proceeds to step S576. In step S576, control unit 510 receives a correction instruction (S268 (FIG. 9)) transmitted from smartphone 200, and returns to step S556. After returning to step S556, control unit 510 repeats the processes from step S556 onwards.

[0166] In step S574, the control unit 510 associates the playlist or the playlist after the playlist has been modified with the image material finally adopted from among the image materials included in the group, stores the associated image material in a predetermined area of ​​the storage unit 520, and terminates the processing of FIG. 10.

[0167] (3) Example of theme "△△ Party" <Playlist creation example> An example of generating a playlist when the theme of the video is "XX Party" will be described. In the case of the theme "△△ Party," although illustration is omitted, the explanation will be given assuming a time axis similar to the time axis illustrated in FIG. 3 for the user who requested the creation of the video and the attendees who attended the meeting with the user. The length of the time axis corresponds to the duration of the "△△ Party." The search targets can then be image materials taken by the user who requested the creation of the video and the attendees during the "△△ Party." If the template contains a scenario introducing the "△△ Party," the search range may include image materials published online. In other words, image materials introducing the "△△ Party" can be searched for regardless of whether they are the user or attendees, and that have been published online by third parties other than the user or attendees.

[0168] As described above in detail, video editing device 50A assumes a time axis similar to the time axis illustrated in FIG. 3 for each of the user who requested video generation and the participants who participated in the event with the user. The length of the time axis may be the duration of the event or the life of the main character of the event. Then, image material taken by the user who requested video generation and the participants during the event or throughout their lives may be search targets. If the template contains a scenario that introduces names, places, etc. related to the theme of the video to be generated, the search range may include image materials that are publicly available on the Internet. In other words, for image materials that introduce names, places, etc., it is possible to search for image materials that are publicly available on the Internet by third parties other than the user or participant, regardless of whether they are users or participants. The video editing device 50A selects image materials including cuts that conform to the template from the searched image materials as candidates for adoption, and extracts cuts from the candidate image materials. The video editing device 50A automatically generates a playlist and / or a video based on the playlist, which describes the display order, display speed, framing, playback order, playback speed, volume during playback, etc. of the cuts.

[0169] <Program> The processes executed by the control unit 510 of the video editing device 50A of the image processing system, i.e., the processes illustrated in Fig. 7 or 10, may be recorded on a computer-readable recording medium, and the program recorded on this recording medium may be supplied to the video editing device 50A. The term "computer-readable recording medium" refers to portable recording media such as flexible disks, magneto-optical disks, optical disks, and memory cards, as well as storage devices such as hard disks built into computer systems. The term "computer system" refers to an operating system (OS) and peripheral hardware, such as the illustrated video editing device 50A.

[0170] Furthermore, the term "computer-readable recording medium" may include a medium that dynamically stores a program for a short period of time, such as a communication line when transmitting a program via a network such as the Internet or a communication line such as a telephone line, or a medium that stores a program for a certain period of time, such as a volatile memory within a computer system that serves as a server or client in such a case. The above-mentioned program may be one that realizes part of the above-mentioned functions, or may realize the above-mentioned functions in combination with a program already stored in the computer system.

[0171] 12 is a diagram illustrating the supply of a program to video editing device 50A. Video editing device 50A can be provided with a program recorded on, for example, CD-ROM 953. Video editing device 50A can also be provided with the program via communication line 900. Computer 952 is a server computer that provides the above program, and stores the program on a recording medium 954 such as a hard disk. Communication line 900 is a communication line such as the Internet or personal computer communication, or a dedicated communication line. Computer 952 reads the program from recording medium 954 and transmits the program to video editing device 50A via communication line 900. In other words, the program is carried as a data signal by a carrier wave and transmitted via communication line 900. Video editing device 50A receives the transmitted program. In this manner, the program can be supplied as a computer-readable computer program product in various forms such as a recording medium or a carrier wave.

[0172] According to the above-described embodiment, the following advantageous effects can be obtained: (1) Video editing device 50A, which generates a playlist as information necessary for playing a video based on a plurality of image materials, includes search unit 513 that searches for a plurality of cuts to be used in the video from a plurality of image materials using information about the theme of the video to be played. With this configuration, the search unit 513 can search for appropriate cuts according to the theme of the video, such as the name of an event, etc. Therefore, the video based on the playlist generated using the searched cuts will be attractive to viewers who expect a video that matches the theme.

[0173] (2) Video editing device 50A includes communication unit 530 as a first input unit that inputs information about the theme of the video, search information setting unit 512 as a first information setting unit that sets attribute information indicating the shooting date, shooting location, shooting venue, photographer, etc. of multiple image materials based on the theme as first information for search, and search information setting unit 512 as a second information setting unit that sets analysis information of the multiple image materials (such as the main subject, scene, framing area, etc.) based on the first information and the theme as second information for search. Search unit 513 searches for multiple cuts from the multiple image materials using the attribute information and analysis information. With this configuration, the search unit 513 can search for more appropriate cuts from among many image materials that match the theme of the video. Therefore, a video based on a playlist generated using more appropriate cuts will be more appealing to viewers who expect a video that matches the theme.

[0174] (3) The search unit 513 of the video editing device 50A searches for a plurality of cuts from a plurality of image materials to which metadata is added as additional information indicating attribute information and analysis information in chronological order. With this configuration, the search unit 513 can find out what (who) is in the image material by analyzing the metadata instead of playing back the image material. By using such metadata, the search unit 513 can search for an appropriate shot that matches the theme more efficiently than if the metadata were not used.

[0175] (4) The video editing device 50A further includes a metadata adding unit 517 that adds additional information to a plurality of image materials. With this configuration, if metadata is not added to an image material uploaded to video editing device 50A, metadata adding unit 517 analyzes the image and can add or append information to the image material as metadata, such as who (or what person) is depicted in the image material, when and where what event was filmed, and by whom, etc. This can improve the efficiency of subsequent cut searches.

[0176] (5) The video editing device 50A described in (2) to (4) above includes a communication unit 530 as a second input unit to which information about the main character of the theme and the main character's family, friends, acquaintances, etc. who are friends of the main character is input. The search unit 513 uses the attribute information and analysis information to search multiple image materials for multiple cuts including a first cut that shows the main character and / or friends, or a second cut that shows the location where the main character and / or friends are staying. With this configuration, for example, if the theme of the video is the name of an event, it is possible to appropriately search for shots showing the main characters or participants of the event, or shots showing the locations where these people were during the event. A video based on a playlist generated using the shots searched in this way will be more appealing to viewers who are looking forward to the video.

[0177] (6) In the video editing device 50A, the attribute information includes information indicating the photographer of multiple image materials, and the search unit 513 searches for the first or second cut from multiple image materials in which the photographer is the main character or a friend of the photographer. With this configuration, for example, if the theme of the video is the name of an event, it is possible to appropriately search for cuts in which the main character or a participant of the event was the photographer. A video based on a playlist generated using the cuts found in this way is more appealing to viewers who are looking forward to the video.

[0178] (7) The search unit 513 of the video editing device 50A searches for the first or second cut from a plurality of image materials in which the schedule of the photographer matches the schedule of the main character or a close friend. With this configuration, for example, if the theme of the video is the name of an event, it is possible to appropriately search for shots taken by people who participated in the event. A video based on a playlist generated using the shots found in this way will be more appealing to viewers who are looking forward to the video.

[0179] (8) In video editing device 50A, the theme is an event, and after the event is held, search unit 513 further searches for cuts to be added to the video to be distributed after the event from among the image materials shot at the event. With this configuration, for example, if the theme of the video is a wedding, it is possible to add shots taken by people at the wedding venue to the video played at the wedding venue.The video generated based on the playlist with these shots added will be more appealing to viewers who are looking forward to the video.

[0180] (9) The video editing device 50A further includes a framing determination unit 515 that determines the playback order, display speed (or display time), and framing of multiple cuts to be used in the video based on a template prepared in advance and associated with a theme. With this configuration, inappropriate playback of moving images, such as a significant imbalance in display time among a plurality of cuts or an extremely small proportion of the screen occupied by the main subject, can be avoided compared to when a template is not used, and more attractive moving images can be obtained for viewers who expect moving images.

[0181] (10) A video editing device 50A that edits a video to be played back based on a plurality of image materials includes a communication unit 530 as an input unit to which information on the theme of the video to be played is input, a search information setting unit 512 as a first information setting unit that sets first information as attribute information indicating the shooting time, shooting location, shooting venue, photographer, etc. of the plurality of image materials based on the theme, a search information setting unit 512 as a second information setting unit that sets second information as analysis information of the plurality of image materials (main subject, scene, framing area, etc.) based on the first information and the theme, a search unit 513 that uses the attribute information and analysis information to search for multiple cuts to be used in the video from the plurality of image materials, and an extraction unit 514 and a framing determination unit 515 as editing units that edit using the multiple cuts searched by the search unit 513. With this configuration, the search unit 513 searches for more appropriate cuts from among many image materials according to the theme of the video, and the extraction unit 514 and the framing determination unit 515 can perform appropriate editing using the searched cuts. As a result, a video that is more appealing to viewers who are looking forward to the video can be obtained.

[0182] The following modifications are also within the scope of the present invention, and one or more of the modifications may be combined with the above-described embodiment. (Variation 1) In the above-described embodiment, a smartphone 200 or the like is exemplified as a user terminal, but the user terminal may also be a tablet terminal connected via a communication line, a personal computer, or a television connected via a communication line, etc. Furthermore, although the video editing device 50A has been described as being configured by a server on the cloud 50, it may also be configured by a personal computer or the like located near the user instead of the cloud 50.

[0183] (Variation 2) 8(b), a user X may instruct the deletion of a cut constituting the video being played back by a pinch-in operation, and may instruct the addition of another cut by a pinch-out operation. A pinch-in operation is an operation of narrowing the distance between two fingers touching the screen of the display / operation unit 210 of the smartphone 200, and a pinch-out operation is an operation of widening the distance between two fingers touching the screen.

[0184] When a pinch-in operation is performed, control unit 220 determines that an instruction to delete a cut corresponding to a scene displayed in window 85 has been input. Conversely, when a pinch-out operation is performed, control unit 220 determines that an instruction to add a cut related to a scene displayed in window 85 has been input.

[0185] When control unit 220 determines that an instruction to delete or add a cut has been input, it transmits the correction instruction to video editing device 50A via communication unit 240 in step S218 (FIG. 6), and proceeds to step S220. Upon receiving this correction instruction (step S526 (FIG. 7)), search unit 513 of video editing device 50A returns to step S506 (FIG. 7) and deletes the cut, or searches for and extracts the cut again. In this way, the user can issue instructions to delete or add a cut that constitutes the video being played back by using a pinch-in or pinch-out operation other than operating the re-edit button 87. The addition of a cut may be performed based on an operation other than a pinch-out operation. For example, when an arbitrary material is selected from the searched image materials and tapped, the tapped material may be added to the extracted materials. Furthermore, the deletion of a cut may be performed based on an operation other than a pinch-in operation. For example, when an arbitrary material is selected from the extracted materials and a flick operation is performed, the flicked material may be deleted from the extracted materials.

[0186] Although various embodiments and modifications have been described above, the present invention is not limited to these. Aspects in which the configurations shown in the embodiments and modifications are used in combination are also included within the scope of the present invention. Other aspects conceivable within the scope of the technical concept of the present invention are also included within the scope of the present invention. [Explanation of symbols]

[0187] 1...video editing system, 50A...video editing device, 100...overhead display, 200...smartphone, 210...display / input unit, 220...control unit, 240...communication unit, 260...shooting unit, 510...control unit, 511...template selection unit, 512...search information setting unit, 513...search unit, 514...extraction unit, 515...framing determination unit, 516...playlist generation unit, 517...metadata assignment unit, 520...storage unit, 530 ...Communication section, 601, 602, 603, 604, 605, 611, 612, 613, 614, 615, 616, 621...Image materials, 650, 660, 670, 680, 690, 700, 710, 720, 750, 760...Group, 650A, 660A, 670A, 680A, 690A, 700A, 710A, 720A, 750A, 760A...Playlist, 900...Communication line, 952...Computer, 953...CD-ROM

Claims

[Claim 1] A playback information generating device that generates information necessary for playing back a video based on a plurality of image materials, comprising: a first input unit for inputting information about the theme of the video to be played; a first information setting unit that sets attribute information of the plurality of image materials as first information based on the theme input by the first input unit; a second information setting unit that sets analysis information of the plurality of image materials as second information based on the attribute information and the theme; a search unit that searches the plurality of image materials for a plurality of cuts to be used in the video, using the first information and the second information that are respectively set based on the theme information; A playback information generating device comprising:

Images