Photo movie generating system, photo movie generating device, photo movie generating method, and program

Abstract

To provide a photo movie generating system or the like, capable of making a user sufficiently enjoy memories such as a theme park.SOLUTION: In a photo movie generating system 1, a photo movie generation device 2 and a user terminal 3 are communicably connected via a network 7. The photo movie generation device 2 comprises: imaging acquisition means of acquiring an image photographed by picture photographing devices 4a, 4b, and 4c, ... of photographing spots A, B, and C, ... of a theme park by a user when receiving a generation request of a photo movie from a user terminal 3; and photo movie generation means of generating the photo movie reproduced in the user terminal 3 with music related to the photographing spots A, B, and C, ..., that is, the theme part by using the image acquired by the image acquisition means and transmitting it to the user terminal 3.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a photomovie generating system for generating and providing a photomovie. [Background technology]

[0002] Patent Document 1 proposes an image print system as an example of a photography service at a tourist spot or theme park, in which a photo rally is held in which users are photographed by cameras installed at multiple locations at the tourist spot, etc., and the photographed images are printed out after the rally is over. By using this photography service, users can preserve their memories in photographs.

[0003] Also, a cloud photo storage service provides a service that automatically creates a photo movie using photos stored in the cloud storage. For example, in the cloud photo storage service of Non-Patent Document 1, a user selects photos and videos that he / she wants to make into a movie from the images stored in the cloud storage, and selects a desired theme from several themes prepared in advance, and the movie is automatically generated on the service side. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2018-074281 A [Non-patent literature]

[0005] [Non-Patent Document 1] Google Photos, "Create a movie", [online], [Searched on July 8, 2020], Internet <URL: https: / / photos.google.com / movies / create> Summary of the Invention [Problem to be solved by the invention]

[0006] If users could be provided with movies with music (BGM) added using photos taken with photography services at theme parks, event venues, tourist spots, etc., they would be able to enjoy the photos even more, which would increase the appeal of the photography service.

[0007] Regarding the music to be added to the movie, the movie creation function of Non-Patent Document 1 allows the user to select the music, but the music that can be selected is either a default category (dramatic, electronic, rockin', upbeat, etc.) or music that is DRM (Digital Rights Management) compatible with the music app in the user's terminal. The service only allows songs that are not protected by Digital Rights Management (DDM), so users cannot select songs related to the places they visited, which is not enough to enjoy memories of theme parks, etc.

[0008] The present invention has been made in consideration of the above problems, and has an object to provide a photo movie creating system and the like that allows a user to fully enjoy memories of a theme park or the like. [Means for solving the problem]

[0009] The first invention for solving the above-mentioned problems is a photo movie production system in which a photo movie production device and a user terminal are connected to each other via a network so as to be able to communicate with each other, the photo movie production device comprising: storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit; an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal; The photo movie generating means generates a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquiring means, and when generating a new photo movie to be played along with the music, judges whether or not the movie configuration information of the photo movie to be played along with the same music previously generated exists in the storage unit, and if the movie configuration information exists, generates a new photo movie having a pattern different from the pattern of the movie configuration information. The photo movie generating system is characterized by the above. For example, the shooting spot is set up in a theme park, a tourist attraction, or an event venue, and the photo movie generation system further includes an image storage that stores images taken by the user at the shooting spot in association with the user's user ID, and a photography device that is set up at the shooting spot and takes photos of the user, the photography device reads the user ID when taking photos of the user, and if the user visits a theme park, tourist attraction, or event venue multiple times, the image storage stores the images taken by the photography device during each visit in association with the same user ID. It is also preferable that the image acquisition means acquires images provided by businesses of the visited places, in addition to images taken by the user at photography spots in the visited places.

[0010] According to the first invention, when a request to generate a photo movie is sent from a user terminal to a photo movie generating device, the photo movie generating device acquires images of the user taken at a photography spot, generates a photo movie from the images to be played along with music related to the photography spot, and sends the photo movie to the user terminal. This allows the photo movie generating system to provide the user with a photo movie in which images taken at a photography spot such as a theme park are played along with music related to the theme park, allowing the user to fully enjoy his or her memories and improving the appeal of the photography service.

[0011] In the first invention, You can create impressive photo movies in which the screen changes according to the characteristics of the music.

[0012] In the first invention, Since it is possible to provide photo movies with different expressions for the same music, it is possible to provide a variety of entertainment to users.

[0013] It is also desirable that the photo movie generation means generates the photo movie without adding the audio of the song, links it to identification information of the song and transmits it to the user terminal, and that the user terminal is equipped with a playback means for acquiring audio data of the song corresponding to the identification information and playing the acquired audio data in synchronization with the photo movie. This makes it possible to generate photo movies with smaller file sizes than photo movies with music, thereby saving on storage space and communication bandwidth.

[0014] The photo movie generating device further includes: Cut out the object in It is also preferable that the image processing device further comprises image material generating means for generating image materials, and the photo movie generating means generates the photo movie using the image materials generated by the image material generating means. By generating a photo movie using image material generated from an image, a more interesting and enjoyable photo movie can be provided.

[0015] It is also preferable that the user terminal comprises a music selection means for accepting a selection of music to be used in the photo movie and transmitting the selection information of the music to the photo movie generating device together with a request to generate the photo movie. Furthermore, it is preferable that the user terminal comprises an image designation means for accepting designation of images to be used in the photo movie and transmitting the image designation information to the photo movie production device together with a request to produce the photo movie. This allows the user to specify his / her favorite music and images and generate a photo movie.

[0016] A second invention is a photo movie generating device communicably connected to a user terminal via a network, storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit; an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal; The photo movie generating means generates a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquiring means, and when generating a new photo movie to be played along with the music, judges whether or not the movie configuration information of the photo movie to be played along with the same music previously generated exists in the storage unit, and if the movie configuration information exists, generates a new photo movie having a pattern different from the pattern of the movie configuration information. The photo movie generating device is characterized by the above.

[0017] N A user terminal connected to the photo movie generating device via a network so as to be able to communicate with the photo movie generating device. teeth The present invention is characterized by comprising: a photo movie generation requesting means for transmitting a request for generating a photo movie to the photo movie generating device; and a receiving means for receiving from the photo movie generating device a photo movie that is generated using images taken by a user at a shooting spot and is to be played on the user terminal together with music related to the shooting spot. do.

[0018] A third invention is a photo movie generating method in a photo movie generating system in which a photo movie generating device and a user terminal are communicably connected via a network, comprising: Movie configuration information including effect patterns showing the composition of the changes in the screen in sync with the music in the generated photo movie is stored in the storage unit. When the photo movie generating device receives a request to generate a photo movie from the user terminal, the photo movie generating device acquires images taken by the user at a photography spot to be used in the photo movie; receiving a request to generate a photo movie from the user terminal, acquiring acoustic feature data of a song related to the photography spot; generating a photo movie using the images to be played on the user terminal together with music related to the photography spot, and transmitting the photo movie to the user terminal; The photo movie generating device generates a photo movie in which a screen changes in accordance with music based on the acoustic feature data, and when generating a new photo movie to be played along with the music, determines whether or not the movie configuration information of the photo movie to be played along with the same music previously generated exists in the storage unit, and if the movie configuration information exists, generates a new photo movie having a pattern different from the pattern of the movie configuration information. The photo movie generating method is characterized by the above.

[0019] The fourth invention is a system for controlling a user terminal connected to a network so as to be capable of communicating with the user terminal. The generated photo movie is stored in a storage unit with movie configuration information including effect patterns that indicate the configuration of the screen changes in sync with the music. the photo movie generating device, when receiving a request to generate a photo movie from the user terminal, acquiring images taken by the user at a photography spot to be used in the photo movie; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the generated photo movie to the user terminal. the photo movie generating means generates a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquiring means, and when generating a new photo movie to be played along with the music, determines whether or not the movie configuration information of the photo movie to be played along with the same music previously generated exists in the storage unit, and if the movie configuration information exists, generates a new photo movie having a pattern different from the pattern of the movie configuration information. This is a program that functions as a

[0020] centre A user terminal communicably connected to the photo movie generating device via a network The program is run on the user's terminal. and a receiving means for receiving, from the photo movie generating device, a photo movie that is generated using images taken by a user at a photo spot and that is to be played on the user terminal together with music related to the photo spot. do. Effect of the Invention

[0021] According to the present invention, it is possible to provide a photo movie generating system that allows a user to fully enjoy memories of a theme park or the like. [Brief description of the drawings]

[0022] [Figure 1] 1 is a diagram showing the overall configuration of a photo movie production system 1. [Diagram 2] FIG. 2 shows the hardware configuration of the photomovie generating apparatus 2. [Diagram 3] FIG. 2 shows the hardware configuration of a user terminal 3. [Figure 4] FIG. 2 shows the data structure of an image storage 5. [Diagram 5] FIG. 2 shows the data structure of a music database 60. [Figure 6] 3A shows an example of the data structure of music information 63, and FIG. 3B shows an example of the data structure of acoustic feature data 64. [Figure 7] FIG. 1 shows the functional configuration of a photo movie generation system 1. [Figure 8] 3 is a flowchart showing the flow of processing executed by the photomovie production system 1. [Figure 9] FIG. 4 is a diagram showing an example of an operation screen 8. [Figure 10] FIG. 2 shows an example of an effect template 221. [Figure 11] FIG. 4 shows an example of an effect correspondence table 222. [Figure 12] 11 is a flowchart showing the flow of a photo movie generating process. [Figure 13] 1A to 1C are diagrams showing examples of photo movies 71, 72, and 73. [Figure 14] FIG. 2 shows an example of movie configuration information 223. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0023] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[0024] 1 is a diagram showing the overall configuration of a photo movie production system 1 according to an embodiment of the present invention. The photo movie production system 1 generates a photo movie using images (photos) taken by a user at a photography service such as a theme park, and provides the photo movie to the user.

[0025] Photography services for theme parks, etc. are services that photograph users with photography devices installed in places that users visit, such as theme parks such as amusement parks, tourist spots, or event venues (these places are collectively referred to as theme parks, etc.). With this photography service, the photographed images can be printed out and sold to users, or the image data can be uploaded to storage so that it can be downloaded later on the user's smartphone, etc.

[0026] In this embodiment, it is assumed that photography spots A, B, C, ... are provided in a theme park, and photography services are provided by photography devices 4a, 4b, 4c, ... installed at the photography spots A, B, C, .... The photography devices 4a, 4b, 4c, ... are, for example, stationary photography devices equipped with a camera, a user ID reader, a communication interface, etc.

[0027] As shown in FIG. 1, in the photo movie production system 1, a photo movie production device 2, a user terminal 3, an image storage 5, a music distribution server 6, etc. are connected via a network 7 so as to be able to communicate with each other.

[0028] In this embodiment, the photography devices 4a, 4b, 4c, etc. are also connected to the image storage 5 via the network 7 so as to be able to communicate with each other. The user is issued a user ID by the theme park by performing user registration in advance, and when taking pictures with the photography devices 4a, 4b, 4c, etc., the photography devices 4a, 4b, 4c, etc. read the user ID displayed as a two-dimensional code or the like on the display screen of the user terminal 3 or on the face of the ticket. This allows the images taken with the photography devices 4a, 4b, 4c, etc. to be linked to the user ID and stored in the image storage 5. If the user has visited the theme park multiple times, the images taken with the photography devices 4a, 4b, 4c, etc. during each visit can be linked to the same user ID and stored.

[0029] In response to a request from the user terminal 3, the photo movie generating device 2 generates a photo movie using the above images and transmits it to the user terminal 3.

[0030] Fig. 2 is a diagram showing the hardware configuration of the photo movie production device 2. As shown in Fig. 2, the photo movie production device 2 can be realized by, for example, a server computer configured by connecting a control unit 201, a storage unit 202, a communication unit 203, etc. via a bus 204, etc. However, the present invention is not limited to this, and various configurations can be appropriately adopted, and the photo movie production device 2 can also be configured by multiple computers.

[0031] The control unit 201 includes a CPU (Central Processing Unit). The control unit 201 is made up of a CPU, a read only memory (ROM), a random access memory (RAM), etc. The CPU loads programs stored in the storage unit 202, ROM, and other recording media into a work memory area on the RAM, executes them, and drives and controls each unit connected via a bus 204. The ROM permanently stores programs such as the computer's boot program and BIOS, as well as data. The RAM temporarily stores loaded programs and data, and also has a work area used by the control unit 201 to perform various processes. The control unit 201 functions as each means of the photo movie creation device 2 by reading and executing the programs.

[0032] The storage unit 202 is a recording medium such as a hard disk drive, a solid state drive, a flash memory, etc. The storage unit 202 stores programs executed by the control unit 201, data required for executing the programs, an operating system, etc. The storage unit 202 also stores application programs for performing processes (S105 to S106, S109 to S110, and S113 to S116 in FIG. 8) described below, and the control unit 201 of the photomovie generating device 2 performs processes described below in accordance with these application programs.

[0033] 7, the storage unit 202 of the photo movie generating device 2 stores data related to the generation of a photo movie, such as an effect template 221, an effect correspondence table 222, and movie configuration information 223. The details of these data will be described later.

[0034] The communication unit 203 has a communication control device, a communication port, etc., and controls communication via the network 7. The network 7 may be a LAN (Local Area Network) or a WAN (Wide Area Network) that is connected over a wider area. The network 7 includes public communication lines such as the Internet, base stations, etc. The network 7 may be wired or wireless.

[0035] The bus 204 is a path for transmitting control signals, data signals, and the like between the various components of the photomovie production apparatus 2 .

[0036] Returning to the explanation of Fig. 1, the user terminal 3 is a terminal used by a user, and may be, for example, a smartphone, tablet, PC, music player, game console, etc. The user terminal 3 is equipped with a dedicated application program (hereinafter referred to as a photo movie app) for using the photo movie production system 1. Alternatively, the user terminal 3 may be equipped with a browser capable of browsing a website set up on the network 7 by the photo movie production device 2, and perform processing related to the generation of a photo movie via the website.

[0037] 3 is a diagram showing a hardware configuration of the user terminal 3. As shown in the figure, the user terminal 3 can be realized by a computer or the like configured by connecting, for example, a control unit 301, a storage unit 302, a communication unit 303, a display unit 304, an input unit 305, a peripheral device I / F unit 306, and an audio output unit 307 via a bus 308 or the like. However, the present invention is not limited to this, and various configurations can be used as appropriate.

[0038] The control unit 301, the storage unit 302, the communication unit 303, and the bus 308 are substantially similar to the control unit 201, the storage unit 202, the communication unit 203, and the bus 204 of the photo movie production device 2. Note that the storage unit 302 of the user terminal 3 stores a photo movie application for performing processes (S101 to S104, S117, S118, S121, S122 in FIG. 8) described later, and the control unit 301 of the user terminal 3 executes the processes described later in accordance with the photo movie application.

[0039] The display unit 304 is composed of a display device such as a liquid crystal panel or a CRT monitor, and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display device, and displays display information input under the control of the control unit 301 on the display device. Note that the display unit 304 may be a touch panel display in which a touch panel, which is the input unit 305, is integrally provided with the display device.

[0040] The input unit 305 is, for example, an input device such as a touch panel, a keyboard, a pointing device such as a mouse, or a numeric keypad, and outputs input data to the control unit 301.

[0041] The peripheral device I / F (interface) unit 306 is a port for connecting a peripheral device, and data is sent and received from the peripheral device via the peripheral device I / F unit 306. The peripheral device I / F unit 306 is configured with a USB or the like, and usually has multiple peripheral device I / Fs. The connection with the peripheral device may be wired or wireless.

[0042] The audio output unit 307 outputs the audio data input from the control unit 301 from a speaker.

[0043] Returning to the explanation of Fig. 1, as described above, the image storage 5 is a storage that accumulates and stores images of users taken by the photography devices 4a, 4b, 4c, .... In the image storage 5, images taken by users by the photography devices 4a, 4b, 4c, ... are stored in association with user IDs.

[0044] Fig. 4 is a diagram showing the data structure of the image storage 5. As shown in Fig. 4, in the image storage 5, image data 53 is stored in association with a user ID 51 and an image ID 52.

[0045] The user ID 51 is user identification information issued by the theme park upon the user registration described above.

[0046] The image ID 52 is identification information of the image data 53, and is issued by the photographing devices 4a, 4b, 4c, . . . or the image storage 5.

[0047] The image data 53 is an image of the user taken by the photography devices 4a, 4b, 4c, . . .

[0048] In addition, image storage 5 can also store image data 53 (images of theme park buildings, scenery, characters, etc.) provided by the theme park operator as material for the photo movie, linked to image ID 52.

[0049] Returning to the explanation of Fig. 1, the music distribution server 6 transmits music sound source data and the like in response to requests from the photo movie generating device 2 and the user terminal 3. The music distribution server 6 stores music related to theme parks in a music database 60.

[0050] As shown in FIG. 5, the music database 60 stores music sound source data 62, music information 63, and acoustic feature data 64 in association with a music ID 61.

[0051] The music ID 61 is identification information for the music, and is issued by the music distribution server 6.

[0052] The sound source data 62 is sound source data of music related to the theme park, such as theme songs, character songs, sound effects, background music, etc. of the theme park.

[0053] As shown in FIG. 6(a), the music information 63 is additional information about the music, such as the song title, artist name, lyrics information, and playback time.

[0054] As shown in FIG. 6(b), the acoustic feature data 64 is a feature of the sound source data 62, such as the music composition (A melody, B melody, chorus, etc.), the tempo of the music (BPM; Beats Per Minute), beat (time signature), the timing of the pronunciation of a particular instrument, chord progression (the flow of the melody of the music), volume change (audible volume change), frequency center of gravity (degree of excitement of the music), and pitch power separation (complexity of the music). The acoustic feature data 64 is obtained by known analysis methods (for example, CRI Middleware Co., Ltd., "Ultra-High Speed ​​and High Precision Music Analysis Middleware BEATWIZ (registered trademark)", [onine], [searched on July 8, 2020], Internet, <URL: https: / / www.cri-mw.co.jp / product / amusement / beatwiz / index.html>, or "Song The items of the music information 63 and the acoustic feature data 64 shown in FIG. 6 are merely examples, and the present invention is not limited to these.

[0055] Next, the functional configuration of the photomovie production system 1 will be described with reference to FIG.

[0056] 7, in the photo movie production system 1, the photo movie production device 2 includes, in addition to the above-mentioned storage unit 202, a music acquisition unit 21, an image acquisition unit 23, an image material generation unit 24, a photo movie production unit 25, etc. The user terminal 3 includes a music designation unit 31, an image designation unit 32, a photo movie production request transmission unit 33, a reception unit 34, a sound source data acquisition unit 35, a playback processing unit 36, etc.

[0057] The music selection unit 31 (music selection means) of the user terminal 3 accepts the selection of music to be used in the photo movie. The music selection unit 31 accepts the selection of music from among the music stored in the music distribution server 6 by the user's operation on the operation screen 8 (see FIG. 9) described later, and transmits the music ID 61 (selection information) to the photo movie production device 2 together with a request to produce a photo movie.

[0058] The image designation unit 32 (image designation means) accepts designation of images to be used in the photo movie. The image designation unit 32 accepts designation of an image from among images stored in the image storage 5 by a user's operation on the operation screen 8, and transmits the image ID 52 (designation information) to the photo movie production device 2 together with a photo movie production request.

[0059] The photo movie generation request sending unit 33 (photo movie generation request means) sends a request to generate a photo movie to the photo movie generation device 2 together with the above-mentioned music ID 61 and image ID 52. If a photo movie type (such as "photo movie where the screen changes along with music" or "slide show with BGM") described later is specified on the operation screen 8, the photo movie type specification information is also sent to the photo movie generation device 2 together with the request to generate a photo movie.

[0060] When the music acquisition unit 21 (music acquisition means) of the photo movie generation device 2 receives a request to generate a photo movie from the user terminal 3, it acquires music-related data such as sound source data 62, music information 63, and acoustic feature data 64 linked to the above-mentioned music ID 61 from the music distribution server 6.

[0061] When the image acquisition unit 23 (image acquisition means) receives a request to generate a photo movie from the user terminal 3, the image acquisition unit 23 acquires image data 53 linked to the image ID 52 from the image storage 5.

[0062] The image material generating unit 24 (image material generating means) generates image material to be used in the photo movie from the image data 53 acquired by the image acquiring unit 23. The image material generating unit 24 detects objects such as people, animals, objects, etc. from the image data 53 such as a photograph, cuts out the detected objects, and creates individual image material for each object. The image material generating unit 24 outputs the generated image material to the photo movie generating unit 25.

[0063] The photo movie generating section 25 (photo movie generating means) uses the image data 53 acquired by the image acquiring section 23 and the image material generated by the image material generating section 24 to generate a photo movie.

[0064] Here, the photo movie generating unit 25 can use the music information 63 and the acoustic feature data 64 to generate a "slide show with background music" or a "photo movie in which the screen changes in time with the music" according to the specified photo movie type.

[0065] A "slide show with BGM" is a photo movie in which a specified song is used as BGM and specified images are replaced and displayed in a slide show format. The photo movie generating unit 25 generates the "slide show with BGM" based on the song information 63. Specifically, for example, the "play time" of the song information 63 is divided equally by the number of images, and images are replaced every equally divided time.

[0066] The "photo movie whose screen changes in sync with the music" is a photo movie generated by replacing images and applying effects such as movement and rotation in response to changes in the music detected from the "music composition" and "timing of the pronunciation of specific instruments" in the acoustic feature data 64. The photo movie generating unit 25 generates the "photo movie whose screen changes in sync with the music" by referring to the effect correspondence table 222 and effect template 221 stored in the storage unit 202.

[0067] In this embodiment, the photo movie generated by the photo movie generating unit 25 is a photo movie without music. A photo movie without music is a photo movie with only images without the addition of music sound source data 62. As will be described later, the photo movie without music is played back in the user terminal 3 in synchronization with the music sound source data 62.

[0068] When the photo movie generating unit 25 generates the photo movie, it compresses and encodes the data using the MPEG4 format or the like, and converts it into a data format that can be played back on the user terminal 3. Then, it links movie configuration information 223, which is information that indicates the configuration of the photo movie, with the music ID 61 and stores it in the storage unit 202, and it also links the photo movie data with the music ID 61 and transmits it to the user terminal 3. The movie configuration information 223 will be described later.

[0069] The receiving section 34 (receiving means) of the user terminal 3 receives the photo movie sent from the photo movie production device 2.

[0070] The audio source data acquisition section 35 (audio source data acquisition means) acquires, from the music distribution server 6, audio source data 62 of a song corresponding to a song ID 61 linked to a photo movie.

[0071] The playback processing unit 36 ​​(playback means) plays back the photo movie. In this embodiment, the playback processing unit 36 ​​synchronously plays back the acquired sound source data 62 and the photo movie (photo movie without music) received from the photo movie generating device 2.

[0072] Next, the flow of the process executed by the photomovie production system 1 will be described with reference to FIG.

[0073] When a user launches a photo movie application on the user terminal 3 and inputs a user ID 51, the user terminal 3 displays an operation screen 8 as illustrated in FIG. 9, and accepts the specification of images to be used in the photo movie (step S101), the specification of music (step S102), and the specification of the photo movie type (step S103).

[0074] The images that can be specified in step S101 are image data 53 stored in the image storage 5, and the user can specify image data 53 taken by the photography devices 4a, 4b, 4c, ... as well as image data 53 provided by the theme park operator. On the operation screen 8, thumbnails of these images are displayed in a horizontally scrollable image list 83 along with a message 81 to specify an image, such as "Please select an image to be used in the photo movie." If the user visits the theme park multiple times and uses the photography service, it is desirable to be able to specify image data 53 taken by the photography devices 4a, 4b, 4c, ... not only during the most recent visit but also during past visits. The image data 53 specified by the user is displayed in the selected image display area 82.

[0075] In addition to the user's designation, the user may be allowed to select "automatic" to leave the designation of images to the photo movie generating device 2. In the case of "automatic", the photo movie generating device 2 extracts a predetermined number of images from the image data 53 taken by the photographing devices 4a, 4b, 4c, ... based on, for example, the image rating results (quality of smile level, contrast, brightness, composition, etc.), and displays them in the selected image display area 82 in descending order of rating or in chronological order.

[0076] The songs that can be specified in step S102 are those stored in the music database 60 of the music distribution server 6, and a message 84 is displayed on the operation screen 8, such as "Please select music to be used in the photo movie", prompting the user to select a song, along with lists 86, 87, 88 of these songs in a pull-down format.

[0077] The songs that can be specified are displayed by category, such as song list 86 being "theme songs," song list 87 being "character songs," and song list 88 being "instrumental." Alternatively, the songs that can be specified may be displayed in alphabetical order or alphabetical order. Songs specified by the user are displayed in selected song display area 85. The "Preview" button 85a in FIG. 9 is the button that is operated when previewing the specified song.

[0078] As described above, in addition to the user's selection, a "leave it to me" option may be selected to leave the selection of music to the photo movie generating device 2. In the "leave it to me" option, the photo movie generating device 2 randomly selects music from the music stored in the music database 60, or selects music predetermined by the theme park operator depending on the time of year, such as "Character Song 2" from June to August.

[0079] The photo movie types that can be specified in step S103 are the above-mentioned "photo movie with screen changes in sync with music" and "slide show with BGM," and the operation screen 8 displays a message 89 such as "Please select the type of photo movie" to request the selection of the photo movie type, along with radio buttons 91 and 92. The photo movie type selected by the user is displayed in a selected type display area 90.

[0080] Once the images, music, and photo movie type to be used in the photo movie have been specified, the user terminal 3 sends a request to generate a photo movie to the photo movie generation device 2, along with the music ID 61 of the specified music, the image ID 52 of the specified image, and the specified photo movie type (step S104).

[0081] When the photo movie generation device 2 receives a photo movie generation request (step S105), it requests the music distribution server 6 to transmit the song information 63 and acoustic feature data 64 corresponding to the specified song ID 61 (step S106). When the music distribution server 6 receives a request from the photo movie generation device 2 (step S107), it responds to this request by transmitting the song information 63 and acoustic feature data 64 corresponding to the song ID 61 to the photo movie generation device 2 (step S108). The photo movie generation device 2 receives and acquires the song information 63 and acoustic feature data 64 (step S109).

[0082] The photo movie production device 2 also requests the image storage 5 to transmit image data 53 corresponding to the specified image ID 52 (step S110). When the image storage 5 receives the request from the photo movie production device 2 (step S111), it responds by transmitting the image data 53 corresponding to the image ID 52 to the photo movie production device 2 (step S112). The photo movie production device 2 receives and acquires the image data 53 (step S113).

[0083] When the image data 53 is acquired, the photo movie production device 2 executes an image material generation process for the acquired image data 53 as necessary, and extracts an image material from the image data 53 (step S114). Note that the received image data 53 may be used as an image material as it is.

[0084] Next, the photo movie generating device 2 generates a photo movie of the specified type (step S115).

[0085] If the specified photo movie type is "slide show with background music," the photo movie generating device 2, for example, divides the "playback time" of the music information 63 equally by the number of images, and generates a photo movie in slide show format in which images are replaced every equally divided time.

[0086] When the specified photo movie type is "a photo movie in which the screen changes in sync with the music", the photo movie generating device 2 generates a photo movie by replacing images and applying effects such as movement and rotation according to changes in the music detected from the "music composition" and "timing of the sounding of specific instruments" in the acoustic feature data 64. The effects to be used are defined in the effect correspondence table 222 and the effect template 221 in the storage unit 202.

[0087] The effect template 221 is a program for adding a predetermined processing (effect) to an image, and as shown in FIG. 10, a plurality of effect templates 221 are stored in association with an effect number and the number of images to be used.

[0088] Effect templates 221 include, for example, an "image switch" for switching between multiple displayed images in sequence, a "monochrome to color" for changing an image from monochrome to color, a "transmittance change" for changing the transmittance of an image, a "screen move" for moving the display position of an image within the screen, a "rotate" for rotating an image, a "enlarge" for enlarging an image, a "reduce" for reducing an image, a "display ratio change" for changing the display ratio of an image within the display screen, a "appear" for making an image appear, a "enlarge move" for moving an image while enlarging it, a "enlarge rotate" for rotating an image while enlarging it, a "reduce move" for moving an image while reducing it, a "reduce rotate" for moving an image while reducing it, a "small image increase" for gradually increasing the number of small images, a "multiple image single screen display" for displaying multiple images within a single screen, a "multiple image single screen display + slide" for sliding multiple images displayed within a single screen, and an "alternate display" for displaying images alternately within the screen.

[0089] 10 is an example, and may include other effects. Also, multiple patterns of parameters such as the amount of movement and the speed of change may be prepared for each effect. For example, for the effect "enlarge", "enlarge 1", "enlarge 2", "enlarge 3", ... may be prepared according to the enlargement rate, and for the effect "transmittance change", "transmittance change 1", "transmittance change 2", "transmittance change 3", ... may be prepared according to the speed of change.

[0090] When generating a photo movie, the photo movie generating device 2 determines which effect to use by referring to the effect correspondence table 222. Fig. 11 is a diagram showing an example of the effect correspondence table 222. The effect correspondence table 222 is a table that associates music composition elements (intro, A melody, B melody, C melody, solo, chorus, etc.) with effect templates 221 suitable for them, and as shown in Fig. 11, a number of patterns are defined.

[0091] In the example of FIG. 11, in the case of pattern (1), the effect "monochrome to color" is applied to the "intro" of the song, the effect "move" is applied to the "verse" of the song, the effect "small image" is applied to the "verse" of the song, the effect "multiple image display" is applied to the "verse" of the song, the effect "enlarge" is applied to the "solo" of the song, and the effect "image switch" is applied to the "chorus" of the song.

[0092] 11, the effect "image switching" is applied to the "intro" of the song, the effect "reduce and rotate" is applied to the "verse verse" of the song, the effect "expand and rotate" is applied to the "verse verse" of the song, the effect "screen movement" is applied to the "verse verse" of the song, the effect "expand" is applied to the "solo" of the song, and the effect "image switching" is applied to the "chorus" of the song. In this way, a plurality of patterns of effect application examples are defined in the effect correspondence table 222.

[0093] In step S115, the photo movie production device 2 determines which pattern of the effect correspondence table 222 is to be used to produce a photo movie, for example, by the process shown in FIG.

[0094] 12, the photo movie generating device 2 first judges whether the movie configuration information 223 corresponding to the music ID 61 specified by the user is stored in the storage unit 202 (step S201). The movie configuration information 223 is data indicating the configuration of the screen changes according to the music, such as images to be used in the photo movie, the timing of switching between them, effects to be applied to the images, etc. For music for which a photo movie has already been generated by the photo movie generating device 2, the movie configuration information 223 is stored in the storage unit 202 in association with the music ID 61.

[0095] In step S201, if the movie composition information 223 corresponding to the song ID 61 specified by the user is not stored in the memory unit 202 (step S201; No), there is no history of creating a photo movie for that song, so the photo movie generating device 2 reads out an arbitrary pattern from the effect correspondence table 222 (step S202) and generates a photo movie in accordance with the effect correspondence table 222 of the read pattern (step S203).

[0096] On the other hand, if in step S201 the movie configuration information 223 corresponding to the song ID 61 specified by the user is stored in the storage unit 202 (step S201; Yes), the photo movie generating device 2 reads an unused pattern from the effect correspondence table 222 (step S204), and generates a photo movie according to the effect correspondence table 222 of the read pattern (step S203). By referring to the movie configuration information 223 in this way, a photo movie with a different configuration is created from the second time onwards for a song for which a photo movie has been created before. This makes it possible to generate different movie expressions for the same song, providing the user with a variety of enjoyment.

[0097] FIG. 13 shows examples of photo movies 71, 72, and 73 generated in S203.

[0098] Photo movie 71 in FIG. 13(a) is generated using images 531, 532, 533, ... acquired in S113 as is, and the images displayed are changed when the composition of the music changes (time t1 when the A melody changes to the B melody, time t2 when the B melody changes to the chorus, etc.).

[0099] The photo movie 72 in FIG. 13(b) is produced by applying the effect template 221 to images 531, 532, 533, ... obtained in S113. The images are changed when the music composition changes, and predetermined effects are applied to the images in each section (t0~t1, t1~t2, t2~).

[0100] In the example of FIG. 13(b), the effect "monochrome to color" is applied to image 531 in the section t0 to t1 of the "A melody," and the monochrome image 531 is gradually changed to the color image 531. Next, at time t1 when the A melody changes to the B melody, the image is switched and the effect is also changed to "shrink and rotate." In the section t1 to t2, image 531 is used in the background and image 532 in the foreground, and an animation is generated in which image 532 in the foreground shrinks and rotates. Next, at time t2 when the B melody changes to the chorus, the image is switched. The effect remains "shrink and rotate," but in the section t2 onwards, image 531 is used in the background and image 533 in the foreground, and an animation is generated in which image 533 in the foreground shrinks and rotates.

[0101] Photo movie 73 in FIG. 13(c) was generated using images 534, 536, 539, ... acquired in S113 as background images and image materials 535, 537, 540, 541 generated in S114 as foreground images, with the background image changing when the music composition changes and predetermined effects being applied to the foreground images in each section (t0~t1, t1~t2, t2~).

[0102] In the example of FIG. 13(c), image 534 is displayed as the background during section t0-t1 of the A melody, and the effect "move" is applied to foreground image material 535, causing it to move as indicated by the arrow. Next, at time t1 when the A melody changes to the B melody, the background image is switched to image 536, and the effect "move + color change" is applied to foreground image material 537, generating an animation in which the image changes color and moves as indicated by the arrow. Next, at time t2 when the B melody changes to the chorus, the background image is switched to image 539, and the foreground images are also changed to image materials 540 and 541. During section t2-, image materials 540 and 541 are displayed alternately in sync with the timing of the bass drum sound.

[0103] When the photo movie generating device 2 generates a photo movie, it links the movie configuration information 223 of the generated photo movie with the music ID 61 and stores it in the storage unit 202. As shown in Fig. 14, the movie configuration information 223 is information that links with the music ID 61 and records the music configuration, time, effect template, number of images used, and file names of the images used.

[0104] The example in Fig. 14 is movie configuration information 223 for a photo movie that uses a song with a song ID of "00001" as background music. The duration of the section of the song configuration "intro" is "0-15", and the effect template "monochrome to color" is applied. The number of images used in this effect template is one, and the image file name used is "image15".

[0105] The duration of the "A-melody" section of the song is "15-50", and the effect template "Enlarge Rotate" is applied. This effect template uses two images, and the image file names used are "image8" for the foreground and "image2" for the background.

[0106] In addition, the duration of the "B melody" section of the song is "50-70", and the effect template "shrink and rotate" is applied. This effect template uses two images, and the image file names used are "image8" for the foreground and "image17" for the background.

[0107] In addition, the duration of the "chorus" section of the song is "70-120", and the effect template "Image Switch" is applied. This effect template uses five images, and the file names of the images used are "image10" for the first image, "image11" for the second image, "image12" for the third image, "image13" for the fourth image, and "image14" for the fifth image.

[0108] Additionally, the times (hours) of the song composition "bass drum timing" are "70, 110, 111, 112, 113," and the effect template "image display" is applied at these times. This effect template uses six images, and the image file names used are "image15" for the background, and the image "image1" is displayed at the first (1st) bass drum timing, the image "image2" at the second (2nd) bass drum timing, the image "image3" at the third (3rd) bass drum timing, the image "image4" at the fourth (4th) bass drum timing, and the image "image5" at the fifth (5th) bass drum timing.

[0109] The items "music composition" and "time" in the movie composition information 223 correspond to the acoustic feature data 64 of the music. The item "effect template" is obtained from the effect correspondence table 222 shown in Fig. 11, and the item "number of images used" is obtained from the effect template 221 shown in Fig. 10. Which effect correspondence table 222 to use is decided by the photo movie generating device 2 executing the process of Fig. 12. Also, the item "file name of image used" is arbitrarily decided by the photo movie generating device 2 from the image data 53 obtained in S113 and the image material generated in S114.

[0110] Returning to the description of Fig. 8, when the photo movie is generated by the process of step S115, the photo movie generating device 2 transmits the generated photo movie to the user terminal 3 in association with the specified music ID 61 (step S116).

[0111] When the user terminal 3 receives the photo movie sent from the photo movie generating device 2 (step S117), it plays back the photo movie.

[0112] Here, the user terminal 3 requests the music distribution server 6 to transmit the audio source data 62 of the above-mentioned music ID 61 used in the photo movie (step S118). When the music distribution server 6 receives the transmission request for the audio source data 62 from the user terminal 3 (step S119), it transmits the requested audio source data 62 of the music ID 61 to the user terminal 3 (step S120).

[0113] When the user terminal 3 receives the sound source data 62 (step S121), it plays the photo movie in synchronization with the sound source data 62 (step S122). The user terminal 3 plays the photo movie in synchronization with the sound source data 62, thereby playing a photo movie in which the screen changes in time with the music.

[0114] As described above, in the photo movie generating system 1 of this embodiment, when a request to generate a photo movie is sent from the user terminal 3 to the photo movie generating device 2, the photo movie generating device 2 acquires images of the user taken at photo spots A, B, C, ... from the image storage 5 or the like, generates a photo movie from the images to be played along with music related to photo spots A, B, C, ..., i.e., theme parks, and sends it to the user terminal 3. Since the photo movie combines images taken at a theme park with music related to the theme park, it is possible to strongly remind the user of the theme park, allowing the user to fully enjoy their memories.

[0115] Furthermore, the photo movie generating device 2 of this embodiment can generate impressive photo movies in which the screen changes in accordance with the characteristics of the above-mentioned music, and can generate image materials to be used in the photo movie from the user's images, thereby providing the user with a more interesting and enjoyable photo movie.

[0116] Furthermore, the photo movie generating device 2 accumulates and stores movie configuration information 223 for music for which a photo movie has been generated previously, and generates a photo movie with a different configuration, so that it can provide photo movies with different expressions for the same music, thereby providing the user with a variety of enjoyment.

[0117] Furthermore, the photo movie generating device 2 generates a photo movie without music and plays it synchronously with the sound source data 62 on the user terminal 3 side, thereby making it possible to reduce the file size since the sound source data 62 is not added to the photo movie, thereby reducing storage capacity and communication capacity.

[0118] Furthermore, since the user terminal 3 can specify music and images to be used in the photo movie, a photo movie can be generated using music and images that the user likes.

[0119] However, the present invention is not limited to the above embodiment. For example, in this embodiment, the photo movie generating device 2 transmits a photo movie without music to the user terminal 3, but a photo movie with music may be transmitted to the user terminal 3. A photo movie with music is a photo movie to which audio data 62 of a piece of music specified by the user terminal 3 is added. In other words, it is a photo movie in which the audio data 62 is played together with the video.

[0120] In this case, the photo movie generating device 2 receives and acquires the audio data 62 corresponding to the music ID 61 specified by the user terminal 3 from the music distribution server 6 in S109, and generates a photo movie in S115 in which the music and video of the audio data 62 are synchronously played back. The user terminal 3 plays back this photo movie, and the playback method may be streaming playback or download playback.

[0121] In addition, in this embodiment, the music distribution server 6 stores the acoustic feature data 64 obtained by advance analysis, but the photo movie generation device 2 may perform acoustic analysis each time it acquires sound source data 62 to obtain the acoustic feature data 64.

[0122] Also, the operation screen 8 in Fig. 9 is an example, and the operation screen for specifying images, etc. is not limited to this. For example, in Fig. 9, the image, music, and photo movie type are specified on the same screen, but they may be specified on different screens.

[0123] Furthermore, the photo movie production system 1 of the present invention can be applied not only to theme parks, but also to tourist spots, event venues, etc., and can be applied to any place or area where photography is done at a photography spot, without any particular limitations.

[0124] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0125] 1: Photo movie generation system 2: Photo movie generator 3: User terminal 4a, 4b, 4c: Photographic equipment 5: Image storage 6:Music distribution server 7: Network 21: Music Acquisition Department 23: Image acquisition unit 24: Image material generation section 25: Photo movie generator 31: Song selection section 32: Image specification section 33: Photo movie creation request sending unit 34: Receiving unit 35: Sound source data acquisition section 36: Playback processing section 51: User ID 52: Image ID 53: Image data 60: Song Database 61: Song ID 62: Sound source data 63: Song Information 64: Acoustic feature data 71, 72, 73: Photo Movie 221: Effect Template 222:Effects table 223: Movie configuration information

Claims

1. A photo movie production system in which a photo movie production device and a user terminal are communicably connected via a network, The photo movie generating device comprises: storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit; an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal; Equipped with The photo movie generating means includes: generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means; When generating a new photo movie to be played along with the music, the photo movie generation system determines whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, generates a new photo movie with a pattern different from the pattern of the movie configuration information.

2. The photography spot is set up in a theme park, a tourist spot, or an event venue, The photo movie production system includes: An image storage that stores an image taken by a user at the photography spot in association with the user ID of the user; A photography device is provided at the photography spot and takes a photograph of the user; Further comprising: The photographing device reads the user ID when photographing the user, The photo movie generation system described in claim 1, characterized in that when a user visits a theme park, tourist attraction, or event venue multiple times, the image storage stores images taken with the photography device during each visit, linked to the same user ID.

3. The photo movie generation system of claim 1 or claim 2, characterized in that the image acquisition means acquires images provided by businesses of the visited locations in addition to images taken by the user at photo spots in the user's visited locations.

4. the photo movie generating means generates the photo movie without adding a sound source of the music, associates the photo movie with identification information of the music, and transmits the photo movie to the user terminal; A photo movie production system as described in any one of claims 1 to 3, characterized in that the user terminal is equipped with a playback means for acquiring audio data of a song corresponding to the identification information and playing the acquired audio data in synchronization with the photo movie.

5. The photo movie generating device further comprises image material generating means for extracting an object from the image acquired by the image acquiring means and generating an image material; 5. The photo movie generating system according to claim 1, wherein the photo movie generating means generates the photo movie using the image material generated by the image material generating means.

6. The photo movie production system described in any one of claims 1 to 5, characterized in that the user terminal is equipped with a music designation means for accepting a designation of music to be used in the photo movie and transmitting music designation information to the photo movie production device together with a request to generate the photo movie.

7. The photo movie production system described in any one of claims 1 to 6, characterized in that the user terminal is equipped with an image designation means for accepting designation of images to be used in the photo movie and transmitting image designation information to the photo movie production device together with a request to generate the photo movie.

8. A photo movie generating device communicably connected to a user terminal via a network, storing movie configuration information including effect patterns, which indicates a configuration of changes in the screen of the generated photo movie in accordance with the music, in a storage unit; an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal; Equipped with The photo movie generating means includes: generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means; When generating a new photo movie to be played along with the music, the photo movie generating device determines whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, generates a new photo movie with a pattern different from the pattern of the movie configuration information.

9. A photo movie generating method in a photo movie generating system in which a photo movie generating device and a user terminal are communicably connected via a network, comprising: The photo movie generating device stores movie configuration information including effect patterns, which indicates a configuration of changes in screens in sync with music in the generated photo movie, in a storage unit. receiving a request to generate a photo movie from the user terminal, acquiring images taken by the user at a photography spot to be used in the photo movie; receiving a request to generate a photo movie from the user terminal, acquiring acoustic feature data of a song related to the photography spot; generating a photo movie using the images to be played on the user terminal together with music related to the shooting spot, and transmitting the photo movie to the user terminal; Run The photo movie generating device comprises: generating a photo movie in which the screen changes in accordance with music based on the acoustic feature data; A photo movie generation method characterized in that, when generating a new photo movie to be played along with the music, it is determined whether the movie configuration information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie configuration information exists, a new photo movie having a pattern different from the pattern of the movie configuration information is generated.

10. a photo movie generating device that is communicably connected to a user terminal via a network and stores movie configuration information including effect patterns that indicate a configuration of screen changes in sync with music in a generated photo movie; an image acquisition means for acquiring images taken by a user at a photography spot for use in the photo movie when a request for generating the photo movie is received from the user terminal; a music acquisition means for acquiring acoustic feature data of music related to the photography spot when a photo movie generation request is received from the user terminal; a photo movie generating means for generating a photo movie to be played on the user terminal together with music related to the shooting spot using the images acquired by the image acquiring means, and transmitting the photo movie to the user terminal; A photo movie generating device comprising: The photo movie generating means includes: generating a photo movie in which a screen changes in accordance with music based on the acoustic feature data acquired by the music acquisition means; A program for functioning as a photo movie generating device, characterized in that when generating a new photo movie to be played along with the music, the program determines whether the movie composition information of a previously generated photo movie to be played along with the same music exists in the memory unit, and if the movie composition information exists, generates a new photo movie with a pattern different from the pattern of the movie composition information.

Images