Amusement image capture system

Abstract

To provide techniques that enable the release of a video generated by an amusement image capture device, in a manner that is not unwanted by a user.SOLUTION: An amusement image capture system comprises: an image capture unit that captures an image of a user as a subject; a frame selection unit that selects a representative frame from a plurality of frames constituting a moving image obtained by the capture; and a display unit that displays the selected representative frame in association with the moving image. The frame selection unit selects as the representative frame a frame other than an initial frame of the moving image based on a set condition.SELECTED DRAWING: Figure 8

Description

[Technical field]

[0001] The present invention relates to a display of game images, and in particular to a display technique for captured moving images. [Background technology]

[0002] A play image capture device called Purikura (registered trademark) is extremely popular, especially among people in their teens and twenties. Users take a picture of themselves and decorate the captured image in various ways on an editing display. The decorated image is printed on a sticker mount (see Patent Document 1). The play image capture device also uploads the captured image to an external server. Users can also download edited images by accessing the server from a user terminal such as a smartphone.

[0003] In recent years, a game image capturing device has been proposed that captures not only still images of a user but also moving images and uploads them to a server (see Patent Document 2). Users can download the moving images from the server and publish them on SNS (Social Networking Service), further increasing the utility value of the game image capturing device. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2016-184287 A [Patent Document 2] JP 2019-115029 A Summary of the Invention [Problem to be solved by the invention]

[0005] The video images generated by the game image capture device are published in the form of thumbnails so that the contents can be understood at a glance. That is, when a video image is generated, one of the frames that make up the video image, such as the first frame, is automatically selected as the thumbnail. When the video image is posted on an SNS, a thumbnail of the video image is displayed on the user's posting screen. For this reason, it is important that the thumbnail is an image that looks good to the user.

[0006] The present invention has been made based on the above-mentioned point of view, and its main objective is to provide technology that makes it possible to make moving images generated by a game image capturing device public in a manner that does not displease the user. [Means for solving the problem]

[0007] A game image capturing system according to an embodiment of the present invention includes a capturing unit that captures a user as a subject, a frame selection unit that selects a representative frame from a plurality of frames that constitute a moving image obtained by capturing the user, and a display unit that displays the selected representative frame in association with the moving image. The frame selection unit selects a frame different from the first frame of the moving image as the representative frame based on a set condition.

[0008] Another aspect of the game image capturing system includes a capturing unit that captures the user as a subject, a representative candidate extraction unit that extracts representative candidates from a plurality of frames that make up the moving image obtained by capturing, an input unit that accepts selection input by the user, and a display unit that displays the frame selected by the user from the representative candidates as a representative frame in association with the moving image. Effect of the Invention

[0009] According to the present invention, a technique can be provided that makes it possible to make videos generated by a game image capture device public in a manner that does not displease the user. [Brief description of the drawings]

[0010] [Figure 1] FIG. 1 is an overall configuration diagram of a game image capturing system. [Diagram 2] FIG. 2 is a perspective view of the appearance of the game image capturing device. [Diagram 3] FIG. [Figure 4] 13 is a flowchart showing the flow of play in the game image shooting device. [Diagram 5] FIG. 2 is a plan view of the game image capturing device seen from above. [Figure 6] FIG. 2 is a functional block diagram of the game image capturing device. [Figure 7] 4 is a time chart showing the process of the photographing process. [Figure 8] 1A to 1C are diagrams illustrating a method for generating a captured video and a method for adjusting image data. [Figure 9] FIG. 2 is a diagram illustrating an example of frames constituting an edited video. [Figure 10] FIG. 2 is a diagram illustrating an example of frames constituting an edited video. [Figure 11] FIG. 13 is a diagram illustrating an example of a download screen displayed on a user terminal. [Figure 12] 5 is a flowchart showing the image data adjustment process in S18 of FIG. 4. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] The game image shooting device in this embodiment includes a shooting booth and an editing booth. The user takes a photo of himself / herself in the shooting booth. Hereinafter, an image of the subject (user) taken in the shooting booth is referred to as a "taken image". In the shooting booth, the game image shooting device applies image correction to the shot image, such as whitening the skin and making the hair shiny (hereinafter, such image correction is referred to as "pre-correction"). In the shooting booth, the game image shooting device in this embodiment takes not only still images but also videos. Of the shot images, those acquired as videos are referred to as "shot videos", and those acquired as still images are referred to as "shot still images".

[0012] The captured image that has been pre-edited is sent to the editing booth. In the editing booth, the user can decorate the captured image as desired, such as by attaching a stamp image or doodling with a stylus pen (such image decoration will hereinafter be referred to as "decoration processing" or "editing"). The captured image that is the subject of editing in the editing booth is called the "target image." Among the target images, those that have been edited from a captured video are called "target video images," and those that have been edited from a captured still image are called "target still images." The target image after editing is called the "edited image," and among the edited images, those that have been edited from a target video are called "edited video images," and those that have been edited from a target still image are called "edited still images."

[0013] The game image capture device prints the edited still images on a sticker mount (external medium). The game image capture device also uploads the edited moving images and edited still images to a server (hereinafter also referred to as a "dedicated server"). Users can also download the edited images (edited moving images and edited still images) by accessing the server from a user terminal such as a smartphone. Users can also upload the downloaded edited images to an SNS server (hereinafter referred to as an "SNS server") and make them public.

[0014] The game image photographing device of this embodiment adjusts image data so that thumbnails that satisfy a predetermined evaluation criterion are displayed for the edited video. This prevents undesired thumbnails from being displayed on the selection screen for edited videos when the user accesses a dedicated server. In addition, undesired thumbnails are also prevented from being displayed when the edited video is made public on an SNS. Details of this will be described below.

[0015] FIG. 1 is a diagram showing the overall configuration of a game image capturing system 200. As shown in FIG. In the game image photographing system 200, the game image photographing device 100 is connected to a server 202 and a user terminal 206 (mobile device) via the Internet 204. A user photographs himself / herself using the game image photographing device 100. The game image photographing device 100 obtains a photographed image including a subject (user), applies advance correction to the photographed image as necessary, and accepts decorative processing by the user.

[0016] The edited image after decoration processing is uploaded from the game image capturing device 100 to the server 202 together with the user ID. The server 202 is a dedicated server that manages images generated by the game image capturing device 100, and stores the user ID, capture date and time, and edited images (edited moving images and edited still images) in association with each other. The user can download the edited image from the server 202 by accessing the server 202 by specifying the user ID. The user can also upload the downloaded edited image to an SNS server (not shown) and make it public.

[0017] FIG. 2 is a perspective view of the appearance of the game image capturing device 100. As shown in FIG. The game image capturing device 100 comprises a reception device 102, a photo booth 104, an editing booth 106, and a printing device 110. A user reserves the use of the game image capturing device 100 at the reception device 102. The user inputs a user ID at the reception device 102 and pays a play fee in cash or electronic money. The reception device 102 issues a reservation number to the user. The reception device 102 displays the number of people waiting and the reservation number to be called on the reception monitor 108. When the user's reservation number is displayed on the reception monitor 108, the user enters the photo booth 104.

[0018] A user takes a photograph in the photographing booth 104. After finishing photographing, the user moves to the editing booth 106. An editing device 126 is installed in the editing booth 106. The editing device 126 includes an editing monitor 120. A target image (a photographed image to be subjected to decorative processing) is displayed on the editing monitor 120. In this embodiment, the editing monitor 120 is formed by a capacitive touch panel. Two stylus pens 122a and 122b are provided for one editing monitor 120. The user decorates the target image using the stylus pen 122 and his / her finger. After finishing editing, the printing device 110 prints the edited still image on a sticker mount. In addition, the editing device 126 uploads the edited image to the server 202.

[0019] FIG. 3 is a perspective view showing the appearance of the photography booth 104. As shown in FIG. The photographing booth 104 is shielded from the outside by a curtain 124. A photographing device 128 is installed in the photographing booth 104. The photographing device 128 includes three lights 116 that illuminate the subject user from three directions, a camera 118 for acquiring photographed still images and photographed moving images, a first photographing monitor 112, and a second photographing monitor 114.

[0020] The first photographing monitor 112 displays a preview of the photographed images (photographed moving images and photographed still images) of the user. The second photographing monitor 114 displays photographed images (hereinafter referred to as "sample images") of a person other than the user who is the subject, for example, a model. The user takes a pose of their choice in front of the camera 118 while referring to the sample images displayed on the second photographing monitor 114. Various explanations regarding the photographing method are also displayed on the second photographing monitor 114. As described above, photographed moving images and photographed still images can be acquired in the photographing booth 104.

[0021] Furthermore, the first capture monitor 112 displays a live view image (full-length mirror image) that allows the user to check his or her own appearance, as well as a countdown image that indicates the timing of capture.

[0022] Fig. 4 is a flow chart showing the flow of play in the game image photographing device 100. Fig. 5 is a plan view of the game image photographing device 100 as seen from above. When the user reserves the use at the reception device 102, the reception device 102 executes reception processing (S10). After the reception, the user enters the photo booth 104. The photographing device 128 acquires the user's photographed images (S12). In the photo booth 104, the user can acquire a predetermined number of photographed images. A photographed video and a photographed still image are acquired at the same time, and a pair of a photographed video and a photographed still image is counted as one set of photographed images. In this embodiment, five sets of photographed images (a set of a photographed video and a photographed still image) are acquired. When taking a photograph, the user can select a method of pre-correction from a plurality of pre-correction menus provided by the photographing device 128. The user can also select the background color of the photographed image at the photographing device 128. The photographing device 128 executes pre-correction every time a photographed image is acquired. The pre-correction menu may be selectable at the reception device 102 before playing.

[0023] After shooting, instruction information instructing one of the two editing booths 106 is displayed on the first shooting monitor 112. The user moves to one of the two editing booths 106a, 106b that is specified by the first shooting monitor 112. The shot image (target image) is transferred from the shooting device 128 to the editing device 126. The user applies decorative processing to the shot image (target image) transferred in the editing device 126 (S14).

[0024] After editing, the editing device 126 sends the edited image to the printing device 110, which prints the edited image on a sticker mount (S16). Furthermore, prior to uploading to the server 202, the editing device 126 executes an image data adjustment process for setting a thumbnail of the edited image (S18). Details of this image data adjustment process will be described later. Thereafter, the editing device 126 uploads the edited image to the server 202 via the Internet 204 (S20). A user can download the edited image from the server 202 to a user terminal 206 by accessing the server 202.

[0025] In general, editing takes longer than photographing. In the game image photographing device 100 of this embodiment, two editing booths 106a, 106b (editing devices 126a, 126b) are provided for one photographing booth 104, thereby increasing customer turnover.

[0026] FIG. 6 is a functional block diagram of the game image capturing device 100. As shown in FIG. Each component of the game image capture device 100 is realized by hardware including computing units such as a CPU (Central Processing Unit) and various coprocessors, storage devices such as memory and storage, and wired or wireless communication lines connecting them, and software stored in the storage devices and supplying processing instructions to the computing units. The computer program may be composed of device drivers, an operating system, various application programs located at higher layers, and libraries that provide common functions to these programs. As described above, the game image capture device 100 is composed of a collection of the capture device 128, the reception device 102, the editing device 126, and the printing device 110. Each block described below shows a functional block rather than a hardware unit configuration.

[0027] The game image photographing device 100 includes a user interface processing unit 130, a data processing unit 132, a communication unit 134, and a data storage unit 136. The user interface processing unit 130 is responsible for user interface processing via the reception monitor 108, the editing monitor 120, and the like. The communication unit 134 is responsible for communication processing with the server 202 via the Internet 204. The communication unit 134 can also directly communicate with the user terminal 206 by short-range wireless communication such as Wi-Fi (registered trademark). For example, an edited image may be directly transmitted from the communication unit 134 to the user terminal 206. The data storage unit 136 stores various data. The data processing unit 132 executes various processes based on the data acquired by the communication unit 134 and the data stored in the data storage unit 136.

[0028] The user interface processing unit 130 includes an input unit 140 and an output unit 142. The input unit 140 accepts operational input by a user. The input unit 140 detects operational input by the user to a touch panel installed in the reception device 102, the photographing device 128, the editing device 126, etc. The input unit 140 also detects operational input by the stylus pen 122. The input unit 140 also detects a user ID in the reception device 102.

[0029] The output unit 142 presents information to the user. The output unit 142 executes various output processes such as image display on the reception device 102, the photographing device 128, the editing device 126, etc., as well as audio output and print output.

[0030] The output unit 142 includes a display unit 144. During shooting, the display unit 144 displays a live view image (full-length mirror image) of the user (subject) on the first shooting monitor 112. The user performs shooting while checking his / her own image in the live view image as if looking at a mirror.

[0031] The data processing unit 132 includes an image capture processing unit 150, an edit processing unit 152, a reception processing unit 154, a print processing unit 156, and an image processing unit 158. The image capture processing unit 150 controls settings related to lighting, such as turning on / off, light intensity, and color of the light 116 in the image capture booth 104, and controls image capture by the camera 118.

[0032] In the shooting booth 104, still images and videos are shot with the user as a subject. The shooting processing unit 150 includes a video shooting unit 160 and a still image shooting unit 162. The video shooting unit 160 controls video shooting, and the still image shooting unit 162 controls still image shooting. In this embodiment, videos are shot before and after the still image shooting timing. Hereinafter, the video recorded before the shooting timing is called the "first half video", and the video recorded after the shooting timing is called the "second half video". The video shooting unit 160 generates a shot video by connecting the first half video and the second half video. The still image shooting unit 162 notifies the user of the still image shooting timing by instructing the output unit 142 to display a countdown and notify by voice.

[0033] The output unit 142 displays the photographed image as a target image on the editing monitor 120 (a capacitive touch panel). The editing processing unit 152 decorates the target image according to operational input from the user. The reception processing unit 154 manages the payment of play fees by users at the reception device 102. The reception processing unit 154 guides users who have made reservations to the photography booth 104 every time photography work is completed in the photography booth 104. The print processing unit 156 controls printing of the edited image in the printing device 110.

[0034] The image processing unit 158 ​​executes image data adjustment processing for setting a thumbnail to be associated with the edited video. The image processing unit 158 ​​includes an image recognition unit 164, a frame selection unit 166, and a data adjustment unit 168. The image recognition unit 164 recognizes a face image in a frame constituting the edited video. The frame selection unit 166 extracts a frame suitable for a thumbnail as a representative frame from a plurality of frames constituting the edited video based on the recognized face image. The data adjustment unit 168 adjusts image data so as to associate the representative frame with the edited video, and transmits the image data to the server 202. A specific example of this image data adjustment processing will be described later.

[0035] The communication unit 134 includes a receiving unit 170 and a transmitting unit 172. The receiving unit 170 receives data from an external device such as a server 202 or a user terminal 206. The transmitting unit 172 transmits data to the external device. The transmitting unit 172 transmits an edited image to be uploaded to the server 202.

[0036] 7 is a time chart showing the process of the image capture process, where the horizontal axis indicates the passage of time. When the user touches a shooting start button (not shown) in the shooting booth 104, shooting begins. At this time, a pose instruction image that prompts the user to pose for still image shooting and a sample still image are displayed on the second shooting monitor 114. The pose instruction image is, for example, an image that indicates a notification by text information.

[0037] A countdown to still image capture begins 4 seconds after capture begins. This countdown informs the user of the timing to capture a still image. The count starts 3 seconds before the still image is captured, and the still image is captured when the count reaches 0. Prior to this, video recording begins 1 second before the countdown begins. By starting video recording during the countdown to still image capture, the movements and state of the user until they strike a pose to capture the still image can be casually captured.

[0038] The video recording is stopped once before a still image is captured. After the video recording is stopped, the still image is captured. After the still image is captured, the video recording is resumed. This is to prevent the video from being affected by the flash of the strobe.

[0039] When the recording of the moving images is completed, the captured still images and captured moving images are pre-corrected. The pre-corrected captured still images and captured moving images are displayed on the first capture monitor 112. Note that the first capture monitor 112 displays a live view image from the start to the end of capture.

[0040] FIG. 8 is a diagram showing a method for generating a captured video and a method for adjusting image data. The upper part of the figure shows the video capture period (video recording period) based on the still image capture timing. The middle part of the figure shows the frame acquisition process from the start of video capture in relation to the upper part. The lower part of the figure shows the frame composition of the captured video to be generated.

[0041] In this embodiment, as described above, video recording starts 4 seconds before the still image capture timing, and the countdown starts 3 seconds before. Statistically, it is thought that the user is most motivated to capture a still image, that is, about 1 second before the still image capture timing, when they try to make a good expression (not show a lazy expression). For this reason, the strobe is fired 1 second before the still image capture timing, and the shutter of the camera 118 is released at the still image capture timing.

[0042] The video recording is stopped once before the still image is captured, and resumed after the still image is captured. In this example, the recording of the first half of the video is stopped immediately before the still image is captured. The recording of the second half of the video starts 0.5 seconds after the still image is captured, and continues for 1 second. The reason for leaving a certain time before the video recording resumes is to eliminate the influence of remaining reflected light.

[0043] In the illustrated example, the frame rate of the video is set to 30 fps, and 120 frames of the first half video and 30 frames of the second half video are obtained. Of these, a predetermined number of frames at the end of the first half video are removed as a frame-dropping range, and the first half video and the second half video are combined to obtain a captured video. The frame-dropping range corresponds to a "preparation period" for still image capture, and is set in advance as a frame range that is not suitable for video capture, such as a range in which a flash may be captured in the video. By removing the frame-dropping range in this way, the quality of the image during video playback can be maintained. This captured video does not include images captured as still images, so smooth video playback can be achieved.

[0044] In this embodiment, up to the 90th frame, which is one second before the still image capture timing, is used in the first half of the video, and the 91st frame and after are removed. All frames are used for the second half of the video. As described above, the user's facial expression is at its best when the countdown reaches one second, so this is to avoid missing any recording of that range. In the illustrated example, a captured video of 120 frames is generated by combining the first half of the video and the second half of the video. Editing processing (decoration processing) is reflected in this captured video.

[0045] The image processing unit 158 ​​sets a thumbnail to be displayed when the edited video thus obtained is uploaded to the server 202. The frame selection unit 166 extracts a frame suitable for a thumbnail as a representative frame from the multiple frames constituting the edited video. However, as shown in the lower part of Fig. 8, the range that can be selected as a representative frame (hereinafter referred to as the "frame selection range") is the range constituting the first half of the video. Since it is considered that the user will look careless after capturing a still image, the range of the first half of the video before capturing a still image is selected.

[0046] The image recognition unit 164 sequentially extracts frames of the edited video starting from the first frame, recognizes the user's facial image in each frame, and identifies the position and expression of the face. The image recognition unit 164 determines whether the user's entire face is included in the angle of view in each frame and whether the user's facial expression satisfies a predetermined evaluation criterion.

[0047] 9 and 10 are diagrams showing examples of frames constituting an edited video. Fig. 9 shows an example in which the user's facial expression satisfies the evaluation criteria, and Fig. 10 shows an example in which the user's facial expression does not satisfy the evaluation criteria. In the illustrated example, multiple (two) users are captured, and as shown in FIG. 9, if the entire face of each user is within the angle of view and the eyes of each user are wide open (eye opening is a predetermined value or more), the evaluation criteria are met. As shown in FIG. 10, if any of the users' eyes are half-closed, rolled up, or closed, the evaluation criteria are not met. Also, if any of the users' faces are partially cut off (protruding from the angle of view), the evaluation criteria are not met. The frame selection unit 166 sets a frame that meets the evaluation criteria as a representative frame.

[0048] In other words, the representative frame's "setting conditions" include satisfying a predetermined evaluation criterion. In this embodiment, the conditions for satisfying the evaluation criterion are that the degree of opening of the eyes of all users is equal to or greater than a reference value, that the position of the pupil is within a reference range of the white of the eye, and the like, on the premise that the entire faces of all users appearing in the edited video are included in the video.

[0049] Regarding the evaluation criteria, whether or not the entire face is contained within the angle of view can be determined, for example, as follows. When a face is detected in image recognition, the determination may be made based on whether both eyes and a mouth are included. Alternatively, the determination may be made based on whether the ratio of the size of the face area to the entire screen is equal to or greater than a predetermined value.

[0050] The eye opening can be determined, for example, as follows. First, an area corresponding to a face image (face area) is detected from the frame. From that face area, an area corresponding to the right eye and an area corresponding to the left eye are detected. The opening degree of the right eye area (degree of eye opening) is then quantified. For example, a right eye image with a maximum opening degree and a right eye image with a minimum opening degree may be prepared, and the opening degree of the right eye area may be quantified from the vertical width of the pupil area in the right eye area to be determined. The same applies to the opening degree of the left eye area.

[0051] If the first frame of the edited video satisfies the evaluation criteria, the frame selection unit 166 sets the first frame as a representative frame and a thumbnail. If the first frame does not satisfy the evaluation criteria, the next frame and subsequent frames are judged in sequence, and if a frame satisfies the evaluation criteria, it sets the representative frame and thumbnail. The data adjustment unit 168 adjusts the image data to include setting information indicating the correspondence between the edited video and its thumbnail. This "setting information" may be, for example, information that associates the ID of the edited video with the number of the frame corresponding to the thumbnail of the edited video. The transmission unit 172 transmits the adjusted image data to the server 202. When a user accesses the server 202 to download the edited video, a display unit (not shown) of the server 202 displays a thumbnail corresponding to the edited video on a download screen.

[0052] FIG. 11 is a diagram showing an example of a download screen displayed on the user terminal 206. As shown in FIG. By accessing the server 202, the user can display a download screen 210 on the user terminal 206. On the download screen 210, thumbnails corresponding to the edited moving images uploaded from the game image shooting device 100 are displayed. By selecting one of the thumbnails, the user can download the corresponding edited moving image to the user terminal 206.

[0053] FIG. 12 is a flowchart showing the image data adjustment process in S18 in FIG. In the image data adjustment process, the image processing unit 158 ​​acquires the first frame of the edited video (S30). The image recognition unit 164 performs image recognition on the first frame to identify the user's face image (S32) and to identify the facial expression (S34). Then, it is determined whether the above-mentioned evaluation criteria are satisfied (S36).

[0054] If the first frame of the edited video meets the evaluation criteria (Y in S36), the frame selection unit 166 sets the first frame as the thumbnail (S38). If the first frame does not meet the evaluation criteria (N in S36) and the next frame is within the frame selection range (Y in S40), the image processing unit 158 ​​acquires the next frame (S44) and returns to S32.

[0055] In this way, the image recognition unit 164 sequentially judges the next frame onwards, and if there is a frame that satisfies the evaluation criterion (Y of S36), it sets it as the thumbnail (S38). If none of the frames in the frame selection range meets the evaluation criterion (N of S40), the first frame is set as the thumbnail (S44). The data adjustment unit 168 updates the image data to include setting information for the edited video and its thumbnail (S46).

[0056] The edited video with the thumbnail set in this manner is uploaded to the server 202. The transmission unit 172 of the game image shooting device 100 may upload the edited video together with the user ID to the server 202. The server 202 stores the edited images (edited video, edited still images) in association with the user ID.

[0057] A user can check edited videos stored in the server 202 by accessing the server 202 using a user ID. Thumbnails associated with each edited video are displayed on a download screen displayed by the server 202. A user can download the edited video from the server 202 to the user terminal 206 by selecting any of the thumbnails. The server 202 transmits to the user terminal 206 the edited image (edited video, edited still image) that is associated with the user ID and selected by the user. A user can also publish the edited video by uploading the downloaded edited video to an SNS server.

[0058] The game image capturing system 200 has been described above based on the embodiment. According to this embodiment, a thumbnail of the captured video generated by the game image capturing device 100 is set by extracting a frame that satisfies a predetermined evaluation criterion. The conditions for satisfying this evaluation criterion include that the entire face image of the user is contained within the angle of view, and that the user's facial expression is good (satisfying the predetermined setting criterion). Therefore, when the captured video is uploaded to the server 202, it is possible to prevent an image that is not desirable to the user from being displayed as a thumbnail. In addition, when the captured video is uploaded to an SNS, it becomes easier to publish the captured video in a form that meets the user's wishes.

[0059] In addition, if the captured video includes multiple users, a frame that satisfies the evaluation criteria for all users is set as the thumbnail. This makes it possible to provide a service that takes into consideration all users who download the captured video from the server 202.

[0060] The present invention is not limited to the above-mentioned embodiment and modified examples, and the components can be modified without departing from the gist of the present invention. Various inventions can be formed by appropriately combining multiple components disclosed in the above-mentioned embodiment and modified examples. In addition, some components can be deleted from all the components shown in the above-mentioned embodiment and modified examples.

[0061] Although the play image capturing system 200 has been described as being configured by the play image capturing device 100, the server 202, and the user terminal 206, particularly the play image capturing device 100 and the server 202, some of the functions of the play image capturing device 100 may be realized by the server 202, or some of the functions of the server 202 may be assigned to the play image capturing device 100. Also, a third device other than the play image capturing device 100 and the server 202 may assume some of the functions of these devices. How to allocate a plurality of functions required to realize the present invention to one or more pieces of hardware may be determined in consideration of the processing capacity of each piece of hardware, the specifications required for the play image capturing system 200, and the like.

[0062] [Variations] In the above embodiment, an example has been shown in which the image data adjustment process is executed by the game image capturing device 100. In a modified example, the image data adjustment process may be executed by the server 202. Specifically, the server 202 is equipped with an image processing unit. The game image capturing device 100 uploads the generated edited images (edited moving images) directly to the server 202. The image processing unit of the server 202 may select representative frames from the edited moving images that satisfy a predetermined evaluation criterion, and display them as thumbnails.

[0063] Alternatively, dedicated application software for performing image data adjustment processing may be installed in the user terminal 206. When a user registers as a member with the server 202, this application software can be downloaded from the server 202 as member software.

[0064] The edited images (edited moving images) generated by the game image capturing device 100 are uploaded to the server 202 without any adjustment to the thumbnails. The user downloads the edited moving images to the user terminal 206. Alternatively, the edited images generated by the game image capturing device 100 may be sent directly to the user terminal 206. The user terminal 206 uses the member software to perform image data adjustment processing on the edited moving images similar to that of the above embodiment, and displays the adjusted thumbnails on the screen. The user can upload the image data with the adjusted thumbnails to the SNS server.

[0065] In the above embodiment, a representative frame is selected from the captured video and displayed as a thumbnail. In a modified example, a captured still image may be selected as a representative frame and displayed as a thumbnail of the captured video. Since a countdown is performed until the still image is captured, the user's facial expression is likely to be at its best at the time of capturing the still image. This makes it easier to publish a thumbnail that meets the user's wishes.

[0066] In the above embodiment, an example has been shown in which image data adjustment processing is performed on an edited video that reflects editing in an editing booth. In a modified example, a similar image data adjustment processing may be performed on a shot video that does not reflect editing (i.e., the video itself shot in the shooting booth), such as when editing was not performed in the editing booth. Although not mentioned in the above embodiment, when image data adjustment processing is performed on an edited video, the processing may be performed during the period the user stays in the editing booth. When image data adjustment processing is performed on a shot video that does not reflect editing, the processing may be performed during the period the user stays in the shooting booth.

[0067] Furthermore, the editing processing unit 152 may be configured to turn off the decorative processing from the edited image and return it to the original photographed image when instructed by the user. The image processing unit 158 ​​may perform the above-mentioned image data adjustment processing on the original photographed moving image.

[0068] In the above embodiment, an example was shown in which a representative frame was selected from a plurality of frames constituting a captured moving image, and the representative frame was displayed as a thumbnail to be associated with the moving image. In a modified example, the image processing unit may associate the selected representative frame with the moving image by moving the frame to the beginning. Alternatively, the representative frame may be associated with the moving image by replacing the beginning frame with the representative frame. By making a representative frame that satisfies a predetermined evaluation criterion the beginning frame, it becomes easier to make the screen that first attracts attention when playing the moving image match the user's intention.

[0069] When the image processing unit extracts the representative frame by sequentially determining from the first frame, the image processing unit may invalidate (prohibit display) frames prior to the representative frame in the captured video. The data adjustment unit may reconstruct the captured video (edit data) by deleting frames prior to the representative frame. By thus preventing frames prior to the representative frame from being displayed, the quality of the entire video that is subsequently played can be improved.

[0070] In the above embodiment, as the "setting condition", an example was shown in which it is judged whether the evaluation criterion is satisfied from the first frame (first frame) in order, and a frame that satisfies the predetermined evaluation criterion itself is selected as a representative frame. In a modified example, the degree of conformity with a predetermined evaluation criterion may be judged, and a frame whose degree of conformity is equal to or greater than a predetermined value may be selected as a representative frame. For example, the evaluation criterion may be quantified, and even if 100% of the evaluation criterion is not satisfied, a frame that satisfies a predetermined percentage or more of the evaluation criterion (for example, 75% or more) may be adopted as a representative frame.

[0071] A predetermined number (other than the first) frame from the beginning of the multiple frames may be set as the representative frame. The "predetermined number" is a value that makes the user's facial expression look good, which is set in advance based on statistics, empirical rules, etc. Alternatively, a frame that is a predetermined number of frames before the still image capture timing may be set as the representative frame. The "predetermined number" may be a frame that corresponds to a predetermined time before (for example, 1 second before) the still image capture timing. Again, a value that makes the user's facial expression look good is set in advance based on statistics, empirical rules, etc.

[0072] In the above embodiment, an example was shown in which the frame selection range is from the first frame of the first half video to just before the frame skip range (just before the preparation period). In a modified example, a predetermined range just before the frame skip range may be set as the frame selection range, and a representative frame may be selected from that range. Since the user's facial expression is likely to be at its best just before the frame skip range, it is highly likely that a representative frame that matches the user's intentions can be selected. Alternatively, a frame just before the frame skip range (the 90th frame in the above embodiment) may be selected as the representative frame. If there is statistical support, a specific range in the captured video may be set as the frame selection range, not just just before the frame skip range, and a representative frame may be selected from that range.

[0073] In the above embodiment, the selection of the representative frame is determined based on the degree of eye opening to determine whether the evaluation criteria are met, but the selection may be determined based on other factors, such as whether the corners of the mouth obtained by face recognition are turned up or whether the face is tilted.

[0074] In the above embodiment, the game image shooting device is exemplified as a device that shoots still images and videos in one shooting session, such as shooting a still image when video shooting is paused (in other words, shooting a video before and after shooting a still image). In a modified example, still images and videos may be shot separately. Alternatively, still images may be shot at a predetermined timing during video shooting without stopping video shooting. In these cases, image data adjustment processing similar to that in the above embodiment may be performed on the shot video images and edited video images.

[0075] In the above embodiment, when a captured video image includes multiple users, a frame that satisfies the evaluation criteria for all users is set as a thumbnail. In a modified example, a thumbnail may be set for each of the multiple users. For example, as shown in Figs. 9 and 10, assume that a first user and a second user take a video image together. Assume that the first user uses the game image capture device using the user ID of the first user.

[0076] In this case, for one captured video, the frame selection unit sets the frame selected by the above method for the first user as the first representative frame, while setting the frame selected by the above method for the second user as the second representative frame. The face image of the second user does not need to be taken into consideration when selecting the first representative frame. Similarly, the face image of the first user does not need to be taken into consideration when selecting the second representative frame.

[0077] The data adjustment unit adjusts the image data so that multiple thumbnails are associated with one captured video. That is, a first thumbnail is set with the first representative frame as the thumbnail, and a second thumbnail is set with the second representative frame as the thumbnail. The first thumbnail, the second thumbnail, and the captured video are associated with the user ID of the first user.

[0078] As a result, when the first user accesses the server 202 using his / her user ID, the first thumbnail and the second thumbnail are displayed. By selecting the first thumbnail, the first user can download the captured video associated with the first thumbnail. Also, by selecting both the first thumbnail and the second thumbnail, the first user can download the captured video associated with both thumbnails. The first user can also provide the downloaded captured video that corresponds to the second thumbnail to the second user.

[0079] With this configuration, when multiple users take pictures of their actions together, it is no longer necessary to capture the moment when both users look good and use it as a thumbnail, and it is expected that the processing load on the game image capture device will be reduced. Each user can also have a thumbnail that suits their needs (at least not their unwillingness) for the shared captured video.

[0080] In the above embodiment, an example was shown in which the game image capturing device 100 automatically selected a representative frame and set it as a thumbnail. In a modified example, the representative frame may be selected by the user. The image processing unit 158 ​​of the game image capturing device 100 includes a representative candidate extraction unit in addition to the functions shown in Fig. 6. The representative candidate extraction unit extracts a plurality of representative candidates that can become representative frames from a plurality of frames constituting the captured video image.

[0081] The representative candidate extraction unit sequentially judges whether or not a frame satisfies the evaluation criteria, for example, starting from the first frame, and extracts the frames judged to satisfy the criteria as representative candidates. In this way, a set number of representative candidates (for example, 5 frames, etc.) are obtained. The image processing unit 158 ​​causes the display unit 144 to display a list screen of the representative candidates, for example, in the form of thumbnails. When one of the representative candidates is selected by the user's operation input via the input unit 140, the frame selection unit 166 sets the selected representative candidate as a representative frame. This representative frame is displayed as a thumbnail of the captured video, as in the above embodiment. With this configuration, the captured video can be made public in a form that is more in line with the user's wishes.

[0082] When a captured video includes multiple users, each user may be allowed to select a representative frame. Here, it is assumed that the users include a first user and a second user. The representative candidate extraction unit extracts multiple (a set number) frames (for example, 10 frames) that satisfy the evaluation criteria, starting from the first frame, as representative candidates. The representative candidates include at least one frame that satisfies the evaluation criteria for the first user and at least one frame that satisfies the evaluation criteria for the second user. Alternatively, the ratio between the two may be preset, such as making the number of frames that satisfy the evaluation criteria for the first user and the number of frames that satisfy the evaluation criteria for the second user equal among the set number.

[0083] The image processing unit 158 ​​displays a list screen of the extracted representative candidates on the display unit 144. Each of the first user and the second user can select one of the representative candidates via the input unit 140. The representative candidate selected by the first user is set as the first representative frame, and the representative candidate selected by the second user is set as the second representative frame. If both the first user and the second user have registered as members and have their own user IDs, the data adjustment unit 168 adjusts the image data so that the representative frame selected by each user corresponds to the user ID of each user. The first representative frame is set as the first thumbnail, and the second representative frame is set as the second thumbnail. In other words, a thumbnail that is in line with the intention (at least not unwilling) of each of the multiple users is set.

[0084] As a result, when a first user accesses server 202 using his / her user ID, a first thumbnail is displayed and the captured video can be downloaded by selecting it. When a second user accesses server 202 using his / her user ID, a second thumbnail is displayed and the captured video can be downloaded by selecting it. In addition, when each user uploads the captured video to an SNS, it becomes easier to make the captured video public in a manner that meets the user's wishes.

[0085] In addition, the representative candidate extraction unit may set representative candidates by sequentially determining whether or not the frames satisfy the evaluation criteria from the first frame in a predetermined frame selection range, such as a specified range immediately before a frame drop range, rather than the first frame of the captured video.

[0086] In the above embodiment, the game image capture system has been described as the main example of a service in which videos and photos are captured and printed on stickers or uploaded to a server for users to download. However, the system may also function as the game image capture system of the above embodiment when capturing images of gameplay on gaming machines such as video games, medal games, and prize games.

[0087] In the above embodiment, as shown in Fig. 8, an example has been shown in which a first half video and a second half video are recorded separately before and after still image shooting. In a modified example, a series of videos may be recorded without dividing the videos into a first half video and a second half video, and still image shooting may be performed in the middle of the videos. A camera for shooting videos and a camera for shooting still images may be provided. It is assumed that the video thus obtained includes frames in which the image is still (hereinafter referred to as "dropped frames").

[0088] In this modified example, the first half video and the second half video are not shot separately, but the video portion recorded before the still image shooting is called the "first half video," and the video portion recorded after the still image shooting is called the "second half video." A predetermined number of frames (for example, a total of 90 frames) are extracted from the first half video and the second half video that remain after removing the dropped frames from the series of videos (for example, 120 frames) acquired from the camera as described above, and used as the shot video.

[0089] In addition, the extraction mode may be selectable between a "pre-shutter mode" and a "pre- and post-shutter mode." When the pre-shutter mode is selected, a predetermined frame may be extracted from the first half of the video. When the pre- and post-shutter mode is selected, a predetermined frame may be extracted by combining the first half of the video and the second half of the video. The number of frames in the first half of the video and the second half of the video may be equal.

[0090] For the captured video extracted in this way, it is not necessary to provide a non-selection range as shown in Fig. 8. In that case, in the image data adjustment process shown in Fig. 12, S40 can be replaced with "Is the next frame not the last frame?". If it is not the last frame of the captured video at S40 (N in S40), the next frame is obtained and processing continues (S42). If it is the last frame (Y in S40), the first frame may be set as the thumbnail (S44). [Explanation of symbols]

[0091] 100 game image photographing device, 102 reception device, 104 photographing booth, 106 editing booth, 108 reception monitor, 110 printing device, 112 first photographing monitor, 114 second photographing monitor, 116 light, 118 camera, 120 editing monitor, 126 editing device, 128 photographing device, 130 user interface processing unit, 132 data processing unit, 134 communication unit, 136 data storage unit, 140 input unit, 142 output unit, 144 display unit, 150 photographing processing unit, 152 editing processing unit, 154 reception processing unit, 156 print processing unit, 158 image processing unit, 160 video photographing unit, 162 still image photographing unit, 164 image recognition unit, 166 frame selection unit, 168 data adjustment unit, 200 game image photographing system, 202 server, 204 Internet, 206 user terminal.

Claims

1. An imaging unit that captures video and still images with the user as the subject; An image processing unit that selects a representative frame from the images obtained by imaging and sets it as a thumbnail of the moving image obtained by imaging; A display unit that displays the set thumbnail in association with the moving image; Comprising: The image processing unit selects the still image obtained by the imaging as the representative frame, and a gaming image capturing system characterized by this.

2. The gaming image capturing system according to claim 1, wherein the image processing unit sets the selected representative frame as the first frame of the moving image.

Images