Gameplay image capture system

The game image capture system addresses the issue of unfavorable video thumbnails by selecting a representative frame that meets user-friendly criteria, ensuring videos are shared in a considerate manner.

JP2026183003APending Publication Date: 2026-11-19SEGA CORP
View PDF 2 Cites 0 Cited by

Patent Information

Application Number
JP2026202400
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2026-09-08
Publication Date
2026-11-19

AI Technical Summary

Technical Problem

Existing game image capture devices generate video thumbnails that may not represent the user favorably, potentially causing offense when shared on social media.

Method used

A game image capture system that selects a representative frame from a moving image based on predetermined evaluation criteria, ensuring the frame displays the user in a favorable light, and associates it with the video for public release.

Benefits of technology

Enables the public release of videos generated by game image capture devices in a manner that respects user preferences, minimizing the likelihood of offensive thumbnails.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2026183003000001_ABST
    Figure 2026183003000001_ABST
Patent Text Reader

Abstract

This technology provides a way to publish videos generated by a game image capture device in a manner that does not antagonize the user. [Solution] The game image capture system comprises a capture unit that captures the user as the subject, a frame selection unit that selects a representative frame from a plurality of frames that constitute the motion image obtained by capture, and a display unit that displays the selected representative frame in association with the motion image. The frame selection unit selects a frame different from the first frame of the motion image as the representative frame based on set conditions.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to display of game images, and particularly to a technology for displaying captured moving images.

Background Art

[0002] Amusement image photographing devices called Purikura (registered trademark) enjoy enduring popularity, especially among people in their teens to twenties. Users photograph themselves, and apply various decorations to the photographed images on an editing display. The decorated images are printed on sticker mount (see Patent Document 1). The amusement image photographing device also uploads the captured images to an external server. A user can also download the edited images by accessing the server from a user terminal such as a smartphone.

[0003] In recent years, amusement image photographing devices that capture not only still images but also moving images of users and upload the moving images to a server have also been proposed (see Patent Document 2). Users can download the moving images from the server and publish them on SNS (Social Networking Service), which further enhances the utility value of the amusement image photographing devices.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Patent Document 2

Summary of Invention

Problems to be Solved by the Invention

[0005] The video footage generated by the game image capture device is published as a thumbnail so that its content can be understood at a glance. In other words, when the video is generated, one of the multiple frames that make up the video, such as the first frame, is automatically selected as the thumbnail. When that video is posted to social media, the thumbnail of that video is displayed on the user's posting screen. Therefore, it is important whether or not the thumbnail is an image that looks good to the user.

[0006] This invention is based on the above-mentioned perspective, and its main objective is to provide a technology that enables the public release of moving images generated by a game image capture device in a manner that does not cause offense to the user. [Means for solving the problem]

[0007] A game image capture system according to one aspect of the present invention comprises a capture unit that captures a user as the subject, a frame selection unit that selects a representative frame from a plurality of frames constituting a moving image obtained by capture, and a display unit that displays the selected representative frame in association with the moving image. The frame selection unit selects a frame different from the first frame of the moving image as the representative frame based on set conditions.

[0008] Another embodiment of the game image capture system includes a shooting unit that captures the user as the subject, a representative candidate extraction unit that extracts representative candidates from multiple frames constituting the motion image obtained by shooting, an input unit that accepts selection input from the user, and a display unit that displays the frame selected by the user from among the representative candidates as the representative frame and associates it with the motion image. [Effects of the Invention]

[0009] According to the present invention, it is possible to provide a technology that enables the public release of videos generated by a game image capture device in a manner that does not cause offense to the user. [Brief explanation of the drawing]

[0010] [Figure 1] This is an overall diagram of the game image capture system. [Figure 2] This is a perspective view of the game image capture device. [Figure 3] This is a perspective view of the exterior of the photo booth. [Figure 4] This is a flowchart showing the gameplay flow in a game image capture device. [Figure 5] This is a top-down plan view of the game image capture device. [Figure 6] This is a functional block diagram of the game image capture device. [Figure 7] This is a time chart showing the process of the image capture. [Figure 8] This diagram illustrates the method for generating captured video footage and the method for adjusting image data. [Figure 9] This diagram shows an example of frames that make up an edited video. [Figure 10] This diagram shows an example of frames that make up an edited video. [Figure 11] This is a diagram illustrating an example of a download screen displayed on a user's terminal. [Figure 12] This is a flowchart showing the image data adjustment process in S18 of Figure 4. [Modes for carrying out the invention]

[0011] The game image capture device in this embodiment includes a shooting booth and an editing booth. The user takes a picture of themselves in the shooting booth. Hereinafter, the image of the subject (user) taken in the shooting booth will be referred to as the "captured image". The game image capture device applies image corrections such as skin whitening and hair shininess to the captured image in the shooting booth (hereinafter, such image corrections will be referred to as "pre-corrections"). The game image capture device in this embodiment captures not only still images but also videos in the shooting booth. Of the captured images, those acquired as videos will be referred to as "captured videos", and those acquired as still images will be referred to as "captured still images".

[0012] The captured image that has been subjected to pre-correction is sent to an editing booth. In the editing booth, the user can apply any arbitrary decoration to the captured image, such as attaching a stamp image or doodling with a stylus pen (hereinafter, such image decoration is referred to as "decoration processing" or "editing"). The captured image that is the editing target in the editing booth is referred to as the "target image". Further, among the target images, those whose editing target is a captured moving image are referred to as "target moving images", and those whose editing target is a captured still image are referred to as "target still images". The target image after editing is referred to as an "edited image", and among the edited images, those obtained by editing the target moving image are referred to as "edited moving images", and those obtained by editing the target still image are referred to as "edited still images".

[0013] The amusement image capturing apparatus prints the edited still image on a sticker mount (external medium). In addition, the amusement image capturing apparatus uploads the edited moving image and the edited still image to a server (hereinafter also referred to as "dedicated server"). The user can also download the edited images (the edited moving image and the edited still image) by accessing the server from a user terminal such as a smartphone. The user can also upload the downloaded edited images to an SNS server (hereinafter referred to as "SNS server") for publication.

[0014] The amusement image capturing apparatus of the present embodiment adjusts the image data so as to display a thumbnail that satisfies a predetermined evaluation criterion for the edited moving image. This prevents an undesired thumbnail from being displayed on the edited moving image selection screen when the user accesses the dedicated server. In addition, an undesired thumbnail is prevented from being displayed even when the edited moving image is published on an SNS. The details will be described below.

[0015] Figure 1 is an overall configuration diagram of the amusement image capturing system 200. In the amusement image photographing system 200, the amusement image photographing apparatus 100 is connected to the server 202 and the user terminal 206 (portable device) via the Internet 204. A user photographs themselves at the amusement image photographing apparatus 100. The amusement image photographing apparatus 100 acquires a captured image including a subject (the user), applies pre-correction to the captured image as necessary, and accepts decoration processing from the user.

[0016] The edited image after decoration processing is uploaded from the amusement image photographing apparatus 100 to the server 202 together with the user ID. The server 202 is a dedicated server that manages images generated by the amusement image photographing apparatus 100, and stores the user ID, photographing date and time, and edited images (edited moving image and edited still image) in association with one another. The user can download the edited image from the server 202 by accessing the server 202 with the user ID specified. The user can also upload the downloaded edited image to an unillustrated SNS server for publication.

[0017] Figure 2 is an external perspective view of the amusement image photographing apparatus 100. The amusement image photographing apparatus 100 includes a reception device 102, a photographing booth 104, an editing booth 106, and a printing device 110. A user reserves use of the amusement image photographing apparatus 100 at the reception device 102. The user inputs a user ID at the reception device 102, and pays a play fee with cash or electronic money. The reception device 102 issues a reservation number to the user. The reception device 102 causes a reception monitor 108 to display the number of waiting people and the reservation number to be called. When the user's own reservation number is displayed on the reception monitor 108, the user enters the photographing booth 104.

[0018] In the shooting booth 104, the user takes a photograph. After the shooting is complete, the user moves to the editing booth 106. The editing booth 106 is equipped with an editing device 126. The editing device 126 includes an editing monitor 120. The editing monitor 120 displays the target image (the photographed image to be decorated). In this embodiment, the editing monitor 120 is formed by a capacitive touch panel. Two stylus pens 122a and 122b are provided for each editing monitor 120. The user uses the stylus pen 122 and their finger to decorate the target image. After editing is complete, the printing device 110 prints the edited still image onto sticker paper. The editing device 126 also uploads the edited image to the server 202.

[0019] Figure 3 is an external perspective view of the shooting booth 104. The shooting booth 104 is shielded from the outside by a curtain 124. A shooting device 128 is installed in the shooting booth 104. The shooting device 128 includes three lights 116 that illuminate the user being photographed from three directions, a camera 118 for acquiring still images and moving images, a first shooting monitor 112, and a second shooting monitor 114.

[0020] The first shooting monitor 112 displays a preview of the user's captured images (both video and still images). The second shooting monitor 114 displays images of a person other than the user who is the subject, such as a model (hereinafter referred to as "sample images"). The user takes any shooting pose in front of the camera 118, referring to the sample images displayed on the second shooting monitor 114. Various explanations regarding the shooting method are also displayed on the second shooting monitor 114. As described above, video and still images can be acquired in the shooting booth 104.

[0021] In addition, the first shooting monitor 112 displays a live view image (mirror image) for the user to check their own appearance, as well as a countdown image to indicate the timing of the shot.

[0022] Figure 4 is a flowchart showing the flow of gameplay in the game image capture device 100. Figure 5 is a top view of the game image capture device 100. When a user makes a reservation at the reception device 102, the reception device 102 performs the reception process (S10). After reception, the user enters the shooting booth 104. The shooting device 128 acquires the user's images (S12). In the shooting booth 104, the user can acquire a predetermined number of images. Video and still images are acquired simultaneously, and a pair of video and still images is counted as one set of images. In this embodiment, five sets of images (sets of video and still images) are acquired. When shooting, the user can select a pre-correction method from several pre-correction menus provided by the shooting device 128. The user can also select the background color of the images at the shooting device 128. The shooting device 128 performs pre-correction each time an image is acquired. The pre-correction menu may also be selected at the reception device 102 before playing.

[0023] After shooting, the first shooting monitor 112 displays instruction information indicating which of the two editing booths 106 to use. The user moves to editing booth 106, which is one of the two editing booths 106a and 106b, as specified by the first shooting monitor 112. The captured image (target image) is transferred from the shooting device 128 to the editing device 126. The user then applies decorative processing to the transferred captured image (target image) in the editing device 126 (S14).

[0024] After editing, the editing device 126 sends the edited image to the printing device 110, which prints the edited image onto a sticker sheet (S16). Prior to uploading to the server 202, the editing device 126 performs an image data adjustment process to set a thumbnail for the edited image (S18). Details of this image data adjustment process will be described later. Subsequently, the editing device 126 uploads the edited image to the server 202 via the internet 204 (S20). The user can download the edited image from the server 202 to the user terminal 206 by accessing the server 202.

[0025] Generally, editing takes more time than shooting. In the game image shooting device 100 of this embodiment, customer turnover is increased by providing two editing booths 106a and 106b (editing devices 126a and 126b) for each shooting booth 104.

[0026] Figure 6 is a functional block diagram of the game image capture device 100. Each component of the game image capturing device 100 is realized by hardware including a CPU (Central Processing Unit) and various coprocessors, memory and storage devices, and wired or wireless communication lines connecting them, as well as software stored in the storage devices that supplies processing instructions to the arithmetic units. The computer program may consist of device drivers, an operating system, various application programs located at a higher layer, and libraries that provide common functions to these programs. As described above, the game image capturing device 100 is configured as a collection of a capturing device 128, a receiving device 102, an editing device 126, and a printing device 110. The blocks described below represent functional units, not hardware units.

[0027] The game image capture device 100 includes a user interface processing unit 130, a data processing unit 132, a communication unit 134, and a data storage unit 136. The user interface processing unit 130 is responsible for user interface processing via the reception monitor 108, the editing monitor 120, etc. The communication unit 134 is responsible for communication processing with the server 202 via the internet 204. The communication unit 134 can also communicate directly with the user terminal 206 using short-range wireless communication such as Wi-Fi (registered trademark). For example, the communication unit 134 may directly transmit the edited image to the user terminal 206. The data storage unit 136 stores various types of data. The data processing unit 132 performs various processes based on the data acquired by the communication unit 134 and the data stored in the data storage unit 136.

[0028] The user interface processing unit 130 includes an input unit 140 and an output unit 142. The input unit 140 receives user input. The input unit 140 detects user input to touch panels installed in the reception device 102, the shooting device 128, the editing device 126, etc. The input unit 140 also detects input using a stylus pen 122. The input unit 140 also detects the user ID in the reception device 102.

[0029] The output unit 142 provides information to the user. In addition to displaying images on the reception device 102, the shooting device 128, the editing device 126, etc., the output unit 142 performs various output processes such as audio output and print output.

[0030] The output unit 142 includes a display unit 144. During shooting, the display unit 144 displays a live view image (mirror image) of the user (subject) on the first shooting monitor 112. The user performs the shooting while checking their appearance as if looking in a mirror using the live view image.

[0031] The data processing unit 132 includes a shooting processing unit 150, an editing processing unit 152, a reception processing unit 154, a printing processing unit 156, and an image processing unit 158. The shooting processing unit 150 controls the lighting settings in the shooting booth 104, such as the on / off status, light intensity, and color of the light 116, and controls the shooting by the camera 118.

[0032] In the shooting booth 104, still images and videos are taken with the user as the subject. The shooting processing unit 150 includes a video shooting unit 160 and a still image shooting unit 162. The video shooting unit 160 controls video shooting, and the still image shooting unit 162 controls still image shooting. In this embodiment, video is shot before and after the still image shooting timing. Hereinafter, the video recorded before the shooting timing will be called the "first half video," and the video recorded after the shooting timing will be called the "second half video." The video shooting unit 160 generates the captured video by concatenating the first half video and the second half video. The still image shooting unit 162 notifies the user of the still image shooting timing by instructing the output unit 142 to display a countdown and provide audio notification.

[0033] The output unit 142 displays the captured image as the target image on the editing monitor 120 (capacitive touch panel). The editing processing unit 152 modifies the target image according to user input. The reception processing unit 154 manages user payment of play fees at the reception device 102. The reception processing unit 154 guides reserved users to the shooting booth 104 each time a shooting session is completed. The printing processing unit 156 controls the printing of the edited image at the printing device 110.

[0034] The image processing unit 158 ​​performs image data adjustment processing to set a thumbnail to be associated with the edited video. The image processing unit 158 ​​includes an image recognition unit 164, a frame selection unit 166, and a data adjustment unit 168. The image recognition unit 164 recognizes face images in the frames that make up the edited video. Based on the recognized face images, the frame selection unit 166 extracts a frame suitable for a thumbnail from among multiple frames that make up the edited video as a representative frame. The data adjustment unit 168 adjusts the image data so that the representative frame is associated with the edited video and sends it to the server 202. A specific example of this image data adjustment processing will be described later.

[0035] The communication unit 134 includes a receiving unit 170 and a transmitting unit 172. The receiving unit 170 receives data from external devices such as a server 202 and a user terminal 206. The transmitting unit 172 transmits data to external devices. The transmitting unit 172 transmits the edited image to be uploaded to the server 202.

[0036] Figure 7 is a time chart showing the process of image processing. The horizontal axis represents the passage of time. When the user touches the start shooting button (not shown) in the shooting booth 104, shooting begins. At this time, a pose instruction image prompting the user to pose for still image capture, and a sample still image are displayed on the second shooting monitor 114. The pose instruction image is, for example, an image representing a notification via text information.

[0037] Four seconds after shooting begins, a countdown to still image capture starts. This countdown informs the user of the timing for taking the still image. The countdown starts three seconds before the still image is to be taken, and the still image is captured when the count reaches zero. Prior to this, video recording begins one second before the countdown starts. By starting video recording during the countdown to still image capture, it is possible to discreetly capture the user's movements and actions as they pose for the still image.

[0038] Video recording is paused before still image capture. Still image capture takes place after video recording has stopped. Video recording resumes after still image capture. This is to minimize the impact of the strobe flash on the video.

[0039] Once video recording is complete, the captured still image and video are pre-corrected. The pre-corrected captured still image and video are displayed on the first shooting monitor 112. The first shooting monitor 112 also displays the live view image from the start to the end of recording.

[0040] Figure 8 illustrates the method for generating captured video footage and the method for adjusting image data. The top section of the diagram shows the video recording period (video recording period) based on the timing of still image capture. The middle section shows the frame acquisition process from the start of video recording in relation to the top section. The bottom section shows the frame structure of the generated video.

[0041] In this embodiment, as described above, video recording begins 4 seconds before the still image capture timing, and the countdown begins 3 seconds before. Statistically, it is thought that the user is most focused on capturing a still image, that is, they are trying to make a good expression (avoiding a slack expression), around 1 second before the still image capture. For this reason, the strobe is fired 1 second before the still image capture timing, and the shutter of camera 118 is released at the still image capture timing.

[0042] The recording of video is paused before still image capture and resumed afterward. In this example, the recording of the first half of the video is paused immediately before still image capture. Recording of the second half of the video begins 0.5 seconds after still image capture and continues for 1 second. This predetermined delay before resuming video recording is to eliminate the effects of residual reflected light.

[0043] In the illustrated example, the video recording frame rate is set to 30fps, resulting in 120 frames for the first half of the video and 30 frames for the second half. A predetermined number of frames at the end of the first half of the video are removed as dropped frames, and the first and second half of the video are combined to create the recorded video. The dropped frames correspond to the "preparation period" for still image capture and are pre-set as frames unsuitable for video recording, such as frames where a flash may be captured in the video. By removing the dropped frames in this way, the image quality during video playback can be maintained. Since this recorded video does not contain still images, smooth video playback can be achieved.

[0044] In this embodiment, the first half of the video footage uses up to frame 90, which is one second before the still image capture timing, and frames from frame 91 onwards are removed. All frames are used in the second half of the video footage. As mentioned above, the user's facial expression is at its best around the time the countdown reaches one second, so this is to ensure that no footage is missed in that range. In the illustrated example, a 120-frame video footage is generated by combining the first and second half of the video footage. Editing (decorative processing) is then applied to this video footage.

[0045] The image processing unit 158 ​​sets the thumbnail to be displayed when the edited video obtained in this way is uploaded to the server 202. The frame selection unit 166 extracts a frame suitable for the thumbnail from among the multiple frames that make up the edited video as a representative frame. However, as shown in the lower part of Figure 8, the range that can be selected as a representative frame (hereinafter referred to as the "frame selection range") is the range that makes up the first half of the video. This is because it is thought that the user will have a relaxed expression after taking a still image, so the range of the first half of the video before taking a still image is selected.

[0046] The image recognition unit 164 sequentially extracts frames from the edited video starting from the first frame, recognizes the user's face image in each frame, and identifies the position and expression of the face. The image recognition unit 164 determines whether the entire user's face is included in the field of view in each frame, and whether the user's expression meets predetermined evaluation criteria.

[0047] Figures 9 and 10 are diagrams illustrating examples of frames that make up an edited video. Figure 9 shows an example where the user's facial expression meets the evaluation criteria, and Figure 10 shows an example where the user's facial expression does not meet the evaluation criteria. In the illustrated example, multiple (two) users are visible, but as shown in Figure 9, the evaluation criteria are met if each user's entire face fits within the frame and their eyes are wide open (eye opening is above a predetermined value). As shown in Figure 10, the evaluation criteria are not met if any user's eyes are half-closed, showing the whites of their eyes, or closed. Also, the evaluation criteria are not met if any part of any user's face is cut off (outside the frame). The frame selection unit 166 sets the frame that meets the evaluation criteria as the representative frame.

[0048] In other words, the "setting conditions" for the representative frame include meeting predetermined evaluation criteria. In this embodiment, assuming that the entire faces of all users appearing in the edited video are included, the conditions for meeting the evaluation criteria include that the degree of eye opening of all users is above a certain standard value, and that the position of the pupils is within the standard range of the whites of the eyes.

[0049] Regarding the evaluation criteria, determining whether the entire face fits within the field of view can be achieved, for example, as follows: When a face is detected in image recognition, the determination may be made by checking whether both eyes and the mouth are included. Alternatively, the determination may be made by checking whether the proportion of the face area to the entire screen is greater than or equal to a predetermined value.

[0050] Determining the degree of eye opening can be achieved, for example, as follows: First, the region corresponding to the face image (face region) is detected from the frame. From that face region, the region corresponding to the right eye and the region corresponding to the left eye are detected. Then, the degree of opening of the right eye region (how wide the eye is open) is quantified. For example, one could prepare a right eye image with maximum opening and a right eye image with minimum opening, and quantify the degree of opening of the right eye region from the vertical size of the pupil region in the right eye region to be determined. The same applies to the degree of opening of the left eye region.

[0051] The frame selection unit 166 sets the first frame of the edited video as the representative frame and thumbnail if the first frame meets the evaluation criteria. If the first frame does not meet the evaluation criteria, it sequentially checks the subsequent frames and sets any frames that meet the evaluation criteria as the representative frame and thumbnail. The data adjustment unit 168 adjusts the image data to include setting information that indicates the correspondence between the edited video and its thumbnail. This "setting information" may be, for example, information that associates the ID of the edited video with the frame number corresponding to the thumbnail of that edited video. The transmission unit 172 transmits the adjusted image data to the server 202. A display unit (not shown) of the server 202 displays the thumbnail corresponding to the edited video on the download screen when a user accesses it to download the edited video.

[0052] Figure 11 shows an example of a download screen displayed on user terminal 206. The user can access server 202 to display the download screen 210 on the user terminal 206. The download screen 210 displays thumbnails corresponding to each edited video uploaded from the game image capture device 100. The user can select any of the thumbnails to download the corresponding edited video to the user terminal 206.

[0053] Figure 12 is a flowchart showing the image data adjustment process in S18 in Figure 4. In the image data adjustment process, the image processing unit 158 ​​acquires the first frame of the edited video (S30). The image recognition unit 164 performs image recognition on the first frame to identify the user's face image (S32) and identify the facial expression (S34). Then, it determines whether or not the above-mentioned evaluation criteria are met (S36).

[0054] The frame selection unit 166 sets the first frame of the edited video as a thumbnail (S38) if the first frame meets the evaluation criteria (Y in S36). If the first frame does not meet the evaluation criteria (N in S36), and the next frame is within the frame selection range (Y in S40), the image processing unit 158 ​​acquires the next frame (S44) and returns to S32.

[0055] Thus, the image recognition unit 164 sequentially checks the subsequent frames, and if there is a frame that meets the evaluation criteria (Y in S36), it sets it as the thumbnail (S38). If none of the frames in the frame selection range meet the evaluation criteria (N in S40), the first frame is set as the thumbnail (S44). The data adjustment unit 168 updates the image data to include the setting information for the edited video and its thumbnail (S46).

[0056] In this way, the edited video with a thumbnail is uploaded to the server 202. The transmission unit 172 of the game image capture device 100 may also upload the edited video to the server 202 along with the user ID. The server 202 saves the edited images (edited video, edited still image) in association with the user ID.

[0057] Users can access server 202 using their user ID to view the edited videos stored on server 202. The download screen displayed by server 202 shows a thumbnail associated with each edited video. Users can download the edited video from server 202 to their user terminal 206 by selecting one of these thumbnails. Server 202 sends the edited video (edited video, edited still image) selected by the user, associated with the user ID, to the user terminal 206. Users can also make their edited videos public by uploading them to an SNS server.

[0058] The game image capture system 200 has been described above based on the embodiments. According to this embodiment, the thumbnails for the captured video images generated by the game image capture device 100 are set by extracting frames that satisfy predetermined evaluation criteria. The conditions for satisfying these evaluation criteria include that the user's entire face is within the field of view and that the user's facial expression is good (satisfying predetermined setting criteria). Therefore, when the captured video is uploaded to the server 202, it is possible to suppress the display of images that are undesirable to the user as thumbnails. Furthermore, when the captured video is uploaded to SNS, it becomes easier to publish the captured video in a way that is in line with the user's wishes.

[0059] Furthermore, if the recorded video contains footage from multiple users, the frame that meets the evaluation criteria will be set as the thumbnail for all users. This ensures that a considerate service can be provided to any user downloading the recorded video from server 202.

[0060] It should be noted that the present invention is not limited to the embodiments and modifications described above, and the components can be modified and implemented without departing from the spirit of the invention. Various inventions may be formed by appropriately combining the multiple components disclosed in the embodiments and modifications described above. In addition, some components may be deleted from all the components shown in the embodiments and modifications described above.

[0061] Although the game image capture system 200 has been described as being composed of the game image capture device 100, server 202, and user terminal 206, and in particular the game image capture device 100 and server 202, some of the functions of the game image capture device 100 may be implemented by the server 202, and some of the functions of the server 202 may be assigned to the game image capture device 100. Furthermore, a third device other than the game image capture device 100 and server 202 may perform some of the functions of these devices. How to distribute the multiple functions necessary to realize the present invention to one or more hardware devices should be determined in consideration of the processing capacity of each hardware device and the specifications required for the game image capture system 200.

[0062] [Differentiation] In the above embodiment, an example was shown in which the image data adjustment process is performed by the game image capture device 100. In a modified example, the image data adjustment process may be performed by the server 202. Specifically, the server 202 is equipped with an image processing unit. The game image capture device 100 uploads the generated edited image (edited video) directly to the server 202. The image processing unit of the server 202 may select a representative frame that satisfies predetermined evaluation criteria from the edited video and display it as a thumbnail.

[0063] Alternatively, dedicated application software for image data adjustment processing may be installed on the user terminal 206. This application software can be downloaded from server 202 as member software when the user registers as a member with server 202.

[0064] The edited images (edited video) generated by the game image capture device 100 are uploaded to the server 202 without any thumbnail adjustments. The user downloads the edited video to the user terminal 206. Alternatively, the edited images generated by the game image capture device 100 may be sent directly to the user terminal 206. The user terminal 206 uses member software to perform the same image data adjustment process on the edited video as in the above embodiment and displays the adjusted thumbnail on the screen. The user can upload the image data with the adjusted thumbnail to the SNS server.

[0065] In the above embodiment, a representative frame was selected from the captured video footage and displayed as a thumbnail. In a modified example, a captured still image may be selected as a representative frame and displayed as a thumbnail for the captured video footage. Since a countdown is performed for still image capture, the user's facial expression is likely to be at its best at the time of still image capture. Therefore, it becomes easier to publish a thumbnail that aligns with the user's intentions.

[0066] The above embodiment shows an example of performing image data adjustment processing on edited video footage that reflects editing done in the editing booth. In a modified example, the same image data adjustment processing may be performed on recorded video footage that does not reflect editing (i.e., the video footage itself that was shot in the shooting booth), such as when no editing was done in the editing booth. Although not mentioned in the above embodiment, when performing image data adjustment processing on edited video footage, the processing may be performed during the period the user is in the editing booth. When performing image data adjustment processing on recorded video footage that does not reflect editing, the processing may be performed during the period the user is in the shooting booth.

[0067] Furthermore, the editing processing unit 152 may also be configured to turn off the decorative processing from the edited image and return it to the original captured image when instructed by the user. The image processing unit 158 ​​may perform the above-described image data adjustment processing on the original captured video image.

[0068] In the above embodiment, an example was shown in which a representative frame is selected from multiple frames constituting the captured video and associated with the video by displaying that representative frame as a thumbnail. In a modified example, the image processing unit may associate the selected representative frame with the video by moving it to the beginning of the frame. Alternatively, it may associate the video with the video by replacing the first frame with the representative frame. By making the representative frame that satisfies predetermined evaluation criteria the first frame, it becomes easier to make the screen that the user first notices when playing the video more in line with the user's intentions.

[0069] The image processing unit may, after sequentially determining and extracting a representative frame from the first frame, invalidate (hide) frames in the captured video that precede the representative frame. Alternatively, the data adjustment unit may reconstruct (edit) the captured video by deleting frames that precede the representative frame. By preventing the display of frames that precede the representative frame in this way, the overall quality of the video played afterward can be improved.

[0070] In the above embodiment, the "setting conditions" include sequentially determining whether the evaluation criteria are met starting from the first frame, and selecting a frame that satisfies the predetermined evaluation criteria itself as the representative frame. In a modified example, the degree of conformance to the predetermined evaluation criteria may be determined, and a frame whose degree of conformance is equal to or greater than a predetermined value may be selected as the representative frame. For example, the evaluation criteria may be quantified, and even if 100% of the evaluation criteria are not met, a frame may be adopted as the representative frame if it is equal to or greater than a predetermined percentage of the evaluation criteria (for example, 75% or more).

[0071] A representative frame may be a predetermined frame (other than the first) from among multiple frames. The "predetermined frame" is set in advance based on statistics, empirical rules, etc., to determine a value that results in a good user expression. Alternatively, a representative frame may be a frame that precedes a predetermined frame from the time the still image was taken. The "predetermined frame" may be a frame that corresponds to a predetermined time (for example, 1 second before) from the time the still image was taken. In this case as well, a value that results in a good user expression is set in advance based on statistics, empirical rules, etc.

[0072] In the above embodiment, an example was shown where the frame selection range was set from the first frame of the first half of the video to just before the frame drop range (just before the preparation period). In a modified example, a predetermined range just before the frame drop range may be set as the frame selection range, and a representative frame may be selected from that range. Since the user's facial expression is most likely to be at its best just before the frame drop range, there is a high probability that a representative frame that matches the user's intentions can be selected. Alternatively, the frame just before the frame drop range (frame 90 in the above embodiment) may be selected as the representative frame. If there is statistical support, the frame selection range may be set to a specific range in the captured video, not limited to just before the frame drop range, and a representative frame may be selected from that range.

[0073] In the above embodiment, an example was shown in which the selection of a representative frame was determined based on the degree of eye opening to determine whether or not the evaluation criteria were met. However, the determination may also be based on other factors. For example, the determination may be based on whether or not the corners of the mouth are turned up or whether or not the face is tilted, as obtained from face recognition.

[0074] In the above embodiment, the game image capture device is exemplified as a device that captures both still images and video in a single shooting session, such as capturing still images at the timing of a pause in video recording (in other words, capturing video before and after capturing still images). In modified examples, still images and video may be captured separately. Alternatively, still images may be captured at predetermined timings during video recording without stopping video recording. In these cases as well, the same image data adjustment processing as in the above embodiment may be performed on the captured video and edited video.

[0075] In the above embodiment, an example was shown in which, when multiple users are included in the captured video, one frame that satisfies the evaluation criteria is set as a thumbnail for all users. In a modified example, it may be possible to set individual thumbnails for each of the multiple users. For example, consider the case where the first user and the second user are filming together, as shown in Figures 9 and 10. Assume that the game video capture device is used with the first user's user ID.

[0076] In this case, with respect to a single captured video image, the frame selection unit sets the frame selected by the above method for the first user as the first representative frame, while setting the frame selected by the above method for the second user as the second representative frame. The face image of the second user does not need to be considered when selecting the first representative frame. Similarly, the face image of the first user does not need to be considered when selecting the second representative frame.

[0077] The data adjustment unit then adjusts the image data so that multiple thumbnails are associated with a single captured video. Specifically, it sets a first thumbnail using the first representative frame and a second thumbnail using the second representative frame. The first user ID is associated with the first thumbnail, the second thumbnail, and the captured video.

[0078] As a result, when the first user accesses server 202 using their user ID, the first and second thumbnails will be displayed. The first user can download the video footage associated with the first thumbnail by selecting it. Alternatively, by selecting both the first and second thumbnails, the first user can download the video footage associated with both thumbnails. The first user can also provide the second user with the video footage corresponding to the second thumbnail from among the downloaded video footage.

[0079] This configuration eliminates the need to capture a moment where both users' expressions are good for the thumbnail when multiple users are filming together, which is expected to reduce the processing load on the game image capture device. Individual users can also save thumbnails for the same captured video that are to their liking (or at least not undesirable).

[0080] In the above embodiment, an example was shown in which the game image capture device 100 automatically selects a representative frame and sets it as a thumbnail. In a modified example, the user may be able to select the representative frame. The image processing unit 158 ​​of the game image capture device 100 includes a representative candidate extraction unit in addition to the functions shown in Figure 6. The representative candidate extraction unit extracts a plurality of representative candidates that can become the representative frame from a plurality of frames that constitute the captured video image.

[0081] The representative candidate extraction unit sequentially determines, for example, whether each frame satisfies the evaluation criteria starting from the first frame, and extracts the frames that are determined to satisfy the criteria as representative candidates. In this way, a set number of representative candidates (for example, 5 frames) are obtained. The image processing unit 158 ​​displays a list of these representative candidates on the display unit 144, for example, in the form of thumbnails. When a representative candidate is selected by user input via the input unit 140, the frame selection unit 166 sets the selected representative candidate as the representative frame. Displaying this representative frame as a thumbnail of the captured video is the same as in the above embodiment. With this configuration, captured videos can be published in a way that better suits the user's intentions.

[0082] If the captured video footage includes multiple users, each user may be allowed to select a representative frame. Here, let's assume that there are two users, a first user and a second user. The representative candidate extraction unit extracts, for example, multiple frames (a set number) that meet the evaluation criteria (e.g., 10 frames) from the first frame as representative candidates. These representative candidates should include at least one frame that meets the evaluation criteria for the first user and at least one frame that meets the evaluation criteria for the second user. Alternatively, the ratio of the number of frames that meet the evaluation criteria for the first user to the number of frames that meet the evaluation criteria for the second user may be set in advance, for example, so that the number of frames that meet the evaluation criteria for the first user and the number of frames that meet the evaluation criteria for the second user are equal out of the set number of frames.

[0083] The image processing unit 158 ​​displays a list of extracted representative candidates on the display unit 144. The first user and the second user can each select a representative candidate via the input unit 140. The representative candidate selected by the first user is designated as the first representative frame, and the representative candidate selected by the second user is designated as the second representative frame. If both the first and second users are registered members and each has a user ID, the data adjustment unit 168 adjusts the image data so that the representative frame selected by each user corresponds to each user's user ID. The first representative frame is set as the first thumbnail, and the second representative frame is set as the second thumbnail. In other words, a thumbnail that is appropriate (or at least not undesirable) is set for each of the multiple users.

[0084] This system allows the first user to access server 202 using their user ID, where the first thumbnail will be displayed, and selecting it will allow them to download the recorded video. Similarly, the second user to access server 202 using their user ID will see the second thumbnail, which they can then select to download the recorded video. Furthermore, when each user uploads the recorded video to social media, it becomes easier to publish the video in a way that aligns with their own intentions.

[0085] The representative candidate extraction unit may also determine whether a frame satisfies the evaluation criteria by sequentially checking the first frame in a predetermined frame selection range, such as a predetermined range immediately before the frame drop range, rather than the first frame of the captured video, and set the representative candidate accordingly.

[0086] In the above embodiment, the game image capture system was mainly described as a service that captures videos and photos of gameplay and prints them on stickers or uploads them to a server for users to download. However, it is not limited to this, and the system may also function as the game image capture system of the above embodiment when capturing gameplay in game machines such as video games, medal games, and prize-winning games.

[0087] In the above embodiment, as shown in Figure 8, an example was shown in which the first half of the video and the second half of the video were recorded separately before and after still image capture. In a modified example, a series of videos may be recorded without dividing them into first and second halves, and still images may be captured in the middle of the video. A camera for video recording and a camera for still image capture may be provided. Assume that the video image acquired in this way contains frames in which the image is frozen (hereinafter referred to as "dropped frames").

[0088] In this modified version, the first and second halves of the video are not filmed separately, but the video portion recorded before still image capture is called the "first half video," and the video portion recorded after still image capture is called the "second half video." From the series of video images (for example, 120 frames) acquired from the camera as described above, dropped frames are removed, and a predetermined number of frames (for example, a total of 90 frames) are extracted from the remaining first and second half video images to be used as the captured video.

[0089] Furthermore, the system may offer two extraction modes: "pre-shutter mode" and "pre- and post-shutter mode." If pre-shutter mode is selected, a predetermined frame may be extracted from the first half of the video. If pre- and post-shutter mode is selected, a predetermined frame may be extracted from both the first and second half of the video. The number of frames in the first and second half of the video may be equal.

[0090] For the captured video images extracted in this way, it is not necessary to define a non-selected area as shown in Figure 8. In that case, in the image data adjustment process shown in Figure 12, S40 can be replaced with "Is the next frame not the final frame?". If the next frame is not the final frame of the captured video images (N in S40), the next frame is acquired and processing continues (S42). If it is the final frame (Y in S40), the first frame may be set as the thumbnail (S44). [Explanation of symbols]

[0091] 100 Game image capture device, 102 Reception device, 104 Shooting booth, 106 Editing booth, 108 Reception monitor, 110 Printing device, 112 First shooting monitor, 114 Second shooting monitor, 116 Light, 118 Camera, 120 Editing monitor, 126 Editing device, 128 Shooting device, 130 User interface processing unit, 132 Data processing unit, 134 Communication unit, 136 Data storage unit, 140 Input unit, 142 Output unit, 144 Display unit, 150 Shooting processing unit, 152 Editing processing unit, 154 Reception processing unit, 156 Printing processing unit, 158 Image processing unit, 160 Video shooting unit, 162 Still image shooting unit, 164 Image recognition unit, 166 Frame selection unit, 168 Data adjustment unit, 200 Game image capture system, 202 Server, 204 Internet, 206 User terminal.

Claims

[Claim 1] The photography department photographs users as subjects, A frame selection unit that selects a representative frame from multiple frames that make up the video image obtained during shooting, A display unit that displays the selected representative frame in association with the aforementioned video image, Equipped with, The game image capture system is characterized in that the frame selection unit selects a frame different from the first frame of the video as the representative frame based on the set conditions.

Citation Information

Patent Citations

  • JP2016‐184287A

  • JP2019‐115029A