Information processing device and game image display method

Abstract

In the present invention, a game execution unit 110 executes a game program on the basis of a user's operation, and generates a game image that includes a game character. A recording unit 140 records video image data 144 obtained by extracting a game character image from a past game video. A display processing unit 130 superimposes the game character image, which is included in the video image data 144, on the game image generated by the game execution unit 110.

Description

[Technical field]

[0001] The present disclosure relates to a technique for superimposing an image of a game character previously operated by a user on an image of a game being played by a user. [Background technology]

[0002] Games have been available in which a user can compete against a ghost character based on control information of a game character operated in the past. The game synchronously processes the control information of the game character operated by the user and the control information of the game character operated in the past, allowing the user to enjoy competing against the ghost character. The control information of the past game character may be, for example, control information played by the user or a friend, or control information played by a world ranker. Summary of the Invention [Problem to be solved by the invention]

[0003] However, if the game does not support processing of control information for the ghost character, the user cannot enjoy racing against the ghost character. Therefore, it is preferable to provide a mechanism for allowing the user to virtually race against the ghost character even in a game that does not support control of the ghost character.

[0004] The present disclosure has been made in consideration of these problems, and has as its objective to provide a technique for superimposing an image of a game character that has been previously operated on a game image. [Means for solving the problem]

[0005] In order to solve the above problems, an information processing device according to one aspect of the present disclosure includes a game execution unit that executes a game program based on user operation to generate a game image including a game character, a recording unit that records video data in which game character images are extracted from past game videos, and a display processing unit that superimposes the game character image included in the video data on the game image generated by the game execution unit.

[0006] A game image display method according to another aspect of the present disclosure includes the steps of executing a game program based on user operation to generate a game image including a game character, recording video data in which game character images are extracted from past game videos, and superimposing the game character image included in the video data on the game image.

[0007] In addition, any combination of the above components, and conversions of the expressions of the present disclosure between methods, devices, systems, computer programs, recording media on which computer programs are readably recorded, data structures, etc. are also valid aspects of the present disclosure. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a diagram showing an information processing system according to an embodiment. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of an information processing device. [Diagram 3] FIG. 2 is a diagram illustrating functional blocks of the information processing device. [Figure 4] FIG. 13 is a diagram showing an example of a game image. [Diagram 5] FIG. 13 is a diagram showing an example of a system image superimposed on a game image. [Figure 6] FIG. 13 is a diagram showing another example of a system image superimposed on a game image. [Figure 7] FIG. 13 is a diagram showing an example of a battle screen with a ghost character. [Figure 8] FIG. 13 shows a scene in which the countdown to the start of the race has begun. [Figure 9] FIG. 13 shows a scene where the race has started. [Figure 10] FIG. 13 shows a scene in which the user's game character reaches a checkpoint. [Figure 11] FIG. 13 shows a scene in which the user's game character reaches the goal. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] 1 illustrates an information processing system 1 according to an embodiment of the present disclosure. The information processing system 1 includes an information processing device 10 and a server device 12. An access point (AP) 8 has the functions of a wireless access point and a router, and the information processing device 10 is connected to a network 3, such as the Internet or a LAN (Local Area Network), via the AP 8, and is communicatively connected to the server device 12.

[0010] The information processing device 10 is connected wirelessly or wired to an input device 6 operated by a user, and the input device 6 transmits user operation information to the information processing device 10. The information processing device 10 reflects the user operation information in system software and game processing, and outputs the processing result from the output device 4. The information processing device 10 of the embodiment is a terminal device such as a game device or a personal computer that executes a game, and the input device 6 may be a device that supplies user operation information to the information processing device 10, such as a game controller.

[0011] The input device 6 is configured to have a plurality of input sections, such as a plurality of push-type operation buttons, an analog stick that can input an analog amount, and a rotary button. The camera 7, which is an imaging device, is provided near the output device 4 and captures an image of the space around the output device 4. While FIG. 1 shows an example in which the camera 7 is attached to the top of the output device 4, it may be disposed on the side of the output device 4, and in any case, it is disposed in a position where it can capture an image of a user playing a game in front of the output device 4. The camera 7 may be a stereo camera.

[0012] The auxiliary storage device 2 is a large-capacity storage device such as an HDD (hard disk drive) or an SSD (solid state drive), and may be an internal storage device or an external storage device connected to the information processing device 10 via a USB (Universal Serial Bus) or the like. The output device 4 may be a television having a display for outputting images and a speaker for outputting audio, or may be a head-mounted display. The output device 4 may be connected to the information processing device 10 via a wired cable or wirelessly.

[0013] The server device 12 provides a network service to a user of the information processing system 1. The server device 12 manages a network account that identifies the user, and the user signs in to the network service provided by the server device 12 using the network account. By signing in to the network service from the information processing device 10, the user can register in the server device 12 game save data and trophies, which are virtual rewards acquired during game play.

[0014] In the information processing system 1 of the embodiment, the server device 12 has a function of generating video data by extracting game character images from past game videos and providing the generated video data to the information processing device 10. The information processing device 10 displays the game character images included in the video data superimposed on the image of the game being played by the user, allowing the user to enjoy a simulated battle with a ghost character. The server device 12 may provide the information processing device 10 with data associated with the video data, for example data indicating the time it takes to pass one or more checkpoints, together with the video data.

[0015] 2 shows a hardware configuration of the information processing device 10. The information processing device 10 includes a main power button 20, a power ON LED 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

[0016] The main system 60 comprises a main CPU (Central Processing Unit), a memory and a memory controller serving as a primary storage device, a GPU (Graphics Processing Unit), etc. The GPU is mainly used for arithmetic processing of game programs. The main CPU has a function of starting up system software and executing game programs installed in the auxiliary storage device 2 in the environment provided by the system software. The subsystem 50 comprises a sub CPU, a memory and a memory controller serving as a primary storage device, etc., but does not comprise a GPU.

[0017] While the main CPU has the function of executing a game program installed in the auxiliary storage device 2, the sub-CPU does not have such a function. However, the sub-CPU has the function of accessing the auxiliary storage device 2 and the function of sending and receiving data to and from the server device 12. The sub-CPU is configured with only such limited processing functions, and therefore can operate with less power consumption compared to the main CPU. These functions of the sub-CPU are executed when the main CPU is in standby mode.

[0018] The main power button 20 is an input unit through which a user inputs operations, is provided on the front of the housing of the information processing device 10, and is operated to turn on or off the power supply to a main system 60 of the information processing device 10. The power ON LED 21 lights up when the main power button 20 is turned on, and the standby LED 22 lights up when the main power button 20 is turned off. The system controller 24 detects that the main power button 20 has been pressed by the user.

[0019] The clock 26 is a real-time clock that generates current date and time information and supplies it to the system controller 24 , the subsystem 50 , and the main system 60 .

[0020] The device controller 30 is configured as an LSI (Large-Scale Integrated Circuit) that executes the transfer of information between devices, like a south bridge. As shown in the figure, devices such as the system controller 24, media drive 32, USB module 34, flash memory 36, wireless communication module 38, wired communication module 40, subsystem 50, and main system 60 are connected to the device controller 30. The device controller 30 absorbs differences in electrical characteristics and data transfer speeds of the respective devices and controls the timing of data transfer.

[0021] The media drive 32 is a drive device that is driven by mounting a ROM medium 44 on which application software such as games and license information are recorded, and that reads out programs, data, and the like from the ROM medium 44. The ROM medium 44 is a read-only recording medium such as an optical disk, a magneto-optical disk, or a Blu-ray disk.

[0022] The USB module 34 is a module that connects to an external device via a USB cable. The USB module 34 may be connected to the auxiliary storage device 2 and the camera 7 via a USB cable. The flash memory 36 is an auxiliary storage device that constitutes an internal storage. The wireless communication module 38 wirelessly communicates with, for example, the input device 6 using a communication protocol such as Bluetooth (registered trademark) protocol or IEEE802.11 protocol. The wired communication module 40 communicates with an external device via a wired connection, and connects to the network 3 via, for example, an AP 8.

[0023] 3 shows functional blocks of the information processing device 10. The information processing device 10 includes a processing unit 100, a communication unit 102, and a reception unit 104. The processing unit 100 includes a game execution unit 110, a system image generation unit 112, a data acquisition unit 114, an image analysis unit 120, a time measurement unit 122, a display processing unit 130, and a recording unit 140.

[0024] 3, each element described as a functional block performing various processes of the information processing device 10 can be configured in hardware as a circuit block, memory, or other LSI, and in software as a game software or system software loaded into memory, etc. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various forms by hardware alone, software alone, or a combination thereof, and are not limited to any one of them.

[0025] The recording unit 140 is configured from the auxiliary storage device 2 or the flash memory 36, and records the game software 142. The game software 142 includes at least a game program, and image data and sound data that constitute the game. The game program performs calculation processing to move a game character in a virtual space. In the embodiment, the game character means a game object that is the subject of operation by the user, and includes not only virtual people and animals, but also vehicles and the like.

[0026] The game execution unit 110 executes a game program based on user operations of the input device 6. The game execution unit 110 has a GPU that executes rendering processing and the like, and generates game images including a game character operated by the user. The game execution unit 110 also generates game sound, but in the following description, the processing of the game sound will be omitted.

[0027] 4 shows an example of a game image displayed on the output device 4. When a user selects the game title "ABC Car Race" on a menu screen on which icon images of multiple games are arranged, the game execution unit 110 executes game software 142 for the game title "ABC Car Race" to generate a game image including a game character 80. The game execution unit 110 supplies the generated game image to the display processing unit 130, and the display processing unit 130 displays the game image on the output device 4.

[0028] The game software 142 does not have a function for moving a ghost character in a game based on control information of a game character previously operated. Therefore, in the information processing device 10 of the embodiment, the system software displays a ghost character superimposed on a game image, so that the user can virtually fight against the ghost character.

[0029] When a predetermined button (for example, a home button) on the input device 6 is operated, the reception unit 104 receives it as a display operation for a system image, and the system image generation unit 112 generates a system image that can be operated by the user. The system image generation unit 112 is realized by system software. The system image generation unit 112 supplies the generated system image to the display processing unit 130, and the display processing unit 130 displays the system image on the output device 4 by superimposing it on the game image.

[0030] FIG. 5 shows an example of a system image superimposed on a game image. The system image generating unit 112 generates a system image 70a including a card-shaped GUI element, and the display processing unit 130 superimposes the system image 70a on the game image and displays it on the output device 4. The system image 70a includes information about the game title "ABC Car Race". In this example, the top three friends with the fastest times are displayed together with their fastest times, and the user's own fastest time is ranked second. The system image generating unit 112 acquires this information from the server device 12. When the user selects the "Play Against Rival" item in the system image 70a, the receiving unit 104 receives the selection operation of the item, and the system image generating unit 112 generates a system image for selecting a ghost player.

[0031] FIG. 6 shows another example of a system image superimposed on a game image. The system image generating unit 112 generates a system image 70b for selecting a ghost player, and the display processing unit 130 superimposes the system image 70b on the game image and displays it on the output device 4. The system image 70b includes options for ghost players who will be the user's opponents. In the example shown in FIG. 6, "Smith", the world record holder, "Tatsuya", who has the fastest time among the friends, and the user himself are displayed as options for ghost players. In this embodiment, the user selects "Tatsuya".

[0032] When the reception unit 104 receives the selection operation of "Tatsuya", the data acquisition unit 114 accesses the server device 12 and acquires the video data when "Tatsuya" achieved the fastest time "01:15:12" in the ABC car race, the passing time data associated with the video data, and the checkpoint specific image. The data acquisition unit 114 records the acquired video data 144, passing time data 146, and checkpoint specific image 148 in the recording unit 140.

[0033] In the information processing system 1, the server device 12 is provided with past game videos from a plurality of users. The server device 12 may function as a distribution server that distributes the game videos provided by a plurality of users. Users can access the server device 12 and enjoy the game videos posted by various users.

[0034] In addition to its function as a distribution server, the server device 12 of the embodiment has a function of processing game videos provided by users and generating video data for ghost play. Specifically, the server device 12 generates video data by extracting game character images from past game videos and registers the data in a database. In processing the game video of the ABC car race, the server device 12 generates video data by extracting only the cars, which are game characters, from the game video (i.e., deleting images other than the cars). At this time, the server device 12 may process the game characters to be semi-transparent. The start point of the video data may be the timing when the race starts, and the end point may be the timing when the game characters reach the finish line, but may also be another timing as described below.

[0035] The server device 12 also analyzes the game video to determine the time taken for the game character to pass one or more checkpoints (the time elapsed from the start to the checkpoint). Here, a checkpoint is defined as a specific scene in the game video. In the ABC car race, a scene in which a specific tree placed beside the course or a specific signboard is displayed may be defined as a checkpoint. The checkpoint may be manually defined by a curator. A specific image for detecting a checkpoint (a checkpoint specific image) is associated with the game and registered in a database.

[0036] The server device 12 performs image analysis of the game video to determine the passing time at the defined checkpoints. In the embodiment, a first checkpoint, a second checkpoint, and a third checkpoint are defined, and the server device 12 performs image analysis of the game video of the ABC car race to detect checkpoint-specific images and determine the passing time at each checkpoint. If a checkpoint is set to a "tree" placed beside the course, the server device 12 analyzes each frame of the game video in order and determines the passing time at the checkpoint based on the frame number of the frame in which the set "tree" is first detected. The server device 12 associates a semi-transparent game character image with the extracted video data, and registers passing time data indicating the passing time at each checkpoint in the database.

[0037] The server device 12 receives a request to acquire video data and passing time data associated with the video data from the information processing device 10. In the above example, the server device 12 receives a request to acquire video data and passing time data when "Tatsuya" achieved the fastest time of "01:15:12". The server device 12 reads out the corresponding video data and passing time data, and the checkpoint specific image of the ABC car race from the database, and transmits them to the information processing device 10.

[0038] In this way, the data acquisition unit 114 acquires from the server device 12 the video data 144 and passing time data 146 when “Tatsuya” achieved the fastest time of “01:15:12”, as well as the checkpoint specific image 148, and records them in the recording unit 140.

[0039] FIG. 7 shows an example of a battle screen with a ghost character. The display processing unit 130 reads out the moving image data 144 from the recording unit 140, and superimposes a game character image included in the moving image data 144 on a game image generated by the game executing unit 110. FIG. 7 shows a state in which a ghost character 82 is displayed superimposed on a game image. This ghost character 82 is a game character image included in the start frame of the moving image data 144 (i.e., the frame at the timing when the race starts), and playback of the moving image data 144 has not started at this point. Since the ghost character 82 is made semi-transparent in the moving image data 144, the user can easily recognize that the ghost character 82 is an opponent game character.

[0040] The display processing unit 130 provides a passing time display area 90 on the left side of the screen, which displays the time when the ghost player "Tatsuya" passed through the checkpoint. By providing the passing time display area 90, the display processing unit 130 reduces the area in which the game image is displayed. Note that the passing time display area 90 may be displayed superimposed on the game image, in which case the display area of ​​the game image does not need to be reduced.

[0041] The display processing unit 130 reads out the passing time data 146 from the recording unit 140, and displays the passing times of the checkpoints in the passing time display area 90. In this embodiment, the passing times of "Tatsuya" at the first checkpoint, the second checkpoint, and the third checkpoint, as well as the finish time, are displayed in the passing time display area 90.

[0042] 8 shows a scene where the countdown to the start of the race has begun. In the ABC car race, a countdown of "3," "2," "1," and "GO" is displayed before the start of the race. The image analysis unit 120 performs image analysis on the game images generated by the game execution unit 110, and detects a start image for determining the timing to start playing the video data 144.

[0043] The image analysis unit 120 analyzes the frame images generated by the game execution unit 110 to detect the countdown display "3". The countdown display "3" indicates that the race will start in 3 seconds, and serves as a start image for determining the timing to start playing the video data 144. When the image analysis unit 120 detects "3" in the game image, it notifies the display processing unit 130 that it has detected "3". The display processing unit 130 determines the timing to start playing the video data 144 from the timing at which the image analysis unit 120 detects the start image. If the start image is the countdown display "3", the timing to start playing the video data 144 is 3 seconds after the image analysis unit 120 detects "3".

[0044] In the embodiment, the start image is set to "3" in the countdown display, but the start image may be set to "2" or "1" in the countdown display. The display processing unit 130 determines the timing to start playing the video data 144, and starts playing the video data 144 at the determined timing.

[0045] In the embodiment, the server device 12 generates the video data 144 whose starting point is the timing when the race starts, but it may also generate the video data 144 whose starting point is the timing when the countdown display "3" is displayed. If the start point of the video data 144 is set to the timing when the countdown display "3" is displayed, the display processing unit 130 may determine the timing when the image analysis unit 120 detects the countdown display "3" as the timing to start playing the video data 144, and may immediately start playing the video data 144 when notified by the image analysis unit 120 that "3" has been detected.

[0046] 9 shows a scene where the race has started. When the race starts, the user uses the input device 6 to operate the game character 80. At the same time, the display processing unit 130 starts playing the video data 144, and displays a semi-transparent ghost character 82 superimposed on the game image generated by the game execution unit 110. By looking at the semi-transparent ghost character 82, the user can enjoy a simulated race against "Tatsuya." Below the passing time display area 90, a time display area 92 showing the elapsed time from the start of the race is displayed.

[0047] FIG. 10 shows a scene in which the user's game character 80 has reached the first checkpoint. The image analysis unit 120 analyzes the frame image generated by the game execution unit 110 to check whether or not the frame image contains a checkpoint identifying image 148. Since the first checkpoint, second checkpoint, and third checkpoint appear in this order on the race course, the image analysis unit 120 checks whether or not the frame image contains an image for identifying the first checkpoint (first checkpoint identifying image 148a). Here, the first checkpoint identifying image 148a is an image of a "tree" placed on the right side of the race course.

[0048] When the image analysis unit 120 performs image analysis on the game image and confirms that the first checkpoint identification image 148a is included, it detects that the game character 80 operated by the user has passed the first checkpoint. When passing through the first checkpoint is detected, the time measurement unit 122 acquires the time at which the game character 80 passed through the first checkpoint. In the example shown in Fig. 10, the time measurement unit 122 measures that the game character 80 passed the first checkpoint at "00:16:02".

[0049] The time measurement unit 122 refers to the passing time data 146 recorded in the recording unit 140 and calculates the time difference between the time when the game character 80 operated by the user passes the first checkpoint and the passing time of the first checkpoint included in the passing time data 146 associated with the video data 144. Here, since the passing time of the first checkpoint included in the passing time data 146 is "00:15:21", the time measurement unit 122 calculates the time difference at the first checkpoint as "+00:00:81". Note that "+" in the time difference indicates that the game character 80 is slower than the ghost character 82, and "-" in the time difference indicates that the game character 80 is faster than the ghost character 82. The time measurement unit 122 provides the calculated time difference to the display processing unit 130, and the display processing unit 130 superimposes information related to the calculated time difference on the game image. The user can check whether the game character 80 is ahead of or behind the ghost character 82 by looking at the time difference information 150. The time measurement unit 122 may display the time difference at the first checkpoint in the passing time display area 90.

[0050] For the subsequent second and third checkpoints, the image analysis unit 120 performs image analysis on the game image to detect the second and third checkpoint images, and the time measurement unit 122 calculates the time difference at each checkpoint. Every time the game character 80 reaches a checkpoint, the display processing unit 130 displays time difference information 150 superimposed on the game image, allowing the user to check whether the game character 80 is ahead of or behind the ghost character 82 at each checkpoint.

[0051] FIG. 11 shows a scene where the user's game character 80 has reached the finish line. The image analysis unit 120 analyzes the frame images generated by the game execution unit 110, and when it detects that the game image contains "FINISH", it determines that the race has ended and notifies the time measurement unit 122 and the display processing unit 130. The time measurement unit 122 stops the timer counting up, calculates the finish time, and notifies the display processing unit 130. When the display processing unit 130 receives the notification that the race has ended, it ends the playback of the video data 144 and ends the superimposed display of the ghost character 82. When the display processing unit 130 receives the notification of the finish time, it displays the finish time in the time display area 92. Note that the finish time may be provided by the game execution unit 110, since the game knows the finish time.

[0052] The present disclosure has been described above based on examples. The examples are merely illustrative, and it will be understood by those skilled in the art that various modifications are possible in the combination of each component and each treatment process, and that such modifications are also within the scope of the present disclosure.

[0053] In the embodiment, the start timing of the race is determined by detecting "3" on the countdown display, and the end timing of the race is determined by detecting "FINISH." The images of "3" on the countdown display and "FINISH" may be included in the checkpoint specific image 148, and the image analysis unit 120 may perform image analysis by referring to the checkpoint specific image 148.

[0054] In the course of the game, the game character 80 may not be able to reach the goal, for example, by going off course, crashing, etc. In that case, an image to be displayed when the goal is not reached may be registered as a checkpoint specific image 148, and when the image analysis unit 120 detects the image, the display processing unit 130 may forcibly terminate the playback of the video data 144. [Industrial Applicability]

[0055] The present disclosure can be applied to the technical field of superimposing an image of a game character that has been previously operated on an image of a game being played by a user. [Explanation of symbols]

[0056] 1···information processing system, 10··information processing device, 12···server device, 100··processing unit, 102···communication unit, 104··reception unit, 110··game execution unit, 112···system image generation unit, 114···data acquisition unit, 120···image analysis unit, 122···time measurement unit, 130···display processing unit, 140···recording unit, 142···game software, 144···moving image data, 146···passing time data, 148···checkpoint specific image.

Claims

1. a game execution unit that executes a game program based on a user operation to generate a game image including a game character; a recording unit that records video data obtained by extracting game character images from past game videos; an image analysis unit that analyzes the game image and detects a start image for determining a timing to start playing the video data; a display processing unit that superimposes a game character image included in the moving image data on the game image generated by the game executing unit, the display processing unit determines a timing to start playing the video data from a timing when the image analysis unit detects the start image, and starts playing the video data at the determined timing.

23. An information processing apparatus comprising:

2. the image analysis unit analyzes the game image to detect that a game character operated by a user has passed a predetermined checkpoint; 2. The information processing apparatus according to claim 1,

3. The game device further includes a time measurement unit that acquires the time when the game character operated by the user passes through the checkpoint.

3. The information processing apparatus according to claim 2.

4. the recording unit records a passing time of a checkpoint associated with the video data, the time measurement unit calculates a time difference between a time at which a game character operated by a user passes through a checkpoint and a time at which the game character passes through the checkpoint associated with the video data; the display processing unit superimposes information indicating the calculated time difference on the game image; 4. The information processing apparatus according to claim 3.

5. Executing a game program based on a user operation to generate a game image including a game character; A step of recording moving image data obtained by extracting game character images from past game moving images; analyzing the game image to detect a start image for determining a timing to start playing the video data; determining a timing to start playing the video data from a timing at which the start image is detected; starting playback of the moving image data at the determined timing and superimposing a game character image included in the moving image data on the game image; A game image display method comprising the steps of:

6. On the computer, A function of executing a game program based on a user operation to generate a game image including a game character; A function for recording video data that extracts game character images from past game videos; a function of analyzing the game image and detecting a start image for determining a timing to start playing the video data; a function of determining a timing to start playing the video data from a timing at which the start image is detected; a function of starting playback of the video data at the determined timing and superimposing a game character image included in the video data on the game image; A program to achieve this.

Images