Information processing device, image generation program, and control program

Abstract

An information processing device (3) comprises: a memory (33) for storing a program; and at least one processor (34) for executing the program. The at least one processor functions as: an operation signal acquisition means (35) for acquiring, from an operation device (2), an operation signal indicating details of an input operation to a plurality of buttons provided in the operation device (2); an application execution means (37) for executing an application on the basis of the acquired operation signal; an image generation means (39) for generating a first image (P1) of a screen at the time of execution of the application and a second image (P2) indicating the details of the input operation indicated by the acquired operation signal; and an image output means (41) for outputting the first image (P1) and the second image (P2) generated by the image generation means (39).

Description

Information processing device, image generation program, and control program

[0001] The present invention relates to an information processing apparatus, an image generating program, and a control program.

[0002] Conventionally, a gaming system including a gaming device, a controller for operating the gaming device, and a display device has been known (see, for example, Patent Document 1). In the gaming system described in Patent Document 1, an operation signal corresponding to an operation by a player is transmitted from the controller to the gaming device, and the gaming device executes a game program in response to the received operation signal to progress the game. The gaming device also transmits a game screen showing the progress of the game in progress to the display device, and displays the game screen.

[0003] US Patent Application Publication No. 2009 / 0131171

[0004] In certain game genres, such as fighting games and shooting games, users often rapidly press multiple buttons on a controller. However, even if a game device is capable of recording video of the game, it generally does not display the input operations performed on the controller, making it difficult for users to review the input operations they performed. Furthermore, for game developers, knowing the input operations performed when a bug occurs can be useful in identifying and resolving bugs in the game. However, when a bug is discovered by a user, it is difficult to know the input operations performed when the bug occurred, which can result in a problem of time required to resolve the bug.

[0005] In addition, in some game genres, such as fighting games, commands for special moves and the like may be input by combining input operations on multiple buttons. However, beginners and other inexperienced users have difficulty inputting these commands, which can prevent them from enjoying the game.

[0006] The present invention aims to solve at least part of the above problems, and has a first object to provide an information processing device and a program that can confirm the content of input operations on an operating device, and a second object to provide an information processing device and a program that can be easily operated.

[0007] An information processing device according to a first aspect of the present invention is an information processing device comprising a memory for storing a program and at least one processor for executing the program, wherein the at least one processor functions as an operation signal acquisition means for acquiring from the operation device an operation signal indicating the content of an input operation for a plurality of buttons on the operation device, an application execution means for executing an application based on the acquired operation signal, an image generation means for generating a first image of a runtime screen of the application and a second image indicating the content of the input operation indicated by the acquired operation signal, and an image output means for outputting the first image and the second image generated by the image generation means.

[0008] An image generation program according to a second aspect of the present invention causes at least one processor to function as an operation signal acquisition means for acquiring, from an operation device, operation signals indicating the content of input operations performed on a plurality of buttons on the operation device; an application execution means for executing an application based on the acquired operation signals; and an image generation means for generating a first image of a runtime screen of the application and a second image indicating the content of the input operations indicated by the acquired operation signals.

[0009] An information processing device according to a third aspect of the present invention is an information processing device comprising a memory for storing a program and at least one processor for executing the program, wherein the at least one processor functions as: operation signal acquisition means for acquiring from the operation device an operation signal indicating the content of an input operation for a plurality of buttons of the operation device; application execution means for executing an application based on the acquired operation signal; target button setting means for setting at least one button of the plurality of buttons as a target button; input content recording means for recording the content of the input operation indicated by the acquired operation signal; target operation setting means for setting a portion of the recorded input operation recorded by the input content recording means as a target operation; and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to input of the target button is acquired.

[0010] A control program according to a fourth aspect of the present invention causes at least one processor to function as operation signal acquisition means for acquiring from an operation device an operation signal indicating the content of an input operation for a plurality of buttons of the operation device, application execution means for executing an application based on the acquired operation signal, a target button setting unit for setting at least one button of the plurality of buttons as a target button, input content recording means for recording the content of the input operation indicated by the acquired operation signal, target operation setting means for setting a portion of the recorded input operation recorded by the input content recording means as a target operation, and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to input of the target button is acquired.

[0011] 1 is a schematic diagram showing the configuration of an information processing system according to an embodiment. A front view showing an operation device according to an embodiment. A block diagram showing the configuration of an information processing device according to an embodiment. A functional block diagram showing the configuration of a processor according to an embodiment. A diagram showing an example of an image in which a first image and a second image are superimposed according to an embodiment. A diagram showing an example of a second image generated in a first mode according to an embodiment. A diagram showing an example of a second image generated in a second mode according to an embodiment. A diagram showing an example of a second image generated in a third mode according to an embodiment. A diagram showing an example of a second image generated in a fourth mode according to an embodiment. A diagram showing an example of a comparison image in which the contents of an input operation can be compared according to an embodiment. A diagram showing an example of an image of a target button setting screen according to an embodiment. A diagram showing an example of an image of a target operation setting screen according to an embodiment.

[0012] An embodiment of the present invention will be described below with reference to the drawings. [Configuration of Information Processing System] Fig. 1 is a schematic diagram showing the configuration of an information processing system 1 according to this embodiment. As shown in Fig. 1, the information processing system 1 according to this embodiment includes an operation device 2, an information processing device 3 that executes predetermined processing in response to an operation signal received from the operation device 2, and a display device DP that displays an image received from the information processing device 3. Each component of the information processing system 1 will be described below.

[0013] [Configuration of Controller] FIG. 2 is a front view of the controller 2. As shown in FIG. 2, the controller 2 is a controller configured in an inverted U shape when viewed from the front, and is a type of pointing device. The controller 2 has gripping portions 2L and 2R at the left and right ends, respectively, which are held by the player's left and right hands. Such a controller 2 has a plurality of input-capable controls. Specifically, a directional button 21 capable of inputting up, down, left, and right directions is disposed on the left side of the front portion 2A of the controller 2, and four buttons 22, 23, 24, and 25 are disposed on the right side. Also disposed on the front portion 2A are a two-point detection touchpad 2T and left and right sticks 2SL and 2SR, sandwiched between the area where the directional button 21 is disposed and the areas where the buttons 22 to 25 are disposed. In the following description, the right portion of the directional button 21 is referred to as the right button 21R, the lower portion as the down button 21B, the left portion as the left button 21L, and the upper portion as the up button 21T. Furthermore, when the right button 21R and the down button 21B are input simultaneously, this is the same as when the lower right portion of the directional button 21 is input, so the lower right portion of the directional button 21 is referred to as the lower right button 21BR. Similarly, the lower left portion of the directional button 21 is referred to as the lower left button 21BL, the upper left portion as the upper left button 21TL, and the upper right portion as the upper right button 21TR.

[0014] Buttons 26 and 27 are provided on the left and right sides of the touch pad 2T, and a speaker 2SP is provided below it. A button 28 is provided near the left end of the top surface 2U of the operation device 2, and a button 29 is also provided near the right end.

[0015] The operation device 2 outputs, through the speaker 2SP, a sound corresponding to the audio information received from the information processing device 3. The operation device 2 transmits an operation signal indicating the type of input key or button in response to input from the direction button 21, the various buttons 22 to 29, the touch pad 2T, and the sticks 2SL and 2SR.

[0016] [Configuration of Information Processing Device] Fig. 3 is a block diagram showing the configuration of the information processing device 3. The information processing device 3 executes an OS and applications, and can be configured, for example, by a PC (Personal Computer), a game device, etc. As shown in Fig. 3, the information processing device 3 has a transmitting / receiving unit 31, a communication unit 32, a memory 33, and a processor 34, which are connected to each other by a bus line BL.

[0017] [Configuration of the Transmitter / Receiver] The transmitter / receiver 31 is an interface connected to the display device DP and the controller device 2. For example, the transmitter / receiver 31 transmits an image generated by the processor 34 to the display device DP. Also, for example, the transmitter / receiver 31 communicates with the controller device 2 to receive an operation signal including type information indicating the type of an operator operated on the controller device 2. Furthermore, the transmitter / receiver 31 transmits an audio signal generated by the processor 34 to the controller device 2.

[0018] [Configuration of Communication Unit] The communication unit 32 communicates with the server SV on the network N shown in Fig. 1 under the control of the processor 34. The communication unit 32 downloads various programs and data from the server SV and stores them in the memory 33. Under the control of the processor 34, the communication unit 32 can upload data stored in the memory 33 to the server SV. Such data includes recorded operation information indicating recorded operations, which will be described later.

[0019] [Memory configuration] The memory 33 is configured with a disk-type recording medium such as an HDD (Hard Disk Drive) or a non-volatile memory such as an SSD (Solid State Drive), and a volatile memory, and holds various data as well as basic software such as an OS that controls the operation of the information processing device 3 and programs such as applications.

[0020] [Processor Configuration] Fig. 4 is a functional block diagram showing the configuration of the processor 34. The processor 34 is at least one processor such as a CPU (Central Processing Unit), and controls the operation of the information processing device 3. For example, the processor 34 executes a program such as basic software stored in the memory 33, thereby functioning as an operation signal acquisition means 35, an input content recording means 36, an application execution means 37, a mode setting means 38, an image generation means 39, a record information acquisition means 40, an image output means 41, a target button setting means 42, a command information acquisition means 43, a target operation setting means 44, and an execution control means 45 shown in Fig. 4 .

[0021] [Overview of Image Generation Processing] The operation signal acquisition means 35, input content recording means 36, application execution means 37, mode setting means 38, image generation means 39, recorded information acquisition means 40, and image output means 41 perform the image generation processing by the processor 34 executing a program recorded in the memory 33. The image generation processing is processing that generates a first image of the application runtime screen and a second image showing the content of the input operation on the controller device 2 operated by the user, thereby making it possible to check the content of the input operation on the controller device 2 while the application is running.

[0022] [Configuration of Operation Signal Acquisition Means and Input Content Recording Means] The operation signal acquisition means 35 receives an operation signal from the operation device 2 via the transmission / reception unit 31 and acquires the received operation signal. The input content recording means 36 records an input operation corresponding to the operation signal acquired by the operation signal acquisition means 35 in the memory 33. Specifically, the input content recording means 36 records the type of operator, such as a button, indicated by the acquired operation signal along a time axis. More specifically, the input content recording means 36 records operation record information that associates the elapsed time from a certain point in time with the type of operator indicated by the acquired operation signal. In this embodiment, the input content recording means 36 records operation record information that associates the elapsed time from a certain point in time with the type of button 21 to 25 indicated by the acquired operation signal. At this time, the input content recording means 36 temporarily records the operation record information in the nonvolatile memory of the memory 33, and also records the operation record information in the nonvolatile memory of the memory 33 or an HDD when a user performs an operation to record the input operation content. The input operations recorded by the input content recording means 36 are either input operations based on operation signals acquired between the time when the user performs the recording start operation and the time when the user performs the recording stop operation, or input operations based on operation signals acquired within a predetermined period of time up to the time when the recording operation is performed.

[0023] [Configuration of Application Execution Means and Mode Setting Means] The application execution means 37 reads out an application selected by a user from the memory 33 and executes it. The application execution means 37 then progresses the application based on the acquired operation signal. For example, if the application being executed is a game application, the application execution means 37 progresses the game related to the running game application by, for example, moving a character in response to the acquired operation signal. The mode setting means 38 sets a display mode for input operations. Four display modes, a first mode, a second mode, a third mode, and a fourth mode, are set in the information processing device 3, and the mode setting means 38 acquires an operation signal corresponding to an input operation by the user on the operation device 2 and sets the display mode of the information processing device 3 from the first to fourth modes based on the acquired operation signal.

[0024] [Configuration of Image Generating Means] The image generating means 39 generates an image of a runtime screen of an application executed by the application executing means 37. Furthermore, when an operation signal that activates a display mode that displays the content of input operations on the controller device 2 is acquired from the controller device 2, the image generating means 39 generates an image showing the content of input operations on the multiple operators of the controller device 2. In this embodiment, the image generating means 39 generates various images in addition to an image showing the content of input operations on the multiple buttons 21 to 25. Hereinafter, an image of a runtime screen of an application will be referred to as a first image, and an image showing the content of input operations on the multiple buttons 21 to 25 will be referred to as a second image.

[0025] FIG. 5 is a diagram illustrating an example of a display image DS generated by the image generating means 39, in which a first image P1 and a second image P2 are superimposed. The display image DS illustrated in FIG. 5 is a display image DS generated in the first mode. When the first mode is set by the mode setting means 38, the image generating means 39 generates a display image DS including a first image P1 and a second image P2, as illustrated in FIG. 5. The first image P1 included in the display image DS illustrated in FIG. 5 is an image of a screen displayed when a fighting game application is executed, and includes two characters CH1 and CH2 that can move within the screen displayed when the application is executed. The display image DS illustrated in FIG. 5 is an image in which the second image P2 is superimposed on the first image P1. However, the present invention is not limited to this. The image generating means 39 may generate a display image DS in which the first image P1 and the second image P2 are separated.

[0026] [Display Image Generated in First Mode] Figure 6 is a diagram showing an example of a second image P21 generated in the first mode. Of the second images P2, the second image P21 generated in the first mode is an image in which one of the vertical and horizontal axes represents a time axis, and figures representing the types of input buttons that are input from among the multiple buttons 21 to 25 are arranged along the other axis, as shown in an example in Figure 6. Specifically, the second image P21 is an image in which the vertical axis represents a time axis, and figures F representing the types of input buttons are arranged along the horizontal axis. When a new operation signal is acquired, the image generating means 39 scrolls figures F representing previously input buttons upward within the second image P2, and arranges figures F representing input buttons indicated by the newly acquired operation signal in the bottom row. Note that the second image P21 may be an image in which figures F representing the most recent operation signal are arranged in the top row, and figures F representing input buttons whose input time has elapsed longer are arranged downward.

[0027] The figures F indicating the type of input button include figures F01 to F08, which indicate the input direction of the directional button 21 when the directional button 21 is input, as well as figures F09 to F12, which indicate that buttons 22 to 25 are input. Figure F01 is a figure imitating the right button 21R, and figure F02 is a figure imitating the bottom right button 21BR. Figure F03 is a figure imitating the bottom button 21B, and figure F05 is a figure imitating the left button 21L. Although not shown in FIG. 6 , figure F04 is a figure imitating the bottom left button 21BL, and figure F06 is a figure imitating the top left button 21TL. Figure F07 is a figure imitating the top button 21T, and figure F08 is a figure imitating the top right button 21TR.

[0028] The figure F09 is a figure that resembles the button 22, and the figure F10 is a figure that resembles the button 23. The figure F11 is a figure that resembles the button 24, and the figure F12 is a figure that resembles the button 25. The second image P2 also includes an indicator LK that connects the figures F that indicate the types of input buttons that have been continuously pressed for a predetermined period of time or more among the plurality of buttons 21 to 25.

[0029] The image generating means 39 updates the second image P21 when at least one of the buttons 21 to 25 is pressed, or at predetermined intervals. In the former case, when a new button is pressed and an operation signal corresponding to that button is acquired, the image generating means 39 moves the graphic F associated with the previously input button up one row and places the graphic F associated with the new input button at the bottom row. In the latter case, after a predetermined period of time has elapsed, the image generating means 39 moves the graphic F associated with the previously input button up one row and places the graphic F associated with the button input during the immediately preceding period at the bottom row. Note that if an operation signal associated with at least one of the buttons 21 to 25 was not acquired during the immediately preceding period, the image generating means 39 places a graphic indicating that no operation signal was acquired at the bottom row, although this is not shown. Then, when a new operation signal is acquired, the image generating means 39 moves the graphic F associated with the previously input button up one row and places the graphic F associated with the new input button at the bottom row, similar to the former case. In this way, the image generating means 39 updates the second image P21.

[0030] [Second Image Generated in Second Mode] Figure 7 is a diagram showing an example of a second image P22 generated in the second mode. Of the second images P2, the second image P22 generated in the second mode is an image in which, as shown in Figure 7, figures indicating the types of input buttons among the multiple buttons 21 to 25 are arranged along one of the vertical and horizontal axes, and gauges indicating the input times for the input buttons are arranged along the other axis. Specifically, the second image P22 is an image in which a figure F indicating the type of input button and the input time TM for the input button are arranged along the vertical axis, and a gauge GA indicating the input time for the input button is arranged in association with the figure F along the horizontal axis. In the example of Figure 7, the second image P22 includes figures F09, F10, and F11 corresponding to the buttons 22, 23, and 24, the input times TM1, TM2, and TM3 for the buttons 22 to 24, and gauges GA1, GA2, and GA3 indicating the input times for the buttons 22 to 24. The second image P22 shows the state in which the buttons 22, 24, and 23 are pressed in this order, and the buttons 22 to 24 are continuously pressed.

[0031] For example, while the button 22 is being pressed, the image generating means 39 extends the gauge GA associated with the figure F09 in the second image P22 and displays the elapsed time at the input time TM1. When the input time for the button 22 has elapsed a predetermined time, the extension of the gauge GA stops, but the display of the elapsed time at the input time TM1 continues. On the other hand, when the button 22 input is released, the image generating means 39 fades out the figure F09, input time TM1, and gauge GA1 associated with the button 22, for example. The image generating means 39 updates the second image P22 at short intervals, such as every 50 msec. That is, the image generating means 39 updates the second image P22 every predetermined time.

[0032] [Second Image Generated in Third Mode] Figure 8 is a diagram showing an example of a second image P23 generated in the third mode. Of the second images P2, the second image P23 generated in the third mode is an image in which one of the vertical and horizontal axes is a time axis TA, and along the other axis, a figure F indicating the type of input button that was pressed among the multiple buttons 21 to 25 and an input time bar TB indicating the input time of the input button are arranged, as shown in an example in Figure 8. More specifically, the second image P23 is an image in which the horizontal axis is the time axis TA, figures F relating to the input buttons are arranged along the vertical axis, and an input time bar TB is arranged extending from the figure F along the time axis TA.

[0033] The second image P23 includes a current time bar BA that extends perpendicular to the time axis TA extending along the horizontal axis and along the vertical axis. The current time bar BA indicates the current time point. The position of the current time bar BA does not change on the time axis TA, while the graphic F associated with the input button and the input time bar TB indicating the input time scroll to the left of the current time bar BA as time passes. During this process, while input to the input button continues, the input time bar TB extends to the left from the current time bar BA while remaining in contact with the current time bar BA. That is, the input time bar TB extends relative to the current time bar BA, while the extension of the input time bar TB stops when the input to the input button is released. When an operation signal corresponding to a new input button is acquired, the image generating means 39 places the graphic F associated with the other input button on the current time bar BA and extends the input time bar TB while the input to the other input button continues. In this way, as time passes, the figure F and the input time bar TB included in the second image P23 scroll leftward relative to the current time bar BA.

[0034] In the example of FIG. 8 , the second image P23 includes figures F09 to F11 associated with buttons 22 to 24, figure F01 associated with the right button 21R, figure F05 associated with the left button 21L, and input time bars TB1, TB2, TB3, TB4, and TB5 associated with the figures F09 to F11, F01, and F05, respectively. The input time bars TB1 to TB5 are extended according to the input time of the corresponding input button. The second image P23 shown in FIG. 8 indicates that the right button 21R was pressed while button 23 was being pressed, and then button 22 was pressed while buttons 23 and 21R were being pressed, and then button 24 was pressed. The second image P23 shown in FIG. 8 indicates that the input for button 23 was first released, then the input for right button 21R was released, and then the input for button 24 was released. 8 shows that the left button 21L is being newly input while the button 22 is still being input. The area to the left of the current time bar BA in this second image P23 scrolls leftward as time passes, as described above. This second image P23 allows the user to see which of the buttons 21 to 25 was input, during what period, and at what timing, while also being able to see the input status of the other buttons.

[0035] [Second Image Generated in Fourth Mode] Figure 9 is a diagram showing an example of a second image P24 generated in the fourth mode. Of the second images P2, the second image P24 generated in the fourth mode is an image including a plurality of figures F arranged along one of the vertical and horizontal axes and representing the plurality of buttons 21-25, and a display PT arranged along the other axis corresponding to at least one of the figures F and indicating the input state of an input button among the plurality of buttons 21-25 that has been pressed. More specifically, the second image P24 includes a plurality of figures F01-F12 arranged along the horizontal axis and representing the plurality of buttons 21-25, and a display PT arranged along the vertical axis corresponding to at least one of the figures F01-F12 and indicating the input state of one of the input buttons 21-25. While the display PT is a circle in the example of Figure 9, it may be a polygon or other shape. In the example of Figure 9, the second image P24 shows that the most recent input operation was simultaneous input of button 23 and the right button 21R, and that immediately before that, the bottom right button 21BR was input.

[0036] When a new operation signal is acquired, the image generating means 39 scrolls downward in the second image P2 the rows in which the display PT is arranged according to the figures F relating to the past input buttons, and arranges the figure F relating to the new input button in the top row. Note that the second image P24 may be an image in which the figures F relating to the most recent input button are arranged in the bottom row, and figures F relating to input buttons whose time since input has been longer are arranged upward.

[0037] As in the first mode, the image generating means 39 updates the second image P24 when at least one of the buttons 21 to 25 is pressed, or at predetermined intervals. In the former mode, when a new button is pressed and an operation signal corresponding to that button is acquired, the image generating means 39 moves the currently displayed rows down one row and places a display PT in the top row at a position corresponding to the graphic F corresponding to the newly input button. In the latter mode, after a predetermined period has elapsed, the image generating means 39 moves the currently displayed rows down one row and places a display PT in the top row at a position corresponding to the graphic F corresponding to the button pressed within the immediately preceding period. Note that if an operation signal corresponding to at least one of the buttons 21 to 25 was not acquired within the immediately preceding period, the image generating means 39 moves the currently displayed rows down one row and places a graphic indicating that no operation signal was acquired in the top row, although this is not shown in the figure. Then, when a new operation signal is acquired, the image generating means 39, as in the former case, moves down each row in which the figure F corresponding to the previous input button is displayed, and places the display PT in the top row at a position corresponding to the figure F corresponding to the new input button. In this way, the image generating means 39 updates the second image P24. Note that, as with the second image P21, the second image P24 includes an indication LK that connects the figures F indicating the types of buttons among the plurality of buttons 21 to 25 that have been continuously input for a predetermined period of time or more.

[0038] [Configuration of Recorded Information Acquisition Means] The recorded information acquisition means 40 communicates with the server SV via the communication unit 32 and acquires operation information recorded in the server SV. Specifically, the recorded information acquisition means 40 acquires from the server SV operation record information recorded when the same application was executed as the operation record information recorded in the memory 33 by the input content recording means 36. More specifically, the recorded information acquisition means 40 acquires from the server SV operation record information that records input operations by other users.

[0039] FIG. 10 shows an example of a comparison image CM that allows comparison of the contents of input operations. The image generating means 39 then generates a comparison image CM that allows comparison of the operation record information acquired by the record information acquiring means 40 with the operation record information recorded in the memory 33. As shown in FIG. 10 , in the upper section CM1 of the comparison image CM, a figure F indicating the contents of input operations based on the operation record information acquired from the server SV is arranged along the time axis TA shown in the middle section CM2. Note that in the upper section CM1, the names of players who performed input operations based on the operation record information acquired from the server SV are displayed in a player display section CM11. In the lower section CM3 of the comparison image CM, a figure F indicating the contents of input operations based on the operation record information recorded in the memory 33 is arranged along the time axis TA shown in the middle section CM2. Note that in the lower section CM3, the names of players who performed input operations based on the operation record information recorded in the memory 33 are displayed in a player display section CM31. In the comparison image CM, figures F representing multiple input buttons pressed at the same time are arranged in a direction perpendicular to the time axis TA. For this reason, in the example of Fig. 10, the upper row CM1 and the lower row CM3 are provided with sections in which figures F01 and F12 are arranged along an axis perpendicular to the time axis TA. By checking such a comparison image CM, it is possible to compare the content of input operations by other users with the content of input operations by oneself.

[0040] [Configuration of Image Output Means] The image output means 41 outputs the image generated by the image generation means 39 via the transmission / reception unit 31. When the transmission / reception unit 31 is connected to the display device DP, the image generated by the image generation means 39 is transmitted to the display device DP by the image output means 41, and the display device DP displays the received image.

[0041] [Overview of Execution Control Processing] The operation signal acquisition means 35, input content recording means 36, application execution means 37, target button setting means 42, command information acquisition means 43, target operation setting means 44, and execution control means 45 are executed by the processor 34 reading out a program recorded in the memory 33. The execution control processing is processing in which at least one button out of the plurality of buttons 21 to 25 is set as a target button, and when the set target button is input, the application execution means 37 executes processing according to the preset target operation.

[0042] [Configuration of Target Button Setting Means] FIG. 11 is a diagram showing an example of an image ST of a target button setting screen. The target button setting means 42 sets at least one button of the plurality of buttons 21 to 25 as a target button. When the target button setting means 42 sets a target button, the image generation means 39 generates the image ST of the target button setting screen shown in FIG. 11. The image ST includes a display area ST1 in which a design resembling the operation device 2 is arranged, and an explanatory text ST2 for the setting screen. The image ST generated by the image generation means 39 is output to an external device such as a display device DP by the image output means 41. When a target button setting screen including such an image ST is displayed, if an operator signal corresponding to input of at least one of the plurality of buttons 21 to 25 is acquired, the target button setting means 42 sets the button indicated by the acquired operation signal as the target button. Note that if operation signals corresponding to input of two of the plurality of buttons 21 to 25 are acquired, the target button setting means 42 sets those two buttons as target buttons based on the acquired operation signals.

[0043] [Configuration of Command Information Acquisition Means] The command information acquisition means 43 shown in Fig. 4 acquires command information from the server SV via the communication unit 32. More specifically, the command information acquisition means 43 acquires command information indicating a specific command in an application executable by the application execution means 37. Note that a command is an input operation on the multiple buttons 21 to 25 to execute a specific function, and is input by successively operating two or more of the multiple buttons 21 to 25. The command information indicates the content of the input operation corresponding to the specified command.

[0044] [Configuration of Target Operation Setting Means] The target operation setting means 44 sets, as a target operation, a part of the recorded input operations, which are input operations recorded by the input content recording means 36. There are two methods for the target operation setting means 44 to set the target operation.

[0045] FIG. 12 is a diagram illustrating an example of an image TP of a target operation setting screen. The first setting method is a setting method in which a range of input operations selected by the user from among recorded input operations is set as target operations. When the target operation setting means 44 sets a target operation using the first setting method, the image generation means 39 generates the image TP of the target operation setting screen shown in FIG. 12. The image TP has a time axis TA on one of its vertical and horizontal axes, and figures F indicating the type of input button are arranged along the other axis based on the content of the recorded input operation. In the example of FIG. 12, time moves backward toward the left along the time axis TA, and moves closer to the present toward the right. The image TP includes a selection range TP1 for selecting multiple figures F. The user selects the input operation to be set as the target operation by changing the position or size of the selection range TP1. The target operation setting means 44 sets the input operation included in the selection range TP1 as the target operation. In the example of FIG. 12, the target operation setting means 44 sets an input operation of sequentially inputting the down button 21B, the lower right button 21BR, the right button 21R, and the button 24 as the target operation.

[0046] The second setting method is a setting method in which, among the recorded input operations, an input operation corresponding to a command corresponding to the command information acquired by the command information acquisition means 43 is set as the target operation. In the second setting method, for example, the target operation setting means 44 compares the recorded input operations with the command information and extracts commands included in the recorded input operations. Then, if there are multiple extracted commands, the target operation setting means 44 sets the command selected by the user as the target operation. On the other hand, if there is only one extracted command, the target operation setting means 44 sets that command as the target operation. Note that if there is no extracted command, the target operation setting means 44 may set the target operation using the above-mentioned first setting method, or may not set a target operation.

[0047] [Configuration of Execution Control Means] When the execution control means 45 acquires an operation signal corresponding to input of a target button set by the target button setting means 42, it causes the application execution means 37 to execute processing corresponding to the target operation. That is, when the acquired operation signal is an operation signal corresponding to input of a target button, the execution control means 45 causes the application execution means 37 executing the application to execute processing corresponding to the target operation. As a result, for example, when the target operation set by the target operation setting means 44 is an input operation for performing a predetermined special move in a game application being executed by the application execution means 37, the execution control means 45 causes the application execution means 37 to execute processing for performing the special move. Note that when multiple buttons are set as target buttons, the execution control means 45 executes processing corresponding to the target operation when the multiple buttons are operated simultaneously. However, this is not limited to this, and the execution control means 45 may execute processing corresponding to the target operation when at least one of the multiple buttons set as target buttons is input.

[0048] Effects of the Embodiment The information processing device 3 according to the present embodiment described above provides the following effects. The information processing device 3 includes a memory 33 and a processor 34. The memory 33 stores a program. The processor 34 executes the program recorded in the memory 33 to function as an operation signal acquisition means 35, an application execution means 37, an image generation means 39, and an image output means 41. The operation signal acquisition means 35 acquires, from the controller device 2, an operation signal indicating the content of an input operation performed on the multiple buttons 21 to 25 of the controller device 2. The application execution means 37 executes an application based on the acquired operation signal. The image generation means 39 generates a first image P1 of a runtime screen of the application executed by the application execution means 37 and a second image P2 indicating the content of the input operation indicated by the acquired operation signal. More specifically, the image generation means 39 generates, as the second image P2, one of second images P21, P22, P23, and P24. The image output means 41 outputs the first image P1 and the second image P2 generated by the image generation means 39 .

[0049] According to this configuration, a second image P2 indicating the content of an input operation performed on the plurality of buttons 21 to 25 of the controller device 2 is output, and the content of the input operation performed on the plurality of buttons 21 to 25 can be confirmed by checking the output second image P2. Furthermore, since not only the second image P2 but also the first image P1 of the application execution screen is output, the content of the input operation and the execution content of the application can be confirmed simultaneously by checking the first image P1 and the second image P2. Furthermore, the second image P2 is generated separately from the first image P1. Therefore, the first image P1 and the second image P2 can be output even if the application executed by the application execution means 37 changes. This improves the convenience of the information processing device 3.

[0050] As shown in FIG. 6 , the second image P21 generated in the first mode is an image in which one of the vertical and horizontal axes represents the time axis, and a graphic F indicating the type of button input from among the multiple buttons 21 to 25 is arranged along the other axis. Specifically, the second image P21 is an image in which the vertical axis represents the time axis, and a graphic F indicating the type of button input from among the multiple buttons 21 to 25 is arranged along the horizontal axis. The image generation means 39 updates the second image P21 either when at least one of the multiple buttons 21 to 25 is input, or every time a predetermined time period elapses. With this configuration, the graphic F indicating the type of input button is arranged along the time axis in the second image P21, making it easy to understand which buttons among the multiple buttons 21 to 25 were input and in what order. Therefore, when playing a game application such as a fighting game or a shooting game in which input operations are performed quickly using the multiple buttons 21 to 25, it is easy to understand the content of the input operations.

[0051] As shown in Fig. 6, the second image P21 generated in the first mode includes a display LK that connects together figures F indicating the types of buttons among the multiple buttons 21 to 25 that have been continuously pressed for a predetermined period of time or more. With this configuration, the display LK is displayed when a button is pressed and held, making it possible to distinguish between a single button press and a long press. Therefore, the content of the input operation can be accurately understood.

[0052] As shown in FIG. 7 , the second image P22 generated in the second mode is an image in which, along one of the vertical and horizontal axes, graphics F indicating the type of input button that was pressed among the multiple buttons 21-25 are arranged, and along the other axis, a gauge GA indicating the input time of the input button is arranged. Specifically, the second image P22 is an image in which the graphics F relating to the input buttons are arranged along the vertical axis, and a gauge GA indicating the input time of the input button is arranged along the horizontal axis. The image generating means 39 updates the second image P22 every time a predetermined time elapses. This configuration makes it easy to grasp the type of button that was pressed and the input time of that button. For example, when executing a game application such as a racing game that requires input of buttons corresponding to the accelerator and brake, the content of the input operation can be easily grasped.

[0053] As shown in FIG. 8 , the second image P23 generated in the third mode is an image in which one of the vertical and horizontal axes represents a time axis TA, and a graphic F indicating the type of input button that is the input button among the multiple buttons 21-25 and an input time bar TB indicating the input time of the input button are arranged along the other axis. Specifically, the second image P23 is an image in which the horizontal axis represents the time axis TA, and a graphic F indicating the type of input button that is the input button among the multiple buttons 21-25 and an input time bar TB indicating the input time of the input button are arranged along the vertical axis. The image generating means 39 updates the second image P23 every time a predetermined time elapses. With this configuration, the user can easily grasp the long press time of a long pressed input button by checking the input time bar TB. Furthermore, even if two or more of the multiple buttons 21-25 are pressed simultaneously, the graphic F indicating the type of each input button is displayed, making it easy to grasp the input timing of each input button.

[0054] 8, in the second image P23, the input time bar TB indicates the current time point on the time axis TA and extends relative to the current time bar BA extending along the vertical axis. With this configuration, when a button is pressed and held down, the input time bar TB extends relative to the current time bar BA, making it easy to confirm which button is currently being pressed and held down.

[0055] As shown in FIG. 9 , the second image P24 generated in the fourth mode includes a plurality of figures F arranged along one of the vertical and horizontal axes, each representing a corresponding one of the buttons 21-25, and a display PT arranged along the other axis corresponding to at least one of the figures F, indicating the input state of an input button that has been pressed among the buttons. The second image P24 includes a plurality of figures F representing each of the buttons 21-25 arranged along the horizontal axis, each corresponding to at least one of the figures F, and a display PT indicating the input state of the input button among the buttons 21-25 arranged along the vertical axis. The image generating means 39 updates the second image P24 either when at least one of the buttons 21-25 is pressed or at predetermined intervals. This configuration allows the user to easily determine which of the buttons 21-25 has been pressed and at what time.

[0056] The second image P24 includes an indicator LK that connects together figures F that indicate the types of buttons that have been pressed continuously for a predetermined period of time among the multiple buttons 21 to 25. This configuration makes it easy to identify the type of button that has been pressed and held down. Therefore, input operations for the multiple buttons 21 to 25 can be easily confirmed.

[0057] In the information processing device 3, the image output means 41 outputs an image in which the second image P2 is superimposed on the first image P1. More specifically, the image output means 41 outputs an image in which one of the second images P21, P22, P23, and P24 is superimposed on the first image P1. With this configuration, the user observes an image in which the second image P2 is superimposed on the first image P1. This allows the user to check the content of the input operation included in the second image P2 while checking the first image P1. This makes it easier to understand the content of the input operation.

[0058] In the information processing device 3, the processor 34 functions as an input content recording means 36 and a recorded information acquisition means 40. The input content recording means 36 records input operations indicated by operation signals in the memory 33. The recorded information acquisition means 40 acquires recorded information of the input operations from an external server SV. The image generation means 39 generates a comparison image CM, an example of which is shown in FIG. 10. The comparison image CM is an image that allows comparison between the content of the input operation indicated by the recorded information acquired by the recorded information acquisition means 40 and the content of the input operation recorded by the input content recording means 36. The image output means 41 outputs the comparison image CM. With this configuration, for example, the content of an input operation by a user can be compared with the content of an input operation by another user acquired from an external source. This improves the convenience of the information processing device 3.

[0059] The information processing device 3 includes a memory 33 and a processor 34. The memory 33 stores a program. The processor 34 executes the program recorded in the memory 33 to function as an operation signal acquisition means 35, an input content recording means 36, an application execution means 37, a target button setting means 42, a target operation setting means 44, and an execution control means 45. The operation signal acquisition means 35 acquires, from the operation device 2, an operation signal indicating the content of an input operation for the plurality of buttons 21 to 25 of the operation device 2. The input content recording means 36 records the content of the input operation indicated by the acquired operation signal. The application execution means 37 executes an application based on the acquired operation signal. The target button setting means 42 sets at least one button of the plurality of buttons 21 to 25 as a target button. The target operation setting means 44 sets a portion of the recorded input operation recorded by the input content recording means 36 as a target operation. When the execution control means 45 acquires an operation signal corresponding to an input from the target button, it causes the application execution means 37 to execute processing corresponding to the target operation.

[0060] With this configuration, for example, when a user presses a target button, the execution control means 45 causes the application execution means 37 to execute processing corresponding to the target operation. By setting a complex input operation as the target operation, the application execution means 37 can be caused to execute processing corresponding to the complex input operation with a simple operation. Furthermore, because the execution control means 45 causes the application execution means 37 to execute the above processing, the application execution means can be caused to execute the above processing even if the application being executed changes.

[0061] In the first setting method, the target operation setting means 44 sets, as the target operation, an input operation selected by the user from among the recorded input operations recorded in the memory 33. With this configuration, the user can set any input operation from the recorded input operations as the target operation.

[0062] In the information processing device 3, the image generating means 39 generates an image TP of a target operation setting screen including at least a part of the recorded input operations. The target operation setting means 44 sets the input operations selected in the image TP as target operations. With this configuration, the user can easily perform the operation of setting the target operation.

[0063] In the second setting method, the target operation setting means 44 extracts a predetermined command from the recorded input operations and sets the extracted command as the target operation. With this configuration, for example, a command composed of complex input operations can be easily set as the target operation.

[0064] The processor 34 functions as a command information acquisition means 43. The command information acquisition means 43 acquires command information indicating the content of an input operation corresponding to a predetermined command. The target operation setting means 44 extracts a predetermined command from the input operations recorded in the memory 33 based on the command information acquired by the command information acquisition means 43. With this configuration, the target operation setting means 44 can automatically set a target operation based on the acquired command information.

[0065] The input operations recorded by the input content recording means 36 are either input operations based on operation signals acquired between the user's start recording operation and the user's stop recording operation, or input operations based on operation signals acquired within a predetermined period before the user's recording operation. With this configuration, if the input operations to be recorded are input operations based on operation signals acquired between the user's start recording operation and the user's stop recording operation, the user can determine the recording period for the input operations. On the other hand, if the input operations to be recorded are input operations based on operation signals acquired within a predetermined period before the user's recording operation, past input operations can be recorded without the user having to perform a start recording operation in advance. This improves the convenience of the information processing device 3.

[0066] [Modifications of the Embodiments] The present invention is not limited to the above-described embodiments, and modifications and improvements within the scope of achieving the object of the present invention are included in the present invention. In the above-described embodiments, the input content recording means 36 records the content of the input operation indicated by the operation signal corresponding to the input of the buttons 21 to 25 of the controller device 2. The image generating means 39 generates the second image P2 including the content of the input operation indicated by the operation signal corresponding to the input of the buttons 21 to 25 of the controller device 2. The target operation setting means 44 sets the target operation based on the content of the input operation recorded in the memory 33, i.e., the content of the input operation performed on the buttons 21 to 25 of the controller device 2. However, this is not limited to this, and the content of the input operation performed on at least one of the other controllers of the controller device 2, such as the buttons 26 to 29, the touchpad 2T, and the sticks 2SL and 2SR, may also be recorded by the input content recording means 36, may be generated as the second image P2 by the image generating means 39, and may be set as the target operation by the target operation setting means 44.

[0067] In the above embodiment, the vertical and horizontal axes of the second images P21 to P24, the comparison image CM, and the image TP of the target operation setting screen generated by the image generating means 39 were defined as described above. However, the vertical and horizontal axes of the second images P21 to P24, the comparison image CM, and the image TP of the target operation setting screen may be reversed. For example, in the second image P21, the vertical axis is the time axis, and the figures F related to the input buttons are arranged along the horizontal axis. However, this is not limiting, and the second image P21 may also be an image in which the horizontal axis is the time axis and the figures F are arranged along the vertical axis.

[0068] In the above embodiment, the information processing device 3 has a function of generating the second image P2 showing the content of the input operation and a function of setting a target operation for a target button. However, this is not limited to this, and the information processing device 3 may have only one of the two functions.

[0069] In the above embodiment, the target operation setting means 44 is capable of setting the target operation using either a first setting method in which an input operation selected by the user from recorded input operations recorded in the memory 33 is set as the target operation, or a second setting method in which an input operation corresponding to an acquired command is extracted from the recorded input operations and the extracted input operation is set as the target operation. However, this is not limiting, and the target operation setting means 44 may set the target operation using either the first setting method or the second setting method.

[0070] In the above embodiment, the program read and executed by the processor 34 is recorded in the memory 33. The memory 33 is configured to include a disk-type recording medium or non-volatile memory and a volatile memory. However, this is not limiting. The program may be recorded on an optical disk, which is a disk-type recording medium, and read from the optical disk when the processor executes the above-mentioned process. The program executed by the processor 34 may also be obtained from a device on the network N.

[0071] [Summary of the Invention] The summary of the invention is provided below: [1] An information processing device including a memory for storing a program and at least one processor that executes the program, wherein the at least one processor functions as: operation signal acquisition means that acquires, from an operation device, operation signals indicating the content of input operations performed on a plurality of buttons of the operation device; application execution means that executes an application based on the acquired operation signals; image generation means that generates a first image of a runtime screen of the application and a second image that indicates the content of the input operation indicated by the acquired operation signals; and image output means that outputs the first image and the second image generated by the image generation means.

[0072] According to this configuration, a second image showing the content of input operations performed on multiple buttons of the operating device is output, so that the content of the input operations performed on the multiple buttons can be confirmed by checking the output second image. Furthermore, since not only the second image but also a first image of the application's runtime screen is output, the content of the input operations and the execution content of the application can be confirmed simultaneously by checking the first and second images. Furthermore, the second image is generated separately from the first image. Therefore, the first and second images can be output even if the application being executed changes. Therefore, the convenience of the information processing device can be improved.

[0073] [2] The information processing device according to [1], wherein the second image is an image in which one of the vertical and horizontal axes represents a time axis and a graphic indicating the type of button input from among the plurality of buttons is arranged along the other axis, and the image generating means updates the second image either when at least one of the plurality of buttons is input or every time a predetermined time has elapsed. With this configuration, the graphic indicating the type of button input is arranged along the time axis in the second image, making it easier to understand which of the plurality of buttons were input and in what order. Therefore, when playing a game application such as a fighting game or a shooting game in which input operations using multiple buttons are performed quickly, it is easier to understand the details of the input operations.

[0074] [3] In the information processing device described in [2], the second image includes a display that connects together figures indicating the types of buttons among the plurality of buttons that have been continuously pressed for a predetermined time or more. With this configuration, the display is made when a button is pressed and held, so it is possible to distinguish between a single input and a long press of the button. Therefore, the content of the input operation can be accurately understood.

[0075] [4] In the information processing device described in [1], the second image is an image in which, along one of a vertical axis and a horizontal axis, a figure indicating the type of input button that is the button that has been pressed among the plurality of buttons is arranged, and along the other axis, a gauge indicating the input time of the input button is arranged, and the image generating means updates the second image every time a predetermined time has elapsed. This configuration makes it easier to grasp the type of button that has been pressed and the input time of that button. For example, when playing a game application such as a racing game that requires input of buttons corresponding to the accelerator and brake, it is easier to grasp the content of the input operation.

[0076] [5] In the information processing device described in [1], the second image is an image in which one of the vertical and horizontal axes represents a time axis, and along the other axis, a graphic indicating the type of input button that is an input button among the plurality of buttons and an input time bar indicating the input time of the input button are arranged, and the image generating means updates the second image every time a predetermined time elapses. With this configuration, by checking the input time bar, the long press time of the input button that was pressed and held can be easily understood. Furthermore, even if two or more of the plurality of buttons are pressed simultaneously, a graphic indicating the type of each input button is displayed, so the input timing of each input button can be easily understood.

[0077] [6] The information processing device according to [5], wherein the input time bar indicates a current point on the time axis and extends relatively together with a current time bar extending along the other axis. With this configuration, when a button is pressed and held down, the input time bar extends together with the current time bar, making it easy to confirm which button is currently being pressed and held down.

[0078] [7] The information processing device according to [1], wherein the second image includes a plurality of figures each representing the plurality of buttons, arranged along one of a vertical axis and a horizontal axis, and a display arranged along the other axis corresponding to at least one of the plurality of figures, and showing an input state of an input button that is an input button among the plurality of buttons, and the image generating means updates the second image either when at least one of the plurality of buttons is input or at predetermined intervals. With this configuration, it is possible to easily grasp which of the plurality of buttons has been input and at what timing.

[0079] [8] The information processing device according to [7], wherein the second image includes a display that connects together figures indicating the types of buttons that have been pressed continuously for a predetermined time or longer among the plurality of buttons. This configuration makes it easier to confirm the type of a button that has been pressed and held down. Therefore, input operations for the plurality of buttons can be easily confirmed.

[0080] [9] In the information processing device described in any one of [1] to [8], the image output means outputs an image in which the second image is superimposed on the first image. With this configuration, the user observes an image in which the second image is superimposed on the first image. Therefore, the user can check the content of the input operation contained in the second image while checking the first image. Therefore, the content of the input operation can be more easily understood.

[0081]

[10] In the information processing device described in any one of [1] to [9], the at least one processor functions as an input content recording means for recording the input operation indicated by the operation signal, and a recorded information acquisition means for acquiring recorded information of the input operation from the outside, the image generation means generates a comparison image that allows a comparison between the content of the input operation indicated by the recorded information acquired by the recorded information acquisition means and the content of the input operation recorded by the input content recording means, and the image output means outputs the comparison image.

[0082] With this configuration, for example, the content of an input operation by a user can be compared with the content of an input operation by another user acquired from outside, thereby improving the convenience of the information processing device.

[0083]

[11] An image generation program that causes at least one processor to function as: operation signal acquisition means that acquires, from an operation device, operation signals indicating the contents of input operations performed on a plurality of buttons of the operation device; application execution means that executes an application based on the acquired operation signals; and image generation means that generate a first image of a runtime screen of the application and a second image indicating the contents of the input operations indicated by the acquired operation signals. Execution of such a control program by at least one processor can produce the same effects as the information processing device described above.

[0084]

[12] An information processing device comprising a memory for storing a program and at least one processor for executing the program, wherein the at least one processor functions as: operation signal acquisition means for acquiring from the operation device an operation signal indicating the content of an input operation for a plurality of buttons of the operation device; application execution means for executing an application based on the acquired operation signal; target button setting means for setting at least one button of the plurality of buttons as a target button; input content recording means for recording the content of the input operation indicated by the acquired operation signal; target operation setting means for setting a part of the recorded input operation recorded by the input content recording means as a target operation; and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to an input of the target button is acquired.

[0085] According to this configuration, for example, when a user inputs a target button, the execution control means causes the application execution means to execute a process corresponding to the target operation. According to this, by setting a complex input operation as the target operation, it is possible to cause the application execution means to execute a process corresponding to the complex input operation with a simple operation. Furthermore, since the execution control means causes the application execution means that executes the application to execute the process, it is possible to cause the application execution means to execute the process even if the application being executed changes.

[0086]

[13] The information processing device according to

[12] , wherein the target operation setting means sets an input operation selected by a user from the recorded input operations as the target operation. With this configuration, the user can set any input operation from the recorded input operations as the target operation.

[0087]

[14] The information processing device according to

[13] , wherein the at least one processor functions as an image generating means for generating an image of a setting screen including at least a part of the recorded input operations, and the target operation setting means sets an input operation within a range selected on the image of the setting screen as the target operation. With this configuration, a user can easily perform an operation for setting the target operation.

[0088]

[15] The information processing device according to

[12] , wherein the target operation setting means extracts a predetermined command from the recorded input operations and sets the extracted predetermined command as the target operation. With this configuration, for example, a command formed by a complex input operation can be easily set as the target operation.

[0089]

[16] The information processing device according to

[15] , wherein the at least one processor functions as command information acquisition means for acquiring command information indicating the content of an input operation corresponding to the predetermined command, and the target operation setting means extracts the predetermined command from the recorded input operations based on the command information acquired by the command information acquisition means. With this configuration, the target operation setting means can automatically set the target operation based on the acquired command information.

[0090]

[17] In the information processing device described in any one of

[12] to

[16] , the input operation recorded by the input content recording means is either an input operation based on the operation signal acquired between the start recording operation and the end recording operation, or an input operation based on the operation signal acquired within a predetermined period before the recording operation. With this configuration, if the input operation to be recorded is an input operation based on an operation signal input between the start recording operation and the end recording operation, the user can determine the recording period for the input operation. On the other hand, if the input operation to be recorded is an input operation based on an operation signal input within the predetermined period before the recording operation, past input operations can be recorded without first performing the start recording operation. This improves the convenience of the information processing device.

[0091]

[18] A control program comprising: at least one processor configured to function as: operation signal acquisition means for acquiring, from an operation device, operation signals indicating the contents of input operations for a plurality of buttons of the operation device; application execution means for executing an application based on the acquired operation signals; target button setting section for setting at least one button of the plurality of buttons as a target button; input content recording means for recording the contents of the input operation indicated by the acquired operation signals; target operation setting means for setting a part of the recorded input operations recorded by the input content recording means as a target operation; and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to an input of the target button is acquired. By having at least one processor execute such a control program, it is possible to achieve the same effects as the above-mentioned information processing device.

[0092] 2...operation device, 21...direction button, 22-25...button, 3...information processing device, 33...memory, 34...processor, 35...operation signal acquisition means, 36...input content recording means, 37...application execution means, 38...mode setting means, 39...image generation means, 40...recorded information acquisition means, 41...image output means, 42...target button setting means, 43...command information acquisition means, 44...target operation setting means, 45...execution control means, BA...current Hour bar, CM...comparison image, F, F01, F02, F03, F04, F05, F06, F07, F08, F09, F10, F11, F12...figure, GA, GA1, GA2, GA3...gauge, P1...first image, P2, P21, P22, P23, P24...second image, TA...time axis, TM1, TM2, TM3...input time, TB, TB1, TB2, TB3...input time bar, TP...image (image of the target operation setting screen), TP1...selection range.

Claims

1. An information processing device comprising a memory for storing a program and at least one processor for executing the program, wherein the at least one processor functions as: operation signal acquisition means for acquiring from the operation device operation signals indicating the content of input operations made to multiple buttons on the operation device; application execution means for executing an application based on the acquired operation signals; image generation means for generating a first image of a runtime screen of the application and a second image indicating the content of the input operations indicated by the acquired operation signals; and image output means for outputting the first image and the second image generated by the image generation means.

2. An information processing device according to claim 1, wherein the second image is an image in which one of the vertical and horizontal axes represents a time axis and a figure indicating the type of button that has been pressed among the plurality of buttons is arranged along the other axis, and the image generating means updates the second image either when at least one of the plurality of buttons is pressed or every time a predetermined time has elapsed.

3. An information processing device according to claim 2, wherein the second image includes a display that connects together figures indicating the types of buttons among the plurality of buttons that have been continuously input for a predetermined period of time or more.

4. An information processing device according to claim 1, wherein the second image is an image in which, along one of the vertical and horizontal axes, figures indicating the type of input button that is the button that has been input from among the plurality of buttons are arranged, and along the other axis, a gauge indicating the input time of the input button is arranged, and the image generating means updates the second image every time a predetermined time has elapsed.

5. An information processing device according to claim 1, wherein the second image is an image in which one of the vertical and horizontal axes is a time axis, and along the other axis, a figure indicating the type of input button that is the button that has been input among the plurality of buttons and an input time bar indicating the input time of the input button are arranged, and the image generating means updates the second image every time a predetermined time has elapsed.

6. An information processing device according to claim 5, wherein the input time bar indicates the current point in time on the time axis and extends relatively together with the current time bar extending along the other axis.

7. An information processing device according to claim 1, wherein the second image includes a plurality of figures arranged along one of the vertical and horizontal axes and representing each of the plurality of buttons, and a display arranged along the other axis according to at least one of the plurality of figures and indicating the input state of an input button that is an input button among the plurality of buttons, and wherein the image generating means updates the second image either when at least one of the plurality of buttons is input or every time a predetermined time has elapsed.

8. An information processing device according to claim 7, wherein the second image includes a display that connects together figures that indicate the types of buttons among the plurality of buttons that have been continuously input for a predetermined period of time or more.

9. An information processing device according to any one of claims 1 to 8, wherein the image output means outputs an image in which the second image is superimposed on the first image.

10. An information processing device according to any one of claims 1 to 8, wherein the at least one processor functions as an input content recording means for recording the input operation indicated by the operation signal, and as a recorded information acquisition means for externally acquiring recorded information that records the input operation, the image generation means generates a comparison image that allows a comparison between the content of the input operation indicated by the recorded information acquired by the recorded information acquisition means and the content of the input operation recorded by the input content recording means, and the image output means outputs the comparison image.

11. An image generation program that causes at least one processor to function as: operation signal acquisition means that acquires, from an operation device, operation signals indicating the content of input operations performed on multiple buttons on the operation device; application execution means that executes an application based on the acquired operation signals; and image generation means that generates a first image of a screen when the application is executed and a second image that indicates the content of the input operations indicated by the acquired operation signals.

12. An information processing device comprising a memory for storing a program and at least one processor for executing the program, wherein the at least one processor functions as: operation signal acquisition means for acquiring from the operation device an operation signal indicating the content of an input operation for a plurality of buttons of the operation device; application execution means for executing an application based on the acquired operation signal; target button setting means for setting at least one button of the plurality of buttons as a target button; input content recording means for recording the content of the input operation indicated by the acquired operation signal; target operation setting means for setting a portion of the recorded input operation recorded by the input content recording means as a target operation; and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to the input of the target button is acquired.

13. An information processing device according to claim 12, wherein the target operation setting means sets an input operation selected by the user from among the recorded input operations as the target operation.

14. An information processing device according to claim 13, wherein the at least one processor functions as an image generating means for generating an image of a setting screen including at least a portion of the recorded input operations, and the target operation setting means sets the input operations in a range selected on the image of the setting screen as the target operations.

15. An information processing device according to claim 12, wherein the target operation setting means extracts a predetermined command from the recorded input operations and sets the extracted predetermined command as the target operation.

16. An information processing device according to claim 15, wherein the at least one processor functions as a command information acquisition means for acquiring command information indicating the content of an input operation corresponding to the specified command, and the target operation setting means extracts the specified command from the recorded input operations based on the command information acquired by the command information acquisition means.

17. An information processing device according to any one of claims 12 to 16, characterized in that the input operation recorded by the input content recording means is either an input operation based on the operation signal acquired between the time when a recording start operation is performed and the time when a recording end operation is performed, or an input operation based on the operation signal acquired within a predetermined period of time up to the time when the recording operation is performed.

18. A control program comprising: at least one processor; operation signal acquisition means for acquiring, from an operation device, an operation signal indicating the content of an input operation for a plurality of buttons on the operation device; application execution means for executing an application based on the acquired operation signal; target button setting section for setting at least one button of the plurality of buttons as a target button; input content recording means for recording the content of the input operation indicated by the acquired operation signal; target operation setting means for setting a portion of the recorded input operation recorded by the input content recording means as a target operation; and execution control means for causing the application execution means to execute processing corresponding to the target operation when the operation signal corresponding to input from the target button is acquired.

Images