Information processing system, program, and computer-implemented method

Abstract

To take a new countermeasure against communication delay.SOLUTION: The information processing system includes a host device and one or more guest devices capable of communicating with the host device. The host apparatus acquires first operation data indicating an operation on the host apparatus, receives second operation data indicating an operation on the guest apparatus from the guest apparatus, and executes game processing based on the first operation data and the second operation data. The host device draws a first game image to be displayed on the host device and a second game image to be transmitted to the guest device such that the second game image is displayed before the first game image, displays the first game image on the display device, and transmits the second game image to the guest device. The guest device receives the second game image transmitted from the host device, and causes the display device to display the second game image. The guest device acquires the second operation data and transmits it to the host device.SELECTED DRAWING: Figure 7

Description

[Technical Field]

[0001] The present invention relates to an information processing system, a program, and a method implemented on a computer for executing a game that involves communication between information processing devices. [Background technology]

[0002] Conventionally, there is a technique for executing a game by communicating between information processing devices (see, for example, Patent Document 1). There is also a technique for dealing with delays that occur in communication between information processing devices during a game (see, for example, Patent Document 2). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6646991 specification [Patent Document 2] Patent No. 6927750 specification Summary of the Invention [Problem to be solved by the invention]

[0004] New countermeasures against communication delays are desired.

[0005] Therefore, an object of the present invention is to provide an information processing system, a program, or a method implemented in a computer that is capable of taking a novel measure against communication delays. [Means for solving the problem]

[0006] In order to solve the above problems, the present invention employs the following configurations (1) to (8).

[0007] (1) One example of the present invention is an information processing system including a host device and one or more guest devices capable of communicating with the host device. The host device includes: means for acquiring first operation data indicating operations performed on the host device; means for receiving second operation data indicating operations performed on the guest device from the guest device; means for executing game processing based on the first operation data and the second operation data; means for rendering a first game image indicating the result of the game processing, the first game image being a game image displayed on the host device, and a second game image indicating the result of the game processing, the second game image being a game image transmitted to the guest device, such that the second game image precedes the first game image; means for displaying the first game image on a display device; and means for transmitting the second game image to the guest device. The guest device includes means for receiving the second game image transmitted from the host device, means for displaying the received second game image on a display device, means for acquiring the second operation data, and means for transmitting the second operation data to the host device.

[0008] According to the above configuration (1), the timing of transmitting the game image to the guest device can be advanced, thereby reducing delays in display on the guest device.

[0009] (2) In the above configuration (1), the second game image may be drawn before the first game image within one frame period.

[0010] According to the above configuration (2), when a first game image and a second game image are generated within the same frame period, the second game image can be displayed earlier on the guest device.

[0011] (3) In the above configuration (1) or (2), the second game image may be transmitted without waiting for the timing of switching the display of the first game image on the host device.

[0012] According to the above configuration (3), the timing of transmitting the game image to the guest device can be accelerated.

[0013] (4) In any of the above configurations (1) to (3), the transmission of the second game image may be performed before the start of drawing of the first game image. Note that this configuration includes an embodiment in which, when a common image is included in the first game image and the second game image, the common image is drawn before the transmission of the second game image.

[0014] According to the above configuration (4), the second game image can be displayed more quickly on the guest device.

[0015] (5) In any of the configurations (1) to (4) above, a plurality of guest devices may be present. In the drawing, the second game images transmitted to each guest device may be drawn before the first game image. In the transmission of game images, the drawn second game image may be transmitted to the corresponding guest device without waiting for the drawing of other second game images to be completed.

[0016] According to the above configuration (5), when there are multiple guest devices, the second game image can be transmitted to each guest device more quickly.

[0017] (6) In any of the configurations (1) to (5) above, the display of the game image on the host device may be performed in accordance with the switching timing of the host device, and the display of the game image on the guest device may be performed in accordance with the switching timing of the guest device.

[0018] According to the above configuration (6), the host device and the guest device can draw and display game images without being subject to the constraints that arise when the switching timing between the two devices is synchronized.

[0019] (7) In any of the configurations (1) to (6) above, the drawing may involve drawing at least a portion of an image common to the first game image and the second game image, then drawing the second game image excluding at least a portion of the common image, and then drawing the first game image excluding at least a portion of the common image.

[0020] According to the above configuration (7), it is possible to efficiently draw game images, thereby reducing the time required for the drawing process.

[0021] (8) In any of the above configurations (1) to (7), the host device may further include means for storing a game program for game processing executed by the computer of the host device and system software separate from the game program. The game program may include a display command indicating that the second game image may be displayed on the display. The system software may include a command for transmitting the game image when the execution of the display command is triggered.

[0022] According to the above configuration (8), even if the game program does not include a transmission command, the system software will transmit at an appropriate timing, thereby improving the efficiency of game program development.

[0023] Another example of the present invention may be a program for causing a computer of an information processing device to execute each of the processes in (1) to (8) above. Another example of the present invention may be a program for causing a computer of an information processing device to execute some of the processes in (1) to (8) above (for example, processes executed in a host device described later, or processes executed by a host program and / or a guest program described later). Another example of the present invention may be a method for executing each of the processes in (1) to (8) above in an information processing system. Another example of the present invention may be an information processing device (for example, a host device or a guest device) included in the information processing system in (1) to (8) above. [Brief explanation of the drawings]

[0024] [Figure 1] FIG. 1 is a block diagram showing an example of the configuration of an information processing system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a block diagram showing an example of a hardware configuration of an information processing device. [Figure 3] FIG. 1 is a functional block diagram showing an example of functions possessed by a host device and a guest device. [Figure 4] FIG. 10 is a diagram showing an example of a flow until game processing is executed using operation data acquired by the host device. [Figure 5] FIG. 10 is a diagram showing an example of each piece of operation data stored in the host device in a certain frame. [Figure 6] FIG. 10 is a diagram showing another example of each piece of operation data stored in the host device in a certain frame. [Figure 7] FIG. 10 is a diagram showing an example of the order of processing for generating, transmitting, and displaying a game image, which is executed within one frame period in the individual image mode. [Figure 8] FIG. 10 is a diagram showing an example of the relationship between the rendering process and the game image used therein. [Figure 9] FIG. 1 is a diagram showing an example of various data used in information processing in a host device; [Figure 10]FIG. 10 is a diagram showing an example of various data used for information processing in a guest device. [Figure 11] A flowchart showing an example of the flow of information processing executed by the CPU of the host device according to the game program. [Figure 12] A flowchart showing an example of the flow of information processing executed by the CPU of the host device according to the game program. [Figure 13] A flowchart showing an example of the flow of graphics processing executed by the GPU of the host device by a game program. [Figure 14] 10 is a flowchart showing an example of the flow of processing related to receiving operation data among information processing executed in the CPU of the host device by the host program. [Figure 15] 10 is a flowchart showing an example of the flow of processing related to displaying and transmitting image data among information processing executed in the CPU of the host device by the host program. [Figure 16] 10 is a flowchart showing an example of the flow of a reception process executed by a guest program in a CPU of a guest device. [Figure 17] 10 is a flowchart showing an example of the flow of drawing and transmission processing executed in the CPU of the guest device by a guest program. DETAILED DESCRIPTION OF THE INVENTION

[0025] [1. Hardware configuration of information processing system] An information processing system according to an example of this embodiment will be described below. FIG. 1 is a block diagram showing an example of the configuration of the information processing system according to this embodiment. As shown in FIG. 1, the information processing system 1 includes a plurality of information processing devices (two information processing devices 2 and 3 in FIG. 1). The number of information processing devices included in the information processing system 1 may be any number equal to or greater than two. The information processing devices 2 and 3 may each be any type of information processing device, such as a game console, a smartphone, a tablet terminal, or a personal computer, regardless of whether it is portable or stationary. Furthermore, the information processing devices 2 and 3 may be of different types or the same type.

[0026] The information processing devices 2 and 3 can communicate with each other. In the information processing system 1, the information processing devices 2 and 3 communicate with each other to exchange data for game processing, thereby executing a game. In this embodiment, among the information processing devices included in the information processing system, an information processing device that stores a game program or has the right to execute the game program (information processing device 2 in the example shown in FIG. 1) operates as a host device, and the other information processing device (information processing device 3 in the example shown in FIG. 1) operates as a guest device. Furthermore, in this embodiment, each information processing device functions as both a host device and a guest device depending on the situation. The host device is an information processing device that executes game processing based on a game program and generates game images that indicate the results of the game processing. The generated game images are transmitted from the host device to the guest device. The guest device is an information processing device that does not need to execute game processing based on a game program and receives game images from the host device. The guest device transmits operation data that indicates user operations on its own device to the host device, and the host device executes the game processing based on the operation data. Note that communication between the information processing devices 2 and 3 may be via any mode of communication, such as Internet communication or short-range wireless communication, and may be performed directly between the devices or via another device (e.g., a server).

[0027] FIG. 2 is a block diagram showing an example of the hardware configuration of the information processing device 2. In this embodiment, the information processing device 2 and the information processing device 3 are the same type, and although not shown, the information processing device 3 also has the same configuration as the information processing device 2. In this embodiment, each of the information processing devices 2 and 3 has both a function for operating as a host device and a function for operating as a guest device. In this embodiment, an information processing device equipped with system software and an OS (operating system) described below operates as a host device by executing a game program. Meanwhile, the information processing device can operate as a guest device even if it does not store a game program. However, in other embodiments, the information processing device may operate as a guest device by executing a game program. In other embodiments, the information processing device included in the information processing system 1 may have only one of a function for operating as a host device and a function for operating as a guest device. That is, in other embodiments, the information processing system 1 may include an information processing device that can operate only as a host device and an information processing device that can operate only as a guest device.

[0028] As shown in FIG. 2, the information processing device 2 includes an SoC (System-on-a-chip) 11 that includes a processor and a memory that executes information processing for a game. For example, the SoC 11 includes a processor such as a CPU (Central Processing Unit) and a GPU (Graphics Processing Unit). The SoC 11 is connected to the units 12-16 shown in FIG. 2. Each process of this embodiment may be executed by multiple processors. Also, for example, some of the processes may be executed by multiple processors located at different locations, such as a server processor.

[0029] The information processing device 2 includes an input unit 12. The input unit 12 detects an operation performed by a user on the information processing device 2, and outputs operation data indicating the detected operation. The input unit 12 may include, for example, an operation member such as an analog stick or a button, or may include a sensor such as a gyro sensor or an acceleration sensor. Note that in another embodiment, the information processing device 2 may be configured to be capable of communicating with the information processing device 2 via short-range wireless communication such as Wi-Fi (registered trademark) or Bluetooth (registered trademark), or in some cases via the Internet, and to acquire operation data from a controller including an input unit.

[0030] The information processing device 2 includes a display 13 for displaying images. Note that in other embodiments, the information processing device 2 may not include the display 13, and may instead display images on a display device that can communicate with the information processing device 2.

[0031] The information processing device 2 includes a communication module 14 for communicating with other information processing devices. In this embodiment, the communication module 14 is capable of performing Internet communication as a first communication mode and local communication as a second communication mode. The first communication mode is a mode in which the information processing device 2 connects to the Internet by connecting to a wireless LAN using a method conforming to the Wi-Fi standard, for example, and communicates with other information processing devices connected to the Internet (whether communication is via a server or direct communication). The second communication mode is a mode in which direct wireless communication is performed with other information processing devices within the communication range of the information processing device 2 using a predetermined communication method (for example, a communication method using a proprietary protocol or a method conforming to the Wi-Fi standard). Note that the communication method between the information processing devices included in the information processing system is arbitrary, and may be wired communication or wireless communication.

[0032] The information processing device 2 includes, as examples of storage units, a DRAM 15 and a flash memory 16. The DRAM 15 stores various data used in information processing for executing a game (for example, game processing for progressing the game, processing for transmitting and receiving data to and from other information processing devices for the game, etc.) and generated by the information processing. The flash memory 16 stores various programs for the processor of the information processing device 2 to execute information processing for executing a game.

[0033] In this embodiment, the flash memory 16 stores a game program and system software separate from the game program. The game program is stored in the flash memory 16 by downloading it from a predetermined server (or may be pre-installed in some cases). The game program may be stored in a storage medium (e.g., a memory card) that is detachably attached to the information processing device 2. In this embodiment, the system software refers to a program that is pre-installed in the information processing device 2 regardless of whether or not a user instructs it to do so. The system software includes a host program that is executed when the information processing device 2 operates as a host device, and a guest program that is executed when the information processing device 2 operates as a guest device. The information processing device 2 can store multiple game programs and execute each game program to run different games. The host program and guest program are programs shared by each game program (i.e., programs that are commonly used when each game program is executed).

[0034] [2. Overview of operations in information processing systems] Below, with reference to FIGS. 3 to 8, an example of the operation performed by each information processing device when a game is executed in the information processing system will be described. Each operation may be implemented by either software or hardware, or a combination of both. Below, an example will be described in which the information processing device 2 operates as a host device and the information processing device 3 operates as a guest device. Note that in the following description, the explanations of each functional block (such as "game processing unit 20" and "drawing unit 21") and terms (such as "frame number") are intended to explain specific implementation examples and operations of each functional block in this embodiment. While the explanations may be specific to this embodiment for convenience in order to facilitate understanding of the embodiment, they do not limit the scope of each functional block or terminology itself.

[0035] 3 is a functional block diagram showing an example of functions possessed by the host device and the guest device 3. The host device 2 executes game processing based on a game program, and has the functions of displaying generated game images on its own device and transmitting them to the guest device 3, which include the units 20-25 shown in FIG.

[0036] The game processing unit 20 is a processing unit that executes a game program, and performs game processing (e.g., processing for controlling each object in the game space and updating parameters indicating the game situation) to progress the game based on operation data from the host device and the guest device. The game processing unit 20 passes game parameters indicating the results of the game processing (e.g., parameters indicating the game situation, such as the position and state of each object in the game space) to the drawing unit 21.

[0037] The rendering unit 21 is a processing unit of the GPU and renders game images based on game parameters received from the game processing unit 20. That is, the rendering unit 21 renders game images generated based on the game parameters in a frame buffer. In an individual image mode, described later, game images for the host are rendered in the frame buffer, and game images for the guest are rendered in a memory area for guest images. As described above, in this embodiment, game images are generated on the host device 2 side. In this embodiment, game images are generated by rendering to the frame buffer, but the manner in which game images are generated on the host device 2 is not limited to this. For example, game images may not be rendered directly in the frame buffer, but may be generated in another memory and then transferred to the frame buffer. The rendering unit 21 generates one or more game images to be displayed on each information processing device (i.e., the host device 2 and the guest device 3). The game image for the host device and the game image for the guest device may be the same or different. For example, in the case of a fighting game, the same game image may be displayed on each information processing device, and in the case of a racing game, different game images may be displayed on each information processing device. If the two are different, the rendering unit 21 generates a game image for each of the host device and the guest device. On the other hand, if the two are the same, the rendering unit 21 only needs to generate one game image. Furthermore, if there are multiple guest devices, the game image may be the same for each guest device, or may be different for each guest device.

[0038] In this embodiment, the host device can operate in either a common image mode or an individual image mode. The common image mode is a mode in which the same image as the game image for the host device is transmitted to the guest device as the game image for the guest device (the game image displayed on the display for the guest device). In the common image mode, a game image is generated in the host device, and the same game image is displayed on the host device and the guest device. The individual image mode is a mode in which an image different from the game image for the host device is transmitted to the guest device as the game image for the guest device. In the common image mode, a game image for the host device and a game image for the guest device are generated in the host device, respectively, and the game image for the host device is displayed on the host device, and the game image for the guest device is transmitted from the host device to the guest device and displayed. Note that in the individual image mode, when the same game image is transmitted to multiple guest devices, the game images displayed on each guest device may be the same or different.

[0039] In this embodiment, whether the host device operates in common image mode or individual image mode depends on the game program. More specifically, some game programs include code for generating images that are common to the host device and guest devices, while other game programs include code for generating images that are individual to the host device and guest devices. For example, a game program for a game played from a common viewpoint by all players includes code for generating a common image mode. Also, a game program for a game played from a different viewpoint by each player includes code for generating individual images. In the host device, the game program may generate game images that are common images or individual images, and may specify either the common image mode or the individual image mode to a system program, which may then transmit the game images to the guest device in accordance with the specification.

[0040] Of the generated game images, the game image for the host device is output to the display 13 of the host device 2. In addition, the rendering unit 21 passes, to the transmission control unit 25, the game image for the guest device (which may be the same as the game image for the host device as described above) of the generated game images.

[0041] The rendering unit 21 repeatedly generates game images in a first cycle. In this embodiment, the period during which one game image for the host device is generated and displayed is defined as one frame. The length of the first cycle (i.e., one frame) is, for example, 1 / 60 seconds. In this embodiment, game images for the guest device may be generated in a second cycle (e.g., 1 / 30 seconds) that is longer than the first cycle. Specifically, game images for the guest device are generated once every two frames (i.e., half the frequency of game images for the host device). In other embodiments, the relationship between the lengths of the first cycle and the second cycle may be arbitrary. For example, the first cycle and the second cycle may be the same. Furthermore, the first cycle does not need to be fixed and may be variable. When the first cycle is variable, the second cycle may be fixed or may be variably set in response to changes in the first cycle.

[0042] The transmission control unit 25 reads the game image for the guest device from the frame buffer (in the common image mode) or from the memory area for the guest image (in the individual image mode), and transmits it to the guest device 3 using the communication module 14. In this embodiment, transmission by the transmission control unit 25 is performed in response to an instruction from the game processing unit 20. In this embodiment, the transmission control unit 25 generates image data for transmission by performing predetermined processing on the game image for the guest device passed from the drawing unit 21. Specifically, the transmission control unit 25 performs processing to reduce the resolution of the game image for the guest device passed from the drawing unit 21, and then encodes the game image after this processing. This reduces the amount of image data transmitted to the guest device 3. Note that in other embodiments, the transmission control unit 25 may not perform the processing to reduce the resolution and the encoding.

[0043] In this embodiment, the transmission control unit 25 associates a frame number with the game image and transmits it to the guest device 3. Here, the frame counter 23 counts the number of frames each time a game image for the host device is generated, and passes the frame number indicating the current frame number to the transmission control unit 25. For example, the game processing unit 20 notifies the frame counter 23 in response to the generation of a game image, and the frame counter 23 counts the number of frames in response to this notification. The transmission control unit 25 includes frame number information passed from the frame counter 23 in the game image data. Note that this frame number indicates the frame number when the game image for the host device corresponding to the associated game image for the guest device was generated (which can also be said to be drawn). Note that when the drawing and display of one game image are performed within one frame period in the host device 2, the frame number can also be said to indicate the frame number at which the game image for the host device corresponding to the associated game image for the guest device is displayed.

[0044] In this embodiment, the transmission control unit 25 associates the frame numbers with the game images by embedding (also called writing) frame number information in the metadata area of ​​the image data of the encoded game images. This allows game images with frame numbers to be efficiently generated and transmitted. Note that in other embodiments, any method can be used to associate frame numbers with game images, and the frame numbers may be transmitted as separate data while being accompanied by information regarding the correspondence between the two.

[0045] The ID registration unit 24 stores the ID of the guest device 3 in the DRAM 15. For example, when a guest device to be a communication partner is determined when a game is started, the ID registration unit 24 stores the ID of the guest device in the DRAM 15. The transmission control unit 25 acquires the ID of the guest device 3 from the ID registration unit 24 and transmits a game image associated with a frame number to the information processing device indicated by the acquired ID. As described above, in this embodiment, a game image for the guest device is generated once every two frames, and the game image is also transmitted to the guest device 3 once every two frames. Note that a game image for the guest device may be generated once per frame and transmitted every two frames (in this case, for example, some of the generated data may not be used or may be combined).

[0046] When the game image is transmitted from the host device 2 to the guest device 3 as described above, the game image is drawn (for example, written to a frame buffer) and displayed on the guest device 3. The guest device 3 also transmits operation data indicating operations performed on the guest device 3 to the host device 2. In this embodiment, the guest device 3 includes the units 31-36 shown in FIG. 3 as functions for drawing the game image and transmitting the operation data to the host device 2.

[0047] The reception control unit 31 receives image data transmitted from the host device 2 using the communication module of the guest device 3. The reception control unit 31 decodes the image data and then passes it to the drawing unit 32, and also passes frame number information embedded in the image data to the transmission control unit 35.

[0048] The rendering unit 32 performs rendering processing on the game image for the guest device 3 received from the reception control unit 31. That is, the rendering unit 32 writes the game image to the frame buffer. The game image written to the frame buffer is output to the display of the guest device 3 once per frame. The game image to be rendered may be any of the game images for the guest device 3 received from the reception control unit 31, i.e., from the host device 2. For example, the rendering unit 32 may render the most recent game image among the game images received from the reception control unit 31. Alternatively, depending on the situation, the rendering unit 32 may select the most recent or slightly older game image from among the game images received from the reception control unit 31 that have not yet been rendered, and use that image as the target of the rendering processing. Note that in this embodiment, as described above, the game image is transmitted to the guest device 3 once every two frames, but the rendering unit 32 renders a game image once per frame. That is, if a new game image has not been received, the rendering unit 32 renders the same game image as in the previous frame. In other embodiments, the drawing unit 32 may draw a game image in response to receiving a new game image, and may not perform drawing processing (i.e., not update the frame buffer) for frames in which no game image has been received.

[0049] The operation data acquisition unit 33 acquires operation data output from the input unit of the guest device 3 and passes it to the transmission control unit 35. In this embodiment, the operation data is acquired in a predetermined cycle. The length of this predetermined cycle is arbitrary, but in this embodiment, the operation data is acquired in the first cycle (i.e., for each frame). The transmission control unit 35 transmits the operation data received from the operation data acquisition unit 33 to the host device 2 using the communication module of the guest device 3.

[0050] In this embodiment, the transmission control unit 35 associates the frame number embedded in the game image at the time of acquiring the operation data with the operation data and transmits the operation data. Note that the "game image at the time of acquiring the operation data" means a game image that is being drawn (i.e., already drawn) at the time of acquisition, a game image that is being drawn at that time (for example, a game image that is in the process of being written to the frame buffer), a game image that has already been received at that time, and a game image that is being displayed at that time. In this embodiment, the transmission control unit 35 associates the frame number last received from the reception control unit 31 with the operation data. Therefore, in this embodiment, the operation data is associated with the frame number associated with the game image that is being drawn in the frame buffer at the time the operation data is acquired. Note that in this embodiment, the operation data is transmitted in the above-mentioned predetermined cycle.

[0051] As will be described in detail later, the operation data of the host device 2 is associated with the frame number (in this embodiment, the frame number corresponds to the game image drawn or displayed on the host device) at the time the operation data (operation data of the host device) was acquired. Therefore, by associating a frame number with the operation data on the guest device 3 side as described above, the operation data of the host device 2 and the operation data of the guest device 3 that are associated with the same frame number are acquired for game images generated at the same timing.

[0052] Furthermore, in this embodiment, the transmission control unit 35 transmits an operation number in addition to the frame number associated with the operation data. Specifically, the operation number counter 36 counts the operation numbers and passes the current operation number to the transmission control unit 35. The transmission control unit 35 transmits the operation number passed from the operation number counter 36 associated with the operation data. The operation number is counted so as to increment when the frame number associated with the operation data to be transmitted this time is the same as the frame number associated with the operation data transmitted previously (this occurs when operation data is acquired multiple times while the same game image is being drawn). Furthermore, the operation number is reset to 0 when the frame number associated with the operation data to be transmitted this time is different from the frame number associated with the operation data transmitted previously (that is, when a game image different from that of the previous frame is drawn). Therefore, with respect to a series of operation data transmitted frame by frame, operation data having the same frame number are associated with different operation numbers (specifically, operation numbers that increment by 1).

[0053] The ID registration unit 34 stores the ID of the host device 2 in DRAM. For example, when a game is started and a host device to be a communication partner is determined, the ID registration unit 34 stores the ID of the host device in DRAM. The transmission control unit 35 acquires the ID of the guest device 3 from the ID registration unit 34 and transmits the above-mentioned operation data to the information processing device indicated by the acquired ID.

[0054] When the operation data is transmitted from the guest device 3 to the host device 2 as described above, the host device 2 executes game processing based on the operation data. In this embodiment, the host device 2 includes the units 20, 26-29 shown in FIG. 3 as functions for executing game processing based on the operation data.

[0055] The reception control unit 26 receives operation data transmitted from the guest device 3 using the communication module 14. The reception control unit 26 passes the operation data and the frame number and operation number associated therewith to the operation data storage unit 27, and also extracts the frame number and operation number from the received data and passes them to the delay countermeasure processing unit 29.

[0056] The operation data storage unit 27 stores the operation data from the guest device 3 and the associated frame number and operation number in the DRAM 15. Every time operation data is received from the guest device 3, the operation data is additionally stored in the DRAM 15, and therefore the operation data is accumulated in the DRAM 15.

[0057] Furthermore, the operation data acquisition unit 28 acquires operation data output from the input unit 12 of the host device 2 and passes it to the operation data accumulation unit 27. At this time, the operation data accumulation unit 27 acquires the frame number at which the operation data of the host device 2 was acquired from the frame counter 23, associates the frame number with the operation data, and additionally stores them in the DRAM 15. As a result, the operation data of the host device 2 is also accumulated in the DRAM 15 in the same way as the operation data of the guest device 3. Note that the operation data of the host device 2 is acquired in the above-mentioned predetermined cycle (i.e., at the same frequency as the acquisition of operation data in the guest device 3).

[0058] As described above, in this embodiment, operation data is acquired and accumulated for each frame of the host device 2 and the guest device 3. However, in other embodiments, operation data of the host device 2 and the guest device 3 may be acquired and accumulated at intervals shorter than one frame (for example, multiple times per frame period).

[0059] The delay countermeasure processor 29 executes countermeasure processing for communication delays occurring between the host device 2 and the guest device 3. In this embodiment, the delay countermeasure processor 29 calculates the amount of delay in communication between the host device 2 and the guest device 3 based on the frame number and operation number received from the reception controller 26 (i.e., the frame number and operation number associated with the operation data from the guest device 3) and the frame number indicating the current frame output from the frame counter 23. A specific method for calculating the amount of delay will be described later. For example, the delay countermeasure processor 29 calculates the amount of delay every time operation data is received from the guest device 3. The timing for calculating the amount of delay is not limited to the above, and in other embodiments, the amount of delay may be calculated once for every predetermined number of frames, or once for every predetermined number of operation data received from the guest device 3.

[0060] Furthermore, in this embodiment, the delay countermeasure processing unit 29 selects operation data to be used in game processing from among the accumulated operation data (i.e., operation data of the host device 2 and operation data of the guest device 3) based on the amount of delay. The method for selecting operation data based on the amount of delay will be described later. Information indicating the selected operation data is passed from the delay countermeasure processing unit 29 to the game processing unit 20. Based on the information received from the delay countermeasure processing unit 29, the game processing unit 20 acquires the selected operation data from the accumulated operation data, and executes game processing based on the acquired operation data.

[0061] As shown in FIG. 3 , in this embodiment, the functions of the units 20-29 included in the host device 2 are executed by cooperation between the game program and system software. For example, when the system software executed by the computer (i.e., the processor provided in the SoC 11) receives a game image for the guest device from the game program, the transmission control unit 25 acquires a frame number from the frame counter 23, performs the embedding process described above, and transmits the image to a destination set in accordance with instructions from the game program at the start of the game. Furthermore, the reception control unit 26 receives operation data and frame numbers from the guest device 3 and transmits the respective data to the game program. Based on the above, the information processing device can have the function of associating frame numbers with game images and transmitting them, and the function of extracting frame numbers (and operation numbers) from operation data received from the guest device 3, without installing a specific game program. This eliminates the need to create these functions for each game program, thereby improving the efficiency of game program development.

[0062] 3, in this embodiment, each of the units 31-35 included in the guest device 3 is realized by a computer (in other words, a processor) executing system software. As described above, an information processing device can operate as a guest device without installing a specific game program. This allows a game to be played using a host device and a guest device only if the game program is installed on the host device, and does not require the game program to be installed on the guest device, thereby improving user convenience.

[0063] Furthermore, in this embodiment, one information processing device has, through system software, the functions of both the transmission control unit 25 and the reception control unit 26 on the host device side and the units 31-35 on the guest device side (see FIGS. 2 and 3). This allows the information processing device to function as either a host device or a guest device. Therefore, when a user plays a game based on a game program stored in his / her own information processing device, he / she can play the game with multiple people, with his / her own information processing device as the host device and another user's information processing device as the guest device. Furthermore, when a user plays a game based on a game program stored in another user's information processing device, he / she can play the game with multiple people, with the other user's information processing device as the host device and his / her own information processing device as the guest device. This further improves user convenience.

[0064] It should be noted that which functions of the information processing device are realized by the game program and which functions are realized by the system software is arbitrary. For example, in another embodiment, the function of receiving operation data in the host device 2 may be realized by the game program, or all of the functions shown in Figure 3 may be realized by the game program.

[0065] Next, an example of a method for calculating the amount of delay will be described with reference to Fig. 4. Fig. 4 is a diagram showing an example of the flow up to the execution of game processing using operation data acquired in the first and second frames, where a certain frame in the host device 2 is set as the first frame.

[0066] 4, image data of a game image that is drawn in the host device 2 or transmitted to the guest device 3 in the nth frame (n is a natural number) in the host device 2 is indicated as "image data (n)." Also, operation data of the host device acquired when the game image of the nth frame is drawn is indicated as "operation data H(n)." Also, operation data of the guest device associated with the nth frame (n is a natural number) and associated with an operation number m (m is a natural number) is indicated as "operation data G(n,m)."

[0067] 4, in the nth frame in the host device 2, operation data H(n) of the host device 2 is acquired, and a game image is generated based on the game processing. As described above, the acquired operation data H(n) is accumulated in the host device 2, and the generated image data (n) for the guest device is transmitted to the guest device 3.

[0068] Also, in the (n+1)th frame in the host device 2, similar to the nth frame, operation data of the host device 2 is acquired and a game image based on the game processing is generated. The operation data H(n+1) acquired in the second frame is accumulated in the host device 2. Here, in this embodiment, a game image for the guest device is generated once every two frames, so in the example shown in FIG. 4, a game image for the guest device is not generated or transmitted in the (n+2)th frame.

[0069] Due to a communication delay, the guest device 3 receives the image data (n) transmitted from the host device 2 at a timing different from the timing at which the image data (n) is rendered on the host device 2, and the image data (n) is rendered on the guest device 3. At this time, the operation data acquired in the frame in which the image data (n) is first rendered is "operation data G(n,0)" associated with the frame number "n" and the operation number "0" (see FIG. 4). In the frame, the guest device 3 transmits the operation data G(n,0) to the host device 2.

[0070] Furthermore, in the frame following the frame in which image data (n) is rendered on the guest device 3, no image data is received from the host device 2, and the image data (n) is rendered again. At this time, the operation data acquired in that frame becomes "operation data G(n,1)" associated with the frame number "n" and the operation number "1" (see FIG. 4). In the above frame, the guest device 3 transmits the operation data G(n,1) to the host device 2.

[0071] In this embodiment, image data is transmitted from the host device 2 to the guest device 3 at a rate of once every two frames, so that the guest device 3 basically draws a new game image at a rate of once every two frames. However, in reality, due to factors such as communication conditions, the guest device 3 does not necessarily receive one image data set every two frames. Therefore, for example, it is possible that the same game image is drawn continuously for three or more frames. In this case, the operation number associated with the operation data will be a value of 2 or greater. Furthermore, depending on the communication conditions, there may be only one piece of operation data for a certain game image, in which case operation data with an operation number of "1" will not be generated.

[0072] When the host device 2 receives operation data transmitted from the guest device 3, it calculates the amount of delay based on the frame number and operation number associated with the operation data and the reception timing of the operation data. Here, the amount of delay can be calculated as, for example, the length of time from when image data is transmitted from the host device 2 to the guest device 3 to when operation data corresponding to the image data (i.e., operation data acquired in the frame in which the image data is drawn) is transmitted from the guest device 3 and received by the host device 2. In other words, the amount of delay can be considered as the sum of the time required for the image data to be transmitted from the host device 2 to the guest device 3 ((a) in FIG. 4)) and the time required for the operation data to be transmitted from the guest device 3 to the host device 2 ((b) in FIG. 4).

[0073] Here, when the host device 2 receives operation data (i.e., operation data with operation number 0) acquired in the frame that is drawn for the first time after the guest device 3 receives image data, the difference obtained by subtracting the value of the frame number associated with the operation data from the current frame number (i.e., the value of the frame counter at the time the operation data is received) is the length of the period corresponding to the delay amount. Therefore, the delay amount can be calculated from the difference. In the example shown in Fig. 4, operation data G(n,0) with frame number n is received in the n+5th frame in the host device 2, so the delay amount at this time can be calculated as 5 (frames).

[0074] On the other hand, when the host device 2 receives operation data acquired in a frame that is drawn for the second or subsequent time after the guest device 3 receives image data (i.e., operation data with an operation number of 1 or greater), the difference is the sum of the time (a) above, the time (b) above, and the time from when the image data is drawn for the first time until the operation data is acquired (i.e., one frame) (i.e., (c) shown in FIG. 3). Therefore, the difference does not accurately indicate the amount of delay. Here, since the operation number associated with the operation data indicates the time (c), in the above case, the amount of delay can be calculated based on the frame number and the operation number. Specifically, in the above case, the amount of delay can be calculated as the difference obtained by subtracting the sum of the frame number and the operation number associated with the operation data from the current frame number. In the example shown in FIG. 4, in the (n+6)th frame in the host device 2, operation data G(n,1) with a frame number of n and an operation number of 1 is received, so the amount of delay at this time can be calculated as 5 (frames).

[0075] From the above, in this embodiment, the delay amount is calculated according to the following equation (1). (Delay amount)=(current frame number)−{(frame number associated with operation data of guest device 3)+(operation number associated with operation data of guest device 3)} (1) In this embodiment, the operation number associated with the operation data obtained in the frame in which image data is first drawn after being received in the guest device 3 is set to 0, so the delay amount can be calculated using the above formula (1) whether operation data with an operation number of 0 is received or whether operation data with an operation number of 1 or greater is received.

[0076] In this embodiment, image data is transmitted every two frames from the host device 2 to the guest device 3, and the same game image is rendered across multiple frames on the guest device 3, so an operation number is assigned to each piece of operation data in order to distinguish between the pieces of operation data acquired over the multiple frames. This allows the host device 2 to calculate the amount of delay based on the frame number and operation number, as described above.

[0077] Note that even when image data is transmitted from the host device 2 to the guest device 3 frame by frame (at the frame rate of the host device 2), the guest device 3 may perform the same operation as above. This allows the host device 2 to calculate the amount of delay based on the frame number and operation number. Note that even in the above case, due to factors such as communication conditions, the guest device 3 may not necessarily receive one image data per frame, and image data transmitted from the host device 2 may be lost without being received by the guest device 3. Therefore, by transmitting operation data associated with a frame number and an operation number from the guest device 3 to the host device 2 as described above, the host device 2 can calculate the amount of delay in the same manner as above.

[0078] In another embodiment, the information processing system 1 may associate only a frame number with operation data without assigning an operation number. In this case, for a frame in which operation data associated with a new frame number is received, the host device 2 may calculate a delay amount based on the frame number, and for a frame in which operation data associated with the same frame number as the previous frame is received, the host device 2 may execute game processing based on the last calculated delay amount without calculating a new delay amount. Furthermore, even without assigning an operation number, the timing of operations performed on the guest device may be managed based on the order in which the operation data is received.

[0079] The method for calculating the delay amount using the frame number is not limited to the above, and other methods may be used. For example, in other embodiments, the host device 2 does not need to calculate the delay amount every time it receives operation data from a guest device, but may calculate the delay amount at a fixed interval longer than one frame. For example, the host device 2 may calculate the current delay amount based on delay amounts calculated multiple times up to the present time (for example, by calculating the average of multiple delay amounts). For example, the host device 2 does not need to repeatedly calculate the delay amount, but may calculate the delay amount at a predetermined timing in the game (for example, at the start of the game). The host device 2 may calculate the delay amount without using the frame number (for example, using a conventional method), or may select operation data using a delay amount not based on the frame number using a method described below (see FIGS. 5 and 6).

[0080] In game processing in the host device 2, operation data to be used in the game processing is selected for each of the operation data of the host device 2 and the guest device 3 based on the amount of delay calculated as described above. As will be described in detail later, in this embodiment, the host device 2 selects operation data that was acquired in a frame that is earlier than the amount of delay from the current frame, such that the frame numbers of the operation data of the host device 2 and the operation data of the guest device 3 are the same. For example, in the example shown in FIG. 4 , the amount of delay is calculated to be 5 frames for the (n+5)th frame, so game processing is executed using operation data H(n) with frame number n and operation data G(1,0). Furthermore, the amount of delay is calculated to be 5 frames for the (n+6)th frame, so game processing is executed using operation data H(n+1) with frame number n+1 and operation data G(n,1). This ensures that the operation data of the host device 2 and the operation data of the guest device 3 used in the game processing are acquired for game images generated at the same time, reducing the possibility of an advantage or disadvantage arising between the host device 2 and the guest device 3 in the game.

[0081] Next, an example of a method for selecting operation data will be described with reference to Figures 5 and 6. In Figures 5 and 6, an example will be described in which the information processing system 1 includes three guest devices AC. In this case, image data and operation data are transmitted and received between the host device 2 and each guest device using the method described above. In Figures 5 and 6, symbols (e.g., H(n)) are assigned to the operation data in the same manner as in Figure 4. However, the operation data of the guest devices AC will be referred to as "operation data GA(n,m)," "operation data GB(n,m)," and "operation data GC(n,m)."

[0082] 5 is a diagram showing an example of each operation data accumulated in the host device 2 in a certain frame. In FIG. 5, for example, the host device 2 is in the (n+6)th frame, and operation data H(n) to H(n+6) have been accumulated as the operation data of the host device 2. Also, at this point in time, operation data GA(n,0) to GA(n+4,0) have been acquired as the operation data of guest device A, operation data GB(n,0) to GB(n+2,1) have been acquired as the operation data of guest device B, and operation data GC(n,0) to GC(n+2,0) have been acquired as the operation data of guest device C.

[0083] When there are multiple guest devices, the host device 2 calculates the amount of delay between itself and each guest device AC. In the example shown in Fig. 5, the amounts of delay between the host device and the guest devices AC are calculated as 2 (frames), 3 (frames), and 4 (frames), respectively, according to the above formula (1).

[0084] In this embodiment, the host device 2 determines the largest delay amount between itself and each guest device AC as the reference delay amount. Then, for the operation data of each information processing device (i.e., the host device and each guest device), operation data associated with a frame number indicating a frame that is the reference delay amount before the current frame (hereinafter referred to as a "reference frame") is selected. In the example shown in Figure 5, the reference delay amount is calculated to be 4 (frames), and as a result, operation data with a frame number n+2 is selected for each information processing device (see Figure 5).

[0085] In this embodiment, since image data is transmitted from the host device 2 to the guest device 3 once every two frames, there is no operation data associated with the frame number of a frame for which no image data is transmitted, with respect to the operation data of the guest device. For example, in this embodiment, image data is not transmitted in either an even-numbered frame or an odd-numbered frame (frame n+1 in the example shown in FIG. 4). For example, if image data is not transmitted in an even-numbered frame, there is no operation data with an even-numbered frame number for the operation data of the guest device, and instead, multiple operation data with odd-numbered frame numbers are accumulated. In the example shown in FIG. 5, there is no operation data with frame number n+1 or n+3 for the operation data of the guest device, and multiple operation data with frame number n or n+2 are accumulated. In view of the above, in this embodiment, if there is no operation data associated with a frame number indicating the reference frame, the host device 2 selects the oldest operation data (i.e., the smallest operation number) that has not yet been selected from the operation data associated with a frame number obtained by subtracting 1 from the frame number of the reference frame. For example, if the reference frame is determined to be the n+3 frame based on the reference delay amount, the oldest operation data that has not yet been selected among the operation data with frame number n+2 of the guest device is selected.

[0086] 5 and 6, the host device 2 may delete operation data selected for game processing in a certain frame from the DRAM 15, so that only operation data that has not yet been selected (i.e., not used in game processing) is stored in the DRAM 15. In this case, the host device 2 can select the operation data with the smallest operation number from among the operation data stored in the DRAM 15, which are associated with the frame number of the reference frame (or the frame number obtained by subtracting 1 from the reference frame), thereby selecting the oldest operation data that has not yet been selected.

[0087] As described above, in this embodiment, the host device 2 selects a pair of operation data associated with synchronized frame numbers from the operation data of the host device 2 and the operation data of the guest device, and executes game processing based on the selected operation data. This reduces the possibility of an advantage or disadvantage in the game occurring between the host device and the guest device, since the game processing is executed using operation data acquired at the same (or nearly the same) time in the game.

[0088] Note that the above "set of operation data associated with frame numbers with aligned timing" does not necessarily mean that the frame numbers are the same in the strict sense, but includes a set of operation data selected so that the frame numbers of each operation data fall within a certain range. For example, the host device 2 may select the oldest operation data that has not yet been selected within a range of frame numbers indicating a predetermined number of frames before and after the reference frame.

[0089] Furthermore, as described above, in this embodiment, operation data is selected based on the largest of the delay amounts between the host device and each guest device (i.e., the reference delay amount). This makes it less likely that operation data associated with the frame number of the reference frame has not yet been received by the host device 2, making it easier to select multiple operation data associated with frame numbers that are synchronized in time. This reduces the possibility of an advantage or disadvantage in the game occurring among multiple guest devices.

[0090] It should be noted that when using a set of operation data associated with frame numbers that are synchronized in time, it is not necessary to calculate and use the amount of delay. However, if the amount of delay for a certain guest device becomes very large, for example, operation data from a long time ago will be used in the game processing, and the operation response of all players will be significantly worsened due to the large amount of delay for that guest device. For this reason, in this embodiment, an allowable value (which can also be considered an upper limit) is set for the reference amount of delay.

[0091] 6 is a diagram showing an example of each operation data accumulated in the host device 2 in a certain frame. In FIG. 6, for example, the host device 2 is in the n+7th frame, and operation data H(n) to H(n+7) have been accumulated as the operation data of the host device 2. Also, at this point in time, operation data GA(n,0) to GA(n+4,0) have been acquired as the operation data of guest device A, operation data GB(n,0) to GB(n+2,1) have been acquired as the operation data of guest device B, and operation data GC(n,0) to GC(n,1) have been acquired as the operation data of guest device C.

[0092] In the example shown in FIG. 6, the delay amounts between the host apparatus and guest apparatus AC are calculated to be 3 (frames), 4 (frames), and 6 (frames), respectively.

[0093] In the example shown in FIG. 6, it is assumed that 5 (frames) is set as the allowable value. The host device 2 determines the above-mentioned reference delay amount with the allowable value as the upper limit. Specifically, the reference delay amount is determined to be the largest of the delay amounts between the host device 2 and each guest device that does not exceed the allowable value. In the example shown in FIG. 6, the delay amount between the host device 2 and guest device C is 6 (frames), which exceeds the allowable value, so the reference delay amount is determined to be 4 (frames) based on the delay amount between the host device 2 and guest device B. The allowable value may be any value and may be set to any value taking into account the game content, etc.

[0094] In this embodiment, the host device 2 excludes guest devices with delays greater than the reference delay, and selects operation data based on the reference delay from the operation data of the other guest devices other than the excluded guest devices (hereinafter referred to as "excluded guest devices") and the operation data of the host device. In the example shown in FIG. 6, guest device C, whose communication delay is 6 (frames), is excluded, and operation data is selected for the host device 2 and guest devices A and B based on the reference delay. As a result, the reference frame is the n+3 frame, and operation data H(n+3), operation data GA(n+3,1), and operation data GB(n+2,1) are selected and used in the game processing. In this case, although the delay between the host device 2 and guest device C is 6 (frames), for guest devices A and B, the difference between the current frame and the frame of the operation data used in the game processing is kept to 4 (frames), and a set of operation data associated with frame numbers with aligned timing is selected.

[0095] As described above, in this embodiment, the maximum delay between the host device and each guest device, excluding delays exceeding the allowable value, is set as the reference delay, and operation data is selected based on the reference delay, thereby reducing the possibility that it will take too long for a change in the game image to occur in response to an operation performed by the user.

[0096] Note that the operation data of the excluded guest device may be selected based on a predetermined rule, and the selected operation data may be used in the game processing. For example, the most recently acquired operation data from the accumulated operation data may be used in the game processing. Alternatively, for example, the operation data of the excluded guest device may not be used in the game processing.

[0097] Even when image data is transmitted from the host device for each frame, the guest device transmits operation data to the host device in the same manner as described above. In this case, the host device can select operation data in the same manner as described above using the methods shown in Figures 5 and 6.

[0098] Next, the timing of transmitting and displaying game images in the host device will be described with reference to Fig. 7. Fig. 7 is a diagram showing an example of the order of processing for generating, transmitting, and displaying game images, which is executed within one frame period in the individual image mode.

[0099] As shown in FIG. 7 , in this embodiment, in the process of generating game images for the host device and the guest device, the host device executes a game program to generate a game image for the guest device before generating a game image for the host device. Specifically, a rendering construction command for drawing a game image for the guest device is issued by the game processing unit 20, and the rendering unit 21, which is a processing unit of the GPU, executes a process of drawing the game image for the guest device in the memory area for the game image for the guest device in accordance with the rendering construction command. The rendering construction command is code included in the game program and is a command that instructs the GPU on the content of the drawing. The memory area for the game image for the guest device is predetermined, and in the transmission process described below, the game image is read from that memory area and transmitted. The game processing unit 20 waits for the execution of the rendering construction command and then executes a display command indicating that the game image generated by the rendering construction command may be displayed on the device corresponding to the game image (specifically, the host device or the guest device). The display command is code within the game program. In other words, this command notifies the system program that the game image has been completed, and also indicates that the game image may be displayed on the display of the corresponding game device. Note that the actual timing of display on the display of the game device is not the timing of issuance of this command, but the timing of the subsequent V-sync (the timing of the change of game frames). The transmission control unit 25, which executes the system program, starts the transmission process of the game image as soon as the display command is executed. In this embodiment, the system software includes a transmission program for executing the transmission process. The transmission program starts execution of the transmission process when triggered by the execution of a display command by the game program. The transmission program includes an instruction to transmit the game image in response to the execution of the display command. The transmission process may include processing of the game image into image data for transmission, such as reducing the resolution of the game image or encoding the game image.The transmission control unit 25 reads the game image after these processes from the memory area for the game image for the guest device and transmits it to the guest device.

[0100] Note that game images may be managed in blocks. Processing related to game images may be performed block by block. For example, the host device may perform a rendering process for a game image for each block, and for a block for which rendering has been completed, perform a process for reducing the resolution and an encoding process for that block without waiting for a display command to be issued (or without waiting for the completion of rendering of other blocks). This can shorten the time from when the generation of a game image for the guest device begins to when transmission to the guest device begins.

[0101] While FIG. 7 illustrates an example in which there is one guest device, if there are multiple guest devices, the host device repeatedly executes the generation and transmission processes for the game images for the guest devices for each game image for the guest device. In this case, the host device transmits the drawn game image to one of the multiple guest devices in response to a game image being drawn for that guest device and a display command being executed for that image. At this time, the host device transmits the game image to a certain guest device in response to a display command for the game image to be transmitted to that guest device, without waiting for a display command for a game image for another guest device. Note that the order in which the generation and transmission processes for each game image for multiple guest devices are executed is arbitrary. For example, the host device may execute the generation and transmission processes in an order specified by the game program, or in an order based on the communication status between the host device and each guest device (e.g., starting with the game image corresponding to the guest device with the largest communication delay). Furthermore, if there are multiple guest devices, the game images for the guest devices may be common to each guest device. At this time, the game image for the guest device generated by the drawing construction command may be transmitted to all guest devices in response to a display command relating to the game image.

[0102] As shown in FIG. 7 , in this embodiment, the host device executes a game program, thereby starting execution of a process for generating game images for the host device after completion of the process for generating game images for the guest device (i.e., this order is defined by the game program code). Specifically, a rendering construction command for drawing a game image for the host device is issued by the game processing unit 20 after execution of a display command for the game image for the guest device. The rendering unit 21 executes a process for drawing the game image in a frame buffer in accordance with the issued rendering construction command. Note that the game images for the host device and the game images for the guest device generated within one frame period are generated, for example, from different viewpoints (using different virtual camera settings) based on the same game processing results. Furthermore, while FIG. 7 illustrates an example in which the process for generating game images for the host device starts after completion of the process for transmitting game images for the guest device, the process for generating game images for the host device may start after the process for transmitting game images for the guest device has started. For example, the host device may start the process for generating game images for the host device before completion of the process for transmitting game images for the guest device, and execute part of the transmission process and the generation process in parallel. The game processing unit 20 may also execute the drawing construction command.

[0103] As with the game images for the guest device, a display command is issued by the game processing unit 20 for the game image for the host device after a drawing construction command, and the host device executes a display process to display the game image for the host device, which has been drawn in the frame buffer and for which the display command has been executed, on the display 13 at the subsequent game image switching timing. Here, in this embodiment, the host device executes the display process for the game image for the host device at the image switching timing (sometimes referred to as V-sync timing) (see FIG. 7). The image switching timing is set in the host device so that it occurs repeatedly at predetermined time intervals (the above-mentioned first cycle). After the display command, the display process is executed upon the arrival of the image switching timing. As a result, the game image displayed on the display 13 of the host device is updated at predetermined time intervals.

[0104] As described above, in this embodiment, game images for the guest devices are transmitted to the guest devices without waiting for the image switching timing on the host device. Furthermore, when game images are generated for each guest device, game images for each guest device are transmitted without waiting for the rendering of game images for other guest devices. This allows the guest devices to receive game images more quickly, thereby speeding up the timing for displaying images on the guest devices.

[0105] Furthermore, in this embodiment, the host device generates game images for the guest device earlier than game images for the host device within one frame period, thereby enabling the host device to transmit the game images to the guest device at an earlier timing, which allows the guest device to receive and display the game images earlier, thereby reducing the effects of communication delays.

[0106] The game image for the host device and the game image for the guest device may include common images. For example, if the game image includes an image representing a virtual space and UI images (e.g., menu images, images showing character abilities and status, images showing user commands, a map image of the game space, etc.), the UI images may be common between the game image for the host device and the game image for the guest device. When such common images are included, the host device may first execute a generation process for at least a portion of the common images, then execute a generation process for the remaining portions of the game images for the guest device, and then execute a generation process for the remaining portions of the game images for the host device. This allows the game image generation process to be executed efficiently and the time required for the process to be shortened. In the above case, as in the case shown in FIG. 7, the host device executes a transmission process for the game image for the guest device after the entire game image for the guest device has been generated, without waiting for the image switching timing. This, as in the case shown in FIG. 7, allows the effects of communication delays to be reduced.

[0107] In the common image mode of this embodiment, the host device first executes a game image generation process, and after the game image is drawn in the frame buffer, executes a transmission process to transmit the game image to the guest device. After the game image is drawn in the frame buffer, the host device executes a display process for the game image at the image switching timing. Thus, in the common image mode of this embodiment, the game program simply draws the game image (via the GPU) in the frame buffer (a memory area in which the game image to be displayed on its own display device is drawn) and executes a display command, and the system program then executes the transmission process to the guest device and the display process on the host device. Note that, as in the individual image mode, in the common image mode, a frame buffer may be provided for each of the host device and the guest device, and the same game image may be drawn in each frame buffer. In this case, the game image generation process, transmission process, and display process may be executed in the same order as shown in FIG. 7.

[0108] Next, the timing at which the drawing of game images begins on the guest device side will be described with reference to Fig. 8. Generally, the drawing process of game images (i.e., writing to the frame buffer) is executed immediately after the image switching timing (sometimes called V-sync timing), which is the timing at which the image written to the frame buffer is displayed on the display device. Furthermore, the image switching timing on the guest device is set, for example, at the beginning of each frame.

[0109] Fig. 8(a) shows a case where the drawing process starts simultaneously with the image switching timing, and Fig. 8(b) shows a case where the drawing process starts at a timing delayed from the image switching timing.

[0110] In the case of (a) shown in FIG. 8, the rendering process starts simultaneously with the image switching timing. Therefore, even if a game image is received before the next image switching timing, the rendering process has already started, so the received game image cannot be rendered and the game image cannot be displayed at the next image switching timing. In other words, in the case of (a) shown in FIG. 8, the game image rendered in the rendering process for a certain frame is the one received in the previous frame. In other words, the received game image is rendered in the rendering process for the next frame period. As such, with the general method, the timing at which the received game image is displayed on the guest device 3 is delayed.

[0111] On the other hand, in the case of (b) shown in FIG. 8, the rendering process starts later than the image switching timing. Therefore, even if the image switching timing is later than the image switching timing, as long as the game image is received by the guest device 3 before the start timing of the rendering process, the received game image can be rendered in that frame. Therefore, in the case of (b) shown in FIG. 8, the timing at which the received game image is displayed on the guest device 3 can be earlier than in the case of (a) shown in FIG. 8. Note that by setting the start timing of the rendering process as late as possible within one frame period, the possibility of rendering the game image received in that frame in that frame can be increased. This allows the timing at which the game image is displayed on the guest device 3 to be earlier, so this embodiment can also take measures against delays.

[0112] As described above, in this embodiment, the rendering unit 32 of the guest device 3 starts the rendering process later than the start of one frame period, as shown in FIG. 8(b). Specifically, the rendering unit 32 starts the rendering process a predetermined time before the end of one frame period (which can also be referred to as a predetermined time before or after the image change timing). The start timing of the rendering process may be any time between the image change timing and the next image change timing. In other words, the predetermined time may be any length greater than 0 and shorter than the length of one frame. In this embodiment, the predetermined time is set to the time required to render the game image (i.e., the time required to render image data in the frame buffer). In other words, the start timing of the rendering process is set to the predetermined time before the image change timing. This allows the rendering process to end before the end of one frame period, thereby increasing the likelihood that the game image received in that frame can be rendered in that frame and displayed at the next image change timing. Note that the time set as the "time required to render the game image" is determined based on the hardware performance of the guest device 3, etc. In other embodiments, the above-mentioned time may be set variably. Furthermore, in other embodiments, the above-mentioned predetermined time may be set to the time required to draw a game image plus a margin of time. This allows for a margin of time between the end of the drawing process and the end of one frame period.

[0113] Furthermore, although not shown, on the host device 2 side, drawing of game images begins at the start timing of one frame period. That is, the drawing unit 21 of the host device 2 begins drawing game images at the image switching timing. This is because the host device 2 does not draw game images received from another device, and the time required to draw game images is relatively long and varies depending on the game situation, so there is little benefit to delaying the drawing process within one frame period. Furthermore, by starting drawing at the start timing of one frame period, the drawing process can be more reliably completed within that frame.

[0114] In this embodiment, the image switching timing in the host device 2 and the image switching timing in the guest device 3 are not synchronized, but are set independently. This allows the host device 2 and the guest device 3 to draw and display game images without being subject to the constraints that would arise if the image switching timings were synchronized between them. For example, this reduces the possibility of delays in the drawing and display of game images due to the constraints. Note that in other embodiments, the image switching timings may be synchronized between the host device 2 and the guest device 3.

[0115] [3. Specific examples of processing in information processing systems] Next, a specific example of information processing executed in the information processing system 1 will be described with reference to FIGS.

[0116] Fig. 9 is a diagram showing an example of various data used for information processing in the host device 2. As shown in Fig. 9, the host device 2 stores communication partner data, accumulated operation data, delay amount data, reference delay amount data, frame number data, and excluded guest data. Each piece of data shown in Fig. 9 is stored in a storage medium (e.g., DRAM 15) accessible by the host device 2.

[0117] The communication partner data indicates the ID of the guest device that is the communication partner.

[0118] The accumulated operation data is operation data of the host device 2 or the guest device 3 accumulated in the host device 2. The host operation data accumulated as operation data of the host device includes data of a frame number associated with the operation data. The guest operation data accumulated as operation data of the guest device includes data of a frame number and an operation number associated with the operation data.

[0119] The delay amount data indicates the amount of delay between the host device 2 and the guest device 3. If there are multiple guest devices, delay amount data is stored for each guest device. The reference delay amount data indicates the above-mentioned reference delay amount.

[0120] The frame number data indicates the current frame number counted by the frame counter 23 .

[0121] The excluded guest data indicates the guest device that is set as the above-mentioned excluded guest device.

[0122] Fig. 10 is a diagram showing an example of various data used for information processing in the guest device 3. As shown in Fig. 10, communication partner data, image data, frame number data, and operation number data are stored in the guest device 3. Each piece of data shown in Fig. 10 is stored in a storage medium (e.g., DRAM 15) accessible by the host device 2.

[0123] The communication partner data indicates the ID of the host device that is the communication partner.

[0124] The image data is image data that is transmitted from the host device 2 and received by the guest device 3.

[0125] The frame number data indicates a frame number associated with the image data received at the guest device 3.

[0126] The operation number data indicates an operation number associated with the operation data transmitted from the guest device 3 to the host device 2.

[0127] An example of information processing executed in the host device 2 and the guest device 3 will be described below with reference to FIGS. 11 to 17. In this embodiment, the processor of the information processing device (i.e., the host device 2 or the guest device 3) executes a program (e.g., a game program, a host program, or a guest program) stored in the information processing device to execute the processing of each step shown in FIGS. 11 to 17. The processor may be the CPU or GPU described above, or a dedicated circuit. Furthermore, if the information processing device is capable of communicating with another information processing device (e.g., a server), some of the processing of each step shown in FIGS. 11 to 17 may be executed in the other information processing device. Furthermore, the processing of each step shown in FIGS. 11 to 17 is merely an example, and the processing order of each step may be reversed, or another process may be executed in addition to (or instead of) the processing of each step, as long as similar results are obtained.

[0128] 11 to 17, the processor executes the processes of the steps shown in Fig. 11 to 17 using a memory (for example, the DRAM 15 or a memory provided in the SoC). That is, the processor 81 stores information (in other words, data) obtained by each processing step in the memory, and when using the information in a subsequent processing step, reads the information from the memory and uses it.

[0129] 11 and 12 are flowcharts showing an example of the flow of information processing executed by the game program in the CPU of the host device 2. The processing shown in Fig. 11 is started, for example, when an instruction to start a game using multiple devices is given by a player during execution of the game program.

[0130] 11, the CPU determines a communication method with the guest device. For example, the CPU determines whether to communicate in the first communication mode (i.e., Internet communication) or in the second communication mode (i.e., local communication) in response to a user instruction. After step S1, the process of step S2 is executed.

[0131] In step S2, the CPU determines a guest device to be used as a communication partner. The method for determining the guest device is arbitrary. For example, when the host device 2 communicates via the Internet, the guest device to be used as a communication partner may be determined by being designated by a server, or may be determined from among candidates presented by the server in response to an instruction from the user of the host device 2. Alternatively, for example, the user of the host device 2 may directly designate the communication partner. Specifically, the user of the host device 2 may designate another user's account, and information about the account may be transmitted to the server. The server may then identify an information processing device corresponding to the user of the account, and the identified information processing device may be determined as the communication partner. Alternatively, the user of the host device 2 may designate an ID of another information processing device to determine the guest device, and the server may receive the ID and establish communication between the other information processing device and the host device 2. For example, when the host device 2 communicates via local communication, the guest device to be used as a communication partner may be determined from among information processing devices capable of communicating with the host device 2 in response to an instruction from the user. Once the communication partner is determined, the CPU stores data indicating the ID of the guest device to be used as communication partner data in memory. Once the communication partner is determined in step S2, the game starts and a series of processes from step S3 onwards are executed. In the example shown in Figures 11 and 12, the processing loop consisting of the series of processes from step S3 to S23 is executed every 1 / 60 [second].

[0132] In step S3, the CPU determines an initial value of the reference delay amount. Specifically, the CPU stores data indicating a predetermined initial value (for example, the same value as the allowable value) in memory as reference delay amount data. After step S3, the process of step S4 is executed.

[0133] In step S4, the CPU resets the counted frame number. Specifically, the CPU stores in memory, as frame number data, data indicating 0. After step S4, the process proceeds to step S5.

[0134] In step S5, the CPU acquires operation data of the own device from the input unit 12 and stores the acquired operation data in association with the frame number. Specifically, the CPU stores data including the acquired operation data and the frame number data stored in the memory as host operation data. After step S5, the process of step S6 is executed.

[0135] In step S6, the CPU determines whether operation data has been acquired from the guest device. Specifically, the CPU determines whether the operation data received from the guest device has been passed from the host program to the game program. If the determination result in step S6 is positive, the process of step S7 is executed. On the other hand, if the determination result in step S6 is negative, the process of step S14 (see FIG. 12), which will be described later, is executed.

[0136] In step S7, the CPU accumulates the operation data acquired in step S6. Specifically, since the acquired operation data is associated with a frame number and an operation number, the CPU stores the operation data including data indicating the frame number and operation number in memory as guest operation data. After step S7, the process of step S8 is executed.

[0137] In step S8, the CPU calculates the amount of delay between the host device and the guest device based on the operation data acquired in step S6. That is, the CPU calculates the amount of delay between the host device and the guest device that sent the operation data according to the method described in FIG. 4. Data indicating the calculated amount of delay is stored in memory as delay amount data. Note that if delay amount data for the same guest device is already stored in memory, the stored delay amount data is overwritten with the newly calculated delay amount data. Following step S8, the process of step S9 is executed.

[0138] In step S9, the CPU determines whether the delay calculated in step S8 exceeds the allowable value. If the determination result in step S9 is positive, the process proceeds to step S10. On the other hand, if the determination result in step S9 is negative, the process proceeds to step S11.

[0139] In step S10, the CPU sets the guest device for which it is determined that the delay amount exceeds the allowable value as an excluded guest device. That is, the CPU stores data indicating the set excluded guest device in memory as excluded guest data. After step S10, the process of step S11 is executed.

[0140] In a case where there are multiple guest devices, operation data from multiple guest devices may be acquired during the determination process of step S6. In this case, the processes of steps S7-S10 are executed for each of the acquired operation data.

[0141] In step S11, the CPU identifies the maximum current delay amount among the delay amounts of the guest devices. That is, the CPU identifies the maximum delay amount among the delay amounts indicated by the delay amount data stored in the memory. After step S11, the process of step S12 is executed.

[0142] In step S12, the CPU updates the reference delay amount. Specifically, the CPU sets the reference delay amount to the smaller of the above-mentioned allowable value and the maximum delay amount identified in step S11. The reference delay amount data stored in the memory is updated to indicate the newly set reference delay amount. Following step S12, the process of step S13 is executed.

[0143] In step S13 shown in FIG. 12, the CPU selects operation data to be used in the game processing based on the reference delay amount updated in step S12. Specifically, the CPU selects operation data associated with a frame number with aligned timing from among the operation data included in the accumulated operation data stored in memory, according to the method described with reference to FIGS. 5 and 6. Note that operation data of an excluded guest device indicated by the excluded guest data stored in memory is not selected in step S13. Furthermore, even if the operation data is of a guest device that is not an excluded guest device, the operation data may not be selected because, for example, operation data corresponding to a reference frame based on the reference delay amount has not been accumulated. Furthermore, in this embodiment, the CPU deletes the selected operation data from the accumulated operation data stored in memory. Following step S13, the process of step S14 is executed.

[0144] In step S14, the CPU determines whether or not it is time to reset the guest device excluded in step S10. This timing may be any timing, and may be, for example, every predetermined time (e.g., 30 seconds) or may be timing according to the game situation (e.g., when one match ends). If the determination result in step S14 is positive, the process of step S15 is executed. On the other hand, if the determination result in step S14 is negative, the process of step S16 is executed.

[0145] In step S15, the CPU resets the current excluded guest device. That is, the contents of the excluded guest data stored in the memory are reset, and a state in which no excluded guest device has been set is established. After step S15, the process of step S16 is executed.

[0146] As described above, in this embodiment, the reset of the excluded guest device is performed at the timing of step S14, but in other embodiments, the reset may be performed at any timing. For example, in other embodiments, if the delay amount calculated based on the operation data received from the excluded guest device does not exceed the allowable value, the host device may reset the exclusion of the guest device.

[0147] In step S16, the CPU determines whether operation data has been selected for each information processing device excluding the excluded guest device in step S13. If the determination result in step S16 is positive, the process proceeds to step S17. On the other hand, if the determination result in step S16 is negative, the process proceeds to step S18.

[0148] In step S17, the CPU executes game processing using the operation data selected in step S13. The specific content of the game processing is arbitrary, and for example, the movement of a player character is controlled based on the selected operation data, or a command in the game is selected based on the selected operation data. After step S17, the processing of step S19 is executed.

[0149] On the other hand, in step S18, the CPU executes game processing without using the operation data. For example, the CPU controls the actions of non-player characters not operated by the user in accordance with predetermined rules, or performs action control of a player character that is performed independently of user input (for example, control of a falling action due to gravity). Following step S18, the processing of step S19 is executed. Note that in step S18, instead of performing game processing based on the operation data, processing is performed to wait until all operation data is collected, and processing that is not directly related to the progress of the game and would be inconvenient if stopped, such as playing background music, can be performed; this can be determined appropriately depending on the content of the game.

[0150] In another embodiment, in a configuration in which there are multiple guest devices, if operation data is not selected for some of the guest devices that are not excluded guest devices in the processing of step S13, the CPU may execute the game processing using only the selected operation data in the above-mentioned step S18. For example, the CPU may control the actions of only the player character corresponding to the guest device for which operation data has been selected, based on the selected operation data.

[0151] In step S19, the CPU increments the frame number. That is, the frame number data stored in the memory is updated to indicate a value that is 1 more than the value before the update. After step S19, the process proceeds to step S20.

[0152] If the game program is a program that generates images that are common to the host device and the guest device, the processes of steps S20 to S22 described below are executed.

[0153] In step S20, the CPU instructs the system software to generate a game image for the guest device based on the results of the game processing in step S17 or S18. Specifically, the CPU passes the above-mentioned rendering construction commands, including graphics parameters used in the rendering process, and the current frame number to the system software. In response to this, the GPU executes a process to render the game image, thereby generating the game image (see FIG. 13). After the CPU has completed execution of all the rendering construction commands for generating the game image, it executes a display command for the game image. Execution of the display command triggers the system software to execute a transmission process to transmit the game image to the guest device (see FIG. 15). Note that if there are multiple guest devices and each guest device has a different game image, the process in step S20 instructs the system software to generate a game image corresponding to each guest device, and once generation of each image is completed, a display command for that image is executed. If there are multiple guest devices, in step S20, an instruction is issued to generate and transmit one game image corresponding to one guest device designated based on a predetermined rule from among the guest devices for which a game image has not yet been generated in the current frame. The predetermined rule is arbitrary and may be determined in the game program. After step S20, the process of step S21 is executed.

[0154] In step S21, the CPU determines whether or not an instruction to generate game images for all guest devices was issued in step S20. If the determination result in step S21 is positive, the process of step S22 is executed. On the other hand, if the determination result in step S21 is negative, the process of step S20 is executed again. Note that if there are multiple guest devices and the game images for the guest devices are common to each guest device, the CPU may execute the process of step S22 without executing the determination process of step S21.

[0155] In step S22, the CPU generates a game image for the host device based on the result of the game processing in step S17 or S18. Specifically, in step S22, as in step S20 above, the CPU passes a drawing construction command and the current frame number to the system software. The game image is drawn in accordance with the drawing construction command (see FIG. 13). In step S22, after all drawing construction commands are executed, a display command is executed. When the display command is executed, the drawn game image is displayed on the host device at the image switching timing (see FIG. 15). Following step S22, the processing of step S23 is executed.

[0156] Note that if the game program generates separate images for the host device and the guest device, the following processing steps are executed instead of the processing of steps S20 to S22 above. Specifically, the CPU performs processing to generate a game image based on the result of the game processing of step S17 or S18. In this processing step, as in the processing of step S20 above, the CPU passes a drawing construction command and the current frame number to the system software. After the drawing construction command causes the GPU to draw a game image in the frame buffer (a memory area in which the game image to be displayed on the display of the host device is drawn) (see FIG. 13), a display command is executed, which triggers the game image drawn in the frame buffer to be sent by the system software to the guest device (see FIG. 15), and is also displayed on the display of the host device at the image switching timing of the host device.

[0157] In the processing examples shown in FIGS. 11 and 12, the host device generates a game image for the guest device every frame and transmits the game image to the guest device every frame. However, as shown in FIGS. 4 to 6, the host device may generate and transmit a game image for the guest device once every several frames (e.g., once every two frames). For example, before the processing of step S20, the CPU may execute a process to determine whether the current frame is a frame for transmitting a game image to the guest device. In this case, the processing flow by the CPU may be configured to proceed to the processing of step S20 if the determination result is positive, and to proceed to the processing of step S22 if the determination result is negative. In another embodiment, when there are multiple guest devices, the host device does not need to transmit game images to all guest devices in the same frame. For example, when there are two guest devices, the host device may transmit game images to the first guest device in odd-numbered frames and to the second guest device in even-numbered frames.

[0158] In step S23, the CPU determines whether or not to end the game that is currently in progress through the game processing in step S16. For example, if a predetermined end condition in the game is met, or if an instruction to end the game is given by the user, the CPU determines that the game should be ended. If the determination result in step S23 is negative, the processing of step S5 is executed again. In this case, the processing of steps S5 to S23 is repeated until the game ends. On the other hand, if the determination result in step S23 is positive, the processing of step S3 is executed again. In this case, a new game will be started. In the above case, the CPU returns to the processing of step S1, for example, and determines the communication method. Alternatively, the process may return to step S2 and the communication partner may be determined again. In the above case, the CPU may terminate the execution of the game program.

[0159] Fig. 13 is a flowchart showing an example of the flow of graphics processing executed by system software on the GPU of the host device 2. Part or all of the graphics processing shown in Fig. 13 may be executed on the CPU of the host device 2. The processing shown in Fig. 13 is started, for example, in response to the start of the processing shown in Figs. 11 and 12. Note that in the example shown in Fig. 13, a processing loop consisting of the series of steps S31-S33 is executed every time a game image is generated, that is, every time an instruction to generate a game image is issued in step S20 or S22.

[0160] In step S31 shown in Fig. 13, the GPU acquires the rendering construction command and the current frame number sent from the game program by the processing of step S20 or S22 above. After step S31, the processing of step S32 is executed.

[0161] In step S32, the GPU draws game images based on the drawing construction command acquired in step S31. That is, the GPU draws game image data generated based on the drawing construction command in a frame buffer. In this embodiment, the storage unit of the host device is provided with a frame buffer for drawing game images to be displayed on the display device of the host device, and in the individual image mode, a memory area for drawing game images to be displayed on the display device of the guest device. In the individual image mode, if there are multiple guest devices and a game image is generated for each guest device, a memory area is provided for each guest device. Note that if the game image to be transmitted to multiple guest devices is the same, the game image may be read from the same memory area and transmitted to each guest device. The game image drawn in each memory area is displayed on the host device and transmitted to the guest devices by processing executed by a host program (described later) (see FIG. 15). In the common image mode, the GPU may draw the game image in any one frame buffer, or may draw the same game image in each frame buffer. Following step S32, the processing of step S33 is executed.

[0162] In step S33, the GPU associates the frame number obtained in step S31 with the game image generated in step S32. Specifically, the GPU writes frame number information into the metadata area of ​​the image data of the game image. After step S33, the process of step S31 is executed again.

[0163] Fig. 14 is a flowchart showing an example of the flow of processing related to receiving operation data among the information processing executed by the CPU of the host device 2 by the host program. The processing shown in Fig. 14 is started in response to the start of the processing shown in Fig. 11 and Fig. 12, for example. In this embodiment, a processing loop consisting of a series of processing steps S41-S42 shown in Fig. 14 is executed every 1 / 60 [second].

[0164] 14, the CPU determines whether operation data from the guest device has been received by the communication module 14. If the determination result of step S41 is positive, the process of step S42 is executed. On the other hand, if the determination result of step S41 is negative, the process of step S41 is executed again.

[0165] In step S42, the CPU passes the received operation data to the game program. Specifically, the operation data associated with the frame number and operation number is passed to the operation data storage unit 27, and the frame number and operation number are extracted and passed to the anti-delay processing unit 29. Following step S42, the processing of step S41 is executed again.

[0166] Fig. 15 is a flowchart showing an example of the flow of processing related to the display and transmission of image data among the information processing executed in the CPU of the host device 2 by the host program. The processing shown in Fig. 15 is started in response to the start of the processing shown in Fig. 11 and Fig. 12, for example. Note that in the host device, the processing shown in Fig. 15 is executed in parallel with the processing shown in Fig. 14.

[0167] In step S51 shown in FIG. 15, the CPU determines whether or not to transmit a game image to the guest device. Specifically, the CPU determines whether or not a display command for a game image for the guest device, which is included in the game program, has been executed by the CPU. In this embodiment, this display command is a hardware-related command, and therefore, strictly speaking, is executed by the CPU via the system program. In the individual image mode, a display command for a game image for the guest device is issued by the processing of step S20 above. Therefore, if the display command is issued and the determination result of step S43 is positive, the processing of step S52 is executed. On the other hand, if the determination result of step S51 is negative, the processing of step S54 is executed.

[0168] In step S52, the CPU generates image data for transmission to the guest device for the game image for which the display command was issued. In this embodiment, the image data for transmission is generated by performing a process of reducing the resolution and an encoding process on the game image drawn in the memory area of ​​each game image in step S32. Specifically, the CPU extracts the frame number from the image data associated with the frame number generated in step S33, and performs a process of reducing the resolution of the image data. Next, the CPU encodes the image data with reduced resolution and writes the frame number in the metadata area of ​​the encoded image data. Following step S52, the process of step S53 is executed.

[0169] In step S53, the CPU transmits the image data to be transmitted generated in step S52 to the guest device using the communication module 14. In the individual image mode, since the guest device is associated with the memory area in which the game image is stored, the destination guest device is the guest device associated with the memory area in which the game image of the image data to be transmitted was stored. In the case of multiple guest devices, if the game image for each guest device is common to all guest devices, the CPU may transmit the image data to each guest device in step S53. In the common image mode, if there are multiple guest devices, the CPU transmits the image data to each guest device in step S53. In this embodiment, the CPU identifies the ID of the destination guest device by referencing the communication partner data stored in memory. Thus, in this embodiment, the system software manages the communication partner determined in step S2. Therefore, the game program does not need to identify the ID of the destination guest device in the instruction in step S20, for example, thereby improving the efficiency of game program development. Following step S53, step S54 is executed.

[0170] In step S54, the CPU determines whether the image switching timing has arrived in the host device. If the determination result in step S54 is positive, the process of step S55 is executed. On the other hand, if the determination result in step S54 is negative, the process of step S51 is executed again.

[0171] In step S55, the CPU displays the game image for the host device on the display 13. Specifically, the CPU displays the game image stored in the frame buffer (in the individual image mode, the frame buffer associated with the host device) by outputting it to the display 13. Following step S55, the process of step S51 is executed.

[0172] Fig. 16 is a flowchart showing an example of the flow of reception processing executed by the CPU of the guest device 3 by a guest program. The processing shown in Fig. 16 is started, for example, when a communication partner of the host device (specifically, a guest device) is determined by the processing of step S2 shown in Fig. 11 and communication between the host device and the guest device is started. At this time, the CPU stores data indicating the ID of the host device that will be the communication partner in memory as communication partner data.

[0173] 16, the CPU determines whether image data from the host device 2 (specifically, the image data transmitted in step S53) has been received by the communication module. If the determination result in step S61 is positive, the process of step S62 is executed. On the other hand, if the determination result in step S61 is negative, the process of step S64, which will be described later, is executed.

[0174] In step S62, the CPU resets the counted operation number. Specifically, the CPU stores data indicating 0 in the memory as the operation number data. After step S62, the process proceeds to step S63.

[0175] In step S63, the CPU decodes the image data received in step S61 and stores the decoded image data in memory. After step S63, the process of step S61 is executed again. Thereafter, the processes of steps S61 to S63 are executed repeatedly.

[0176] Fig. 17 is a flowchart showing an example of the flow of drawing and transmission processing executed by the CPU of the guest device 3 by a guest program. The processing shown in Fig. 17, similar to the processing shown in Fig. 16, is started when a communication partner of the host device is determined by the processing of step S2 shown in Fig. 11 and communication between the host device and the guest device is started. In the guest device, the processing shown in Fig. 17 is executed in parallel with the processing shown in Fig. 16.

[0177] In step S71, the CPU determines whether or not the timing to start drawing in the current frame has arrived. As described above, in this embodiment, this timing is a predetermined time before the end timing of one frame period (see FIG. 8). If the determination result of step S71 is positive, the process of step S72 is executed. On the other hand, if the determination result of step S71 is negative, the process of step S71 is executed again. The determination result of step S71 becomes positive once in one frame period, and as a result, the processes of steps S72 to S76, which will be described later, are executed in a cycle of 1 / 60 [second]. Therefore, in this embodiment, the process of drawing image data and the process of obtaining operation data are performed every 1 / 60 [second].

[0178] In step S72, the CPU extracts the frame number embedded in the image data last stored in the process of step S63. After step S72, the process of step S73 is executed.

[0179] In step S73, the CPU starts drawing the image data for which the frame number was extracted in step S72. That is, the CPU instructs the GPU to execute drawing processing for the image data (specifically, to write the received image data into a frame buffer). This starts the drawing processing, which ends during the current frame period. The drawn game image is displayed by being output to the display in accordance with the image switching timing of the guest device. Following step S73, the processing of step S74 is executed.

[0180] In step S74, the CPU acquires operation data of the own device from the input unit. After step S74, the process proceeds to step S75.

[0181] In step S75, the CPU associates the frame number and the operation number with the operation data and transmits the data to the host device 2. Specifically, the CPU associates the frame number extracted in step S72 and the operation number indicated by the operation number data stored in memory with the operation data acquired in step S74. Next, the CPU transmits the operation data in which the frame number and the operation number are associated to the host device 2 using the communication module. Following step S75, the process of step S76 is executed.

[0182] In step S76, the CPU increments the operation number. That is, the operation number data stored in the memory is updated to indicate a value that is 1 more than the value before the update. After step S76, the process of step S71 is executed again.

[0183] 4. Modifications of this embodiment In the above embodiment, a frame number associated with a game image is transmitted from the host device to the guest device, and a delay countermeasure process is executed based on the operation data associated with the frame number transmitted from the guest device. Here, frame numbers may not be associated with the game image transmitted from the host device to the guest device and the operation data transmitted from the guest device to the host device, and delay countermeasure process based on the operation data from the guest device may not be executed. Even if the delay countermeasure process is not executed, the effects of communication delays can be reduced by transmitting the game image for the guest device to the guest device without waiting for the image switching timing on the host device, thereby taking measures against communication delays. Furthermore, a communication delay can also be countermeasured by generating the game image for the guest device before the game image for the host device within one frame period.

[0184] In the above embodiment, information indicating the frame number assigned to the frame in which the game image was generated was used as the timing information used in the delay countermeasure processing, but the specific content of the timing information is arbitrary. In other embodiments, for example, information indicating time (e.g., a timestamp) may be used as the timing information. Furthermore, the current timing information on the host device is information that can be obtained on the host device regardless of whether time is synchronized between the host device and the guest device, server, etc., and can be information indicating the timing of the game image generated on the host device.

[0185] As the delay countermeasure processing, in the above embodiment, the host device calculates the amount of delay and executes a process for selecting operation data of the host device and guest device to be used in the game processing based on the amount of delay. Here, the content of the process executed based on the amount of delay to counter the delay is arbitrary and is not limited to the above. For example, using the amount of delay calculated by the processing of this embodiment, any delay countermeasure technology can be used, such as the delay countermeasure technologies disclosed in Japanese Patent Application Laid-Open Nos. 2022-160999, 2022-161009, 2022-161000, and Japanese Patent Registration No. 6927750. For example, in another embodiment, the host device may select only the operation data of the host device to be used in the game processing based on the amount of delay. In this case, the operation data of the guest device may be selected based on a rule independent of the amount of delay (e.g., a rule that the last data received by the host device is selected). According to the above, instead of using the latest operation data of the host device (i.e., data obtained in the current frame), data obtained in the past is selected taking into account the amount of delay, thereby preventing the host device from gaining too much of an advantage in the game.

[0186] Furthermore, the specific content of the delay countermeasure processing is arbitrary and is not limited to a process of calculating a delay amount. For example, in another embodiment, the host device may execute a process of selecting operation data to be used in game processing based on timing information associated with the operation data, without calculating a delay amount, as the delay countermeasure processing. Specifically, the host device may select, from the accumulated operation data of the host device and operation data of the guest device, pairs of operation data associated with timing information that is synchronized. Note that, in the above embodiment, "pairs of operation data associated with timing information that is synchronized" refers to pairs of operation data associated with the same frame number or frame numbers whose difference is within a predetermined range. This also reduces the possibility of an advantage or disadvantage in the game occurring between the host device and the guest device.

[0187] Furthermore, when a delay amount is used in the delay countermeasure processing, the delay amount does not need to be based on the frame number as in the above embodiment. For example, in another embodiment, the host device 2 may calculate the delay amount before the start of the game using a method other than the method using the frame number, and perform the delay countermeasure processing during the game using the pre-calculated delay amount.

[0188] Furthermore, although the information processing system in the above embodiment executes a delay countermeasure process based on timing information, the information processing system does not necessarily have to execute such a delay countermeasure process. The information processing system may take measures against communication delays by a process other than the delay countermeasure process described above, or may solve a problem other than communication delays.

[0189] Furthermore, in the information processing system of the above embodiment, the timing at which rendering begins on the guest device is set to a predetermined time before the timing at which the displayed game image is switched. This allows the timing at which the game image is displayed on the guest device to be advanced regardless of the relationship between the first and second cycles, thereby making it possible to take measures against delays. Note that the timing at which rendering begins is arbitrary. In other embodiments, the timing at which rendering begins may be different from the predetermined time before the timing at which the game image is switched, or may be earlier than that timing.

[0190] In other embodiments, the host device does not need to be configured to display game images on its own device as in the above embodiment, and game images may not be displayed on the host device side, i.e., the host device may be configured to generate only game images for the guest device.

[0191] In the above embodiment, the game program is a program for a game executed using multiple information processing devices, but the content of the game is arbitrary. The game may be a multiplayer game in which multiple users each play the game using an information processing device, or a single-player game in which one user plays the game using multiple information processing devices (e.g., viewing multiple displays).

[0192] In other embodiments, the information processing system may not have some of the configurations in the above embodiments, or may not execute some of the processes executed in the above embodiments.

[0193] In this embodiment, the term "program" includes source code, intermediate code, object code, native code, script, etc., and the form of the code is not limited. Furthermore, the term "program" not only refers to an entire application program but also includes a program that realizes some of the functions of the application. Furthermore, each function of this embodiment may be realized by several separate programs. In this case, a collection of several programs can also be considered a program. Furthermore, the program causes a computer, which is configured with a main processor, sub-processors, peripheral circuits, firmware, etc. in an information processing device, to function to realize each function of this embodiment. A single processor may execute the program, or each function of the program may be executed by a different processor. Furthermore, each function realized by the program does not have to be realized solely by the processor's processing, but may also be realized by utilizing functions of the computer (e.g., graphics, wireless, etc.). Furthermore, the computer may further include an interpreter or emulator, in which case the program may run on the interpreter or emulator. [Industrial Applicability]

[0194] The above-described embodiment can be used in, for example, a game system or a game program, with the aim of enabling new countermeasures against communication delays, for example. [Explanation of symbols]

[0195] 1. Information Processing Systems 2. Host Device 3 Guest Device 20 Game Processing Unit 21 Drawing section 23 Frame Counter 24 ID Registration Department 25 Transmission control section 26 Reception control section 27 Operation data storage unit 28 Operation data acquisition unit 29 Delay Countermeasure Processing Unit 31 Reception control section 32 Drawing section 33 Operation data acquisition unit 34 ID Registration Department 35 Transmission control section

Claims

1. An information processing system including a host device and one or more guest devices capable of communicating with the host device, The host device means for acquiring first operation data indicating an operation on the player's own device; means for receiving second operation data indicating an operation on the guest device from the guest device; means for executing game processing based on the first operation data and the second operation data; means for rendering a first game image, which is a game image showing a result of the game processing and is displayed on the host device, and a second game image, which is a game image showing a result of the game processing and is transmitted to the guest device, such that the second game image precedes the first game image; means for displaying the first game image on a display device; means for transmitting the second game image to the guest device; The guest device means for receiving the second game image transmitted from the host device; means for displaying the received second game image on a display device; means for acquiring the second operation data; means for transmitting the second operation data to the host device.

2. In the drawing, the second game image is drawn prior to the first game image within one frame period. The information processing system according to claim 1 .

3. the second game image is transmitted without waiting for a timing at which the display of the first game image is switched on the host device; 3. The information processing system according to claim 1.

4. 3. The information processing system according to claim 1, wherein the second game image is transmitted before the first game image starts to be drawn.

5. There may be a plurality of guest devices; In the drawing, the second game images to be transmitted to each of the guest devices are drawn before the first game images; In transmitting the game image, the rendered second game image is transmitted to the corresponding guest device without waiting for completion of rendering of other second game images.

3. The information processing system according to claim 1.

6. the display of the game image on the host device is performed in accordance with the switching timing on the host device; the display of the game image on the guest device is performed in accordance with the switching timing on the guest device; 3. The information processing system according to claim 1.

7. In the drawing, drawing is performed on at least a part of an image common to the first game image and the second game image, then drawing is performed on the second game image excluding at least a part of the common image, and then drawing is performed on the first game image excluding at least a part of the common image.

3. The information processing system according to claim 1.

8. the host device further comprises means for storing a game program for the game processing and system software separate from the game program, the game program being executed by a computer of the host device; the game program includes a display command indicating that the second game image may be displayed on a display; the system software includes a command to transmit a game image in response to execution of the display command; 3. The information processing system according to claim 1.

9. In an information processing system capable of communicating with each other as a host device or a guest device, a computer of the host device A process of acquiring first operation data indicating an operation on the player's own device; acquiring second operation data indicating an operation on the guest device from the guest device; a process of executing a game process based on the first operation data and the second operation data; a program for executing a process of drawing a first game image, which is a game image showing the result of the game processing and is displayed on the host device, and a second game image, which is a game image showing the result of the game processing and is sent to the guest device, so that the second game image precedes the first game image.

10. The computer further comprises:

10. The program according to claim 9, further comprising: a process for setting the second game image to a state in which it is permitted to be displayed on a display, in order to trigger transmission of the second game image to the guest device.

11. 11. The program according to claim 9, wherein, in the drawing, the second game image is drawn earlier than the first game image within one frame period.

12. 11. The program according to claim 9, wherein the second game image is transmitted before the first game image starts to be drawn.

13. There may be a plurality of guest devices; In the drawing, the second game images to be transmitted to each of the guest devices are drawn before the first game images; 11. The program according to claim 9, wherein, in transmitting the game image, the drawn second game image is brought into a state in which it may be displayed on a display without waiting for completion of drawing of other second game images.

14. 1. A computer-implemented method executed in an information processing system including a host device and one or more guest devices capable of communicating with the host device, comprising: The host device acquiring first operation data indicating an operation on the player's device; receiving second operation data from the guest device, the second operation data indicating an operation on the guest device; executing a game process based on the first operation data and the second operation data; a first game image that is a game image showing a result of the game processing and is displayed on the host device, and a second game image that is a game image showing the result of the game processing and is transmitted to the guest device, the first game image being drawn before the first game image; displaying the first game image on a display device; transmitting the second game image to the guest device; The guest device receiving the second game image transmitted from the host device; displaying the received second game image on a display device; obtaining the second operation data; transmitting the second operation data to the host device.

Images