Information Processing Device and Image Sharing Method

Abstract

The transmission processing unit 166a transmits an image to the distribution server. The status information acquisition unit 164a acquires information indicating the statuses of multiple members from the management server. The room image generation unit 124a generates a member display area based on the information indicating the statuses of multiple members, and the member display area includes information about members who transmit an image and information about members who do not transmit an image in different areas. The room image generation unit 124a displays information about members who are viewing the image transmitted from the transmission processing unit 166a to the distribution server.

Description

Technical Field

[0001] The present disclosure relates to a technique for sharing images among multiple users. Background Art

[0002] PTL 1 discloses an image sharing system in which, in an environment where an information processing device of a host user and an information processing device of a guest user are connected by P2P (peer-to-peer) instead of via a server, an image of a game that the host user is playing is shared with the guest user. In this image sharing system, a sharing mode (shared screen) for allowing the guest user to view the game image, a sharing mode (transfer my controller) for allowing the guest user to play the game instead of the host user, and a sharing mode (transfer another controller) for allowing the guest user to participate in the game as a new player and allowing the host user and the guest user to play the game together are prepared.

[0003] [Citation List]

[0004] [Patent Document]

[0005] [PTL 1]

[0006] JP 2017-35298A Summary of the Invention

[0007] [Technical Problem]

[0008] The network service of a game functions as a communication tool. For example, if multiple users participating in the same chat room share a game image, it is expected that more active communication can be established. Therefore, it is preferable to implement a mechanism for allowing a user who is playing a game to simply share the image of the ongoing game with other users. Not only in games, but also in network services such as a conference system in which multiple information processing devices are connected, if a mechanism for simply sharing images is implemented, it is expected that smooth communication can be supported.

[0009] Therefore, an object of the present invention is to provide a technique for sharing images.

[0010] [Solution to Problem]

[0011] To solve the above problems, an information processing device according to a specific aspect of the present disclosure is connected to a management server that manages the states of multiple members participating in a room and a distribution server that distributes images. The information processing device includes a transmission processing section that sends images to the distribution server; a status information acquisition section that acquires information indicating the states of the multiple members from the management server; and a room image generation section that generates a member display area based on the information indicating the states of the multiple members, wherein information about the members who send images and information about the members who do not send images are included in different areas, and wherein the room image generation section displays information indicating the members who are viewing the images sent from the transmission processing section to the distribution server.

[0012] Another aspect of the present disclosure is an image sharing method, including the steps of sending an image to a distribution server that distributes images, acquiring information indicating the states of multiple members from a management server that manages the states of multiple members participating in a room, generating a member display area based on the information indicating the states of the multiple members, wherein information about the members who send images and information about the members who do not send images are included in different areas, and displaying information indicating the members who are viewing the image sent to the distribution server.

[0013] It should be noted that a method, device, system, recording medium, or computer program obtained by combining any of the above-described constituent elements or expressions in the present disclosure is also effective as an aspect of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] Figure 1 is a diagram depicting an image sharing system according to an embodiment of the present disclosure.

[0015] Figure 2 is a diagram depicting the hardware configuration of an information processing device.

[0016] Figure 3 is a diagram depicting the functional blocks of an information processing device that sends game images.

[0017] Figure 4 is a diagram depicting an example of a game screen.

[0018] Figure 5 is a diagram depicting an example of a system image superimposed on a game image.

[0019] Figure 6 is a diagram depicting the functional blocks of a distribution server.

[0020] Figure 7 is a diagram depicting an example of a system image superimposed on a game image.

[0021] Figure 8It is a diagram showing an example of a system image superimposed on a game image.

[0022] Figure 9 It is a diagram depicting the functional blocks of an information processing device that receives game image information under screen sharing.

[0023] Figure 10 It is a diagram showing an example of a game screen displayed on an output device.

[0024] Figure 11 It is a diagram showing an example of a message displayed on an output device.

[0025] Figure 12 It is a diagram showing an example of a system image superimposed on a game image.

[0026] Figure 13 It is a diagram depicting a display example in full-screen display format.

[0027] Figure 14 It is a diagram depicting a display example in picture-in-picture display form.

[0028] Figure 15 It is a diagram depicting a display example in split-screen display form.

[0029] Figure 16 It is a diagram showing an example of a system image superimposed on a game image.

[0030] Figure 17 It is a diagram showing an example of a message displayed on an output device.

[0031] Figure 18 It is a diagram showing an example of a system image superimposed on a game image.

[0032] Figure 19 It is a diagram showing an example of a system image displayed on an output device.

[0033] Figure 20 It is a diagram depicting an image of a game that a user is playing. Detailed implementation

[0034] Figure 1 It depicts an image sharing system 1 according to an embodiment of the present disclosure. The image sharing system 1 includes a plurality of information processing devices 10a, 10b, 10c, and 10d (hereinafter, when they are not specifically distinguished from each other, referred to as "information processing device 10"), a management server 5, and a distribution server 9, which are connected via a network 3 such as the Internet or a LAN (local area network).

[0035] The information processing devices 10a, 10b, 10c, and 10d are terminal devices operated by respective users and are connected to output devices 4a, 4b, 4c, and 4d (hereinafter, referred to as "output device 4" when not specifically distinguished from each other). The output device 4 may be a television having a display for outputting an image and a speaker for outputting sound, or may be a head-mounted display. The output device 4 may be connected to the corresponding information processing device 10 via a wired cable or wirelessly.

[0036] An access point (hereinafter referred to as "AP") 8 has a wireless access point function and a router function. Each information processing device 10 is connected to the corresponding AP 8 wirelessly or wiredly and is thus communicably connected to a management server 5, a distribution server 9, and other information processing devices 10 on the network 3.

[0037] The information processing device 10 is connected to an input device 6 operated by a user wirelessly or wiredly. The input device 6 outputs operation information indicating the result of a user operation to the information processing device 10. The information processing device 10 receives the operation information from the input device 6 and reflects the operation information in the processing of system software or application software so as to output a processing result through the output device 4. In the image sharing system 1, the information processing device 10 may be a game device that executes a game, and the input device 6 may be a device such as a game controller for providing user operation information to the information processing device 10. The input device 6 may include a plurality of input portions including a plurality of push operation buttons, a joystick through which an analog amount can be input, and a rotatable button.

[0038] The auxiliary storage device 2 is a memory such as an HDD (hard disk drive) or an SSD (solid state drive), and may be an internal memory or an external memory connected to the information processing device 10 via USB (universal serial bus) or the like. A camera 7 as an image capturing device is arranged near the output device 4 and captures an image of the surrounding area of the output device 4.

[0039] In the image sharing system 1, users A, B, C, and D are room members in the same chat room. The users can perform text chat with each other, and if a user has headphones, they can perform voice chat with each other. The chat room is created and managed by the management server 5. The management server 5 receives chat data (text data and / or voice data) sent from each user and transmits the chat data to other users in the room. For example, the management server 5 transmits the chat data sent from user A to users B, C, and D in the room.

[0040] In a chat room, each user can share an image of the ongoing game with other users. An upper limit can be set on the number of users joining the chat room, but there is no necessary limit on the number. It should be noted that the chat room is an example of a virtual room or group where online users gather. Another type of room or group can be used instead.

[0041] The management server 5 is maintained and managed by the management entity of the image sharing system 1 and provides network services including a chat service to the users of the image sharing system 1. The management server 5 manages network accounts for identifying individual users. By using the network account, the user logs in to the network service. After logging in to the network service, the user enters the chat room to be able to communicate with other room members. It should be noted that, for example, after logging in, the user can save data about the game in the management server 5.

[0042] The distribution server 9 is maintained and managed by the management entity of the image sharing system 1 and provides a service for distributing stream data of the image of the game the user is playing to another user participating in the chat room. It should be noted that the stream data of course includes game sounds, but the explanation of the distribution of game sounds will be omitted. The explanation of the distribution of game images will be mainly given below.

[0043] Figure 2 The hardware configuration of the information processing device 10 is depicted. The information processing device 10 includes a main power button 20, a power-on LED (light emitting diode) 21, a standby LED 22, a system controller 24, a clock 26, a device controller 30, a media drive 32, a USB module 34, a flash memory 36, a wireless communication module 38, a wired communication module 40, a subsystem 50, and a main system 60.

[0044] The main system 60 includes a main CPU (central processing unit), a memory serving as a main memory and a memory controller, a GPU (graphics processing unit), etc. The GPU is mainly used for the calculation of game programs. These functions can be implemented by a system on a chip and can be formed on one chip. The main CPU has the function of executing game programs recorded in the auxiliary storage device 2 or the ROM (read only memory) medium 44.

[0045] The subsystem 50 is equipped with a sub CPU, a memory serving as a main memory, and a memory controller, etc., but is not equipped with a GPU and thus does not have the function of executing game programs. The number of gate circuits in the sub CPU is less than the number of gate circuits in the main CPU. The operating power consumption in the sub CPU is less than the operating power consumption in the main CPU.

[0046] The main power button 20 is an input section that receives user operations. The main power button 20 is arranged on the front surface of the housing of the information processing device 10 and is operated to turn on / off the power of the main system 60 of the information processing device 10. When the main power button 20 is turned on, the power LED 21 lights up. When the main power button 20 is turned off, the standby LED 22 lights up.

[0047] The system controller 24 detects that the user has pressed the main power button 20. When the main power button 20 is pressed while the main power is in the off state, the system controller 24 regards this pressing operation as an "on instruction". On the other hand, when the main power button 20 is pressed while the main power is in the on state, the system controller 24 regards the pressing operation as an "off instruction".

[0048] The clock 26 as a real-time clock generates current date and time information and provides this information to the system controller 24, the subsystem 50, and the main system 60. The device controller 30 is formed as an LSI (Large Scale Integration) and exchanges information between devices like a south bridge. As shown in the figure, devices such as the system controller 24, the medium drive 32, the USB module 34, the flash memory 36, the wireless communication module 38, the wired communication module 40, the subsystem 50, and the main system 60 are connected to the device controller 30. The device controller 30 absorbs the differences in electrical characteristics and data transmission speeds between devices and controls the data transmission timing.

[0049] The medium drive 32 is driven by the ROM medium 44 attached thereto. Application software and license information for games and the like are recorded on the ROM medium 44. The medium drive 32 reads out programs, data, etc. from the ROM medium 44. The ROM medium 44 can be a read-only recording medium such as an optical disc, a magneto-optical disc, or a Blu-ray disc.

[0050] The USB module 34 is connected to an external device via a USB cable. The USB module 34 can be connected to the auxiliary storage device 2 and the camera 7 via a USB cable. The flash memory 36 is an auxiliary memory that constitutes the internal memory. The wireless communication module 38 wirelessly communicates with the input device 6 according to a communication protocol such as the Bluetooth (registered trademark) protocol or the IEEE (Institute of Electrical and Electronics Engineers) 802.11 protocol. The wired communication module 40 performs wired communication with an external device to connect to the network 3 via the AP 8.

[0051] In the image sharing system 1, chat room members are allowed to share game images using three sharing modes. Hereinafter, the user who distributes the game images is referred to as the "host" or "host user", and the user who receives the distribution of the game images is referred to as the "guest" or "guest user".

[0052] <First Sharing Mode>

[0053] In the first sharing mode, a guest user watches the game image of the host user. This is called "screen sharing". In screen sharing, the game image of the host user is shared with the guest user via the distribution server 9. That is, the information processing device 10 of the host user sends the game image to the distribution server 9, and the information processing device 10 of the guest user receives the game image from the distribution server 9. The guest user is allowed to watch the game image of the host user, but is not allowed to perform operations in the game.

[0054] <Second Sharing Mode>

[0055] In the second sharing mode, the guest user plays the game on behalf of the host user while watching the game image of the host user. This is called "assistant game". In the assistant game, the game image of the host user is not shared with the guest user through the distribution server 9. That is, a P2P connection is established between the information processing device 10 of the host user and the information processing device 10 of the guest user to share the game image. The host user cannot perform operations in the game, and only the guest user is allowed to perform operations in the game because the host user gives the game operation permission to the guest user.

[0056] <Third Sharing Mode>

[0057] In the third sharing mode, the guest user joins the game as a new player, plays the game in cooperation with the host user while watching the game image of the host user. This is called "cooperative game". In the cooperative game, the game image of the host user is not shared with the guest user through the distribution server 9. That is, a P2P connection is established between the information processing device 10 of the host user and the information processing device 10 of the guest user to share the game image. In the cooperative game, the guest user also uses the game resources on the host user side, and the host user and the guest user respectively become Player 1 and Player 2 to participate in the game, so that the host user and the guest user are allowed to perform operations in the game cooperatively with each other.

[0058] In the second and third sharing modes, a P2P connection is established between the information processing device 10 of the host user and the information processing device 10 of the guest user to share the game image, and at least the guest user has game permissions. Since the guest user is allowed to play the game in the second and third sharing modes, in some cases, the second and third sharing modes are collectively referred to as "shared games".

[0059] When distributing the image of the game that the user is playing, one user becomes the host user, and when viewing the image of the game that another person is playing, further becomes the guest user. Thus, an information processing apparatus 10 includes a transmission-side configuration that becomes the host user and a reception-side configuration that becomes the guest user. Hereinafter, for the sake of explanation, it is assumed that the information processing apparatus 10a of user A has a transmission-side configuration for transmitting game images, and the information processing apparatus 10c of user C has a reception-side configuration for receiving game images, so that the game images are shared. However, it should be understood that both the transmission-side configuration and the reception-side configuration are installed in each information processing apparatus 10.

[0060] Figure 3 The functional blocks of the information processing apparatus 10a that depicts the image of the game that user A is playing are shown. The information processing apparatus 10a includes a processing section 100a, a communication section 102a, and a reception section 104a. The processing section 100a includes an execution section 110a, a system image generation section 120a, an image processing section 140a, a frame buffer 150a, and a sharing processing section 160a.

[0061] The system image generation section 120a includes a report generation section 122a and a room image generation section 124a. The frame buffer 150a includes a game buffer 152a that temporarily stores game image data and a system buffer 154a that temporarily stores system image data. The sharing processing section 160a includes a status information transmission section 162a, a status information acquisition section 164a, a transmission processing section 166a, an invitation transmission section 172a, and a connection processing section 174a. The transmission processing section 166a includes a first transmission processing section 168a and a second transmission processing section 170a.

[0062] In Figure 3 each element shown as a functional block for performing various processes can be formed in terms of hardware by a circuit block, a memory, and any other LSI, and can be implemented in terms of software by system software or a game program loaded in the memory. Thus, those skilled in the art will understand that these functional blocks can be implemented in various configurations, such as a configuration including only hardware, a configuration including only software, and a combination thereof. There is no particular limitation on the implementation of the functional blocks.

[0063] The communication section 102a receives operation information regarding operations performed by user A on the input section of the input device 6. In addition, the communication section 102a receives chat data regarding other room members in the chat room from the management server 5, and also receives information indicating the status of other room members.

[0064] The communication section 102a sends information indicating the status of user A to the management server 5. In addition, the communication section 102a sends the stream data regarding the game image and game sound generated by the processing section 100a to the distribution server 9 and / or a separate information processing device 10. Hereinafter, the stream data for reproducing the game image may be simply referred to as the game image. The functional blocks of the communication section 102a are shown as having Figure 2 the functions of the wireless communication module 38 and the functions of the wired communication module 40 in

[0065] The receiving section 104a is provided between the communication section 102a and the processing section 100a, and exchanges data or information with the communication section 102a and the processing section 100a. When operation information is received from the input device 6 via the communication section 102a, the receiving section 104a provides the operation information to a specified functional block in the processing section 100a.

[0066] The execution section 110a executes a game program (hereinafter, in some cases simply referred to as "game"). The functional block represented here as the execution section 110a is implemented by software such as system software or game software or hardware such as a GPU. The execution section 110a executes the game program to generate game image data and game sound data. It should be noted that the game is an example of an application, and the execution section 110a can execute any application that is not a game.

[0067] During the period when user A plays the game, the execution section 110a executes the game program, and based on the operation information input by user A to the input device 6, performs calculations for generating the movement of the game character in the virtual space. After receiving the calculation result in the virtual space, the GPU generates game image data based on the viewpoint (virtual camera) in the virtual space.

[0068] Figure 4 An example of the game screen displayed on the output device 4a of user A is depicted. User A is playing the game title "Special Football". The execution section 110a generates game image data and provides the game image data to the image processing section 140a. Then, the image processing section 140a temporarily stores the game image data in the game buffer 152a, generates a display image from the image data temporarily stored in the frame buffer 150a, and provides the display image to the output device 4a. Accordingly, the output device 4a outputs the game image. It should be noted that, in reality, the output device 4a additionally outputs the game sound generated by the execution section 110a, and user A operates the input device 6 to play the game title "Special Football" while watching the game image and listening to the game sound output from the output device 4a.

[0069] In the shared processing section 160a, the status information sending section 162a sends information indicating the status of user A to the management server 5. The information indicating the status includes information indicating whether the user is playing a game, and if the user is playing a game, it also includes information indicating the game title, the on / off state of the microphone, and information about image sharing. The information about image sharing includes information about user A as the host user and information about user A as the guest user. The information about user A as the host user includes information indicating that user A starts screen sharing (the first sharing mode), and information indicating an invitation to send an assist game (the second sharing mode) or a cooperative game (the third sharing mode) to room members. The information about user A as the guest user includes information about the sharing mode of the game image distributed by another room member. Specifically, in order to allow another room member to view the image distributed by the distribution server 9, the status information sending section 162a sends information indicating the image to be viewed to the management server 5. In order to accept an invitation to an assist game or a cooperative game from another room member, the status information sending section 162a sends information indicating acceptance of the invitation to the management server 5.

[0070] The function of the status information sending section 162a is also installed in each of the information processing devices 10b to 10d of users B to D. Therefore, each user's information processing device 10 sends information indicating the user's status to the management server 5. Preferably, when the status changes, the information processing device 10 immediately sends information indicating the change to the management server 5. The management server 5 obtains information indicating the user's status from each of the information processing devices 10 and manages the respective current statuses of the users. The management server 5 sends information indicating the users to the information processing devices 10 belonging to the same chat group. The status information obtaining section 164a of the information processing device 10a obtains information indicating the respective statuses of the users.

[0071] The information processing device 10 according to the embodiment provides a mechanism that allows the user A who is playing a game to share an image in a simple manner. When user A briefly presses a prescribed button on the input device 6 during the game, the receiving section 104a receives the button operation and provides the operation information to the system image generation section 120a. The system image generation section 120a obtains the button operation information as a system image display request and calls the status information obtaining section 164a.

[0072] The status information acquisition section 164a acquires information indicating the user's status from the management server 5 and provides this information to the system image generation section 120a. In the system image generation section 120a, the room image generation section 124a generates system image data indicating the status of the chat room in which user A is participating, and provides this system image data to the image processing section 140a. The image processing section 140a temporarily stores the system image data in the system buffer 154a, generates a display image from the image data temporarily stored in the frame buffer 150a, and provides the display image to the output device 4a. Specifically, the image processing section 140a generates a display image by combining the game image data temporarily stored in the game buffer 152a and the system image data temporarily stored in the system buffer 154a, and provides the generated display image to the output device 4a. Accordingly, the output device 4a outputs a display image in which the system image is superimposed on the game image.

[0073] Figure 5 An example of a system image 200 superimposed on a game image is depicted. The room image generation section 124a generates system image data based on the information indicating the user's status acquired by the status information acquisition section 164a. A member display area 202 for indicating the status of the members in the chat room in which user A is participating is provided in the system image 200. The room image generation section 124a generates a member display area 202 including member information according to the information representing the status of a plurality of room members (users). In Figure 5 the member display area 202 shown, information indicating the user icon, user name, title of the game in progress, and whether the microphone is on / off is displayed. Information indicating whether a shared game is being executed may be additionally included in the member display area 202.

[0074] The share start button 204 is an operation element that allows user A to start screen sharing as a first sharing mode. When user A operates the share start button 204, the status information transmission section 162a sends information indicating that user A has started screen sharing (first sharing mode) to the management server 5 and the distribution server 9. Then, the first transmission processing section 168a sends the stream data of the game image that the image processing section 140a has read from the game buffer 152a to the distribution server 9. As described above, the stream data includes game sound data. The first transmission processing section 168a compresses the stream data in a prescribed format and sends the compressed data to the distribution server 9.

[0075] Here, the image processing section 140a only reads out the game image data temporarily stored in the game buffer 152a and provides the game image data to the first transmission processing section 168a. Therefore, the system image data temporarily stored in the system buffer 154a is prevented from being combined with the game image data. As a result, the image data to be distributed does not include the system image data, and thus the game image data can be distributed separately.

[0076] The first transmission processing section 168a determines the resolution of the game image data to be transmitted to the distribution server 9 according to the quality of the connection state between the communication section 102a and the distribution server 9. That is, if the connection state is poor, the first transmission processing section 168a determines to reduce the resolution of the game image data. It should be noted that the execution section 110a generates the game image data at a frame rate of 60 fps (frames per second) or 30 fps, but if the connection state with respect to the distribution server 9 is extremely poor, the first transmission processing section 168a can reduce the frame rate as well as the resolution.

[0077] In the case where the game image data with a resolution of 1080p is temporarily stored in the game buffer 152a, if the connection state is good, the image processing section 140a provides the game image data with a resolution of 1080p to the first transmission processing section 168a. However, if the connection state is not good, the image processing section 140a needs to reduce the resolution of the game image. When it is difficult to transmit the game image with a resolution of 1080p, the image processing section 140a reduces the resolution to 720p. When it is difficult to transmit the game image with a resolution of 720p, the image processing section 140a reduces the resolution to 540p. When it is difficult to transmit the game image with a resolution of 540p, the image processing section 140a reduces the resolution to 360p.

[0078] The first transmission processing section 168a determines the quality of the connection state between the communication section 102a and the distribution server 9 and gives a request to the image processing section 140a to convert the game image resolution suitable for this quality. Therefore, the image processing section 140a reconfigures the game image data at the requested resolution, and the first transmission processing section 168a transmits the reconfigured game image data to the distribution server 9. It should be noted that the first transmission processing section 168a can continuously monitor the connection state, and if the connection state changes, the first transmission processing section 168a can give an instruction to the image processing section 140a regarding the resolution suitable for the change.

[0079] Figure 6Depicts the functional blocks of the distribution server 9. The distribution server 9 includes a control section 300 and a communication section 302. The control section 300 includes an image acquisition section 310, a conversion section 312, and a distribution section 314. The distribution section 314 stores information about the chat room members, and when a viewing request is received from a member, distributes the game image to the information processing device 10 of a certain member in the same chat room.

[0080] In Figure 6 each element illustrated as a functional block for performing various processes can be formed by a circuit block, a memory, and any other LSI on the hardware side, and can be implemented by system software on the software side or a game program loaded in the memory. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various configurations, such as a configuration including only hardware, a configuration including only software, and a combination thereof. There is no particular limitation on the implementation of the functional blocks.

[0081] The image acquisition section 310 acquires the image data transmitted by stream from the information processing device 10a. The resolution of the image data is dynamically set according to the connection state between the information processing device 10a and the distribution server 9. Here, it is assumed that the game image data is transmitted from the information processing device 10 at a resolution of 1080p / 60fps. The conversion section 312 converts the acquired image data into image data of some convertible resolution. Specifically, the conversion section 312 transcodes the acquired image data into image data with a resolution lower than that of the original image data.

[0082] The conversion section 312 according to the embodiment has the function of converting the image data into resolutions of 720p, 540p, and 360p. The conversion section 312 converts the resolution of the image data acquired by the image acquisition section 310 into a lower resolution. Therefore, when the image acquisition section 310 acquires image data of 1080p, the conversion section 312 converts the 1080p image data into 720p image data, 540p image data, and 360p image data.

[0083] Note that when the image acquisition section 310 acquires 720p image data, the conversion section 312 converts the 720p image data into 540p image data and 360p image data. When the image acquisition section 310 acquires 540p image data, the conversion section 312 converts the 540p image data into 360p image data. Whether or not the image data is distributed, the conversion section 312 performs this conversion and waits for a viewing request from other members in the same chat room.

[0084] Figure 7An example of the system image 200 superimposed on the game image of User A is shown. The room image generation section 124a generates system image data based on the information indicating the user state acquired by the state information acquisition section 164a. Compared with the case where the system image 200 is displayed in Figure 5 User A himself starts screen sharing in the chat room. It should be noted that since the information processing device 10a side can recognize that User A has started screen sharing, the state information acquisition section 164a can acquire the information indicating the state of User A as internal information in the sharing processing section 160a.

[0085] The room image generation section 124a generates a member display area 202 based on the information about the states of multiple users, where the information about the user who sends an image and the information about the user who does not send an image are included in different areas. In Figure 7 the example shown, a host display area 206 for displaying the information about the user who sends an image and a non-host display area 208 for displaying the information about the user who does not send an image are provided in the system image 200. As Figure 7 shown, the host display area 206 and the non-host display area 208 are provided as distinguishable different areas. Therefore, User A can easily distinguish which member is performing screen sharing and which member is not. Preferably, the host display area 206 is located above the non-host display area 208 so that User A can preferentially view the information in the host display area 206 because the display area of the system image 200 is limited.

[0086] The viewing member display area 212 is an area for displaying the room members who are viewing the game image under the screen sharing of User A. In Figure 7 the case shown, no member is viewing the game image of User A

[0087] The shared game start button 210 is an operation element for allowing User A to start a shared game as the second sharing mode or the third sharing mode. The room image generation section 124a adds the shared game start button 210 to the system image 200 for performing an operation of inviting room members to participate in the ongoing game and screen sharing. When User A operates the shared game start button 210, a window for inviting other room members to share the game is displayed, so that User A can select the room members to be invited.

[0088] Figure 8 An example of the system image 200 superimposed on the game image of User A is depicted. Compared with Figure 7Compared with the system image 200, the information indicating user D is placed in the host display area 206. This indicates that user D is performing screen sharing. Since the host display area 206 for displaying information about the user sending the image and the non-host display area 208 for displaying information about the user not sending the image are provided in the system image 200, user A can easily identify which member is performing screen sharing and which member is not.

[0089] Figure 9 FIG. depicts the functional blocks of the information processing device 10c that receives game images under screen sharing. The information processing device 10c includes a processing section 100c, a communication section 102c, and a receiving section 104c. The processing section 100c includes an execution section 110c, a system image generation section 120c, an image processing section 140c, a frame buffer 150c, and a sharing processing section 160c.

[0090] The system image generation section 120c includes a report generation section 122c and a room image generation section 124c. The frame buffer 150c includes a game buffer 152c for temporarily storing game image data and a system buffer 154c for temporarily storing system image data. The sharing processing section 160c includes a status information sending section 162c, a status information obtaining section 164c, a request sending section 180c, an acceptance sending section 184c, and an image obtaining section 186c. The image obtaining section 186c includes a first image obtaining section 188c and a second image obtaining section 190c.

[0091] In Figure 9 each element illustrated as a functional block for performing various processes can be formed by a circuit block, a memory, and any other LSI on the hardware side, and can be implemented by system software on the software side or a game program loaded in the memory. Therefore, those skilled in the art will understand that these functional blocks can be implemented in various configurations, such as a configuration including only hardware, a configuration including only software, and a combination thereof. There is no particular limitation on the implementation of the functional blocks. Figure 9 The part in Figure 3 can be the same as the part in

[0092] The communication section 102c receives operation information regarding operations performed by user C on the input section of the input device 6. Additionally, the communication section 102c receives chat data created by other room members in the chat room from the management server 5, and also receives information indicating the status of other room members. Further, the communication section 102c receives streaming data regarding game images from the distribution server 9 and / or another information processing device 10. Additionally, the communication section 102c sends information indicating the status of user C to the management server 5. The functional blocks of the communication section 102c are shown as having Figure 2 the functions of both the wireless communication module 38 and the wired communication module 40 in

[0093] The receiving section 104c is provided between the communication section 102c and the processing section 100c, and exchanges data or information between the communication section 102c and the processing section 100c. When operation information regarding an operation performed on the input device 6 is received via the communication section 102c, the receiving section 104c provides the operation information to a specified functional block of the processing section 100c.

[0094] The execution section 110c executes a game program. Here, the functional block shown as the execution section 110c is implemented by software such as system software or game software, or by hardware such as a GPU. By executing the game program, the execution section 110c generates game image data and game sound data. It should be noted that a game is an example of an application, and the execution section 110c can execute any application that is not a game.

[0095] During the time when user C is playing a game, the execution section 110c executes the game program, and based on the operation information input by user C to the input device 6, performs calculations for generating the movement of game characters in the virtual space. After receiving the calculation result in the virtual space, the GPU generates game image data based on the viewpoint (virtual camera) in the virtual space.

[0096] Figure 10 An example of a game screen displayed on the output device 4c of user C is depicted. User C is playing the game title "Fighting Arena". The execution section 110c generates game image data and provides the game image data to the image processing section 140c. Then, the image processing section 140c temporarily stores the game image data in the game buffer 152c, generates a display image from the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. Thus, the output device 4c outputs the game image. It should be noted that in reality, the output device 4c also outputs the game sound generated by the execution section 110c, and user C operates the input device 6 to play the game title "Fighting Arena" while watching the game image and listening to the game sound output from the output device 4c.

[0097] In the shared processing section 160c, the status information sending section 162c sends information indicating the status of user C to the management server 5. The information indicating the status includes information indicating whether the user is playing a game, and further, if the user is playing a game, includes information indicating the game title and the on / off state of the microphone, and information about image sharing. The information about image sharing includes information about user C as the host user and information about user C as the guest user. The information about user C as the host user includes information indicating that user C starts screen sharing (first sharing mode), and information indicating an invitation to send an assist game (second sharing mode) or a cooperative game (third sharing mode) to the room members. The information about user C as the guest user includes information about the sharing mode of the game image distributed by another room member. Specifically, in order to allow another room member to view the image distributed by the distribution server 9, the status information sending section 162c sends information indicating the image to be viewed to the management server 5. In order to accept an invitation to an assist game or a cooperative game from another room member, the status information sending section 162c sends information indicating acceptance of the invitation to the management server 5.

[0098] Here, the operation of the information processing device 10c when a room member other than user C starts screen sharing will be described. Refer to Figure 5 , when user A operates the sharing start button 204, the status information sending section 162a of the information processing device 10a sends information indicating that user A starts screen sharing (first sharing mode) to the management server 5 and the distribution server 9. In addition, the first sending processing section 168a sends the stream data of the game image that the image processing section 140a has read from the game buffer 152a to the distribution server 9. After user A starts screen sharing, the management server 5 sends a report indicating that user A has started screen sharing to the information processing devices 10b, 10c, and 10d of users B, C, and D who are other room members in the chat room.

[0099] Figure 11An example of the message 220 displayed on the output device 4c of user C is depicted. The report generation section 122c generates system image data including the message 220 based on the report sent from the management server 5, and provides the system image data to the image processing section 140c. The image processing section 140c temporarily stores the system image data in the system buffer 154c, generates a display image from the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. Specifically, the image processing section 140c generates a display image by combining the game image data temporarily stored in the game buffer 152c with the system image data temporarily stored in the system buffer 154c, and provides the display image to the output device 4c. Accordingly, the output device 4c outputs a display image in which the system image is superimposed on the game image. User C sees the message 220 and recognizes that user A has started screen sharing. Note that the report generation section 122c may perform sound output to notify user C that user A has started screen sharing.

[0100] When user C presses a prescribed button on the input device 6 while the message 220 is being displayed, the request sending section 180c sends a viewing request including information for identifying user A to the management server 5. Note that the request sending section 180c may send a viewing request including information for identifying user A to the distribution server 9.

[0101] It should be noted that, for example, the message 220 is only displayed for five seconds. After the message 220 disappears, user C can display a system image to send a request to the management server 5 or the distribution server 9 to view the game image distributed by user A. When user C briefly presses a prescribed button on the input device 6, the receiving section 104c receives the button operation and provides the operation information to the system image generation section 120c. The system image generation section 120c obtains the button operation information as a system image display request and calls the status information acquisition section 164c.

[0102] The status information acquisition section 164c acquires information indicating the user's status from the management server 5 and provides this information to the system image generation section 120c. In the system image generation section 120c, the room image generation section 124c generates system image data indicating the status of the chat room in which the user C is participating and provides this system image data to the image processing section 140c. The image processing section 140c temporarily stores the system image data in the system buffer 154c, generates a display image from the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. Specifically, the image processing section 140c generates a display image by combining the game image data temporarily stored in the game buffer 152c with the system image data temporarily stored in the system buffer 154c and provides the display image to the output device 4c. Accordingly, the output device 4c outputs a display image in which the system image is superimposed on the game image.

[0103] Figure 12 An example of a system image 200 superimposed on a game image is depicted. The room image generation section 124c generates system image data based on the information indicating the user's status acquired by the status information acquisition section 164c. A member display area 202 indicating the status of members in the chat room in which the user has participated is provided in the system image 200. The room image generation section 124c generates the member display area 202 including information about a plurality of room members (users) based on the information indicating the member status. In Figure 12 this, a user icon, a user name, the title of the game being played, and information indicating the on / off state of the microphone are displayed in the member display area 202. In addition, information such as whether a shared game is being executed can be included in the member display area 202.

[0104] The room image generation section 124c generates the member display area 202 based on the information about the status of a plurality of users, where information about the user who sends an image and information about the user who does not send an image are included in different areas. In Figure 12 this example, a host display area 206 for displaying information about the user who sends an image and a non-host display area 208 for displaying information about the user who does not send an image are provided in the system image 200. As Figure 12 shown, the host display area 206 and the non-host display area 208 are provided as distinguishable different areas. Accordingly, the user C can easily distinguish which member is performing screen sharing and which member is not performing screen sharing.

[0105] When user C selects the display area of user A by using the input device 6, the receiving section 104c receives the operation of the user who selects to send the image. Then, the request sending section 180c sends a viewing request including information for identifying the selected user to the management server 5. After receiving the viewing request, the management server 5 sends the viewing request to the distribution server 9 to report that user C wishes to view the game image of user A. It should be noted that the request sending section 180c may send the viewing request directly to the distribution server 9.

[0106] The first image acquisition section 188c determines the resolution of the game image data to be received from the distribution server 9 according to the quality of the connection state between the communication section 102c and the distribution server 9, and adds the resolution of the image to be received to the viewing request. That is, if the connection state is poor, the first image acquisition section 188c determines to receive low-resolution game image data. In the distribution server 9, the conversion section 312 generates image data in multiple resolutions, and the distribution section 314 distributes the image data with a resolution suitable for the connection state with respect to the information processing device 10c to the information processing device 10c.

[0107] Here, it is assumed that game image data with a resolution of 1080p has been sent from the information processing device 10a of user A, and the conversion section 312 has generated image data with a resolution of 720p, image data with a resolution of 540p, and image data with a resolution of 360p. When the connection state between the distribution server 9 and the information processing device 10c is good, the first image acquisition section 188c requests game image data with a resolution of 1080p, and the distribution section 314 distributes the game image data with a resolution of 1080p to the information processing device 10c. However, when the connection state is not good, the distribution section 314 distributes game image data with a resolution lower than 1080p.

[0108] Return reference Figure 9 Then, the first image acquisition section 188c acquires game image data from the distribution server 9. In the information processing device 10c, the format for displaying the game image under screen sharing is determined. In this embodiment, three display formats are prepared: (1) full-screen display, (2) picture-in-picture display, and (3) split-screen display. User C determines any one of the display formats in advance.

[0109] After receiving the image data from the first image acquisition section 188c, the image processing section 140c temporarily stores the image data in the system buffer 154c according to the determined display format, generates a display image according to the image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c.

[0110] Figure 13 depicts a display image in full-screen display format. When the full-screen display format is set, the image processing section 140c temporarily stores the image data acquired by the first image acquisition section 188c in the system buffer 154c, generates a display image based on the temporarily stored image data, and provides the display image to the output device 4c. Accordingly, the output device 4c performs full-screen display of the game image distributed by user A

[0111] Figure 14 depicts a display example in picture-in-picture display format. When the picture-in-picture display format is set, the image processing section 140c reduces the image data acquired by the first image acquisition section 188c and temporarily stores the reduced image data in the system buffer 154c, and generates a display image by combining the game image data of user C temporarily stored in the game buffer 152a and the game image data of user A temporarily stored in the system buffer 154c, and provides the display image to the output device 4c. Accordingly, the output device 4c outputs a display image in which the display area 230 of the game image distributed by user A is superimposed on the image of the game that user C is playing. It should be noted that the position and size of the display area 230 can be freely set by user C in the picture-in-picture display format

[0112] Figure 15 depicts a display example in split-screen display format. When the split-screen display format is set, the image processing section 140c reduces the image data acquired by the first image acquisition section 188c, and temporarily stores the reduced image data in the system buffer 154c, and reduces the game image data generated by the execution section 110c, and temporarily stores the game image data in the game buffer 152c. The image processing section 140c generates a display image by combining the game image data of user C temporarily stored in the game buffer 152a and the game image data of user A temporarily stored in the system buffer 154c, and provides the display image to the output device 4c. In split-screen display, the screen of the output device 4c is divided so that the image of the game that user C is playing and the game image distributed by user A are displayed side by side. It should be noted that in split-screen display, the size of the display area 232 can be freely set by user C

[0113] In picture-in-picture display or split-screen display, the image processing section 140c displays the game image distributed from the distribution server 9 on a part of the display. The distributed image is displayed on a part of the display so that user C can watch the video of user A's game while playing the title "Fighting Arena"

[0114] Note that in picture-in-picture display or split-screen display, the resolution of the game image of user A does not need to be very high because the image is reduced and then displayed on a part of the display. For this reason, the request transmission section 180c may request the transmission of game image data with a resolution lower than 1080p. In this case, the distribution section 314 of the distribution server 9 distributes the image data to the information processing device 10c at a resolution suitable for the display mode in the information processing device 10c. Therefore, communication resources can be used effectively.

[0115] Return reference Figure 3 When user A briefly presses a specified button on the input device 6 during the game, the reception section 104a receives the button operation and provides the operation information to the system image generation section 120a. The system image generation section 120a obtains the button operation information as a system image display request and calls the status information acquisition section 164a.

[0116] The status information acquisition section 164a acquires information indicating the user status from the management server 5 and provides the information to the system image generation section 120a. The room image generation section 124a generates system image data indicating the status of the chat room in which user A is participating and provides the system image data to the image processing section 140a. The image processing section 140a temporarily stores the system image data in the system buffer 154a, generates a display image based on the image data temporarily stored in the frame buffer 150a, and provides the display image to the output device 4a.

[0117] Figure 16 An example of the system image 200 superimposed on the game image of user A is depicted. Compared with the system image 200 depicted in Figure 8 information indicating that user C is watching the game image under the screen sharing of user A is displayed in the viewing member display area 212. User A sees the viewing member display area 212 and can recognize that there is a member watching the game image that user A has distributed.

[0118] As described above, the shared game start button 210 is provided to allow user A to start a shared game as the second shared mode or the third shared mode, and is an operation element for performing an operation to invite members to join an ongoing game. When user A operates the shared game start button 210 to select a member to execute a shared game with user A, the invitation transmission section 172a sends information indicating that the selected member is invited to share the game to the management server 5. The management server 5 sends an invitation to share the game to the information processing device 10 of the selected member. In this embodiment, user A invites user C to share the game.

[0119] Figure 17Depicts an example of message 222 displayed on the output device 4c of user C. It should be noted that the image processing section 140c performs a picture-in-picture display of the game image distributed by user A in the display area 230

[0120] The report generation section 122c generates system image data including message 222 based on the report sent from the management server 5, and provides the system image data to the image processing section 140c. The image processing section 140c temporarily stores the system image data in the system buffer 154c, generates a display image based on the game image data and the system image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c. User C sees message 222 and realizes that user A has invited user C to share the game. Note that the report generation section 122c can perform voice output to notify user C that user A has invited user C to share the game.

[0121] When user C presses a specified button on the input device 6 while message 222 is being displayed, the acceptance sending section 184c sends information for identifying user C and information indicating acceptance of the invitation to the management server 5.

[0122] It should be noted that, for example, message 222 is only displayed for five seconds. After message 222 disappears, user C can display the system image to accept the invitation sent by user A. When user C briefly presses the specified button on the input device 6, the receiving section 104c receives the button operation and provides the operation information to the system image generation section 120c. The system image generation section 120c obtains the button operation information as a system image display request and calls the status information acquisition section 164c.

[0123] The status information acquisition section 164c obtains information indicating the user status from the management server 5 and provides the information to the system image generation section 120c. In the system image generation section 120c, the room image generation section 124c generates system image data indicating the status of the chat room in which user C is participating, and provides the system image data to the image processing section 140c. The image processing section 140c temporarily stores the system image data in the system buffer 154c, generates a display image based on the game image data and the system image data temporarily stored in the frame buffer 150c, and provides the display image to the output device 4c.

[0124] Figure 18 Shows an example of the system image 200 superimposed on the game image of user C. As with Figure 12Compared with the system image 200 in [description], since user C has been invited to participate in the shared game, the information indicating that user A has invited user C to participate in the shared game is displayed in the shared game host display area 216. User C sees the shared game host display area 216 and recognizes that user A has invited user C to participate in the shared game.

[0125] The shared game participation button 214 is provided to allow user C to join the shared game and is the operation element for user C to perform the operation of accepting the invitation sent by user A. When user C operates the shared game participation button 214, the acceptance sending part 184c sends information indicating acceptance of the game invitation sent by user A to the management server 5. The management server 5 gives a report indicating that user C has accepted the invitation to the information processing device 10a of user A.

[0126] Return reference Figure 3 When the receiving part 104a receives the information indicating that user C has accepted the invitation, the connection processing part 174a performs the process of establishing a P2P connection with the information processing device 10c of user C. After establishing the P2P connection, the second sending processing part 170a sends the stream data of the game image that the image processing part 140a has read from the game buffer 152a to the information processing device 10c of the P2P connection. That is, the second sending processing part 170a sends the game image for the game to the information processing device 10c of user C who has accepted the invitation, rather than via the distribution server 9. The stream data to be sent can be the same as the stream data sent to the distribution server 9. Note that if the connection state between the communication part 102a and the distribution server 9 is different from the connection state between the communication part 102a and the information processing device 10c, the stream data can be sent at the corresponding resolution suitable for the connection state.

[0127] Figure 19 An example of the system image displayed on the output device 4c of user C is depicted. In Figure 18 the system image 200 in [description], when user C operates the shared game participation button 214, the process of establishing a P2P connection between the information processing device 10c and the information processing device 10a is performed in the background. Therefore, the system image generation part 120c generates a system image such as Figure 19 shown, so that user C realizes that the connection has been established until the game starts.

[0128] Note that the image processing section 140c continuously displays the game images acquired from the distribution server 9 in the display area 230 until the game is ready. As a result, while waiting for the game to start, user C can watch the game of user A. Note that the expression "until the game is ready" means the period from when the P2P connection is completed to when the stream data regarding the game images is sent from the second transmission processing section 170a.

[0129] Note that even when the distributed image is displayed in a split-screen display format instead of a picture-in-picture display format, the image processing section 140c can continuously display the game images acquired from the distribution server 9 in the display area 232 until the game is ready. In the case where user C operates the shared-game participation button 214 while the distributed image is displayed on a part of the display, the image processing section 140c continuously displays the distributed image in such a manner that user C can watch the image distributed by user A even while waiting for the connection to be established.

[0130] After the P2P connection is established, the second transmission processing section 170a sends the game images for the game to the information processing device 10c of user C who has accepted the invitation. In the information processing device 10c, the second image acquisition section 190c acquires the game images for the game from the information processing device 10a without going through the distribution server 9.

[0131] Figure 20 An image of the game that user C is playing is depicted. Since the message 224 is superimposed on the game image, user C recognizes that the shared game has started. The image processing section 140c temporarily stores the image data acquired by the second image acquisition section 190c in the system buffer 154c, generates a display image based on the temporarily stored image data, and provides the display image to the output device 4c. As a result, the output device 4c displays the game images distributed by user A. User C performs the shared game while watching the displayed game images. So far, the present disclosure has been explained based on the embodiment. This embodiment is illustrative. Those skilled in the art will understand that many modifications can be made to the combination of constituent elements or the combination of processes, and these modifications are also included in the scope of the present disclosure.

[0132] [Industrial Applicability]

[0133] The present disclosure is applicable to the technology of sharing images among multiple users.

[0134] [List of Reference Signs]

[0135] 1: Image sharing system

[0136] 4: Output device

[0137] 5: Management server

[0138] 6: Input device

[0139] 9: Distribution server

[0140] 10a, 10b, 10c, 10d: Information processing devices

[0141] 20: Main power button

[0142] 100a, 100c: Processing parts

[0143] 102a, 102c: Communication parts

[0144] 104a, 104c: Receiving parts

[0145] 110a, 110c: Execution parts

[0146] 120a, 120c: System image generation parts

[0147] 122a, 122c: Report generation parts

[0148] 124a, 124c: Room image generation parts

[0149] 140a, 140c: Image processing parts

[0150] 150a, 150c: Frame buffers

[0151] 152a, 152c: Game buffers

[0152] 154a, 154c: System buffers

[0153] 160a, 160c: Shared processing parts

[0154] 162a, 162c: Status information sending parts

[0155] 164a, 164c: Status information obtaining parts

[0156] 166a: Sending processing part

[0157] 168a: First sending processing part

[0158] 170a: Second sending processing part

[0159] 172a: Invitation sending part

[0160] 174a: Connection processing part

[0161] 180c: Request sending part

[0162] 182c: Invitation receiving part

[0163] 184c: Reception and transmission section

[0164] 186c: Image acquisition section

[0165] 188c: First image acquisition section

[0166] 190c: Second image acquisition section

[0167] 300: Control section

[0168] 302: Communication section

[0169] 310: Image acquisition section

[0170] 312: Conversion section

[0171] 314: Distribution section

Claims

1. An information processing device is connected to a management server that manages the status of a plurality of members participating in a room and a distribution server that distributes images, and the information processing device is configured to: Generate an image of the screen of the user corresponding to the information processing device; Send the image of the user's screen to the distribution server; Obtain information indicating the status of the plurality of members from the management server; Based on the information indicating the status of the plurality of members, generate a first member display area, in which information about members who send images and information about members who do not send images are included in different areas, where the information about members who send images includes information about the user and information about members other than the user who send images, and the images sent by members other than the user are images of the screens of members other than the user; Generate a second member display area, which includes information about members other than the user who are viewing the image sent from the information processing device to the distribution server; and Simultaneously display the image of the user's screen, the first member display area, and the second member display area.

2. The information processing apparatus according to claim 1, wherein The information processing device is further configured to: Display an operation element, and perform an operation of inviting members other than the user to participate in an ongoing game through the operation element.

3. The information processing device according to claim 2, wherein The information processing device is further configured to: create and maintain a peer-to-peer connection with another information processing device, When a member other than the user who is invited accepts the invitation, the information processing device is configured to: via the peer-to-peer connection and without passing through the distribution server, send an image for sharing to other information processing devices of members other than the user who have accepted the invitation.

4. A method for sharing images by an information processing device, including: Generate an image of the screen of the user corresponding to the information processing device; Send the image of the user's screen to a distribution server that distributes images; Obtain information indicating the status of the plurality of members from a management server that manages the status of a plurality of members participating in a room; Based on the information indicating the status of the plurality of members, generate a first member display area, in which information about members who send images and information about members who do not send images are included in different areas, where the information about members who send images includes information about the user and information about members other than the user who send images, and the images sent by members other than the user are images of the screens of members other than the user; Generate a second member display area, which includes information about members other than the user who are viewing the image sent to the distribution server; and Simultaneously display the image of the user's screen, the first member display area, and the second member display area.

5. A recording medium that stores a program to cause a computer to execute the method according to claim 4.

Images