Computer system and method for controlling the generation of viewable video

Abstract

To provide a technique that controls appearance of a character operated by a user in an image that the user views, the user seeing an event held in a virtual space.SOLUTION: A user moves an avatar 4 of the user into a venue of a predetermined event held in a virtual space, and defines the avatar as an audience character 5. The audience character 5 has an index value which is variably set based on the content of control for the audience character, the index value indicating easiness to appear of the character in a viewing video that the user seeing the event views. In generating the viewing video, an application camera-work is applied to common-view cameras Cp (Cp1, Cp2, ...) for capturing images to be used in the viewing video. In the application camera-work, at least one of imaging specifications, such as imaging position, imaging direction, and imaging field of angle, is modified so that an appearance (5Y') of the audience character may be increased / decreased based on the index value of the character.SELECTED DRAWING: Figure 10

Description

Technical Field

[0001] The present invention relates to a computer system or the like that performs control for generating a viewing video. Background Art

[0002] A technique is known in which a plurality of users can simultaneously participate in a virtual space constructed by 3DCG (3 Dimensional Computer Graphics) using their respective avatars, move freely in the virtual space with the avatars, and use various services prepared in the virtual space via the avatars (see, for example, Patent Document 1).

[0003] One of the services in a virtual space is an event that users can participate in as spectators (see, for example, Patent Document 2). The content of events varies, and the interpretation of the term "event" can be broad or narrow; illustrative examples include appreciation of singing, performances, and acting by performers, talk shows, public debates, public lectures, watching game play, sports, and the like. In the case of an idol live (an event where idols sing, dance, and talk), users can obtain a user experience similar to that of a live event in the real world. Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2022-36689 Patent Document 2 Japanese Unexamined Patent Application Publication No. 2021-41241 Summary of the Invention Problem to be Solved by the Invention

[0005] In real-world live events, large screens are set up in the venue to display images of the performers (for example, idols singing and dancing at an idol concert), as well as visual effects (for example, geometric patterns created with CG, beautiful scenery, or imagery), and sometimes even the audience. Finding oneself reflected in the images displayed on the large screen is one of the pleasures for audience members at a live concert. However, there was no known technology to control the appearance of a user's character in the images viewed by users watching events in a virtual space.

[0006] The problem that this invention aims to solve is to provide a technology that controls the appearance of a user's character in images viewed by a user watching an event taking place in a virtual space. [Means for solving the problem]

[0007] The first invention for solving the above-mentioned problems is a computer system that controls the generation of a video for viewing a given event taking place in a virtual space, Based on the actions of the audience user, who is the owner of the audience character that virtually observes the event in the virtual space, the audience character control means (for example, the processing unit 200s and audience character control unit 204 in Figure 14) control the audience character, An index value setting means (for example, the processing unit 200s and index value setting unit 206 in Figure 14) that variably sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video based on the control content of the audience character by the audience character control means, The system includes reflection control means (for example, the processing unit 200s and reflection control unit 208 in Figure 14) that control the reflection of the audience character onto the video being viewed based on the index value of the audience character, It is a computer system equipped with [the following features].

[0008] Furthermore, as another invention, a method for controlling the generation of a viewing video for viewing a given event taking place in a virtual space can be configured, which includes: a viewer character control step that controls a viewer character based on the operation of a viewer user who is the owner of a viewer character that virtually watches the event in the virtual space; an index value setting step that sets an index value indicating how easily the viewer character is reflected in the viewing video, based on the content of the control of the viewer character; and a reflection control step that controls the reflection of the viewer character in the viewing video based on the index value of the viewer character.

[0009] The second invention is a computer system in which the event occurs when the characters appear in the virtual space.

[0010] An "event" is a temporary, virtual event involving an audience. For example, an event could include singing, playing musical instruments, dancing, magic tricks, theatrical performances, demonstrations of creating works of art or crafts, etc. It could also be gameplay or watching sports. It could even be a public filming of a movie or video for online posting in a studio-like setting.

[0011] According to the first invention, a user can watch, observe, and appreciate an event in a virtual space as a spectator through a character they control (spectator character). Each spectator character is assigned an index value, and the appearance of that spectator character is controlled based on that index value. For example, the degree to which a spectator character is visible can be adjusted. For instance, it becomes possible to intentionally include a particular spectator character in the image or to suppress its appearance.

[0012] The third invention is a computer system further comprising the above-described computer system, which includes a main image generation means for each audience user (for example, the processing unit 200s and the main image generation unit 210 for each audience user in Figure 14) that generates an image of the character to be included in the video to be viewed, based on the position of the audience user's character in the virtual space.

[0013] According to the third invention, audience users can change how they view the characters (the images of the characters) by manipulating the posture and position of the audience characters.

[0014] The fourth invention is a computer system in which the reflection control means controls the viewer character and the featured character so that the viewer character and the featured character whose index value meets predetermined conditions appear in the viewing video at the same time (for example, by prioritizing the selection of attribute 533 of the common viewpoint camera definition data 530 in Figure 16, and the attribute in step S30 of Figure 24 that allows the featured character and the viewer character to appear simultaneously).

[0015] According to the fourth invention, the audience user can see an image in which their own audience character and the characters appearing in the show are displayed on the same screen.

[0016] The fifth invention is a computer system in which the reflection control means controls the reflection of the audience characters by controlling a virtual camera (for example, the common viewpoint camera Cp(Cp1, Cp2,...) in Figure 5) that photographs the virtual space.

[0017] According to the fifth invention, one way to control reflections is to control a virtual camera that captures images of a virtual space.

[0018] A sixth aspect of the invention provides a computer system according to the above computer system, wherein the reflection control means controls the reflection of the audience character in the viewing video in a time-division manner by switching photographing targets (for example, step S50 in FIG. 22 based on cut division data 550 in FIG. 16).

[0019] According to the sixth aspect of the invention, by switching photographing targets, it becomes possible to control the reflection of the audience character in the event viewing video in a time-division manner.

[0020] A seventh aspect of the invention provides a computer system according to the above computer system, wherein a video portion of the viewing video when the photographing target is switched in a time-division manner by the reflection control means and the audience character is reflected is common among a plurality of audience users (for example, the large screen 12 in FIG. 4, and the large screen viewing video 36 in FIG. 6 displayed on the multi-image display unit 22).

[0021] According to the seventh aspect of the invention, the video portion of the viewing video in which the audience character is reflected is displayed to be common among the plurality of audience users.

[0022] An eighth aspect of the invention provides a computer system according to the above computer system, wherein the event includes a photographing part in which the appearing character is photographed together with another character, and the reflection control means performs control to include the audience character whose index value satisfies a predetermined condition in said another character and perform photographing together with the appearing character in the photographing part (for example, step S94 in FIG. 23).

[0023] According to the eighth aspect of the invention, the event includes a photographing part in which photographing is performed such that the audience character whose index value satisfies a predetermined condition is captured in the same image as the appearing character. Therefore, for a user of the audience character whose index value satisfies the predetermined condition, the pleasure of finding an image in which their own character is reflected is increased.

[0024] A ninth aspect of the present invention provides the computer system described above, wherein the reflection control means controls reflection of the spectator character by controlling the position of the spectator character in the virtual space (for example, step S21 in FIG. 26, step S78 in FIG. 27).

[0025] According to the ninth aspect of the present invention, a computer system that controls reflection by controlling the position of the spectator character can be configured.

[0026] A tenth aspect of the present invention provides the computer system described above, wherein a virtual display unit is installed in the virtual space, and the reflection control means controls reflection of the spectator character by performing control to display the spectator character on the virtual display unit (for example, large screen 12 in FIG. 4, viewing video 36 for large screen in FIG. 6).

[0027] According to the tenth aspect of the present invention, a virtual display unit is installed in the virtual space, and the spectator character is displayed on the virtual display unit, whereby reflection of the spectator character is controlled.

[0028] An eleventh aspect of the present invention provides the computer system described above, wherein the reflection control means controls reflection of the spectator character by performing control to cause the viewing video to be a multi-image display including an image of the spectator character.

[0029] According to the eleventh aspect of the present invention, the viewing video is set as a multi-image display including an image of the spectator character, whereby reflection of the spectator character is controlled.

[0030] The twelfth invention is a computer system that controls the generation of a viewing video for viewing a given event taking place in a virtual space, comprising: a viewer character control means that controls a viewer character based on the operation of a viewer user who is the owner of a viewer character that virtually watches the event in the virtual space; an index value setting means that sets an index value indicating how easily the viewer character is reflected in the viewing video using the results of a given game played by the viewer user (for example, the game play result point Pg in Figure 7, the index value setting unit 206 in Figure 14, and step S12 in Figure 22); and reflection control means that controls the reflection of the viewer character in the viewing video based on the index value of the viewer character.

[0031] Furthermore, as another invention, a method for controlling the generation of a viewing video for viewing a given event taking place in a virtual space can be configured, which includes: a viewer character control step that controls a viewer character based on the actions of a viewer user who is the owner of a viewer character that virtually watches the event in the virtual space; an index value setting step that sets an index value indicating how easily the viewer character is reflected in the viewing video using the results of a given game played by the viewer user; and a reflection control step that controls the reflection of the viewer character in the viewing video based on the index value of the viewer character.

[0032] An "event" is a temporary, virtual event involving an audience. For example, an event could include singing, playing musical instruments, dancing, magic tricks, theatrical performances, demonstrations of creating works of art or crafts, etc. It could also be gameplay or watching sports. It could even be a public filming of a movie or video for online posting in a studio-like location. A "game" can take place within or outside of an event.

[0033] According to the 12th invention, users can watch, observe, and appreciate events in a virtual space as spectators through a character they control (spectator character). Each spectator character is assigned an index value based on the results of the game played by the user of that spectator character, and the appearance of that spectator character is controlled based on the index value. Thus, the degree to which spectator characters are displayed is adjusted. For example, it becomes possible to intentionally display a particular spectator character or to suppress its display.

[0034] The thirteenth invention is a computer system in which the index value setting means sets the index value based on the behavior of the audience character in the virtual space controlled by the audience character control means (for example, the behavior point Pa in Figure 7).

[0035] According to the 13th invention, it becomes possible to set indicator values ​​based on the actions of audience characters. For example, if the event is an idol live event, improving the visibility of audience characters using cheering items will make the video look better as they are reflected in the viewer's video.

[0036] The fourteenth invention is a computer system in which the index value setting means sets the index value based on the appearance setting of the audience character controlled by the audience character control means (for example, the appearance point Pc in Figure 7).

[0037] According to the 14th invention, it becomes possible to set index values ​​based on the appearance settings of audience characters. For example, if the event is an idol's live performance, improving the visibility of audience characters wearing T-shirts supporting the idol will improve the visual appeal of the video being viewed.

[0038] The fifteenth invention is a computer system in which the index value setting means sets the index value using the results of a given game played by the spectator user (for example, the game play result points Pg in Figure 7).

[0039] According to the 15th invention, it becomes possible to set index values ​​using the game play results of spectator users. For example, it becomes possible to control reflections from a wider range of perspectives, such as improving the visibility of spectator characters with excellent playing performance.

[0040] The sixteenth invention is a computer system in which the reflection control means performs control to select the audience character to appear in the viewing video based on the index value (for example, priority character 5Y in Figures 9 and 10, step S20 in Figure 22, and step S76 in Figure 23).

[0041] According to the 16th invention, the audience character to be targeted for reflection control can be selected based on an index value.

[0042] The 17th invention is a computer system in which the reflection control means controls at least one of the reflection time, reflection position, and reflection image ratio of the audience character reflected in the viewing video based on the index value of the audience character.

[0043] According to the 17th invention, reflection control can be achieved by changing the reflection time, reflection position, and reflection image ratio of the audience character.

[0044] The 18th invention is a computer system further comprising the above-described computer system, which includes reflection result evaluation means for evaluating the results reflected in the viewing video for each of the audience users (for example, the reflection result evaluation unit 212 in Figure 14, the audience character-specific reflection result data 712 in Figure 20, and step S54 in Figure 22).

[0045] According to the 18th invention, it becomes possible to evaluate the results of reflections. By feeding the evaluation results back into the reflection control, the effectiveness of the reflection control can be improved.

[0046] The 19th invention is a computer system further comprising a ranking generation means (for example, a ranking generation unit 214 in Figure 14) that generates a ranking based on the results of the evaluation, in the computer system described above.

[0047] According to the 19th invention, a ranking of reflection results can be generated. [Brief explanation of the drawing]

[0048] [Figure 1] A diagram showing an example configuration of a service delivery system. [Figure 2] A diagram to explain virtual space. [Figure 3] This diagram illustrates an example of a virtual spot configuration that provides a virtual experience of viewing an event as a service. [Figure 4] A diagram illustrating how an event is viewed from a user's perspective. [Figure 5] A diagram illustrating how an event is viewed from a common-view camera perspective. [Figure 6] A diagram illustrating the display on a large screen. [Figure 7] A diagram to explain the indicator values. [Figure 8] A diagram to explain the reflection evaluation value. [Figure 9] A diagram illustrating the selection of preferred characters. [Figure 10] A diagram illustrating the details of reflection control. [Figure 11] A conceptual diagram to explain the changes in camera work. [Figure 12] A diagram showing an example of the data structure for candidate camera work change data. [Figure 13] A diagram showing an example of the data structure for candidate camera work change data. [Figure 14] A block diagram showing an example of the functional configuration of a server-side system. [Figure 15] A diagram showing examples of programs and data stored in the memory of a server-side system. [Figure 16] This diagram shows an example of the data structure for initial settings data for each service. [Figure 17] A diagram showing an example of the data structure of cut-out data. [Figure 18] A diagram showing an example of the data structure of the benefit definition data. [Figure 19] A diagram showing an example of the data structure of user management data. [Figure 20] A diagram showing an example of the data structure for service control data. [Figure 21] A functional block diagram showing an example of the functional configuration of a user terminal. [Figure 22] A flowchart illustrating the processing flow related to service provision performed by the server-side system. [Figure 23] Flowchart continuing from Figure 22. [Figure 24] A flowchart illustrating the process of modifying cutwork. [Figure 25] A diagram showing an example of the data structure for seating area characteristic data in the second embodiment. [Figure 26] A flowchart illustrating the processing flow related to service provision performed by the server-side system in the second embodiment. [Figure 27] Flowchart continuing from Figure 26. [Modes for carrying out the invention]

[0049] Examples of embodiments of the present invention will be described below, but it goes without saying that the embodiments to which the present invention can be applied are not limited to the following embodiments.

[0050] [First Embodiment] Figure 1 shows an example configuration of the service provision system 1000. The service provision system 1000 is a computer system that allows multiple users to simultaneously participate in a virtual space constructed with 3DCG (3-Dimensional Computer Graphics) using their respective user character avatars, move freely within the virtual space using their avatars, and receive various services provided within the virtual space through their avatars.

[0051] The service provision system 1000 is a computer system that includes a server-side system 1010 connected via network 9 for data communication and user-specific user terminals 1500 (1500a, 1500b, ...), with the user terminals 1500 serving as the Man-Machine Interface (MMIF).

[0052] Network 9 refers to a communication path capable of data communication. In other words, Network 9 includes not only LANs (Local Area Networks) using dedicated lines (dedicated cables) or Ethernet (registered trademark) for direct connections, but also telephone communication networks, cable networks, and the Internet.

[0053] The server-side system 1010 is a computer system managed and operated by the operator of the service provision system 1000, and comprises a portal server system 1100P and a plurality of service server systems 1100G (1100Ga, 1100Gb, ...).

[0054] The portal server system 1100P is a computer system that comprehensively manages the virtual space and is the front-end server system that the user terminal 1500 of a user participating in the virtual space first accesses. The portal server system 1100P performs, for example, user registration control, avatar customization control, avatar control within the virtual space, control of the virtual world within the virtual space, and control of the provision of various services according to the position and movement of the avatar. Of course, other controls may be performed depending on the content of the services provided in the virtual space. Furthermore, one or more of these controls may be implemented by a back-end server.

[0055] The service server system 1100G (1100Ga, 1100Gb, ...) is a backend server that provides one or more types of services to user terminals 1500 as clients. The content of the services provided can be configured as appropriate. The content of the "services" can be configured as appropriate. For example, shopping, games, visits to art museums, movies, museums, events, etc.

[0056] An "event" is a temporary occurrence in a virtual space that involves an audience. For example, an event can be a performance such as singing, playing a musical instrument, dancing, magic tricks, acting, or creating artwork or crafts, or it can be a game or sports competition. It can also be the public filming of a movie or a video for online posting. In the following, the services offered will be explained using a live event centered on singing as an example.

[0057] The portal server system 1100P and the service server system 1100G each consist of a main unit 1101, a keyboard, a touch panel, and storage 1140, with a control board 1150 mounted on the main unit 1101. The control board 1150 is equipped with various microprocessors such as a CPU (Central Processing Unit) 1151, a GPU (Graphics Processing Unit), and a DSP (Digital Signal Processor), various IC memories 1152 such as VRAM, RAM, and ROM, and a communication device 1153. Note that part or all of the control board 1150 may be implemented using an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), or a SoC (System on a Chip).

[0058] The portal server system 1100P and the service server system 1100G each realize separate functions through calculation processing performed by the control board 1150 based on predetermined programs and data.

[0059] In Figure 1, the portal server system 1100P and the service server system 1100G are depicted as if they were each a single server device, but they may also be implemented using multiple devices. For example, the portal server system 1100P may be configured with multiple blade servers, each handling different functions, and connected to each other via an internal bus for data communication. The same applies to the service server system 1100G. The portal server system 1100P and the service server system 1100G may also include a database and online storage.

[0060] The user terminal 1500 is a computer system used by users to participate in the virtual space and serves as the human-machine interface in the service provision system 1000. The user terminal 1500 can be used not only for participating in the virtual space, but also for browsing websites and using online services such as online games operated outside of the service provision system 1000.

[0061] User terminals 1500 (1500a, 1500b, ...) are computer systems that can connect to the network 9, such as personal computers, smartphones, wearable computers, portable game consoles, home game consoles, and tablet computers.

[0062] When a primary computer and various devices connected to it perform a single function as a whole, the primary computer and the various devices can be considered as a single user terminal 1500. For example, in Figure 1, user terminal 1500a shows an example where a VR (Virtual Reality) MMIF set 1580 is connected to a desktop PC. The VR MMIF set 1580 includes VR goggles 1581, a VR headset 1583, a VR controller 1585, etc. User terminal 1500b in Figure 1 is exemplified as a smartphone alone, but user terminal 1500b may also be configured to include the VR MMIF set 1580, similar to user terminal 1500a.

[0063] The user terminal 1500 includes an operation input device (e.g., touch panel 1506, keyboard, game controller, mouse, etc.), an image display device (e.g., touch panel 1506, head-mounted display, glasses-type display, etc.), and a control board 1550.

[0064] The control board 1550 is equipped with a CPU 1551, various microprocessors such as a GPU and DSP, various IC memories 1552 such as VRAM, RAM, and ROM, and a communication module 1553 that connects to the network 9. These elements mounted on the control board 1550 are electrically connected via bus circuits and the like, enabling data reading and writing, and signal transmission and reception. Part or all of the control board 1550 may be configured with an ASIC, FPGA, or SoC. The control board 1550 stores programs and various data necessary to realize the functions of a user terminal 1500 in the IC memory 1552. By executing a predetermined application program, the user terminal 1500 realizes the functions of a man-machine interface for the service provision system 1000 and a client when a user receives various services.

[0065] The user terminal 1500 is configured to download application programs and various data necessary for their execution from the portal server system 1100P, but it may also be configured to read them from a storage medium such as a memory card obtained separately by the user.

[0066] Figure 2 is a diagram illustrating the virtual space 3 provided by the service provision system 1000. For ease of understanding, only one avatar 4 is depicted; however, in actual operation, multiple avatars 4 may exist simultaneously within the virtual space 3. Furthermore, the number of virtual spots 6 provided for each service within the virtual space 3 is not limited to the example in Figure 2; many more can be set in actual operation.

[0067] Virtual Space 3 is a digital data space constructed with 3DCG, where multiple users can participate simultaneously using their respective avatars 4, and where each user can act freely with their avatar 4. Virtual Space 3 is a parallel world that has the same flow of time as reality. Even without participating users, Virtual Space 3 continues to exist as data on the server-side system 1010, and time progresses in Virtual Space 3 along with the passage of time in the real world, causing the world represented in Virtual Space 3 to constantly change. On the user terminal 1500, an image of Virtual Space 3 (hereinafter referred to as Virtual Space Image 90) is basically displayed from a first-person perspective as seen from the viewpoint of avatar 4. However, the Virtual Space Image 90 may also be displayed from a third-person perspective.

[0068] Virtual spots 6 (6a, 6b, ...) are set up within virtual space 3 for each service. In the example in Figure 2, virtual spot 6 is depicted as a store, but it may also be set up as an open area like a plaza. Users who have joined virtual space 3 after going through the login procedure become participating users and can enjoy the services provided by virtual spot 6 by moving their avatar 4 to virtual spot 6.

[0069] Figure 3 illustrates an example configuration of virtual spot 6d, which provides a virtual experience of viewing an event as a service. The virtual spot 6d, which provides the event, is designed to resemble an event venue, and includes facility objects such as a stage 10, a large screen 12, and seating areas 14. The large screen 12 is an example of a virtual display unit.

[0070] Users who wish to participate in the event move their avatar 4 to virtual spot 6d, perform the given event participation procedure, and then move to the audience seat 14. The event participation procedure is a procedure in virtual space 3 equivalent to purchasing a live concert ticket or entering a venue in the real world. By going through the event participation procedure, the user becomes an audience user for that event, and avatar 4 becomes an audience character 5.

[0071] When the event begins, character 7 will appear on stage 10 and perform various acts. In the case of a live event, character 7 will be a singer or musician. Character 7 may be a pre-prepared NPC (Non-Player Character) or an avatar 4 of a user. Acts may include singing, playing instruments, dancing, as well as talking and mini-games involving the audience.

[0072] Figure 4 is a diagram illustrating how an event is viewed from a user's perspective. The avatar 4 of the audience user, which is audience character 5, is assigned a virtual camera called User View Camera Cu, which is the viewpoint of that avatar. In the example in Figure 4, only one User View Camera Cu is drawn, but in reality, one is assigned to each of the 5 audience characters.

[0073] On the user terminal 1500 of each spectator user, a main image specific to each spectator user, which is essentially a view of the virtual space 3 captured by the user-viewpoint camera Cu, is displayed as a user-viewpoint viewing video 20. In other words, spectator users can control their avatar 4 to point the user-viewpoint camera Cu in the direction they want to see the event, allowing them to view what is happening in their preferred direction. For example, in the user-viewpoint viewing video 20 example in Figure 4, the characters 7 (7') and a portion of the large screen 12 (12') are visible.

[0074] Furthermore, two display modes are available for the user-viewpoint video 20: a single display mode that displays the user-viewpoint video 20 on its own, and a multi-display mode that displays the multi-image display unit 22 overlaid (or side-by-side) with the user-viewpoint video 20. Viewers can switch between the single display mode and the multi-display mode at will by performing a predetermined display mode switching operation on their user terminal 1500.

[0075] The multi-image display unit 22 is an area that displays sub-images separate from the main image for each audience user. For example, the sub-images can be images displayed on the large screen 12. The number, size, and on-screen layout of the multi-image display units 22 can be configured as appropriate.

[0076] Figure 5 is a diagram illustrating how an event is viewed from a common-view camera perspective. The virtual event venue, spot 6d, has multiple common viewpoint cameras Cp (Cp1, Cp2, ...) set up. These common viewpoint cameras Cp are virtual cameras, and the number of Cp and the type of camera work they are set up can be appropriately configured according to the content and scale of the event. Each common viewpoint camera Cp has its own unique camera work (from where and how to film the subject) pre-configured.

[0077] For example, the common viewpoint cameras Cp1, Cp2, Cp3, Cp4, and Cp5 are cameras that primarily focus on character 7. Specifically, the camera work of common viewpoint camera Cp1 is set to take a bust shot of character 7 from a predetermined relative position (which may include positional fluctuations) diagonally upwards, from the lower left in front of character 7. The camera work of common viewpoint camera Cp2 is set to take a waist shot of character 7 from the right side, capturing character 7 slightly to the left of the center of the screen, with character 7 in the foreground in the depth direction of the screen and audience character 5 in the background. Common viewpoint cameras Cp2 and Cp5 are set so that their shooting positions move along predetermined movement paths.

[0078] Furthermore, the common viewpoint cameras Cp6, Cp7, Cp8, and Cp9 are cameras that primarily focus on audience character 5. Specifically, the camera work of common viewpoint camera Cp9 is set to point the viewpoint towards audience character 5 and to shoot in a long shot from a fixed position in front of the stage, including multiple audience character 5 within the frame.

[0079] Camera work is defined by shooting parameters such as shooting position, shooting orientation (which also determines the shooting direction), and shooting angle of view. The shooting position is defined as the position of virtual spot 6d relative to the local coordinate origin, or the position of the main subject relative to the local coordinate origin. The shooting orientation is defined as the orientation of virtual spot 6d relative to the local coordinate system, or the orientation of the main subject relative to the local coordinate system. In camera work where the shooting position changes, the shooting parameters may also include the shooting start position, shooting end position, and movement path. Furthermore, in camera work where the shooting orientation or shooting angle of view changes during the process of moving the shooting position, the shooting parameters may also include orientation change patterns and angle of view change patterns.

[0080] Images of the event venue captured by the common viewpoint camera Cp are captured using a camera pattern created by switching and combining the common viewpoint camera Cp in a time-division manner, and are provided to the user terminals 1500 of the audience users as a common viewing video 34. In this embodiment, the camera pattern that determines which common viewpoint camera Cp's captured images are used in the common viewing video 34 at which timing is determined by a "cut division" created in advance according to the content of the performance.

[0081] Furthermore, audience users can arbitrarily switch between displaying the common viewing video 34 or the user-perspective viewing video 20 (see Figure 3) on their user terminal 1500 by performing a predetermined viewpoint switching operation.

[0082] Figure 6 is a diagram illustrating the display on the large screen 12. The large-screen viewing video 36 is a different camera pattern from the common viewing video 34, created by combining common viewpoint cameras Cp(Cp1, Cp2,…) in a time-division manner, and is displayed on the large screen 12. The camera pattern for the large-screen viewing video 36 is also determined by a pre-created "cut division" according to the content of the performance.

[0083] Depending on the camera work settings of the common viewpoint camera Cp, the shooting position, shooting posture, and shooting angle of the user viewpoint camera Cu, and the status of the characters 7, the large screen 12 may be within the shooting angle of those virtual cameras. In that case, the event captured by the user viewpoint camera Cu or the common viewpoint camera Cp (the captured image) may include the large screen viewing video 36 displayed on the large screen 12 (see Figure 4).

[0084] Furthermore, the audience character 5 may appear in the event images (common viewing video 34, large screen viewing video 36) captured by the common viewpoint camera Cp. In Figures 4 to 6, the audience character 5, the appearing character 7, and the large screen 12 that appear in the common viewing video 34 are enclosed in single quotes ('). In addition, the audience character 5, the appearing character 7, and the large screen 12 that appear in the large screen viewing video 36 that appear in the common viewing video 34 are enclosed in double quotes (”). The same method will be used in subsequent figures. The service provision system 1000 of this embodiment has a function to automatically control this reflection.

[0085] I will explain in detail how to control reflections. As shown in Figure 7, the service provision system 1000 sets an index value for each of the spectator characters 5 in order to control reflections.

[0086] The "indicator value" is a value that indicates how easily audience characters appear in the common viewing video 34 and the large screen viewing video 36. The indicator value is calculated using variables that include information about the control content of at least the audience character 5 involved in the event. For example, appearance points Pc, action points Pa, and gameplay result points Pg can be used as variables.

[0087] Appearance points Pc are determined based on the appearance of Avatar 4, which is the audience character 5. For example, appearance points Pc are set to increase as the variety and number of items equipped by Avatar 4 increase, and as the rarity of the equipped items increases. It may also be possible to increase the points if items that are support goods or sponsored goods for the character 7 are used. Whether or not an item is support goods or sponsored goods for character 7 is determined by the identification information that is pre-associated with the item indicating that it is support goods or sponsored goods for character 7.

[0088] Action points (Pa) are determined based on the history of actions taken by audience character 5 from the start of the event. For example, actions that can be recorded in the history include cheering for character 7, waving both hands, applauding, giving gifts to character 7 or the event organizer (for example, providing virtual currency or items equivalent to virtual currency that are usable in virtual space 3, so-called tipping), and actions that liven up the event using items (for example, waving a cheering fan, waving a penlight in time with the singing, etc.).

[0089] Action points (Pa) are set to increase as the number of actions, the amount of tips, and the number of items used increases, for example. They may also increase as the rarity of the items used increases. Action points (Pa) may be deducted for certain prohibited actions or actions that violate manners.

[0090] Gameplay result points Pg are determined based on the gameplay results of the games played so far by the user of spectator character 5. The games referenced are mini-games played during the event with spectator character 5 as a participant, and games outside the event that are offered as one of the services in virtual space 3.

[0091] Furthermore, the content of the variables used to calculate the indicator values ​​can be set as appropriate. For example, the cumulative number of times the same character 7 appeared in past events could be used.

[0092] As shown in Figure 8, the service provision system 1000 calculates a reflection evaluation value for each of the 5 audience characters as one of the evaluation values ​​related to reflection control.

[0093] The "reflection evaluation value" is a value that represents the result of reflections in the common viewing video 34. The reflection evaluation value is determined using the following variables: reflection count point Pn, size point Ps, time point Pt, and in-screen position point Pp.

[0094] The reflection count points Pn are calculated so that the more times an object is reflected since the start of the event, the higher the points. The reflection count is increased each time an object is reflected in the common viewing video 34.

[0095] The size points Ps are calculated to be higher the larger the area occupied by the reflected audience character 5 on the screen of the common viewing video 34. The area may be calculated and accumulated for each frame of the common viewing video 34, or if the reflection occurs in consecutive frames, it may be calculated and accumulated in the first frame.

[0096] Time points (Pt) are calculated such that the longer the cumulative time the image is captured, the higher the value.

[0097] The in-screen position point Pp is calculated such that the closer the in-screen position (for example, the position of the face) of the captured audience character 5 is to the center of the common viewing video 34, the higher the value. Furthermore, if character 7 is visible on the screen of the common viewing video 34, the in-screen position of the captured audience character 5 is closer to character 7, the higher the value is calculated.

[0098] Furthermore, the contents of the variables used to calculate the reflection evaluation value can be set as appropriate.

[0099] Figure 9 is a diagram illustrating the selection of "preferred characters". Priority Character 5Y is a character that is given priority in reflection control and is selected from among the 5 audience characters that fall within a predetermined priority application order, starting with the one with the highest index value. Specifically, for each of the 5 audience characters in the priority application order, the rank of the index value and the rank of the reflection evaluation value are compared, and a character whose evaluation value rank is lower than a predetermined allowable rank difference from the rank of the index value is searched for and designated as Priority Character 5Y. In other words, Priority Character 5Y is a character that "should be reflected more often based on its rank of index value, but is not actually reflected."

[0100] Figure 10 is a diagram illustrating the content of reflection control. To increase the likelihood of the priority character 5Y appearing in the image, reflection control is achieved by changing the camera work of the common viewpoint camera Cp. Specifically, this is achieved by changing shooting parameters such as the shooting position, shooting direction, and shooting angle of view.

[0101] In the example shown in Figure 10, with the common viewpoint camera Cp2, in image 40 before the camera work was changed, the priority character 5Y is positioned behind the appearing character 7 and is not visible in the image. However, by changing the shooting position, the character is visible in the modified image 42.

[0102] Figure 11 is a conceptual diagram illustrating the changes in camera work. The initial camera work of the common viewpoint camera Cp is predetermined for each virtual camera to achieve the performance objectives according to the content of the performance. For example, if the common viewpoint camera Cp is set to shoot in a long shot so that as many audience members as possible are included in the image, changing the shooting angle to the equivalent of a medium shot would change the impression given to the audience when the image captured by that virtual camera is used as a common viewing video 34 or a viewing video for a large screen 36, potentially undermining the performance objectives. In other words, depending on the content of the initial camera work, there are shooting parameters that should not be changed, and there are shooting parameters that can only be changed to a limited extent.

[0103] Therefore, each common viewpoint camera Cp is assigned several attributes based on the initial camera work settings. Then, camera work change candidate data 560 is prepared in advance, indicating which shooting parameters should be changed first for each attribute, and reflection control is performed based on this.

[0104] The types of attributes are: The first attribute is "Foreground = Characters appearing in the scene, Background = Audience characters". Second attribute: "Foreground = Audience characters, Background = Characters appearing in the scene," Third attribute: "Close-up of the audience (photographing 1 to 2 audience characters)," Attribute 4: "Audience Middle Shot (Photographing 4 to 5 audience characters)" Attribute 5: "Long shot of the audience (a shot of the entire audience, from half of the 14 seats)." Attribute 6: "Audience Pan (A middle shot of the audience seats, filmed by moving around within seat 14)" Six types will be available.

[0105] The first and second attributes indicate that character 7 and audience character 5 will be shown simultaneously in the same image. Attributes 3 through 6 indicate that audience character 5 will be the main subject. One of these attributes will be set for the common viewpoint camera Cp. The type and content of the attributes can be set as appropriate.

[0106] Reflection control is performed with the aim of increasing the reflection evaluation value of the priority character 5Y. Therefore, the camera to be modified by reflection control is selected as follows: Among the six attributes, the attributes are selected in order from the highest rank (lowest attribute value), and from the common viewpoint camera Cp that includes the camera work of the selected attribute, the camera whose shooting direction is facing the priority character 5Y is selected as the camera to be modified. Then, the camera work modification candidate data 560 that will be applied to the selected attribute is referenced.

[0107] The camera work change candidate data 560 includes an applicable attribute 561 that indicates which attribute the data applies to, and a priority change candidate data 562 that stores candidate changes in order of priority. Priority change candidate data 562 contains one or more candidate changes. Each candidate change indicates which of the initial camera work shooting parameters should be changed and how, in accordance with the staging objectives required for the common viewpoint camera Cp of the attribute indicated by the applicable attribute 561.

[0108] For example, Figure 12 shows an example of camera work change candidate data 560 applied to the attribute "Audience Middle Shot," and Figure 13 shows an example of camera work change candidate data 560 applied to the attribute "Audience Long Shot."

[0109] For the attribute "Audience Middle Shot," "Zoom Up" is specified as the first choice for the change. The first choice allows the priority character 5Y to be shown larger than before the change in the shooting angle, thus efficiently increasing the reflection evaluation value. Since the purpose of the middle shot is to show the audience seats 14 in a larger view than the whole scene, it is unlikely that changing the screen composition by zooming up will detract from the intended effect.

[0110] However, with the attribute "Long Shot of Audience," the objective is to photograph the audience seats 14 as widely as possible, and changing the camera work to zoom in is undesirable. Therefore, in the example in Figure 13, changing the shooting direction, which has little impact on the screen composition, is the first choice, and changing the shooting position is the second choice, and "Zoom In" is not included as a choice or is a lower priority choice.

[0111] The attribute "Audience Bread," which is not illustrated, may include options for changing movement speed or changing movement path.

[0112] In reflection control, it is assumed that the shooting parameters of the camera to be changed have been changed in the order of the candidate data 562 for priority change, and a pre-check is performed to confirm whether the priority character 5Y is indeed captured in the image. For example, if the vector from the common viewpoint camera Cp to the priority character 5Y is within the shooting range, it is considered that the priority character 5Y will indeed be captured in the image after the camera work change. In other words, the pre-check is deemed to have passed.

[0113] If the pre-check passes, the target camera and the proposed changes are adopted as the reflection control. If the pre-check fails, the next candidate change is tried, and if the pre-check fails after trying all candidates, another camera with the selected attribute and shooting direction facing the priority character 5Y is considered the target camera, and the pre-check is performed in the same way. If the pre-check fails for all common viewpoint cameras Cp with the selected attribute, the attribute with the next priority is selected, and the same process is repeated.

[0114] Reflection control is performed for the next performance to be executed (for example, in the case of an idol's live performance, one song in which the idol sings and dances). When the next performance is executed, the modified camera work that has passed the pre-check in the common viewing video 34 of the next performance is applied in place of the default camera work. As a result, compared to shooting with the default camera work, at least one of the following changes occurs: the time the priority character 5Y is reflected, the position in which it is reflected, and the size in which it is reflected (the ratio of the image portion of the priority character 5Y to the image area; the reflection image ratio), making the priority character 5Y more likely to be reflected than before the change.

[0115] Furthermore, the contents of the priority change candidate data 562 are not limited to the examples in Figures 11 to 13, but can be set as appropriate according to the design of the virtual spot 6 (specifically, the positional relationship between the stage 10 and the audience seats 14). For example, in designs where the audience seats 14 are arranged on a flat surface, or in designs like small live music venues where there are no seats and the audience stands on the floor to watch, audience characters 5 located far from the stage 10 will be less likely to be photographed by the common viewpoint camera Cp placed on the stage 10. In such cases, it is preferable to include a change in the vertical position (height) of the common viewpoint camera Cp in the changes to the shooting specifications.

[0116] Next, I will explain the functional configuration. Figure 14 is a block diagram showing an example of the functional configuration of the server-side system 1010, and it is a diagram that shows the functions performed by the portal server system 1100P and the service server system 1100G together.

[0117] The server-side system 1010 can be functionally divided into the portal server system 1100P and the service server system 1100G, but when considered as a single server-side system 1010, it comprises an operation input unit 100s, a processing unit 200s, an audio output unit 390s, an image display unit 392s, a communication unit 394s, and a storage unit 500s.

[0118] The operation input unit 100s is a means for inputting various operations for managing the server-side system 1010 (such as the portal server system 1100P or the service server system 1100G). Examples of this include a keyboard, touch panel, and mouse.

[0119] The processing unit 200s is implemented by electronic components such as IC memory, as well as a processor that serves as an arithmetic circuit, such as a CPU, GPU, ASIC, or FPGA, and controls data input and output with each functional unit, including the operation input unit 100s and the storage unit 500s. It then executes various arithmetic processes based on predetermined programs and data, data received from operation input signals from the operation input unit 100s, etc., and comprehensively controls the operation of the server-side system 1010.

[0120] The processing unit 200s of the server-side system 1010 includes a user management unit 202, a spectator character control unit 204, an index value setting unit 206, a reflection control unit 208, a spectator user-specific main image generation unit 210, a reflection result evaluation unit 212, a ranking generation unit 214, a timing unit 280s, a sound generation unit 290s, an image generation unit 292s, and a communication control unit 294s. Of course, other functional units can also be included as appropriate.

[0121] User Management Unit 202 handles processing related to user registration procedures and stores and manages various information associated with user accounts.

[0122] The spectator character control unit 204 controls the spectator character 5 based on the actions of the spectator user who owns the spectator character 5. In other words, the user can freely control their own avatar 4 and have a virtual experience in the virtual space 3.

[0123] The index value setting unit 206 variably sets an index value indicating how easily the audience character 5 is visible in the viewing video (an image in which the audience user will appear to be watching the event through the audience character 5) based on the control content of the audience character 5 by the audience character control unit 204.

[0124] The reflection control unit 208 controls the reflection of the audience character 5 in the viewing video based on the index value of the audience character 5.

[0125] For example, the reflection control unit 208 controls a virtual camera (common viewpoint camera Cp) that captures the virtual space 3, so that the audience character 5 and the featured character 7, whose index values ​​meet predetermined conditions, appear in the viewing video at the same time. When changing the initial camera work of the common viewpoint camera Cp, this involves prioritizing the testing of candidate changes that result in the priority character 5Y and the featured character 7 appearing in the same image (see Figure 11).

[0126] The audience user-specific main image generation unit 210 generates images of the characters 7 to be included in the viewing video for each audience user, based on the position of that audience user's audience character 5 in the virtual space 3. This includes the generation of the user-viewpoint viewing video 20 (see Figure 4) based on the user-viewpoint camera Cu.

[0127] The reflection result evaluation unit 212 evaluates the results of reflections in the viewed video for each viewer user (each of the 5 viewer characters). This includes calculating and ranking reflection evaluation values ​​(see Figure 8).

[0128] The ranking generation unit 214 generates a ranking based on the evaluation results from the reflection result evaluation unit 212.

[0129] The user management unit 202 and the audience character control unit 204 may be implemented in the portal server system 1100P of the server-side system 1010. The index value setting unit 206, the reflection control unit 208, the audience user-specific main image generation unit 210, the reflection result evaluation unit 212, and the ranking generation unit 214 may be implemented in the service server system 1100G of the server-side system 1010.

[0130] Of course, the distribution of functions is not limited to this example, and functions may be distributed between the portal server system 1100P and the service server system 1100G as appropriate depending on the hardware configuration, etc. Even for a single function, a portion may be handled by one of the portal server system 1100P and the service server system 1100G, and the remainder by the other.

[0131] The timing unit 280s uses the system clock to perform various timing operations, such as determining the current date and time and the time limit.

[0132] The sound generation unit 290s is implemented by executing ICs and software that generate and decode audio data. The sound generation unit 290s outputs the generated audio signal to the sound output unit 390s. The sound output unit 390s is implemented by a speaker or the like and emits sound based on the audio signal.

[0133] The image generation unit 292s generates images for various management screens for system management of the server-side system 1010 and outputs the image data to the image display unit 392s. The image display unit 392s is implemented using a device that displays images, such as a flat panel display, a head-mounted display, or a projector.

[0134] The communication control unit 294s performs data processing related to data communication and enables data exchange with external devices via the communication unit 394s. The communication unit 394s connects to the network 9 to enable communication. This can be achieved by, for example, a wireless communication device, a modem, a TA (terminal adapter), a jack or control circuit for a wired communication cable, etc. In the example in Figure 1, the communication device 1153 corresponds to this.

[0135] Figure 15 shows examples of programs and data stored in the memory unit 500s. The storage unit 500s stores programs and various data for implementing functions that enable the processing unit 200s to comprehensively control the server-side system 1010. The storage unit 500s is also used as a workspace for the processing unit 200s, temporarily storing calculation results and other data executed by the processing unit 200s according to various programs. This function can be implemented using, for example, IC memory such as RAM or ROM, magnetic disks such as hard disks, optical disks such as CD-ROMs or DVDs, or online storage. In the example shown in Figure 1, this corresponds to storage media such as the IC memory 1152 and hard disks installed in the main unit 1101 of the portal server system 1100P and the service server system 1100G. Online storage may also be included in the storage unit 500s.

[0136] The memory unit 500s stores the server program 501, the distribution client program 502, the avatar base model setting data 510, the avatar customization data 512, the virtual space initial setting data 514, the service-specific initial setting data 520, the user management data 600, the service control data 700, the reflected local ranking data 750, the reflected total ranking data 752, and the current date and time 900. The memory unit 500s also stores other programs and data (such as timers, counters, various flags, the current date and time, etc.) as appropriate.

[0137] The server program 501 includes a portal server program 501p for the portal server system 1100P and a service server program 501g for the service server system 1100G.

[0138] The distribution client program 502 is the original client program that is provided to and executed on the user terminal 1500.

[0139] Avatar base setting data 510 is prepared for each type of character that serves as the base for customizing Avatar 4, and stores various initial setting data such as character model data. Avatar customization data 512 is provided for each item used to customize the appearance and abilities of Avatar 4, and stores various initial setting data related to that item. Users can select one of the base avatar models and customize it to give their Avatar 4 various appearance and ability features, thereby giving it individuality.

[0140] The virtual space initial setup data 514 stores various initial setup data for realizing the virtual space 3. This includes data that identifies the reference point or range of the virtual spot 6 in the coordinate system of the virtual space 3.

[0141] The service-specific initial configuration data 520 is prepared for each service provided in virtual space 3 and stores various configuration data necessary to realize the provision of that service. For example, as shown in Figure 16, the service-specific initial setup data 520 includes a unique service ID 521, virtual spot initial setup data 522, character definition data 528, common viewpoint camera definition data 530, performance definition data 540, camera work change candidate data 560, production material data 570, index value calculation criterion data 572, reflection evaluation criterion data 574, and bonus definition data 580. Of course, other data can also be included as appropriate.

[0142] The virtual spot initial setup data 522 stores various initial setup data for the various elements that make up the virtual spot 6 (for example, the stage 10, large screen 12, and seating area 14 in Figure 2). For example, the virtual spot initial setup data 522 stores the seating area initial setup data 523. The seating area initial setup data 523 associates seat numbers with their position coordinates.

[0143] The character definition data 528 is prepared when character 7 (see Figure 2) is designated as an NPC. It stores various setting data for character 7, including the character object model data, texture data, motion data, and voice data for the character.

[0144] The common viewpoint camera definition data 530 is prepared for each common viewpoint camera Cp and stores a unique camera ID 531, a main subject setting 532 indicating the setting of the main subject, camera work attributes 533, and initial camera work 534.

[0145] The performance definition data 540 contains data on the specific content of the service. If the service is a music live performance, the performance definition data 540 will consist of singing, playing instruments, acting, and talk or mini-games that take place in between. One performance definition data 540 includes the performance order 541, song data 542, performance motion data 543 that defines the actions performed by the character 7 in that performance, and cut data 550. Of course, other data can also be included as appropriate.

[0146] The cut division data 550 is data that sets the cut division (pattern of virtual cameras to be used) of the common viewing video 34 and the large screen viewing video 36 for the performance, specifying at what timing images from the common viewpoint camera Cp will be used as images for the common viewing video 34 and the large screen viewing video 36. The cut division data 550 stores, for example, the performance order 552 to which the data applies, the cut specification 555 for the common viewing video and the cut specification 556 for the large screen, which are associated with the performance's elapsed time 554, as shown in Figure 17.

[0147] The cut specification 555 for the shared viewing video specifies which image (e.g., virtual camera) to use as the cut for the shared viewing video 34 that starts at the corresponding performance elapsed time 554. The specification is made by the image file name or the camera ID of the shared viewpoint camera Cp.

[0148] The large screen cut specification 556 specifies which image to use as a cut in the large screen viewing video 36 that starts at the corresponding performance elapsed time 554. The specification is made by the image file name (in the example shown, "Performance Material 01" is one of the performance material data 570) or the camera ID of the common viewpoint camera Cp.

[0149] Returning to Figure 16, the camera work change candidate data 560 defines the candidate ways in which the camera work should be changed as a form of reflection control (see Figures 11 to 13).

[0150] The index value calculation standard data 572 is data referenced for calculating the index value (see Figure 7), and may be described using a predetermined function or table data.

[0151] The reflection evaluation criteria data 574 is data referenced to calculate the reflection evaluation value (see Figure 8), and may be described using a predetermined function or table data.

[0152] A reward definition data 580 is prepared for each type of reward to be granted to the audience character 5. One reward definition data 580 includes, for example, a unique reward ID 581, grant requirements 582 indicating the conditions that must be met for the reward to be granted, and reward content data 583, as shown in Figure 18.

[0153] The granting requirement 582 is described as a combination of AND or OR of one or more subconditions. Subconditions can be set as appropriate, but for example, the performance order condition 582a, index value condition 582b, reflection evaluation value condition 582c, ranking condition 582d, event usage status condition 582e, etc., may be used. Note that any subcondition can be set to "not set" or "unlimited".

[0154] The performance order condition 582a uses performance order 541 (see Figure 16) as the performance ID to indicate which performances should satisfy the granting requirement 582. By setting this condition appropriately, it becomes possible to set it so that the reward is granted to a specific performance during the event if other sub-conditions are met. If performance order condition 582a is set to include all performance orders (or left unset), the reward will be granted if the other sub-conditions are met when all performances of the event have been completed.

[0155] The index value condition 582b indicates the conditions that the index value (see Figure 7) set for the audience character 5 to whom the special benefit is granted must satisfy. For example, it may be described as a threshold or range for the index value.

[0156] The reflection evaluation value condition 582c indicates the conditions that the reflection evaluation value (see Figure 8) set for the audience character 5 to which the reward is granted must satisfy. For example, it is described as a threshold or range for the reflection evaluation value related to the event in question, or the cumulative value of reflection evaluation values ​​previously acquired by the user without distinguishing between events.

[0157] Ranking condition 582d indicates the ranking position based on the reflection evaluation value of the audience character 5 to whom the reward is granted. For example, it describes the ranking position in terms of thresholds or ranges, such as a local reflection ranking based on the reflection evaluation value related to the event, or a total reflection ranking based on the cumulative reflection evaluation value obtained by the user in the past.

[0158] Event usage condition 582e concerns conditions such as the number of times you have participated in the event, the number of times you have used items during the event, and the amount of money you have donated.

[0159] Reward content data 583 is data indicating the content of the rewards to be granted. The reward content can be set as appropriate. For example, it may include item data that can be used in the event, skin data for the avatar 4 to be granted, special custom data for avatar 4, icon data for avatar 4, virtual currency that can be used in virtual space 3, image data to be used for the standby screen of user terminal 1500, etc. Item data may include the right to participate in the event, special bonus points added to the indicator value when calculating the indicator value, etc.

[0160] By appropriately setting the granting requirements 582 and the reward content data 583, it becomes possible to improve the visibility of the audience character 5 after the reward is granted.

[0161] Returning to Figure 15, the user management data 600 is prepared for each registered user of the service provision system 1000 and stores various data related to that user. One user management data 600, for example as shown in Figure 19, includes a user account 601, avatar setting data 602 describing the latest settings of avatar 4, avatar control data 604, game play history data 606, and reflection evaluation value acquisition history 608. Of course, other data can also be included as appropriate.

[0162] The avatar control data 604 corresponds to the character control data of the audience character 5 when the avatar 4 is receiving services at the virtual spot 6. The avatar control data 604 includes, for example, the user account, location information in the virtual space 3, posture information of the avatar 4, user viewpoint control data indicating the position and posture of the user viewpoint camera Cu, motion control data, and the results of playing mini-games conducted during an event in which the audience character 5 is a participant. Of course, other data may also be included as appropriate.

[0163] Game play history data 606 is created each time the user plays a game provided as a service by the service provision system 1000, and stores various data related to that play. One game play history data 606 stores the game name (service ID or game ID may also be used), play date and time, save data, list of owned items, play performance data, etc. Of course, other data may also be stored as appropriate.

[0164] The reflection evaluation value acquisition history 608 stores the date and time, and the value acquired, for each instance when the user's avatar 4 acquired a reflection evaluation value as an audience character 5, in chronological order.

[0165] Returning to Figure 15, the service control data 700 is created for each service provided at the virtual spot 6 and stores various data related to that provision. Figure 20 shows an example of service control data 700 when the service is a live performance by character 7. The service control data 700 includes character control data 701, audience character placement data 703, audience character behavior history data 705, common viewpoint camera control data 706, audience character-specific applicable viewpoint settings 707, audience character-specific display mode settings 708, audience character-specific index values ​​710, audience character-specific reflection result data 712, a priority character user account 714 indicating priority character 5Y, applicable camera work data 720, and viewing video data 730 which are image data such as user viewpoint viewing video 20, common viewing video 34, and viewing video for large screen 36. Of course, other data can also be included as appropriate.

[0166] Audience character placement data 703 is created for each audience character 5, and is stored in association with the user account and the virtual seat number (which may also be the position coordinates in seat 14).

[0167] The audience character behavior history data 705 is created for each audience character 5 and is a history of the actions taken by that audience character 5 from the start to the end of receiving the service. The audience character behavior history data 705 is used to calculate the action points Pa, which are an index value (see Figure 7).

[0168] The common view camera control data 706 is prepared for each common view camera Cp and stores the latest status of the camera and various data used for its control.

[0169] The audience character-specific viewpoint setting 707 is provided for each audience character 5 and indicates whether to display the user-viewpoint viewing video 20 (see Figure 4) or the common viewing video 34 (see Figure 5) on the user terminal 1500 of the user of that character. The setting is switched each time a viewpoint switching operation is performed on the user terminal 1500 related to audience character 5.

[0170] The audience character-specific display mode setting 708 is provided for each of the 5 audience characters and indicates whether to apply single display mode or multi-display mode on the user terminal 1500 of the user of that character. The setting is switched each time a display mode switching operation is performed on the user terminal 1500 related to audience character 5.

[0171] The audience character-specific index value 710 is prepared for each of the 5 audience characters, and stores the latest index value for that character and its ranking among all 5 audience characters.

[0172] The individual audience character reflection result data 712 is prepared for each audience character 5 and stores the latest reflection evaluation value for that character and the reflection evaluation value ranking among all audience characters 5.

[0173] The applied camera work data 720 stores various information about the camera work to be applied in the next performance in order to improve the visibility of the priority character 5Y. The applied camera work data 720 includes the performance to be applied to 721, the elapsed time of the performance to be applied to 722, the camera ID to be applied to 721, and the modified camera work 724. The modified camera work 724 is created by copying the initial camera work 534 (see Figure 16) of the common viewpoint camera Cp to be changed, and modifying a part of the copied shooting parameters with the content of one of the modification candidates in the priority change candidate data 562 (see Figure 11).

[0174] Figure 21 is a functional block diagram showing an example of the functional configuration of the user terminal 1500. The user terminal 1500 comprises an operation input unit 100, a terminal processing unit 200, an audio output unit 390, an image display unit 392, a communication unit 394, and a terminal storage unit 500.

[0175] The operation input unit 100 outputs operation input signals to the terminal processing unit 200 in response to various operation inputs made by the user. This can be achieved, for example, by push switches, joysticks, touchpads, trackballs, accelerometers, gyroscopes, etc.

[0176] The terminal processing unit 200 is implemented by, for example, a microprocessor such as a CPU or GPU, and electronic components such as IC memory, and controls data input and output with each functional unit, including the operation input unit 100 and the terminal storage unit 500. It then controls the operation of the user terminal 1500 by executing various calculation processes based on predetermined programs and data, operation input signals from the operation input unit 100, and various data received from the server-side system 1010.

[0177] The terminal processing unit 200 includes a client control unit 260, a timing unit 280, a sound generation unit 290, an image generation unit 292, and a communication control unit 294.

[0178] The client control unit 260 performs various controls to enable the user terminal 1500 to function as a Man-Machine Interface (MMIF) as a game client in the service provision system 1000. Specifically, the client control unit 260 includes an operation input information provision unit 261 and a display control unit 262.

[0179] The operation input information provision unit 261 controls the transmission of operation input information to the server-side system 1010 in response to input from the operation input unit 100.

[0180] The display control unit 262 performs control to display various images based on data received from the server-side system 1010.

[0181] The timing unit 280 uses the system clock to determine the current date and time, the time limit, and other time information.

[0182] The sound generation unit 290 is implemented by, for example, a digital signal processor (DSP), a processor such as a speech synthesis IC, and an audio codec capable of playing audio files. It generates sound signals for music, sound effects, and various operation sounds and outputs them to the sound output unit 390. The sound output unit 390 is implemented by a device such as a speaker that outputs sound (emits sound) based on the sound signals input from the sound generation unit 290.

[0183] The image generation unit 292 generates and outputs an image signal that displays an image to the image display unit 392 based on the control of the client control unit 260. In the example in Figure 1, this corresponds to a GPU (Graphics Processing Unit), graphics controller, graphics board, etc., mounted on the control board 1550. The image display unit 392 is implemented by a device that displays images, such as a flat panel display, head-mounted display, or projector.

[0184] The communication control unit 294 performs data processing related to data communication and enables data exchange with external devices via the communication unit 394.

[0185] The communication unit 394 connects to the network 9 to enable communication. This is achieved, for example, by a wireless communication device, modem, TA (terminal adapter), jacks and control circuits for wired communication cables, etc. In the example in Figure 1, the communication module 1553 corresponds to this.

[0186] The terminal memory unit 500 stores programs and various data necessary for the terminal processing unit 200 to implement a given function. It is also used as a workspace for the terminal processing unit 200, and temporarily stores calculation results performed by the terminal processing unit 200 according to various programs, as well as input data received from the operation input unit 100. These functions are realized by, for example, IC memory such as RAM or ROM, magnetic disks such as hard disks, and optical disks such as CD-ROMs or DVDs. In the example in Figure 1, the IC memory 1552 mounted on the control board 1550 corresponds to this.

[0187] Specifically, the terminal storage unit 500 stores a client program 800 (application program) for causing the user terminal 1500 to function as a client control unit 260, a user account 802 and login password 804 stored in connection with the user registration procedure for the virtual space 3, and the current date and time 900. Of course, other data can also be stored as appropriate.

[0188] Figures 22 and 23 are flowcharts illustrating the processing flow related to service provision at virtual spot 6, as executed by the server-side system 1010. The event provided is a live event featuring character 7 as the performer.

[0189] Prior to the explanation in the flowchart, each avatar 4 was controlled according to the operations performed on each user's user terminal 1500. Users who wished to participate in the event had already moved their avatar 4 to virtual spot 6d, completed the given event participation procedure, and moved their avatar 4 to the audience seats 14 as they wished. In other words, the avatar 4 of a user who wished to participate in the event was assumed to be an audience character 5. The flowcharts in Figures 22 and 23 show the flow after the event participation procedure has been completed.

[0190] As shown in Figure 22, before starting the event, the server-side system 1010 initializes the audience character-specific viewpoint setting 707 to "user viewpoint" and the audience character-specific display mode setting 708 to "multi-display" (step S10).

[0191] Next, the server-side system 1010 calculates the index value for each audience character 5 (step S12) and initializes the reflection result data 712 for each audience character (step S14). The initial value of the reflection evaluation value is "0".

[0192] Next, the server-side system 1010 initializes the event performance order count to "1" (step S16), and sets the audience character 5 that falls into the top predetermined number of the index value rankings, starting with the one with the highest index value, as priority character 5Y (step S20). Basically, the audience character 5 with the highest index value ranking is set as priority character 5Y. Then, the server-side system 1010 starts the event (step S22).

[0193] When the event starts, the server-side system 1010 starts recording the actions of each audience character 5, which is controlled based on user operations, within their respective events (step S24). Then, if a preferred character 5Y is set (YES in step S26), the system executes a camera work change process related to the next performance to be performed (step S28).

[0194] Figure 24 is a flowchart illustrating the process of changing camera work. The server-side system 1010 first selects an attribute of the common viewpoint camera Cp (step S30). The attribute has a priority order set for that step. When this step is executed for the first time, the attribute with the first priority is selected, and thereafter, each time this step is repeated within the camera work change process, the priority of the selected attribute is lowered by one level.

[0195] Next, the server-side system 1010 refers to the cut data 550 for the next performance to be executed and performs a primary search for a common viewpoint camera with the attributes selected in step S30 (step S32). From the results of the primary search, it performs a secondary search for a virtual camera in which the priority character 5Y exists in the shooting direction (step S34). Then, from the results of the secondary search, it determines the virtual camera (camera to be changed) that will be the target of the camera work change (step S36).

[0196] Next, the server-side system 1010 refers to the camera work change candidate data 560 (see Figure 11) that matches the attribute selected in step S32 and selects a change candidate (step S38). If this step is the first to be executed in this camera work change process, the first candidate is selected. Thereafter, each time this step is executed in this camera work change process, the candidates of the next rank are selected.

[0197] Next, the server-side system 1010 performs a pre-check for reflections (step S40). Specifically, it copies the shooting specifications of the initial camera work 534 (see Figure 16) of the camera to be changed and creates applied camera work data 720 having a modified camera work 724, which is a part of the initial camera work 534 modified with the content of the change candidate selected in step S38 (see Figure 20). Then, it performs a pre-check to see if the virtual camera can capture the priority character 5Y when the applied camera work data 720 is applied to the camera to be changed.

[0198] If the pre-check results show that the priority character 5Y can be photographed, the check is passed (YES in step S40), and the created applied camera work data 720 is left as is, and the camera work change process is terminated.

[0199] If the pre-check results in the inability to photograph the priority character 5Y (NO in step S40), the server-side system 1010 returns to step S30 if there are any camera work change candidate data 560 referenced in step S38 that have not yet been pre-checked (YES in step S42), selects the next highest priority change candidate after the currently selected one, and then executes steps S38 and S40.

[0200] If there are no remaining candidate camera work change data 560 referenced in step S38 that have not been pre-checked (NO in step S42), the server-side system 1010 searches the secondary searched common viewpoint camera Cp for any cameras that have not been pre-checked.

[0201] Then, if there are still cameras that have not been pre-checked (YES in step S44), the server-side system 1010 returns to step S36 and selects a different virtual camera from the secondary searched common view cameras Cp that has not been pre-checked as the camera to be changed, and then executes steps S36 through S42.

[0202] If the pre-check fails even after attempting the pre-check with all the secondary selected common viewpoint cameras Cp (NO in step S44), then search for any remaining attributes that have not yet been pre-checked.

[0203] Then, if any attributes remain (YES in step S46), the server-side system 1010 returns to step S30, selects a different attribute that has not yet undergone a pre-check, and executes steps S32 through S44.

[0204] If the pre-check fails even after trying pre-checks for all attributes (NO in step S46), the server-side system 1010 deletes the applied camera work data 720 created for the pre-check (step S48) and terminates the camera work change process. In this case, it means that a suitable camera work that improves the visibility of priority character 5Y could not be achieved, and the improvement of priority character 5Y's visibility in the next performance will be abandoned.

[0205] Returning to Figure 22, the server-side system 1010 starts executing the next performance (step S50) and starts generating the common viewing video 34 and the large screen viewing video 36 according to the cut data 550 of that performance (step S52).

[0206] During generation, the modified camera work 724 is applied to the common viewpoint camera Cp indicated by the applied camera ID 723 at the applied performance elapsed time 722 specified in the applied camera work data 720, instead of the initial camera work 534. In other words, a common viewing video 34 and a large screen viewing video 36 are generated in which the reflection of the priority character 5Y is improved compared to the initial camera work. In cases where the applied camera work data 720 is not available (for example, if it is deleted in step S48), the common viewing video 34 and the large screen viewing video 36 will be generated according to the original cut division data 550.

[0207] Then, the generation of the common viewing video 34 and the large screen viewing video 36 is started, and the evaluation of the appearance of the audience characters 5 in the generated videos is started (step S54). The appearance evaluation is performed for both the common viewing video 34 and the large screen viewing video 36. In order to calculate the appearance evaluation value, the number of times the characters appear in the image is counted, the size of the appearance is calculated, the duration of appearance is counted, and the position within the image is determined (see Figure 8). Then, the appearance evaluation value and ranking of each audience character 5 are updated.

[0208] Furthermore, the server-side system 1010 starts displaying the common viewing video 34 to the user terminal 1500 of users who have selected the common viewpoint as the viewpoint to be applied to the display on the user terminal 1500 (step S56).

[0209] Furthermore, the generation and display of the user-viewpoint viewing video 20 to the user terminal 1500 of users who have selected the user-viewpoint as the viewpoint to be applied to the display on the user terminal 1500 is started (step S56). That is, an image of the event taken from the user-viewpoint camera of the audience character 5 of the user in question is generated and displayed on the user terminal 1500. At that time, depending on the display mode setting, the large screen viewing video 36 is composited to be displayed as a sub-display.

[0210] As shown in Figure 23, once the currently running performance is finished (step S68), the server-side system 1010 increments the performance order count by "1" (step S70) and deletes the applied camera work data 720 (step S72).

[0211] Next, the server-side system 1010 updates the index value of audience character 5 (step S74) and selects the preferred character 5Y for the next performance (step S76). Specifically, based on the evaluation order of the reflection evaluation value, a predetermined number of top audience characters 5 that have a predetermined priority application order are extracted. For each of the extracted audience characters 5, one audience character 5 whose reflection evaluation value rank is lower than a predetermined allowable rank difference relative to the index value rank, i.e., a character with insufficient reflection, is selected and set as the next priority character 5Y.

[0212] If all performances in the event have not been completed (NO in step S80), the server-side system 1010 returns to step S26.

[0213] Once all performances in the event have finished (YES in step S80), the server-side system 1010 takes a commemorative photo of the event, ensuring that the priority character 5Y is included in the shot (step S92). Specifically, it sets up a virtual camera for the commemorative photo, capturing the performer character 7 in the foreground and the audience seats 14 in the background. The virtual camera for the commemorative photo has a dedicated initial camera work set up. For example, it is set to take a medium shot of the audience character 5 in the audience seats 14 directly in front of the large screen 12, from the side of the screen.

[0214] The server-side system 1010 changes the camera work of the virtual camera for commemorative photos so that the priority character 5Y is visible near the character 7. As a result of this change, the audience seats 14 that appear in the background of the commemorative photo may not be the same seats 14 that are directly in front of the screen as they were initially set to be.

[0215] Next, the server-side system 1010 performs a commemorative photo shoot (step S94). The commemorative photo shoot is one of the photo shoot parts in which the character 7 appears together with the audience character 5. The commemorative photo is saved and may be uploaded separately to a blog or image distribution site, or made public on media in the virtual space 3. Furthermore, commemorative photos may be taken multiple times during the event. Also, the common viewing video 34 is the same in that the commemorative photo shows the featured character 7 together with the audience character 5. Therefore, it is also possible to omit the commemorative photo and replace it with a still image captured from the common viewing video 34. In that case, the generation and display of the common viewing video 34 can be called the "photography part."

[0216] Next, the server-side system 1010 updates the reflection evaluation value acquisition history 608 for each of the 5 audience characters to include the values ​​acquired in this event (step S96). Then, it creates a reflection evaluation ranking and displays it on the user terminal 1500 of the user of each of the 5 audience characters (step S98).

[0217] The reflection evaluation ranking includes a local ranking (reflection local ranking data 750 in Figure 15) that ranks participants based on their reflection evaluation values ​​in the event held, limited to audience character 5, and a total ranking (reflection total ranking data 752 in Figure 15) that ranks all users who have a reflection evaluation value acquisition history 608 and have previously been audience character 5, based on the cumulative reflection evaluation values ​​they have acquired up to that point. The ranking display may include at least the top-ranked participants and a predetermined number of participants from the bottom of the ranking.

[0218] Next, the server-side system 1010 grants a reward to the audience character 5 and notifies them of the grant (step S100). Specifically, for each audience character 5, the server-side system 1010 searches the reward definition data 580 (see Figure 18) for data that satisfies the grant requirement 582, and grants the reward indicated by the reward content data 583 of the corresponding data to the audience character 5. Then, a notification that the reward has been granted is displayed on the user terminal 1500 of the audience character 5.

[0219] Next, the server-side system 1010 notifies all user terminals 1500 of the end of the event (step S102) and terminates the series of processes.

[0220] As described above, according to this embodiment, it is possible to control the appearance of the avatar 4 (audience character 5 in the event) operated by the user in the image viewed by the user who is watching a given event taking place in the virtual space 3.

[0221] The priority character 5Y targeted for reflection control is selected based on the index value set for each of the audience character 5. By appropriately setting the method for calculating the index value, it becomes possible to increase the likelihood that audience character 5 that meet the conditions (for example, audience character 5 with a special appearance, audience character 5 that performs more event-enhancing actions, etc.) will appear in the viewing video of the event.

[0222] According to this embodiment, it becomes possible to enhance the mood of users who look forward to reflections and to improve the user experience related to reflections.

[0223] [Second Embodiment] Next, a second embodiment to which the present invention is applied will be described. This embodiment is basically implemented in the same way as the first embodiment, but differs in that (1) the seat of the spectator character 5 is determined automatically, and (2) the position of the spectator character 5 in the virtual space 3 (specifically, the virtual spot 6 and the seats 14) is changed in relation to reflection control. For components that are the same as in the first embodiment, the same reference numerals as in the first embodiment will be used, and redundant explanations will be omitted, with the focus being on the differences from the first embodiment.

[0224] As shown in Figure 25, the server-side system 1010 of this embodiment stores one or more seating area characteristic data 590. The seating area characteristic data 590 includes an applicable performance order 591 indicating the performance to which the data applies, and area definition data 593.

[0225] The area definition data 593 is the result of a pre-simulation of how much of the audience character 5 in each seat of the audience seats 14 will be visible in the footage, when filming is done according to the initial settings of the cut-out data 550 applied to the performance.

[0226] Figure 25 illustrates the degree of reflection using three rectangular areas (high-probability reflection area 51, medium-probability reflection area 52, and low-probability reflection area 53), but it is not limited to these areas. In reality, the degree of reflection is multi-layered, like contour lines on a map, and the boundaries between areas are also curved, like contour lines.

[0227] The reason for this is that, in the case where character 7 is the user's avatar 4, the position of character 7 on stage 10 during the performance is not necessarily fixed. Also, even when character 7 is an NPC, appropriate "fluctuations" are given to its actions depending on the performance in order to avoid giving the audience a mechanical impression, and even if the same camera work and cuts are used, the distribution of the probability of it appearing in the shot will not be uniform.

[0228] Figures 26 and 27 are flowcharts illustrating the processing flow related to service provision at virtual spot 6, as executed by the server-side system 1010 of this embodiment. In comparison with the flowcharts in Figures 22 and 23 in the first embodiment, the server-side system 1010 assigns seats in areas with a higher probability of being captured in the image to audience characters 5 with higher index rankings (step S21). The audience character placement data 703 (see Figure 20) is then initialized.

[0229] Then, after selecting the priority character 5Y for the next performance, the server-side system 1010 changes the seat assignment for the priority character 5Y to a seat in an area with a higher probability of being captured on camera than the current seat (step S78). This changes the audience character placement data 703 (see Figure 20). Furthermore, when changing seat assignments, it may be possible to automatically control the system so that the priority character 5Y and the audience character 5 who was in the new seat move to swap seats during the performance.

[0230] According to this embodiment, the same effects as in the first embodiment can be obtained.

[0231] [Variation] Although examples of embodiments to which the present invention is applied have been described above, the forms to which the present invention can be applied are not limited to the above forms, and components can be added, omitted, or modified as appropriate.

[0232] (Variation 1) For example, in the above embodiment, only one priority character 5Y is set in steps S20 and S76, but multiple characters can be set and the camera work change process in step S28 can be executed for each priority character 5Y.

[0233] However, depending on the number of priority character 5Y settings, it may be difficult to achieve a change that improves the visibility of multiple priority character 5Y with a single camera work change. Therefore, for a single camera work change, it may be created to improve the visibility of one of the multiple priority character 5Y. In other words, if multiple priority character 5Y are set, the same number of applicable camera work data 720 may be created in step S28. Specifically, for example, in step S36, the common viewpoint camera Cp indicated by the applicable camera ID 723 (see Figure 20) of the applicable camera work data 720 that has already been created at that point may not be selected as the camera to be changed.

[0234] (Variation 2) In the above embodiment, the priority character 5Y was determined based on a comparison of the index value rank and the reflection evaluation value rank, but it may also be determined solely by the index value rank. For example, the top predetermined number of index value ranks (e.g., only the 1st place, or from the 1st to the 3rd place) may be set as the priority character 5Y regardless of the reflection control result.

[0235] (Variation 3) In the above embodiment, the index value is used only for selecting the preferred character 5Y, but it may be used for other purposes as well. For example, in the common viewing video 34 and the viewing video 36 for large screens, the user account of the audience character 5 and its index value may be displayed as text. Specifically, the text may be displayed at the top or bottom edge of the screen, flowing from right to left. At the same time, the audience character 5 designated as preferred character 5Y may be displayed in an identifiable manner.

[0236] This display allows the user of spectator character 5 to understand the situation regarding their own appearance in the event. If the display order is descending order of the metric value ranking, the user can see how much higher their ranking needs to be to become priority character 5Y. Based on this, the user can improve the metric value by manipulating and controlling the actions of spectator character 5, such as using items, to make their spectator character 5 appear more often. For this user, in addition to the enjoyment of watching the event, they can also enjoy the secondary pleasure of how to make their own spectator character 5 appear in the event. In other words, it broadens the range of ways to enjoy the event and improves the quality of the service.

[0237] Furthermore, the display of the metric values ​​for audience character 5 is not limited to the common viewing video 34 or the viewing video 36 for large screens. After the event starts, the metric values ​​may also be displayed on the user terminal 1500 by performing a predetermined metric value display operation on the terminal.

[0238] (Modification #4) In the above embodiment, the priority character 5Y was a control entity, and the user, as a viewer, could not determine which viewer character 5 was the priority character 5Y. However, in the common viewing video 34 and the viewing video 36 for large screens, the priority character 5Y may be displayed in a way that makes it more clearly identifiable. For example, it may be made identifiable by adding an object of a predetermined identification display marker to the priority character 5Y, or by highlighting the outline of the priority character 5Y.

[0239] The identification display may be displayed at all times, or it may be displayed only on the user terminal 1500 after the start of the event by performing a predetermined identification display operation on that terminal.

[0240] (Variation 5) In the above embodiment, reflection control was described as an example in which the reflection of priority character 5Y is improved, but it may also be configured to reduce reflection. For example, the appearance points Pc of the indicator value are set to be deducted if a predetermined dress code is violated, and the behavior points Pa are set to be deducted if a predetermined disruptive behavior is performed. Then, the priority character 5Y is selected from a predetermined number of lower-ranking indicator value characters. In addition, in the priority order change candidate data 562 of the camera work change candidate data 560 (see Figure 11) in this modified example, the changes are set to reduce the visibility of the priority character 5Y, and the applied camera work data 720 is generated.

[0241] When the applicable camera work data 720 is applied during the generation of the common viewing video 34 and the viewing video 36 for large screens, the appearance of priority character 5Y in these viewing videos is reduced, effectively serving as a punishment for disruptive audience character 5. Furthermore, since the likelihood of audience character 5 with potentially offensive appearances appearing in the video is reduced, it is preferable as it helps maintain a consistent level of quality in the event's viewing videos.

[0242] (Variation #6) In the embodiment described above, the process for granting rewards was performed after all performances had finished. However, if the reward definition data 580 (see Figure 18) specifies that rewards should be granted for each performance, step S100 may be executed after each individual performance, for example, between step S76 and step S80. In this case, after each performance, a reward corresponding to the results of the performance in which the audience character 5 appeared is granted. [Explanation of Symbols]

[0243] 3…Virtual space 5…Audience character 5Y…Priority Character 7…Characters 12…Large screen (virtual display unit) 14…Audience seats 20…User-perspective viewing video 22…Multi-image display section 34… Shared viewing videos 36…Video for viewing on a large screen 200s... Processing Unit 204... Audience Character Control Unit 206... Indicator value setting section 208... Reflection Control Unit 210... Main image generation unit for each audience user 212... Reflection Result Evaluation Department 214... Ranking generation unit 500s…Storage part 501…Server program 520... Initial setup data for each service 528... Character definition data 530... Common viewpoint camera definition data 533...Attributes 534...Initial Camera Work 540…Performance definition data 541…Order of performance 550... Cutting data 560...Camera work change candidate data 561…Applicable attribute 562…Priority change candidate data 572…Data for calculating indicator values 574…Reflection Evaluation Criteria Data 600...User management data 602... Avatar settings data 608...History of acquiring reflection evaluation scores 700... Service control data 701...Character control data 706... Common viewpoint camera control data 710…Audience Character-Specific Metrics 712…Reflection result data 714…Priority Character User Account 720...Applicable camera work data 730…Viewing video data 750…Reflection Local Ranking Data 752...Total ranking data for reflections 1000... Service delivery system 1010…Server-side system 1500... User terminal Cp(Cp1,Cp2,…)…Common viewpoint camera cu... User-perspective camera

Claims

1. A computer system that controls the generation of viewing videos for viewing a given event taking place in a virtual space, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means that sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, based on the control content of the audience character by the audience character control means, A reflection control means for controlling the appearance of the audience character in the video being viewed based on the index value of the audience character, the reflection control means for changing the shooting parameters of a virtual camera that photographs the virtual space to shooting parameters that determine that the audience character whose index value satisfies a given priority condition will be captured in the video, A computer system equipped with the following features.

2. The aforementioned event takes place when the characters appear in the aforementioned virtual space. The computer system according to claim 1.

3. As images of the characters to be included in the video to be viewed, a main image generation means for each audience user generates an image of the character based on the position of the audience user's character in the virtual space. The computer system according to claim 2, further comprising:

4. The reflection control means controls the audience character whose index value meets a predetermined condition and the appearing character to appear in the viewing video at the same time. The computer system according to claim 2.

5. A computer system that controls the generation of a video for viewing an event in which characters appear in a virtual space, The aforementioned event includes a photo session where the characters appearing in the event take pictures together with other characters. A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means that sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, based on the control content of the audience character by the audience character control means, A reflection control means for controlling the appearance of the audience character in the video being viewed based on the index value of the audience character, wherein in the shooting part, the reflection control means controls the filming of the audience character whose index value satisfies predetermined conditions to be included with the other characters and filmed together with the appearing characters, A computer system equipped with the following features.

6. The reflection control means controls the reflection of the audience character by controlling the position of the audience character in the virtual space. The computer system according to claim 5.

7. A computer system that controls the generation of a video for viewing a given event taking place in a virtual space where a virtual display unit is installed, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means that sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, based on the control content of the audience character by the audience character control means, A reflection control means for controlling the reflection of the audience character in the viewing video based on the index value of the audience character, wherein the reflection control means controls the reflection of the audience character by performing control to display the audience character on the virtual display unit, A computer system equipped with the following features.

8. A computer system that controls the generation of viewing videos for viewing a given event taking place in a virtual space, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means that sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, based on the control content of the audience character by the audience character control means, A reflection control means for controlling the reflection of the audience character in the viewing video based on the index value of the audience character, wherein the reflection control means controls the viewing video to be a multi-image display including an image of the audience character, and A computer system equipped with the following features.

9. A computer system that controls the generation of viewing videos for viewing a given event taking place in a virtual space, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means for setting an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, using the results of a given game played by the audience user, A reflection control means for controlling the appearance of the audience character in the video being viewed based on the index value of the audience character, the reflection control means for changing the shooting parameters of a virtual camera that photographs the virtual space to shooting parameters that determine that the audience character whose index value satisfies a given priority condition will be captured in the video, A computer system equipped with the following features.

10. The index value setting means sets the index value based on the behavior of the audience character in the virtual space controlled by the audience character control means. The computer system according to any one of claims 1 to 8.

11. The index value setting means sets the index value based on the appearance settings of the audience character controlled by the audience character control means. The computer system according to any one of claims 1 to 8.

12. The indicator value setting means sets the indicator value using the results of a given game played by the spectator user. The computer system according to any one of claims 1 to 8.

13. The reflection control means controls at least one of the reflection time, reflection position, and reflection image ratio of the audience character reflected in the viewing video, based on the index value of the audience character. A computer system according to any one of claims 1 to 9.

14. A computer system that controls the generation of viewing videos for viewing a given event taking place in a virtual space, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means that sets an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, based on the control content of the audience character by the audience character control means, A reflection control means that controls the appearance of the audience character in the viewing video based on the index value of the audience character, A means for evaluating the results of what appears in the video being viewed, for each of the aforementioned viewer users, A computer system equipped with the following features.

15. A computer system that controls the generation of viewing videos for viewing a given event taking place in a virtual space, A spectator character control means controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space. An index value setting means for setting an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video, using the results of a given game played by the audience user, A reflection control means that controls the appearance of the audience character in the viewing video based on the index value of the audience character, A means for evaluating the results of what appears in the video being viewed, for each of the aforementioned viewer users, A computer system equipped with the following features.

16. Ranking generation means for generating a ranking based on the results of the aforementioned evaluation, The computer system according to claim 14 or 15, further comprising:

17. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step, which sets an index value indicating the likelihood of the audience character appearing in the viewing video in a variable manner based on the content of the control of the audience character, A reflection control step for controlling the appearance of the audience character in the viewing video based on the index value of the audience character, the reflection control step including changing the shooting parameters of a virtual camera that photographs the virtual space to shooting parameters that determine that the audience character whose index value satisfies a given priority condition will be captured in the video; A method for controlling the generation of videos for viewing, including the method itself.

18. A method for controlling the generation of a video for viewing an event in which characters appear in a virtual space, The aforementioned event includes a photo session where the characters appearing in the event take pictures together with other characters. A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step, which sets an index value indicating the likelihood of the audience character appearing in the viewing video in a variable manner based on the content of the control of the audience character, A reflection control step for controlling the appearance of the audience character in the viewing video based on the index value of the audience character, the reflection control step including, in the shooting part, controlling the shooting to include the audience character whose index value satisfies predetermined conditions among the other characters and to shoot them together with the appearing characters, A method for controlling the generation of videos for viewing, including the method itself.

19. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step, which sets an index value indicating the likelihood of the audience character appearing in the viewing video in a variable manner based on the content of the control of the audience character, A reflection control step for controlling the reflection of the audience character in the viewing video based on the index value of the audience character, the reflection control step including controlling the reflection of the audience character by performing control to display the audience character on a virtual display unit installed in the virtual space, A method for controlling the generation of videos for viewing, including the method itself.

20. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step, which sets an index value indicating the likelihood of the audience character appearing in the viewing video in a variable manner based on the content of the control of the audience character, A reflection control step for controlling the reflection of the audience character in the viewing video based on the index value of the audience character, the reflection control step including controlling the viewing video to be a multi-image display including an image of the audience character, A method for controlling the generation of videos for viewing, including the method itself.

21. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step in which an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video is set using the results of a given game played by the audience user, A reflection control step for controlling the appearance of the audience character in the viewing video based on the index value of the audience character, the reflection control step including changing the shooting parameters of a virtual camera that photographs the virtual space to shooting parameters that determine that the audience character whose index value satisfies a given priority condition will be captured in the video; A method for controlling the generation of videos for viewing, including the method itself.

22. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step, which sets an index value indicating the likelihood of the audience character appearing in the viewing video in a variable manner based on the content of the control of the audience character, A reflection control step that controls the reflection of the audience character in the viewing video based on the index value of the audience character, A reflection result evaluation step in which the results of appearing in the aforementioned viewing video are evaluated for each of the aforementioned viewer users, A method for controlling the generation of videos for viewing, including the method itself.

23. A method for controlling the generation of a video for viewing a given event taking place in a virtual space, wherein the method controls the generation of a video for viewing a given event taking place in a virtual space, A spectator character control step, which controls a spectator character based on the actions of a spectator user who is the owner of a spectator character that virtually observes the event in the virtual space, An index value setting step in which an index value indicating the likelihood of the audience character appearing in the aforementioned viewing video is set using the results of a given game played by the audience user, A reflection control step that controls the reflection of the audience character in the viewing video based on the index value of the audience character, A reflection result evaluation step in which the results of appearing in the aforementioned viewing video are evaluated for each of the aforementioned viewer users, A method for controlling the generation of videos for viewing, including the method itself.

Images