Content providing system, content providing method, and program product

Abstract

A content providing system according to an embodiment includes at least one processor. The processor generates action log information that records action information indicating an action of a first virtual image operated by a first user during viewing and listening of content, and specific action information indicating a time at which the first virtual image performs a specific action and a content of the specific action. When a second user views and listens to the content after the first user views and listens to the content, the processor reproduces the content and reproduces the action of the first virtual image based on the action information included in the action log information. By referring to the specific action information included in the action log information, the processor configures a display object indicating the content of the specific action in association with the first virtual image at the time at which the first virtual image performs the specific action in the reproduction.

Description

Technical Field

[0001] One aspect of the present invention relates to a content providing system, a content providing method, and a content providing program. Background Technology

[0002] Patent document 1 discloses the following mechanism: viewers who use content in a virtual space (a virtual 3D virtual space) can add their own virtual avatar object in the virtual space, and the added virtual avatar object will be displayed on other viewers' terminals that reproduce the content later.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent No. 6707111 Summary of the Invention

[0006] The technical problem that the invention aims to solve

[0007] According to the above mechanism, when watching or listening to content that has been watched or listened to by other viewers in the past, one can watch or listen to content containing the virtual avatars of those other viewers, thus gaining a sense of connection as if watching or listening to content together with other viewers. However, from the perspective of improving the convenience for users watching or listening to content, there is room for further improvement in the above mechanism.

[0008] One aspect of the present invention is to provide a content delivery system, content delivery method, and content delivery program that can effectively improve the convenience for users who are watching or listening to content.

[0009] Means for solving technical problems

[0010] According to one aspect of the present invention, a content providing system includes at least one processor. The at least one processor performs the following processes: configuring a first virtual avatar corresponding to a first user of prescribed content representing an audiovisual virtual space within the virtual space; generating action log information, the action log information recording action information and specific action information, the action information indicating actions of the first virtual avatar performed by the first user during the first user's audiovisual viewing of the content, the specific action information indicating the time when the first virtual avatar performs a predetermined specific action and the content of that specific action; reproducing the content when a second user views the content after the first user's audiovisual viewing, and reproducing the actions of the first virtual avatar based on the action information contained in the action log information; and configuring a display object representing the specific action as associated with the first virtual avatar at the time when the first virtual avatar performs the specific action in the reproducing content by referring to the specific action information contained in the action log information.

[0011] In the aforementioned content delivery system, for a second user who watches or listens to the same content after a first user who previously watched or listened to it, the actions of the first virtual avatar performed by the first user during the viewing or listening process are reproduced. This allows the second user to experience a sense of connection with the other user (the first user) during the content viewing or listening experience. Furthermore, the display object representing the specific actions performed by the first virtual avatar during the content viewing or listening process is configured in association with the first virtual avatar at the moment that specific action is performed. Thus, the content representing the specific actions performed by the first virtual avatar can be used as reference information about the actions to be taken during the content viewing or listening experience, and can be visually prompted to the second user at appropriate times. As a result, the convenience for the user (the second user) of the viewing or listening content can be effectively improved.

[0012] Invention Effects

[0013] According to one aspect of the present invention, a content delivery system, a content delivery method, and a content delivery program are provided that can effectively improve the convenience for users who are watching or listening to content. Attached Figure Description

[0014] Figure 1 This is a diagram illustrating an example of an application of a content-providing system for implementing a method.

[0015] Figure 2 It means and Figure 1 The content provides a diagram of an example of the hardware architecture associated with the system.

[0016] Figure 3 It means and Figure 1 The diagram illustrates an example of the functional structure of the system that provides virtual avatar recording and processing related content.

[0017] Figure 4 It means Figure 1 The content provides a sequence diagram of an example of the actions processed by the system's virtual avatar recording.

[0018] Figure 5 This is a diagram showing an example of a content image provided to the first user.

[0019] Figure 6 This is an example of a specific action of a virtual character.

[0020] Figure 7 This is an example of a specific action of a virtual character.

[0021] Figure 8 It means and Figure 1 The diagram illustrates an example of the functional structure of the virtual image reproduction processing system provided by the content provider.

[0022] Figure 9 It means Figure 1 The content provides a sequence diagram of an example of the actions processed by the system's virtual image reproduction.

[0023] Figure 10 This is a diagram illustrating an example of a display object shown on a content image.

[0024] Figure 11 This is a diagram showing examples of the first and third virtual images displayed in the content image. Detailed Implementation

[0025] Hereinafter, embodiments of the present invention will be described in detail with reference to the accompanying drawings. Furthermore, in the description of the drawings, the same or equivalent elements are labeled with the same reference numerals, and repeated descriptions are omitted.

[0026] (System Overview)

[0027] The content delivery system involved in this implementation is a computer system that publishes content to users. Content, provided by a computer or computer system, is information that can be recognized by humans. The electronic data representing the content is called content data. The form of content presentation is not limited. Content can be represented, for example, by images (e.g., photographs, videos), documents, sound, music, or a combination of any two or more of these elements. Content can be used for various forms of information transmission or communication. For example, content can be used for various scenarios or purposes such as entertainment, news, education, medical care, games, chat, business transactions, speeches, seminars, and training. Publishing refers to the process of sending information to users via communication networks or broadcast networks.

[0028] Content delivery systems provide content to users (audiophiles) by sending content data to user terminals. In one example, the content is provided by a publisher. The publisher is the person who wants to convey information to the audience; they are the sender of the content. The user (audiophile) is the person who wants to receive that information; they are the user of the content.

[0029] In this embodiment, the content is represented at least using images that depict virtual space. That is, the content data includes at least a content image representing the content. A content image is an image that a person can visually recognize as certain information. The content image can be a moving image (video) or a static image.

[0030] In one example, the content image represents the virtual space in which a virtual object exists. A virtual object is an object that does not exist in the real world but is represented only on a computer system. Virtual objects are represented using image assets independent of live-action images, through 2D or 3D computer graphics (CG). There are no limitations on how virtual objects are represented. For example, virtual objects can be represented using animated assets or based on live-action images to appear nearly real. Virtual space is a virtual 2D or 3D space represented by images displayed on a computer. The content image, for example, is an image representing a landscape as seen from a virtual camera set within the virtual space. The virtual camera is a virtual viewpoint (virtual viewpoint) set within the virtual space in a manner corresponding to the line of sight of the user viewing the content image. The content image or virtual space can also contain real-world objects that actually exist in the real world.

[0031] As an example of a virtual object, there is a virtual avatar that serves as a clone of the user. The virtual avatar is not the real person being filmed, but rather represented using image materials independent of the original image, through 2D or 3D computer graphics (CG). There are no limitations on the method of representing a virtual avatar. For example, a virtual avatar can be represented using animated materials, or it can be based on live-action images to closely resemble the real person. The position and orientation of the virtual camera can be set to align with the viewpoint and line of sight of the virtual avatar. In this case, a first-person viewpoint image is provided to the user. Thus, the user can visually recognize the content image corresponding to the field of vision of the viewpoint (virtual camera) of the virtual avatar positioned within the virtual space. By viewing the content image, the user can experience augmented reality (AR), virtual reality (VR), or mixed reality (MR).

[0032] Content delivery systems can also be used to time-shift content for viewing or listening within a given period after real-time publishing. Alternatively, content delivery systems can be used to publish content on demand, enabling viewing or listening at any time. Content delivery systems publish content represented using content data generated and saved in the past.

[0033] In this invention, the expression "sending" data or information from a first computer to a second computer means the transmission of data or information that ultimately reaches the second computer. That is, the above expression includes cases where other computers or communication devices relay the data or information during the transmission.

[0034] The purpose and usage scenarios of the content are unrestricted. For example, the content can also be educational content, in which case the content data is educational content data. Educational content is, for example, content used by teachers to instruct students. A teacher is someone who teaches academic subjects, skills, etc., and a student is someone who receives that instruction. A teacher is an example of a publisher, and a student is an example of an audience. A teacher can be a qualified teacher or not. Instruction refers to a teacher teaching academic subjects, skills, etc., to students. There are no age or affiliation restrictions for either teachers or students; therefore, the purpose and usage scenarios of educational content are also unrestricted. For example, educational content can be used in various types of schools, such as nurseries, kindergartens, primary schools, secondary schools, higher education institutions, universities, graduate schools, vocational schools, preparatory schools, and online schools, as well as in places or settings outside of schools. Relatedly, educational content can be used for various purposes, such as early childhood education, compulsory education, higher education, and career learning. In one example, educational content includes virtual avatars corresponding to teachers or students, meaning that the virtual avatar appears in at least a portion of the educational content's context.

[0035] [Overall Structure of the Content Provider System]

[0036] Figure 1 This diagram illustrates an example of the application of the content providing system 1 in this embodiment. In this embodiment, the content providing system 1 includes a server 10, user terminals 20 (user terminals 20A, 20B, 20C), a content database 30, and an action log information database 40.

[0037] Server 10 is a computer that publishes content data to user terminals 20. Server 10 is connected to at least one user terminal 20 via a communication network N. A user terminal 20 can be shared by multiple users, or one user terminal 20 can be prepared for each user. In this embodiment, server 10 is connected to at least three user terminals 20A, 20B, and 20C. Server 10 is also connected to a content database 30 and an action log information database 40. The structure of the communication network N is not limited. For example, the communication network N can be configured to include the Internet or an intranet.

[0038] User terminal 20 is a computer used by the viewer / auditor of the content (i.e., the user utilizing the content). In this embodiment, the user corresponds to a student utilizing educational content. User terminal 20 has the function of accessing content providing system 1 (server 10) and receiving and displaying content data. The type and structure of user terminal 20 are not limited. For example, user terminal 20 may be configured as a portable terminal including a high-performance mobile phone (smartphone), tablet terminal, wearable terminal (e.g., head-mounted display (HMD), smart glasses, etc.), laptop computer, mobile phone, etc. Alternatively, user terminal 20 may be configured as a fixed terminal such as a desktop computer. User terminal 20 may also be configured as a combination of two or more terminals exemplified above.

[0039] Users log in to content delivery system 1 via user terminal 20 and can utilize (audiovisual) content. As an example, users can engage in various experiences within a virtual space represented by the content, using their own virtual avatar. In this embodiment, it is assumed that the user of content delivery system 1 is already logged in.

[0040] Content database 30 is a non-volatile storage medium or storage device for storing generated content data. Content database 30 can be considered a library of existing content. Content data is stored in content database 30 via server 10 or any other computer.

[0041] Content data is stored in content database 30 based on a unique content ID associated with the content. In one example, content data includes virtual space data, model data, and scripts.

[0042] Virtual space data is electronic data representing the virtual space that constitutes the content. For example, virtual space data can include information representing the configuration of various virtual objects that constitute the background, the position of a virtual camera, or the position of a virtual light source.

[0043] Model data is electronic data used to define the specifications of virtual objects that constitute content. The specifications of a virtual object refer to the conventions or methods used to control the virtual object. For example, the specifications of a virtual object include at least one of its structure (e.g., shape and size), movement, and sound. The data structure of the model data for a virtual avatar is not limited and can be designed arbitrarily. For example, model data may include information related to multiple joints and bones that constitute the virtual avatar, graphic data representing the appearance design of the virtual avatar, attributes of the virtual avatar, and a virtual avatar ID serving as an identifier for the virtual avatar. As examples of information related to joints and bones, the 3D coordinates of each joint and the combination of adjacent joints (i.e., bones) can be given. However, the structure of this information is not limited to this and can be designed arbitrarily. Attributes of a virtual avatar refer to arbitrary information set to give the virtual avatar characteristics, such as nominal size, voice quality, or personality.

[0044] A script is electronic data that specifies the actions of various virtual objects, virtual cameras, or virtual light sources within a virtual space over time. A script can be information used to determine the story's content. The actions of virtual objects are not limited to visually recognizable movements; they can also include the generation of sounds recognizable by hearing. A script contains motion data indicating how and when each virtual object performs its actions.

[0045] Content data can include information about real-world objects. For example, content data can also include real-life images that depict real-world objects. When content data includes real-world objects, the script can further define where and at what time that real-world object should be projected.

[0046] The action log information database 40 is a non-volatile storage medium or storage device that stores action log information, which records the actions of virtual avatars operated by users in the past when watching or listening to content. Details of the action log information will be described later.

[0047] There are no restrictions on where the databases can be set up. For example, at least one of the content database 30 and the action log information database 40 can be set up in a computer system different from the content providing system 1, or it can be a component of the content providing system 1.

[0048] [Hardware architecture of content delivery system]

[0049] Figure 2 This is a diagram illustrating an example of the hardware structure associated with content providing system 1. Figure 2 This refers to the server computer 100, which functions as a server 10, and the terminal computer 200, which functions as a user terminal 20.

[0050] As an example, the server computer 100 includes a processor 101, a main storage unit 102, an auxiliary storage unit 103, and a communication unit 104 as hardware components.

[0051] Processor 101 is a computing device that executes an operating system and applications. Examples of processors include CPUs (Central Processing Units) and GPUs (Graphics Processing Units), but processor 101 is not limited to these. For example, processor 101 can also be a combination of sensors and dedicated circuitry. Dedicated circuitry can be programmable circuitry such as FPGAs (Field-Programmable Gate Arrays), or other types of circuitry.

[0052] The main storage unit 102 is a device for storing programs used to implement the server 10, calculation results output from the processor 101, and the like. The main storage unit 102 is composed of, for example, ROM (Read Only Memory) or RAM (Random Access Memory).

[0053] The auxiliary storage unit 103 is generally a device capable of storing a larger amount of data than the main storage unit 102. The auxiliary storage unit 103 is constructed from non-volatile storage media such as hard disks or flash memory. The auxiliary storage unit 103 stores the server program P1, which enables the server computer 100 to function as a server 10, and various other data. For example, the auxiliary storage unit 103 may store data related to virtual objects such as virtual avatars and at least one aspect of virtual space. The content provider may be implemented as the server program P1.

[0054] The communication unit 104 is a device that performs data communication with other computers via a communication network N. The communication unit 104 may be composed of, for example, a network interface card (NIC) or a wireless communication module.

[0055] The various functional elements of server 10 are implemented by having processor 101 or main storage unit 102 read server program P1 and execute server program P1. Server program P1 contains code for implementing the various functional elements of server 10. Processor 101 activates communication unit 104 according to server program P1 to read and write data in main storage unit 102 or auxiliary storage unit 103. Through this process, the various functional elements of server 10 are implemented.

[0056] Server 10 can consist of one or more computers. When multiple computers are used, they are interconnected via a communication network, thereby logically forming a server 10.

[0057] As an example, the terminal computer 200 includes a processor 201, a main storage unit 202, an auxiliary storage unit 203, a communication unit 204, an input interface 205, and an output interface 206 as hardware components.

[0058] Processor 201 is a computing device that executes an operating system and applications. Processor 201 may be, for example, a CPU or a GPU, but is not limited to these types.

[0059] The main storage unit 202 is a device for storing programs used to implement the user terminal 20, calculation results output from the processor 201, and the like. The main storage unit 202 is, for example, composed of ROM or RAM.

[0060] The auxiliary storage unit 203 is generally a device capable of storing a larger amount of data than the main storage unit 202. The auxiliary storage unit 203 is constructed from non-volatile storage media such as hard disks or flash memory. The auxiliary storage unit 203 stores the client program P2, which enables the terminal computer 200 to function as a user terminal 20, and various other data. For example, the auxiliary storage unit 203 can store data related to virtual objects such as virtual avatars and at least one aspect of virtual space. The content provider can be implemented as the client program P2.

[0061] The communication unit 204 is a device that performs data communication with other computers via a communication network N. The communication unit 204 may be composed of, for example, a network interface card (NIC) or a wireless communication module.

[0062] Input interface 205 is a device that accepts data based on user operations or actions. For example, input interface 205 may consist of at least one of a controller, keyboard, operation buttons, indicating devices, microphone, sensor, and camera. The keyboard and operation buttons may be displayed on a touch panel. The type of input interface 205 is not limited, nor is the data input to input interface 205. For example, input interface 205 may accept data input or selected by a keyboard, operation buttons, or indicating devices. Alternatively, input interface 205 may accept sound data input by a microphone. Alternatively, input interface 205 may accept image data (e.g., video data or still image data) captured by a camera. Alternatively, input interface 205 may also accept data representing nonverbal activities of the user (e.g., gaze, head movement, movement of a part of the user's body other than the head (e.g., hand, etc.) detected by the motion capture function of a sensor or camera as motion data.

[0063] Output interface 206 is a device for outputting data processed by terminal computer 200. For example, output interface 206 may consist of at least one of a monitor, touch panel, HMD, and speaker. The monitor, touch panel, HMD, and other display devices display the processed data on a screen. In this embodiment, the content image is output and displayed on the HMD. The speaker outputs sound represented by the processed sound data.

[0064] The various functional elements of the user terminal 20 are implemented by having the processor 201 or the main storage unit 202 read and execute the client program P2. The client program P2 contains code for implementing the various functional elements of the user terminal 20. According to the client program P2, the processor 201 activates the communication unit 204, the input interface 205, or the output interface 206 to read and write data from the main storage unit 202 or the auxiliary storage unit 203. Through this process, the various functional elements of the user terminal 20 are implemented.

[0065] At least one of the server program P1 and the client program P2 may be provided on a tangible recording medium such as a CD-ROM, DVD-ROM, or semiconductor memory. Alternatively, at least one of these programs may be provided as a data signal superimposed on a carrier wave via a communication network. These programs may be provided separately or together.

[0066] [Functional details of the content delivery system]

[0067] The content providing system 1 is mainly composed of the ability to perform virtual image recording and virtual image reproduction.

[0068] Virtual avatar recording processing is the process of recording the actions of the first virtual avatar operated by the first user during the period when the first user is watching or listening to the content.

[0069] The virtual avatar reproduction process is as follows: after virtual avatar recording processing for at least one first user is performed, when a second user watches or listens to the content, the content is reproduced in the content image provided to the second user, and the actions of the first virtual avatar corresponding to at least one first user recorded through the virtual avatar recording processing are reproduced.

[0070] The virtual avatar recording process and the virtual avatar reproduction process will be explained in turn below. Furthermore, in this embodiment, the virtual avatar is a humanoid object (see...). Figure 10 as well as Figure 11 However, the form of a virtual image is not limited to a specific form.

[0071] [Functional structure associated with virtual avatar recording and processing]

[0072] Figure 3 This diagram illustrates an example of the functional structure of the content provision system 1 associated with virtual avatar recording and processing. Here, we will use the case of recording the actions of the first virtual avatar corresponding to the first user utilizing the user terminal 20A as an example.

[0073] Server 10, as a functional element associated with virtual avatar recording and processing, includes a receiving unit 11, a content sending unit 12, and a log generation unit 13. The receiving unit 11 receives data signals sent from the user terminal 20. The content sending unit 12 sends content data to the user terminal 20 in response to a request from the user. The log generation unit 13 generates action log information for the virtual avatar based on the action information of the virtual avatar obtained from the user terminal 20 and stores it in the action log information database 40.

[0074] User terminal 20A, as a functional element associated with virtual avatar recording and processing, includes a request unit 21, a receiving unit 22, a display control unit 23, and an action information sending unit 24. The request unit 21 requests various controls related to the content from server 10. The receiving unit 22 receives content data. The display control unit 23 processes the received content data and displays the content on the display device of user terminal 20A. The action information sending unit 24 sends action information to server 10 representing the actions of the first virtual avatar operated by the first user.

[0075] [Actions of the system associated with virtual avatar recording and processing]

[0076] Reference Figure 4 The operation of the content provision system 1 associated with virtual avatar recording and processing will be described, and a portion of the content provision method involved in this embodiment will also be described. Figure 4 This is a sequence diagram representing an example of the actions associated with the virtual avatar recording and processing of content providing system 1.

[0077] In step S101, the request unit 21 of the user terminal 20A sends a content request to the server 10. The content request is a data signal used to request the playback of content (i.e., to start watching or listening to the content) from the server 10. This content request is received by the receiving unit 11 of the server 10.

[0078] In step S102, in response to a content request from user terminal 20A, the content sending unit 12 of server 10 reads content data from content database 30 and sends the content data to user terminal 20A. This content data is received by the receiving unit 22 of user terminal 20A.

[0079] In step S103, the display control unit 23 of the user terminal 20A reproduces the content. In this embodiment, the content published by the content providing system 1 is educational content representing a teaching scene. As an example, the content includes handout data representing a scene of a teacher giving a lesson. The handout data is, for example, 3D (depth) image data. As an example, the virtual space represented by the content includes a background such as a school classroom, and includes an area within the virtual space where the handout data (image) is displayed. In this embodiment, the reproduction of the content refers to reproducing the aforementioned handout data within the virtual space.

[0080] In step S104, the display control unit 23 configures a first virtual image corresponding to the first user within the virtual space. For example, in a virtual space simulating a classroom, the display control unit 23 configures the first virtual image at a position in front of the display area of ​​the handout data (i.e., a position where the handout data can be visually identified).

[0081] In step S105, the display control unit 23 generates a content image (content video) for prompting the first user based on the viewpoint of the first virtual avatar configured in the virtual space (i.e., the virtual viewpoint of the first user set in the virtual space). Furthermore, in this embodiment, the user terminal 20A (display control unit 23) performs the processing (drawing) of generating a content image for the first user based on content data and the first user's virtual viewpoint, but the drawing may also be performed by the server 10.

[0082] Figure 5 This is an example diagram showing the content image IM1 provided to the first user. Figure 5 In this example, the content image IM1 includes a portion of the area 51 that displays the handout data (image) (i.e., the range contained within the field of view from the first user's virtual viewpoint). Additionally, the content image IM1 displays a portion (the hand portion) of the first virtual avatar A1 operated by the first user, and a tool object 52 (as an example, mimicking a book) that can be operated via the first virtual avatar A1.

[0083] In step S106, the first user operates the first virtual avatar A1 by operating the user terminal 20A. The method by which the first user operates the first virtual avatar A1 is not limited to a specific method. For example, the first user can operate the first virtual avatar A1 by operating the controller of the user terminal 20A (an example of the input interface 205). Alternatively, for example, if the user terminal 20A has sensors such as an accelerometer or gyroscope sensor installed on the first user's hand (an example of the input interface 205), the first user can operate the first virtual avatar A1 through gestures (i.e., the movement of the user's hand detected by the aforementioned sensors).

[0084] Examples of operations that can be performed on a virtual avatar include moving the virtual avatar's hand within a virtual space, and performing certain processing on a specified object within a virtual space via the virtual avatar's hand. Furthermore, the parts that can be operated on a virtual avatar are not limited to the virtual avatar's hand. For example, if sensors are installed on multiple parts of the user's body and can sense the movement of each part, the parts that can be operated on the virtual avatar could also be multiple parts of the virtual avatar that move according to the user's body movement (i.e., parts corresponding to the parts of the user's body equipped with sensors).

[0085] In step S107, the motion information sending unit 24 of the user terminal 20A sends motion information representing the action of the first virtual avatar A1 operated in step S106 to the server 10. The motion information is associated with the playback position of the content (lecture notes data) (e.g., the playback time when the start time of the lecture notes data is set to the reference "0:00"). For example, the motion information of the first virtual avatar A1 records the state of the first virtual avatar A1 at each time interval, divided from the start of the playback of the content to the end of the playback (e.g., the position coordinates of each part constituting the first virtual avatar A1). In other words, the motion information of the first virtual avatar A1 is information used to reproduce the movement of the first virtual avatar A1 from the start of the playback of the content to the end of the playback.

[0086] In step S108, the log generation unit 13 of the server 10 generates action log information for the first virtual avatar A1 based on the action information received from the user terminal 20A (action information sending unit 24). The action log information includes not only the action information of the first virtual avatar A1, but also specific action information indicating the time and content of a predetermined specific action performed by the first virtual avatar A1. For example, the log generation unit 13 extracts specific actions from the action information of the first virtual avatar A1 and generates specific action information for each extracted specific action. Then, the log generation unit 13 generates action log information that includes the action information of the first virtual avatar A1 and the specific action information for each extracted specific action.

[0087] As an example, a specific action includes the following: selecting a specified project object from one or more project objects pre-registered for teaching purposes and generating the selected project object in virtual space. (See reference...) Figures 5 to 7 Specific examples of such actions will be provided.

[0088] like Figure 5As shown, firstly, while confirming the content image IM1, the first user aligns the hand of the first virtual avatar A1 with the tool object 52, and performs the operation of selecting the tool object 52. Thus, as... Figure 6 As shown, a list of one or more (eight in this example) project objects 53 pre-registered as projects to be used in the course of learning is displayed. Figure 6 In the example, models of past creatures such as mammoths, ammonites, and Pikachu are registered as item objects 53. The first user, for example, positions the hand of the first virtual avatar A1 towards the desired item object 53 and performs the operation of selecting that item object 53. Thus, as... Figure 7 As shown, the selected item object 53 (in this example, a model of a mammoth) is generated at a position in front of the first virtual avatar A1. The first user can, for example, move the item object 53 freely via the first virtual avatar A1 and observe it from various angles. Furthermore, the types of item objects 53 are not limited to the examples described above. Various types of item objects 53 can be used depending on the content of the lesson. For example, in a scenario where the content (lecture notes) represents a science (biology) lesson, experimental props such as beakers, flasks, and microscopes can be registered as item objects 53. Similarly, in a scenario where the content represents a mathematics lesson, rulers, protractors, compasses, etc., can be registered as item objects 53.

[0089] In this embodiment, when a specific action as described above (i.e., the action of selecting tool object 52 and selecting specified item object 53) is extracted from the action information of the first virtual character A1, the log generation unit 13 generates specific action information that associates the time when the specific action is performed (e.g., the reproduction position of the content when the specific action is performed) with the content of the specific action (in this embodiment, information indicating the selected item object 53).

[0090] In step S109, the log generation unit 13 saves the action log information generated in step S107 in the action log information database 40.

[0091] Through the above processing, the action log information of the first virtual avatar's operation in the content viewing / audiovisualization is saved (i.e., virtual avatar recording processing). Furthermore, this virtual avatar recording processing can be performed on each of the multiple first users who view / audiovisualize the content at any given time. Thus, the action log information of multiple first virtual avatars is saved in the action log information database 40.

[0092] [Functional structure associated with virtual avatar reproduction processing]

[0093] Figure 8This diagram illustrates an example of the functional structure related to the virtual avatar reproduction processing of the content providing system 1. Here, we will take the case where, after the action log information of one or more first virtual avatars has been saved through the virtual avatar recording processing described above (i.e., after one or more first users have watched or listened to the content), a second user watches or listens to the content using the user terminal 20B as an example.

[0094] Server 10, as a functional element related to virtual avatar reproduction processing, includes a virtual avatar information sending unit 14 in addition to the aforementioned receiving unit 11 and content sending unit 12. The virtual avatar information sending unit 14 sends action log information of one or more first users who have previously viewed / listened to the content to user terminal 20. Furthermore, if a third user is simultaneously viewing / listening to the content with a second user, the virtual avatar information sending unit 14 also sends action information of a third virtual avatar corresponding to that third user to user terminal 20. In this embodiment, it is assumed that a third user using user terminal 20C is simultaneously viewing / listening to the content with a second user. In this case, the virtual avatar information sending unit 14 sends the action information of the third virtual avatar received from user terminal 20C to user terminal 20B. In practice, for the third user (user terminal 20C), the same process as for the second user (user terminal 20B) is used, sending the action log information of the first user and the action information of the second user's virtual avatar (second virtual avatar). However, the following explanation focuses on the processing of the second user (user terminal 20B).

[0095] User terminal 20B includes the aforementioned request unit 21, receiving unit 22, and display control unit 23 as functional elements related to virtual avatar reproduction processing. The request unit 21 and receiving unit 22 have the same functions as those in virtual avatar recording processing. The display control unit 23 performs processes specific to virtual avatar reproduction processing, in addition to those described in virtual avatar recording processing.

[0096] [The actions of the system associated with virtual avatar reproduction processing]

[0097] Reference Figure 9 This describes the operation of the content provisioning system 1 associated with virtual avatar reproduction processing, and also describes a part of the content provisioning method of this embodiment. Figure 9 This is a timing diagram illustrating an example of actions associated with the virtual image reproduction processing of content provision system 1.

[0098] In step S201, the request unit 21 of the user terminal 20B sends a content request to the server 10. The content request is a data signal used to request the playback of content (i.e., to start watching or listening to the content) from the server 10. This content request is received by the receiving unit 11 of the server 10.

[0099] In step S202, in response to a content request from user terminal 20B, the content sending unit 12 of server 10 reads content data from content database 30 and sends the content data to user terminal 20B. The content data is then received by the receiving unit 22 of user terminal 20B.

[0100] In step S203, the virtual avatar information sending unit 14 of server 10 selects the virtual avatars to be reproduced. For example, consider a case where there are multiple first users who have previously watched or listened to the same content as the content that the second user wants to watch or listen to, and action log information related to multiple first virtual avatars is stored in action log information database 40. In this case, assuming that all the first virtual avatars are reproduced (displayed) within the content (virtual space) provided to the second user, many first virtual avatars will be displayed in the content image provided to the second user, and the information displayed in the content image may become complex. Therefore, the virtual avatar information sending unit 14 selects a predetermined number (e.g., 3 people) of first virtual avatars as reproduction objects from the multiple first virtual avatars whose action log information is stored in action log information database 40.

[0101] For example, the virtual avatar information sending unit 14 can prioritize the first virtual avatar whose action log information is generated later (i.e., the time when the content was viewed or listened to), and select a predetermined number of first virtual avatars. Alternatively, the virtual avatar information sending unit 14 can prioritize the first virtual avatar whose action log information contains more records of specific action information (i.e., the number of times a specific action was performed during the content viewing or listening), and select a predetermined number of first virtual avatars.

[0102] In step S204, the virtual avatar information sending unit 14 sends the action log information of the first virtual avatar selected in step S203 to the user terminal 20B.

[0103] If a third user is simultaneously watching or listening to the same content as the second user, steps S205 and S206 are executed. Specifically, in step S205, server 10 (for example, virtual avatar information sending unit 14) obtains the action information of the third virtual avatar corresponding to the third user from user terminal 20C. In step S206, server 10 (for example, virtual avatar information sending unit 14) sends the action information of the third virtual avatar to user terminal 20B.

[0104] In step S207, the display control unit 23 of the user terminal 20B reproduces the content. The processing in step S207 is the same as that in step S103.

[0105] In step S208, the display control unit 23 reproduces the actions of each first virtual character in the virtual space based on the action information contained in the action log information of each first virtual character received in step S204. That is, the display control unit 23 configures each first virtual character in the virtual space, causing each first virtual character to move based on the action information contained in its corresponding action log information. Here, the position of each first virtual character can be the actual position where each first virtual character was configured in the past, but there may be interference between each first virtual character and between a first virtual character and a second virtual character (a state of overlapping configuration in the same location). Therefore, the display control unit 23 can configure each first virtual character at a position different from the position where each first virtual character was configured in the past, so that the virtual characters do not interfere with each other (for example, the virtual characters are separated from each other by a certain distance or more).

[0106] In step S209, the display control unit 23, by referring to the specific action information contained in the action log information of each first virtual character received in step S204, configures the display object representing the specific action as associated with each first virtual character when the first virtual character performs a specific action during playback. For example, consider the case where a first user, while watching or listening to content in the past, operates a first virtual character and performs a specific action at the content playback location "30:00" (30 minutes after the playback start time). In this case, when a second user watches or listens to content at the same content playback location ("30:00" in this example), the display control unit 23 configures the display object representing the specific action of that first virtual character as associated with that first virtual character.

[0107] In the event that a third user is simultaneously viewing or listening to the same content as the second user (i.e., when motion information of the third virtual avatar is received in step S206), in step S210, the display control unit 23 configures the third virtual avatar in the virtual space and moves the third virtual avatar based on its motion information. Furthermore, the motion information of the third virtual avatar is periodically sent from the user terminal 20C to the user terminal 20B via the server 10 at predetermined time intervals.

[0108] In step S211, the display control unit 23 generates a content image (content video) for prompting the second user based on the viewpoint of the second virtual avatar configured in the virtual space (i.e., the virtual viewpoint of the second user set in the virtual space). Alternatively, in this embodiment, the user terminal 20B (display control unit 23) performs the processing (drawing) of generating the content image for the second user based on the content data and the second user's virtual viewpoint, but the drawing can also be performed by the server 10.

[0109] In step S212, the display control unit 23 can switch the display of the first virtual avatar from being shown to being hidden based on an instruction from the second user while the second user is watching or listening to the content. For example, if the display control unit 23 receives an instruction to hide the first virtual avatar via a user operation through the controller of the user terminal 20B, it can switch so that the first virtual avatar being displayed in the content image is not shown. Additionally, if the display control unit 23 receives an instruction to re-display the temporarily hidden first virtual avatar, it can switch so that the first virtual avatar is displayed in the content image. Furthermore, when multiple first virtual avatars are selected as playback targets, the switching of the display / hiding of the first virtual avatars can be performed on each first virtual avatar individually or on all first virtual avatars simultaneously.

[0110] Figure 10 This diagram illustrates an example of a display object 54 shown in the content image IM2 provided to the second user. In this example, a display object 54 representing content indicating a specific action, generated in virtual space by the first virtual avatar A1 selecting an item object 53 representing a Pikachu model, is configured in association with the first virtual avatar A1. As an example, the display object 54 is positioned near the first virtual avatar A1 (in this example, in the space in front of the head of the first virtual avatar A1). Figure 10 As shown, the display object 54 may include text information 541 representing the selected item object 53 and an illustrated image 542 representing the selected item object 53. Alternatively, a real-life image such as a photograph may be used instead of the illustrated image 542. If the first virtual avatar A1 is reproduced solely based on its motion information (i.e., only the movement of the first virtual avatar A1's hand is reproduced), it is impossible to grasp what specific actions the first virtual avatar A1 took in the audiovisual content. That is, it is impossible to grasp the actions taken by the first virtual avatar A1 through its tool object 52 (see reference 542). Figure 5 The second user selects which item object 53 based on the operation of the first virtual avatar A1. On the other hand, as described above, when the first virtual avatar A1 performs a specific action (in this embodiment, the operation of selecting a specified item object 53), the display object 54 representing the content of that specific action is configured in association with the first virtual avatar A1, so that the second user can visually grasp the content of the specific action performed by the first user (first virtual avatar A1) at an appropriate time.

[0111] Figure 11 This is a diagram illustrating a display example of the first virtual avatar A1 and the third virtual avatar A3 shown in content image IM2. For example... Figure 11As shown, the display control unit 23 can display the third virtual avatar A3 corresponding to the third user who simultaneously watches and listens to the content with the second user in a different display mode than the first virtual avatar A1 (the virtual avatar being reproduced based on action log information) corresponding to the first user who previously watched and listened to the content. Figure 11 In the example, the display control unit 23 has an inverted triangle-shaped marker object M positioned in the space above the third virtual avatar A3. Based on the presence or absence of this marker object M, the second user can reliably determine whether the virtual avatars configured in the virtual space are the first virtual avatar A1 being reproduced, or the third virtual avatar A3 corresponding to the third user who is providing content in real time. Furthermore, the display format used to distinguish the third virtual avatar A3 from the first virtual avatar A1 is not limited to... Figure 11 The example shown. For example, the display control unit 23 may display the third virtual image A3 in a different color (e.g., a bright color) than the first virtual image A1, or it may display the first virtual image A1 semi-transparently.

[0112] [Effect]

[0113] As described above, one aspect of the content providing system of the present invention includes at least one processor. The at least one processor performs the following processes: configuring a first virtual avatar within a virtual space corresponding to a first user who is viewing or listening to specified content representing the virtual space; generating action log information, the action log information recording action information indicating actions of the first virtual avatar operated by the first user during the period when the first user is viewing or listening to the content, and specific action information indicating the time and content of a predetermined specific action performed by the first virtual avatar; reproducing the content when a second user views or listens to the content after the first user has viewed or listened to it, and reproducing the actions of the first virtual avatar based on the action information contained in the action log information; and configuring a display object representing the specific action as associated with the first virtual avatar at the time when the first virtual avatar performs the specific action in the reproducing content, by referring to the specific action information contained in the action log information.

[0114] One aspect of the present invention relates to a content provision method comprising the following steps: configuring a first virtual avatar within a virtual space corresponding to a first user who is viewing or listening to specified content representing the virtual space; generating action log information, the action log information recording action information indicating actions of the first virtual avatar operated by the first user during the period when the first user is viewing or listening to the content, and specific action information indicating the time and content of a predetermined specific action performed by the first virtual avatar; reproducing the content after the first user has viewed or listened to the content, and reproducing the actions of the first virtual avatar based on the action information contained in the action log information; and configuring a display object representing the specific action as associated with the first virtual avatar at the time when the first virtual avatar performs the specific action in the reproducing content, by referring to the specific action information contained in the action log information.

[0115] One aspect of the present invention relates to a content provider that causes a computer to perform the following steps: configuring a first virtual avatar within a virtual space corresponding to a first user who is viewing or listening to specified content representing the virtual space; generating action log information, the action log information recording action information indicating actions of the first virtual avatar performed by the first user during the period when the first user is viewing or listening to the content, and specific action information indicating the time and content of a predetermined specific action performed by the first virtual avatar; reproducing the content when a second user views or listens to the content after the first user has viewed or listened to it, and reproducing the actions of the first virtual avatar based on the action information contained in the action log information; and configuring a display object representing the specific action performed by the first virtual avatar in the reproducing process as associated with the first virtual avatar by referring to the specific action information contained in the action log information.

[0116] In this respect, it achieves the following effect: For a second user who watches or listens to the same content after the first user who previously watched or listened to it, the actions of the first virtual avatar A1 performed by the first user during the viewing or listening process are reproduced. Thus, the second user can experience a sense of connection with the other user (the first user) during the viewing or listening process. Furthermore, the display object 54 representing the specific actions performed by the first virtual avatar A1 during the viewing or listening process is configured in association with the first virtual avatar A1 at the moment that specific action is performed. Therefore, the content representing the specific actions performed by the first virtual avatar A1 can be visually prompted to the second user at appropriate times as reference information about the actions to be taken during the viewing or listening process. As a result, the convenience for the user (the second user) of the viewing or listening content can be effectively improved.

[0117] In another aspect of the content publishing system, the content can be educational content representing a teaching scenario, and the specific action can be an action used to select a specified project object from one or more pre-registered learning projects for teaching and generate the selected project object in virtual space. According to the above structure, for the second user, appropriate project objects 53 can be generated in the virtual space based on the progress of the teaching within the virtual space and by referring to the actions of other users (the first user) (i.e., the specific actions of the first virtual avatar A1). As a result, the efficiency of learning using virtual space can be effectively improved.

[0118] In another aspect of the content publishing system, the displayed object may include a graphic image representing the project object. Based on the above structure, the content of the project object 53 retrieved by the first virtual avatar A1 can be easily and visually grasped through the graphic image 542.

[0119] In another aspect of the content publishing system, when a second user is watching or listening to content, and a third user is simultaneously watching or listening to the content with the second user, at least one processor can display a third virtual avatar corresponding to the third user in a different display mode than the first virtual avatar. Based on this structure, it is possible to identify the first virtual avatar A1 of the first user who previously watched or listened to the content and the third virtual avatar A3 of the third user who is simultaneously watching or listening to the content with the second user in real time. As a result, the second user can easily communicate with the third user who is watching or listening to the content in real time.

[0120] In other content publishing systems, at least one processor can switch the display of the first virtual avatar from being displayed to being hidden, based on instructions from the second user, during the second user's viewing or listening to the content. If the number of first virtual avatars A1 displayed in the virtual space is too large, the information displayed in the content image IM2 provided to the second user may become complex. According to the above structure, in such a case, the second user can freely switch the first virtual avatar to be hidden.

[0121] In other content publishing systems, where at least one processor generates action log information for multiple first virtual avatars corresponding to multiple first users when a second user watches or listens to content, a predetermined number of first virtual avatars can be selected as the reproduction objects. Based on the above structure, by suppressing the number of first virtual avatars A1 displayed in the virtual space to below a certain number (a predetermined number), it is possible to prevent the information displayed in the content image IM2 provided to the second user from becoming complex.

[0122] In other aspects of the content publishing system, at least one processor can prioritize the first virtual avatar whose action log information was generated later, and select a predetermined number of first virtual avatars. It is believed that users who watch / listen to content while referring to the actions of users (virtual avatars) who have previously watched / listened to content are more likely to perform appropriate virtual avatar actions during content viewing / listening. Based on the above structure, by prioritizing the display of the first virtual avatar A1 of the first user who most recently watched / listened to the content in the content image IM2 provided to the second user, the quality of the reference information (specific actions of the first virtual avatar A1) provided to the second user can be improved.

[0123] In other aspects of the content publishing system, at least one processor can prioritize the first virtual avatar with the most records of specific action information contained in the action log information to select a predetermined number of first virtual avatars. The first user corresponding to the first virtual avatar with the most records of specific action information is more likely to be a user actively engaged in learning within the content's audiovisual content. By prioritizing the first virtual avatar A1 of such users as the reproduction object, the amount of reference information (the specific actions of the first virtual avatar A1) provided to the second user can be increased.

[0124] [Variation Example]

[0125] The present invention has been described in detail above based on the embodiments. However, the present invention is not limited to the above embodiments. Various modifications can be made to the present invention without departing from its spirit.

[0126] For example, some of the functions of the server 10 described above can also be performed by the user terminal 20. Conversely, some of the functions of the user terminal 20 described above can also be performed by the server 10. For example, in the above embodiment, the display control unit 23 of the user terminal 20 mainly performs the reproduction of content, the reproduction of virtual images, and the generation of content images, but these processes can also be performed by the server 10. In this case, the user terminal 20 only needs to perform the display process of receiving the content images processed and generated by the server 10 and displaying the received content images on the display.

[0127] In this invention, the statement "at least one processor executes a first process, executes a second process, ... executes an nth process," or its corresponding statement, includes the concept of cases where the executing entity (i.e., the processor) of the n processes from the first process to the nth process changes midway. That is, this statement includes both the case where all n processes are executed by the same processor and the case where the processor changes in any way among the n processes.

[0128] The processing order of the method executed by at least one processor is not limited to the examples in the above embodiments. For example, some of the above steps (processes) may be omitted, or the steps may be executed in a different order. In addition, any two or more of the above steps may be combined, or a part of the steps may be modified or deleted. Alternatively, other steps may be performed based on the above steps.

[0129] Marker description

[0130] 1…Content providing system, 10…Server, 11…Receiving unit, 12…Content sending unit, 13…Log generation unit, 14…Virtual avatar information sending unit, 20, 20A, 20B, 20C…User terminal, 21…Request unit, 22…Receiving unit, 23…Display control unit, 24…Action information sending unit, 30…Content database, 40…Action log information database, 53…Project object, 54…Display object, 542…Illustrated image, 101…Processor, A1…First virtual avatar, A3…Third virtual avatar, IM1, IM2…Content image, P1…Server program, P2…Client program.

Claims

1. A content delivery system comprising at least one processor, said at least one processor performing the following processes: Based on the content data stored in the content database, educational content representing a virtual space and a teaching scenario is displayed. This content data includes virtual space data, model data, and scripts. The first virtual avatar, corresponding to the student (i.e., the first user) of the educational content described in the audiovisual presentation, is configured within the virtual space. Action log information is generated and stored in an action log information database. The action log information records action information and specific action information. The action information represents the actions of the first virtual character operated by the first user during the period when the first user is watching or listening to the educational content. The specific action information represents the time when the first virtual character performs a predetermined specific action and the content of that specific action. When a second student (different from the first user) listens to or views the educational content after the first user has listened to or viewed the educational content, the educational content is reproduced in the virtual space. Furthermore, based on the action information contained in the action log information stored in the action log information database, the first virtual image is reproduced in the virtual space, and the first virtual image moves based on the action information. By referring to the specific action information contained in the action log information, when the first virtual character in the reproduction performs the specific action, the display object representing the content of the specific action is configured in association with the first virtual character.

2. The content provisioning system according to claim 1, wherein, The specific action is an action used to select a specified project object from one or more project objects pre-registered for the learning projects to be taught, and to generate the selected project object in the virtual space.

3. The content provisioning system according to claim 2, wherein, The display object includes an image representing the project object.

4. The content delivery system according to any one of claims 1 to 3, wherein, When a third user is simultaneously viewing or listening to the educational content while the second user is viewing or listening to the educational content, the at least one processor will display the third virtual avatar corresponding to the third user in a display mode different from the first virtual avatar.

5. The content delivery system according to any one of claims 1 to 3, wherein, During the period when the second user is watching or listening to the educational content, the at least one processor switches between displaying and not displaying the first virtual avatar based on instructions from the second user.

6. The content delivery system according to any one of claims 1 to 3, wherein, When the second user is listening to the educational content, if the action log information of multiple first virtual images corresponding to multiple first users is generated, the at least one processor will select a predetermined number of the first virtual images as the reproduction objects.

7. The content delivery system according to claim 6, wherein, The at least one processor prioritizes the first virtual avatar whose time of generating the action log information is later, in order to select the predetermined number of the first virtual avatars.

8. The content delivery system according to claim 6, wherein, The at least one processor prioritizes the first virtual avatar with the most records of the specific action information contained in the action log information to select the predetermined number of first virtual avatars.

9. A content delivery method, comprising the following steps: Based on the content data stored in the content database, educational content representing a virtual space and a teaching scenario is displayed. The content data includes virtual space data, model data, and scripts. The first virtual avatar, corresponding to the student (i.e., the first user) of the educational content described in the audiovisual presentation, is configured within the virtual space. Action log information is generated and stored in an action log information database. The action log information records action information and specific action information. The action information represents the actions of the first virtual character operated by the first user during the period when the first user is watching or listening to the educational content. The specific action information represents the time when the first virtual character performs a predetermined specific action and the content of that specific action. When a second student (different from the first user) listens to or views the educational content after the first user has listened to or viewed the educational content, the educational content is reproduced in the virtual space. Furthermore, based on the action information contained in the action log information stored in the action log information database, the first virtual image is reproduced in the virtual space, and the first virtual image moves based on the action information. By referring to the specific action information contained in the action log information, when the first virtual character in the reproduction performs the specific action, the display object representing the content of the specific action is configured in association with the first virtual character.

10. A program product comprising a content provider that causes a computer to perform the following steps: Based on the content data stored in the content database, educational content representing a virtual space and a teaching scenario is displayed. The content data includes virtual space data, model data, and scripts. The first virtual avatar, corresponding to the student (i.e., the first user) of the educational content described in the audiovisual presentation, is configured within the virtual space. Action log information is generated and stored in an action log information database. The action log information records action information and specific action information. The action information represents the actions of the first virtual character operated by the first user during the period when the first user is watching or listening to the educational content. The specific action information represents the time when the first virtual character performs a predetermined specific action and the content of that specific action. When a second student (different from the first user) listens to or views the educational content after the first user has listened to or viewed the educational content, the educational content is reproduced in the virtual space. Furthermore, based on the action information contained in the action log information stored in the action log information database, the first virtual image is reproduced in the virtual space, and the first virtual image moves based on the action information. By referring to the specific action information contained in the action log information, when the first virtual character in the reproduction performs the specific action, the display object representing the content of the specific action is configured in association with the first virtual character.

Images