Program, content linking method, terminal device, and educational system

Abstract

We provide technology that facilitates communication through educational content. [Solution] A program that enables the execution of educational content composed of multiple scenes by having one or more processors control the execution of educational content, and when communication with a user of a second account is initiated by a connection request from a user of a first account, the program implements the following process: the first object that controls the progress of the educational content is not displayed in the scene on the terminal operated by the user of the first account, while the first object that controls the progress of the educational content is displayed in the scene on the terminal operated by the user of the second account.

Description

Technical Field

[0001] The present disclosure relates to a program, a content linking method, a terminal device, and an early childhood education system.

Background Art

[0002] The application of network technology to early childhood education has been put into practical use. For example, there is a service where parents and children can enjoy the images and voices of picture books provided online. There is also a service where two points or multiple points can interact bidirectionally while viewing the images of the same picture book. In these services, the progress of the picture book is automatically controlled.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In a two-way service, it is desirable to be able to adjust the progress of the content flexibly. For example, there may be a desire to delay the progress of the content according to the user's reaction. Also, in a slide with a production effect, there may be a desire to temporarily stop the progress of the content or delay the progress.

[0005] One of the objects of the present disclosure is to provide a program, a content linking method, a terminal device, and an early childhood education system that smooth communication through early childhood education content.

Means for Solving the Problems

[0006] One form of this disclosure is a program that causes one or more processors to control the execution of educational content consisting of multiple scenes, and when communication with a user of a second account is initiated by a connection request from a user of a first account, the program implements a process in which the first operation object that controls the progress of the educational content is not displayed in the scene on the terminal operated by the user of the first account, and the first operation object that controls the progress of the educational content is displayed in the scene on the terminal operated by the user of the second account. [Effects of the Invention]

[0007] According to one form of this disclosure, it is possible to enable a second account user to control the progress of educational content. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example configuration of the educational system envisioned in Embodiment 1. [Figure 2] This diagram illustrates an example of the functional configuration of a user terminal in Embodiment 1. [Figure 3] This is a diagram illustrating an example of the structure of educational content. [Figure 4] This is a diagram illustrating an example of a slide. [Figure 5] This diagram illustrates the real-time image synthesis process for the first user. [Figure 6] This diagram illustrates other image synthesis processes for the first user's real-time images. [Figure 7] This diagram illustrates the positional relationship in the depth direction between the background and objects when compositing the first user's real-time image onto a slide. [Figure 8] This diagram illustrates an example of how the display area of ​​the "Grandpa" object overlaps with the display area of ​​the first user's real-time image. [Figure 9] This diagram illustrates how the position of the real-time image of the first user (e.g., grandchild), which is determined for each scene (or slide), changes. [Figure 10] This is a diagram for explaining the change in the display size of the real-time image of the first user (e.g., grandson). [Figure 11] This is a diagram for explaining an example of the role setting sequence among users. [Figure 12] This is a diagram for explaining an example of the connection sequence after role setting. [Figure 13] This is a diagram for explaining an example of the home screen display before connection start. [Figure 14] This is a diagram for explaining examples of the invitation screen and the confirmation screen. [Figure 15] This is a diagram for explaining an example of the confirmation screen display. [Figure 16] This is a diagram for explaining the setting sequence executed immediately after connection start. [Figure 17] This is a diagram for explaining an example of the home screen display after connection start. [Figure 18] This is a diagram for explaining an example of the picture book content selection screen. [Figure 19] This is a diagram for explaining an example of the picture book content scene display. [Figure 20] This is a diagram for explaining an example of the display of other scenes of the picture book content. [Figure 21] This is a diagram for explaining the control sequence according to the user's role. [Figure 22] This is a diagram for explaining the control sequence when the grandparents' terminal receives a scene switching operation. [Figure 23] This is a diagram for explaining an example of scene switching. [Figure 24] This is a diagram for explaining another example of scene switching. [Figure 25] This is a diagram for explaining an example of slide switching within the same scene. [Figure 26] This is a diagram for explaining another example of slide switching within the same scene. [Figure 27] This is a diagram for explaining another example of slide switching within the same scene. [Figure 28]This diagram illustrates the control sequence when a grandchild terminal receives an operation on an object. [Figure 29] This diagram illustrates an example of a visual effect that appears when a grandchild interacts with the "rice ball" object within the slide. [Figure 30] This diagram illustrates an example of a visual effect that appears when grandparents interact with the "rice ball" object in the slide. [Figure 31] This diagram illustrates an example of a visual effect that appears when a grandchild interacts with the "Grandpa" object within a slide. [Figure 32] This diagram illustrates an example of a visual effect that appears when grandparents interact with the "Grandpa" object in the slide. [Figure 33] This diagram illustrates an example of a visual effect that appears when the grandchild manipulates the "dust cloud" object within the slide. [Figure 34] This diagram illustrates an example of a visual effect that appears when grandparents interact with the "dust cloud" object within the slide. [Figure 35] This diagram illustrates an example of a visual effect that appears when a grandchild interacts with the "mouse" object within the slide. [Figure 36] This diagram illustrates an example of a visual effect that appears when a grandchild manipulates the "gold coin" object within the slide. [Figure 37] This diagram illustrates the control sequence when an operation object, which is only displayed on the grandparent's device, is manipulated. [Figure 38] This diagram illustrates an example of how support information is displayed when the first operation object, which is only shown on the grandparents' device, is operated. [Figure 39] This diagram illustrates other examples of how support information appears when the first operation object, which is only displayed on the grandparents' device, is operated. [Figure 40] This diagram illustrates another example of a control sequence when an operation object, which is only displayed on the grandparent's terminal, is manipulated. [Figure 41]This diagram illustrates an example of the display of a first operation object on the grandparents' terminal resulting from the grandchild's response to a question. [Figure 42] This diagram illustrates an example of the display of visual effects when grandparents manipulate the first control object. [Figure 43] This diagram illustrates another example of a control sequence when an operation object, which is only displayed on the grandparent's terminal, is manipulated. [Figure 44] This diagram illustrates an example of the display of other visual effects when grandparents manipulate the first control object. [Figure 45] This diagram illustrates an example of the display of other visual effects when grandparents manipulate the first control object. [Figure 46] This is a sequence diagram explaining how special effects are triggered in response to the actions of the first user (e.g., a grandchild). [Figure 47] This diagram illustrates an example of an object following the movements of the first user (e.g., a grandchild). [Figure 48] This diagram illustrates another example of an object following the actions of the first user (e.g., a grandchild). [Figure 49] This is a diagram illustrating the snapshot function. [Figure 50] This is a diagram illustrating the snapshot recording process. [Figure 51] This is a diagram illustrating an example of presenting a snapshot. [Figure 52] This figure shows an example configuration of the educational system envisioned in Embodiment 2. [Figure 53] This diagram illustrates an example of the functional configuration of the management server in Embodiment 2. [Figure 54] This diagram illustrates an example of user information data. [Figure 55] This diagram illustrates an example of how to assign control rights when there are multiple grandparent terminals connected to one or more grandchild terminals. [Modes for carrying out the invention]

[0009] Embodiments of the present invention will be described below with reference to the drawings. The embodiments described later are merely examples of forms for carrying out the present invention, and the embodiments of the present invention are not limited to the embodiments described later. Therefore, the technical scope of the present invention is not limited to the scope described in the embodiments described later. For example, various modifications or improvements made to the contents described in the embodiments are also included in the technical scope of the present invention. Furthermore, the various functional components described later are implemented through the execution of programs by the processor.

[0010] <Terminology> First, let's explain the terminology used in the embodiments. The term "computer" includes not only processors as hardware, but also combinations of software programs and hardware processors. Computers may also include general-purpose computers, computers designed for specific purposes, workstations, or other systems capable of performing each type of processing.

[0011] A "processor" includes an integrated circuit that performs various processes in cooperation with a program. The processor can function as the various parts (Units) and means (Means) described in the embodiments below. The execution order of the processes performed by the processor is not limited to the order described in the embodiments and can be changed as necessary. A processor can be configured with one or more hardware components. The types of hardware that make up the processor are not limited to any particular type.

[0012] The processor can be, for example, a CPU (=Central Processing Unit), an MPU (=Micro Processing Unit), a programmable logic device such as an FPGA (=Field Programmable Gate Array), a dedicated circuit for executing specific processing such as an ASIC (=Application Specific Integrated Circuit), a GPU (=Graphics Processing Unit), or hardware such as an NPU (=Neural Processing Unit) or TPU (Tensor Processing Unit).

[0013] A processor can be a combination of multiple hardware components of different types, not just multiple hardware components of the same type. When multiple hardware components are configured to perform one or more processes of a given processor, these components may reside in physically separate devices or in the same device. Hardware is composed of circuits, which are combinations of circuit elements such as semiconductor devices.

[0014] A "program" includes the OS (Operating System), firmware, and other software such as microcode. The program may be, for example, a group of program modules. Each function constituting the group of program modules may be implemented by a processor configured to execute each function. The program in each embodiment may be program code or multiple code segments stored in one or more non-temporary computer-readable media (e.g., semiconductor memory, magnetic or optical storage media, or other storage).

[0015] A program may be divided and stored on multiple non-temporary computer-readable media located on devices that are physically separated from each other. Program code and multiple code segments can be represented as any combination of procedures, functions, subprograms, routines, subroutines, modules, software packages, classes, instructions, data structures, and program statements. Program code and multiple code segments may be connected to other code segments or hardware circuits by sending and receiving information, data, arguments, parameters, or memory contents.

[0016] "Educational content" is a form of learning support content, and includes, for example, content that supports the development of children's intelligence and cognitive abilities, as well as content that supports education. The content of educational content varies depending on the target age. For example, educational content for toddlers under 10 years old and elementary school students includes picture books, word games, number games, coloring books, and pretend play. Educational content includes content that children can use alone, content that families can enjoy together while viewing the same screen, and content that requires two-way communication between multiple users (user accounts) connected online. Educational content includes content that is streamed to users and content that is played on the user's device.

[0017] An "educational app" is a type of program that has the function of playing learning support content, and includes programs that have the function of playing educational content. An educational app may be a program dedicated to playing educational content, or it may be a general-purpose program that can play educational content. An educational app may have a function to combine and display the user's real-time images with educational content. Educational apps may have a function to synchronize the playback of educational content, etc., among online connected users. Methods for connecting multiple users over a network include, for example, the P2P (Peer to Peer) method and the client-server method. In either method, it is possible to send and receive user voice, user real-time images, and operation information in real time.

[0018] "Educational services" are a form of learning support service, and include, for example, the provision of educational content via a server, the provision of educational apps via a server, the distribution of educational content via a server, and the synchronized playback of educational content across multiple user terminals. In this specification, the synchronized playback of educational content is referred to as "content linking," and the technology for doing so is referred to as "content linking method." "Educational services" can also be included as a form of image-sharing service.

[0019] <Embodiment 1> <System Configuration> Figure 1 shows an example of the configuration of the educational system 1 assumed in Embodiment 1. The educational system 1 shown in Figure 1 includes two user terminals 10A and 10B. When user terminals 10A and 10B are not distinguished, they are referred to as user terminal 10.

[0020] The two user terminals 10 are connected via network N. Network N can be, for example, the internet, a LAN (Local Area Network), or a mobile communication system such as 4G or 5G. Network N can be a wired network, a wireless network, or a hybrid of wired and wireless networks. The educational system 1 is an example of an information processing system that includes multiple user terminals 10. The educational system 1 is used to provide educational services. The provision of educational services is achieved, for example, through the playback of educational content using an educational app.

[0021] User terminal 10A is a terminal operated by the user of the first account. User terminal 10A is an example of the first terminal device. The user of the first account and the user operating the user terminal 10A may be the same person or different. For example, if a child logs into the educational service from user terminal 10A using their own account, the user of the first account and the user operating user terminal 10A will be the same person. For example, if a parent logs into the educational service using their account, the user of the first account and the user operating user terminal 10A (e.g., the child) will be different people. In this embodiment, there is no distinction made between whether the user of the first account is a parent or a child; the user who is logged into the educational service using the first account is referred to as the "first user."

[0022] Note that the first user may also be the user of the group's representative account. The user of the group's representative account may include, for example, the group's representative or person in charge (e.g., a teacher, supporter, or instructor). Groups managed by a representative account may be based on educational institutions such as nurseries, kindergartens, and schools. Alternatively, groups may be based on classes or other components of educational institutions. Furthermore, groups may be based on members belonging to groups of educational institutions or classes, or on small groups spanning these groups. Groups may also be based on age groups. Additionally, groups may be based on sports teams or other organizations. Finally, groups may be based on families.

[0023] In this embodiment, the first user of the educational content is a single child. However, this does not prevent the first account using the educational content from having multiple users. For example, the users of the first account using the educational content may be a group of children, as described above. Also, if the educational content is learning content, the users of the first account using the educational content may include one or more adults.

[0024] User terminal 10B is a terminal operated by a user with a second account. User terminal 10B is an example of a second terminal device. The user of the second account may also differ depending on the user using the educational content. For example, the user of the second account may be the grandparents of the child operating user terminal 10A. In this case, the child is the grandchild of the grandparents.

[0025] The user of the second account and the user operating user terminal 10B may be the same person or different. For example, if grandparents log in to the educational service using their own accounts, the user of the second account and the user operating user terminal 10B will be the same person. For example, if a grandmother operates user terminal 10B that is logged in to the educational service using the grandfather's account, the user of the second account and the user operating user terminal 10B will be different people. In this embodiment, a user who is logged into the educational service using a second account is referred to as the "second user."

[0026] The grandparents mentioned earlier are an example of a second user. However, the second user could also be the parent, uncle, or aunt of the child using the educational content. Additionally, the second user may also be the user of the group's representative account. The user of the group's representative account may include, for example, the group's representative or person in charge (e.g., a teacher, supporter, or instructor). The groups managed by the representative account here may be based on educational institutions such as nurseries, kindergartens, and schools. Alternatively, the groups managed by the representative account may be based on classes or other components of educational institutions. Furthermore, the groups managed by the representative account may be based on members belonging to groups of educational institutions or classes, or on small groups spanning these groups. Additionally, the groups managed by the representative account may be based on age groups. Furthermore, the groups managed by the representative account may be based on sports teams or similar organizations.

[0027] In this specification, the first user is also referred to as "the other user," "the child," "the user receiving support," "the user receiving protection," "the student," etc., in contrast to the second user. Conversely, the second user is sometimes referred to as "the other user," "the adult," "the supporting user," "the protective user," or "the teacher," in contrast to the first user.

[0028] In this embodiment, the user who requests a connection with the other user through an educational app that plays educational content is referred to as the first user, and the user who is requested to connect is also referred to as the second user. In this embodiment, the roles between two users communicating through the educational app are determined in advance. Specifically, when a grandchild and grandparents use the educational system 1, the user inviting other users (e.g., the grandchild's parent or the grandchild themselves) becomes the first user, and the user receiving the invitation (e.g., the grandchild's grandparents) becomes the second user. After the roles are determined, the first user can call the second user from the educational app, but the second user can no longer call the first user from the educational app in relation to the first user.

[0029] User terminals 10A and 10B, shown in Figure 1, are connected via P2P through an educational app and play the same educational content in sync. The user terminal 10 may be, for example, a smartphone, a tablet computer, a notebook or desktop computer, a game console, a head-mounted device (so-called headset), or a glasses-type device (so-called smart glasses). Incidentally, smart glasses are glasses-type devices that include a small display and a light guide component that focuses the image displayed on the display onto the retina. In the case of Figure 1, there are two user terminals 10, but there may be three or more.

[0030] <Device Hardware Configuration> The user terminal 10 includes a processor 11 for running educational applications, a semiconductor memory 12, an auxiliary storage device 13, a touch panel 14, a camera 15, a microphone 16, a speaker 17, and a communication interface 18. The processor 11 and each device are connected to each other via buses and other signal lines. The processor 11 is, for example, a CPU. The semiconductor memory 12 includes ROM (Read Only Memory) on which UEFI (Unified Extensible Firmware Interface), etc., is recorded, and RAM (Random Access Memory) used as the work area of ​​the processor 11.

[0031] The auxiliary storage device 13 is, for example, a hard disk drive or a semiconductor storage device. The auxiliary storage device 13 stores game programs, various application programs, and data related to the progress of games and various applications. The touch panel 14 is a device in which, for example, a light-transmitting contact detection sensor is placed on the surface of a display. The display may be, for example, a liquid crystal display or an organic EL (=Electro-Luminescence) display. For the contact detection sensor, for example, a capacitive touch sensor may be used.

[0032] The camera 15 is an imaging device, and for example, a CMOS (Complementary Metal Oxide Semiconductor) image sensor is used. The camera 15 may be integrated with the main body of the user terminal 10, or it may be attached externally to the main body of the user terminal 10. The microphone 16 is a device that converts the voice of the user operating the user terminal 10 into an electrical signal. Speaker 17 is a device that converts audio data into sound. The communication interface 18 is a device that enables communication with terminals connected to the network N (for example, other user terminals 10). The communication interface 18 is required to have communication functions that are compatible with the network N used for communication.

[0033] <User terminal functional configuration> Figure 2 illustrates an example of the functional configuration of the user terminal 10 in Embodiment 1. Note that the functional configuration shown in Figure 2 is an example of a functional unit realized through program execution. The user terminal 10 includes a control unit 101 and a storage unit 110. The storage unit 110 is implemented by a semiconductor memory 12 (see Figure 1) and an auxiliary storage device 13 (see Figure 1).

[0034] The control unit 101 operates as various functional units, as described later, through the execution of the educational application 111. Specific examples of the functional units realized by the control unit 101 through the execution of the educational application 111 will be explained as needed. The memory unit 110 shown in Figure 2 stores the educational application 111 and the educational content 112. Needless to say, the educational application 111 and the educational content 112 are just examples, and this does not prevent other data (for example, data captured by the camera 15 or audio data picked up by the microphone 16 (see Figure 1)) from being stored.

[0035] Figure 3 illustrates an example of the structure of educational content 112. The educational content 112 used in this embodiment consists of multiple scenes. These scenes are sometimes referred to as pages. The educational content 112 shown in Figure 3 consists of "Scene 1," "Scene 2," "Scene 3," "Scene 4," ... "Scene N-1," and "Scene N," which switch sequentially as time progresses. The aforementioned control unit 101 (see Figure 2) controls the execution of educational content, which consists of multiple scenes, through the playback of educational content by the educational app.

[0036] Each scene contains one or more slides. In this embodiment, slides may also be referred to as sequences or timelines. For example, "Scene 1" contains n slides. A slide is, for example, a still image and is also called a frame. In each scene, the story may unfold through one or more slides. The unfolding of the story through a single slide is expressed by the appearance and movement of characters, other text, images, etc., within the slide. Text and images are examples of components of a slide. The story unfolds through multiple slides, and this is represented by the animation playback of multiple slides. In other words, multiple slides are automatically switched and displayed on the screen at predetermined time intervals.

[0037] Scene switching is performed at the user's instruction. In this embodiment, only grandparents (users of the second account) can switch scenes. Scene switching is one example of controlling the progress of educational content. Until the user switches from the current scene to the previous or next scene, for example, the display of the last slide of the current scene may continue, or the multiple slides that make up the current scene may be played repeatedly in order from the beginning. In other words, the multiple slides that make up a scene may be played in a so-called loop.

[0038] Figure 4 illustrates an example of a slide. Slide SLD1 shown in Figure 4 represents a scene from a picture book content intended for read-aloud sessions. Slide SLD1 may include, for example, a background, an object, a camera image, and text. The background is a still image representing a scene from a picture book. Objects are elements other than the background that make up a scene in a picture book. The slide shown in Figure 4 includes the objects of "clouds," "rice ball," "grandmother," "grandfather," "beard," and "hat."

[0039] Each object is independent. Objects may be assigned movements that are appropriate to the scene. For example, a "cloud" object may be assigned a movement (animation) that moves in one direction within a slide (for example, from left edge to right edge) within a single scene. An animation refers to a continuous movement observed, for example, across one or more slides. The movement appears automatically based on pre-configured settings, independent of user interaction.

[0040] For example, the "rice ball" object may be assigned movements such as "shaking up and down in small increments" or "rolling." However, assigning movements is not mandatory; the "rice ball" object in a given scene may not have any movements assigned to it. In other words, in a given scene, the "rice ball" object may remain stationary. For example, you could assign a movement such as waving to each of the "grandmother" and "grandfather" objects.

[0041] The "beard" and "hat" objects are decorative objects that are composited onto the first user's real-time image. For example, the "beard" and "hat" objects are one of the mechanisms to enhance the first user's (in this case, the grandchild's) immersion in the educational content. In Figure 4, the "beard" and "hat" objects are provided in the educational content as distinctive parts of the "grandfather" object that appears in the content. Note that the shape, size, and color of the objects composited onto the first user may be the same as the objects that appear in the target scene, but they may also be different, as shown in Figure 4.

[0042] Furthermore, objects for the presentation may be prepared for each scene. For example, distinctive parts of a "mouse" object may be prepared. Specifically, objects for "mouse ears," "mouse nose," and "mouse whiskers" may be prepared. For example, in a scene in which a mouse appears, the first user may also be displayed in a mouse costume, which can enhance the first user's immersion in the educational content.

[0043] These decorative objects are displayed through the functions of the educational app. One of the display functions for visual effects is the ability to superimpose specific visual effects (for example, an "umbrella" object or a "raincoat" object in a rain scene) onto the real-time image of the first user (for example, a grandchild). One of the display functions for visual effects is to superimpose specific visual effects (such as a "hat" object and a "beard" object) associated with a specific object that appears in educational content (such as a "grandfather" object) onto the real-time image of the first user (e.g., a grandchild). Alternatively, an object with the same "clothing" as the "grandfather" object could be superimposed onto the real-time image of the first user (e.g., a grandchild).

[0044] Another feature of the display function for the visual effects is the ability to change the display size of specific visual effects (for example, the "hat" object and the "beard" object) according to the display size of the first user's (for example, the grandchild's) real-time image. In slides SLD1 and SLD1A (see Figure 10), which will be discussed later, the display sizes of the "hat" object and the "beard" object also change in conjunction with the size of the first user.

[0045] For example, if the display size of the first user's real-time image within the slide decreases, the display sizes of the "hat" object and the "beard" object will also decrease proportionally. Conversely, if the display size of the first user's real-time image within the slide increases, the display sizes of the "hat" object and the "beard" object will also increase proportionally. As a result, even if the distance between the user terminal 10A and the first user (e.g., grandchild) changes, the resulting image after compositing the visual effects objects will not appear unnatural. These features can enhance the immersion of first users (e.g., grandchildren) into educational content.

[0046] Objects may incorporate mechanisms (i.e., visual effects) that respond to user actions (e.g., taps, swipes). The content of these visual effects may be changed depending on the user performing the action. For example, if the operation is performed by a child (the user of the first account), a special visual effect (second visual effect) prepared for the object being operated on may be displayed. The special visual effect may include, for example, the appearance of one or more objects or animations, or the generation of sound effects.

[0047] On the other hand, when the operation is performed by grandparents (users of the second account), a mechanism may be adopted that does not display special visual effects (second visual effects). Furthermore, when the operation is performed by grandparents, different types of visual effects may be displayed than when the child is performing, or the same types of visual effects may be used, but the size of the objects associated with the effects may be smaller than when the child is performing, and the speed of movement and changes of the objects associated with the effects may be slower than when the child is performing. If the grandparents (users of the second account) manipulate a specific object (the first operation object), the same visual effect (the first visual effect) may be displayed on both the grandchild's terminal and the grandparent's terminal.

[0048] For animations, you might prepare both dedicated animations (also called gimmicks) and general-purpose animations. Specialized animations include, for example, the automatic execution of animations prepared only for specific objects. Specialized animations also include, for example, effect effects. Effect effects include, for example, special movements of objects (e.g., waving hands, scaling up and down, changing color, changing brightness), the appearance of new objects on the screen, and the disappearance of predetermined objects. General-purpose animations include, for example, movement in one direction. The text includes, for example, the narrative text of a picture book or story. However, the text may also include, for example, dialogue, conversations, and explanatory text.

[0049] The camera image includes a real-time image of the first user and a real-time image of the second user. In this embodiment, communication between the first user and the second user is initiated by a connection request from the first user to the second user. More specifically, it is initiated when the second user responds to the connection request from the first user. In this case, the real-time image of the second user (e.g., grandparents) is displayed in a rectangular area fixedly positioned in the upper left corner of each slide. In this embodiment, the position and size of this rectangular area are the same for all slides. In this embodiment, the rectangular area displays the central portion of the image (an image of a predetermined size) remaining after cropping the left and right sides of the camera image, including the background of the second user. The predetermined size is given, for example, with an aspect ratio of 1:1. Note that the predetermined size image is displayed on the slide in a reduced size. The rectangular area reserved in the upper left corner of each slide is an example of a specific area for educational content. In other words, this specific area is located in a position common to multiple slides (in this case, the upper left corner).

[0050] In this embodiment, the size of the rectangular area is fixed. Therefore, even if the second user moves away from the camera 15 and the size of the subject in the camera image decreases, the size of the rectangular area reserved within the slide does not change. Consequently, as the size of the subject decreases, the area of ​​the background within the rectangular area increases accordingly. Conversely, even if the second user moves closer to camera 15 and the size of the subject increases, the size of the rectangular area reserved within the slide does not change. Therefore, as the size of the subject increases, the area of ​​the background within the rectangular area decreases accordingly. When the distance between the second user and the camera changes, the ratio of the second user's display may be enlarged or reduced so that the size of the subject is displayed within a predetermined range within the rectangular area.

[0051] On the other hand, the real-time image of the first user (e.g., grandchild) is displayed in a designated area for each scene (or slide). This designated area for each scene is set to exclude the aforementioned specific area. The designated area for each scene is an example of a predetermined area in educational content. In this embodiment, the area determined for each scene is pre-set for each educational content as an area that does not obstruct or has little impact on the display of the aforementioned objects such as "clouds," "rice balls," "grandmother," and "grandfather." The first user's real-time image is preferably placed in a predetermined area set during content playback, taking into account the movements assigned to the objects as part of the performance, or the performance effects prepared for the objects. In other words, the display position of the first user's real-time face image is determined in relation to the story development of the picture book. In Figure 4, a real-time image of the grandchild (with a hat and beard added as a visual effect) is displayed in the background to the right of "Grandpa".

[0052] In Figure 4, the rectangular area (hereinafter referred to as the "maximum range") that gives the maximum size of the area used to display the first user's real-time image is shown by a dashed line. The size of the maximum range may be the same for all slides of the educational content, or it may be set for each scene (or each slide). In this embodiment, as shown in Figure 4, the outer edge of the first user's real-time image is not rectangular, but is defined by the contour of the first user (i.e., the contour of the human body) captured in the camera image. By making the outer edge of the first user's real-time image a shape that follows the contour of the first user, the first user's sense of immersion in the educational content can be enhanced.

[0053] This is because, like the "grandmother" object and the "grandfather" object, they are observed as if they were part of the scene. Note that the layout of the predetermined area and the specific area shown in Figure 4 is the same not only for the user terminal 10A operated by the first user (e.g., grandchild), but also for the user terminal 10B operated by the second user (e.g., grandparent). In this embodiment, the layout of the predetermined area and the specific area is the same whether communication with the second user is initiated by a connection request from the first user, or by receiving a connection request from the first user.

[0054] Figure 5 illustrates the synthesis process of the first user's real-time image 151A. First, the processor 11 (see Figure 1) extracts the contour 152 of the first user (e.g., grandchild) from the camera image 151 which includes the first user as the subject. In other words, the processor 11 extracts the contour 152 of the human body. If even a part of the human body is visible, the outline 152 of the human body will be extracted. Therefore, even if only a hand is visible and the face is not, the outline 152 of the hand will be extracted.

[0055] If no part of a human body is visible in the camera image 151, the extraction of the contour 152 will not be performed. An existing algorithm will be used to extract contour 152. Contour 152 will target areas such as the head, hands, and chest. The human body outline 152 also includes clothing, glasses, accessories, etc., worn by the first user. Next, the processor 11 extracts the image portion enclosed by the contour 152 from the camera image 151 and composites it onto the slide as the first user's real-time image 151A. Alternatively, the processor 11 masks the area outside the contour 152 with the slide's background image.

[0056] Figure 6 illustrates another compositing process for the first user's real-time image 151A. In the compositing process described in Figure 5, the boundary between the first user's real-time image 151A and the background image, etc., is clear. However, in Figure 6, the outer edge of the first user's (e.g., grandchild's) real-time image 151A is blurred. By blurring the outer edge, the boundary between the first user's real-time image 151A and the background image becomes less distinct. As a result, the sense of unity between the first user's real-time image 151A and the slide is enhanced, further increasing the first user's immersion in the educational content.

[0057] Figure 7 illustrates the positional relationship in the depth direction between the background and objects when compositing the first user's real-time image 151A onto a slide. If the display area of ​​an object such as a background (hereinafter also referred to as the "second object") 151B, which is not associated with any special effects, overlaps with a predetermined area where the first user's real-time image 151A is displayed, the processor 11 places the first user's real-time image 151A in front of the second object 151B, such as the background. Furthermore, if the display area of ​​the "grandfather" object (first object) 151C, to which movement and visual effects are associated, overlaps with a predetermined area where the first user's real-time image 151A is displayed, the processor 11 places the first user's real-time image 151A behind (towards the back of) the "grandfather" object 151C.

[0058] Figure 8 illustrates an example of display when the display area of ​​the "grandfather" object 151C and the display area of ​​the first user's real-time image 151A overlap. Figure 8 is denoted with corresponding reference numerals for parts that correspond to those in Figure 7. In Figure 8, the left hand of the "grandfather" object overlaps with the right hand of the first user's (e.g., grandchild's) real-time image. In this case, as shown in the enlarged view using the speech bubble, the left hand of the "grandfather" object 151C is displayed in front of the right hand of the first user's real-time image 151A. In other words, the right hand of the first user's real-time image 151A is displayed behind the left hand of the "grandfather" object 151C. This positional relationship satisfies the positional relationship described in Figure 7.

[0059] In this embodiment, if possible, a portion of the outer edge of the real-time image 151A of the first user (e.g., grandchild) displayed in a predetermined area is positioned to overlap with the display area of ​​the "grandfather" object 151C, etc. (i.e., the first object to which the performance effect is associated). By placing a portion of the outer edge of the first user's real-time image 151A behind an object 151C of a "grandfather," which is one of the components of the educational content, the first user can be made to feel as if they have become part of the educational content. As a result, the first user's sense of immersion in the educational content can be enhanced.

[0060] Figure 9 illustrates the change in the position where the real-time image 151A of the first user (e.g., grandchild), which is determined for each scene (or slide), is placed. Figure 9 shows, as an example, three scenes that appear in chronological order (i.e., Scene P, Scene Q, and Scene R). In scenes P and Q shown in Figure 9, the first user's real-time image 151A is displayed to the right of the "grandfather" object 151C.

[0061] However, in the same area of ​​Scene R shown in Figure 9, an object 151D of a "hole" from which the "rice ball" object rolls is displayed. This "hole" object 151D is essential for the story's progression. Therefore, in Scene R, the first user's real-time image 151A is displayed in the background area to the left of the "grandfather" object 151C. Thus, the predetermined area (indicated by the dotted line) in which the first user's real-time image 151A is displayed is determined according to the content of the scene (or slide).

[0062] In other words, the arrangement of a predetermined area for displaying the first user's real-time image 151A is determined so as not to interfere with the display of the first object, such as the "hole" object 151D to which the visual effects are associated. The location information (e.g., coordinates within a slide) of the area where the determined first user's real-time image 151A is displayed is stored in association with each scene (or slide) of the educational content.

[0063] Figure 10 illustrates the change in the display size of the real-time image 151A for the first user (e.g., grandchild). Figure 10 is denoted with corresponding reference numerals for parts that correspond to those in Figure 9. The three scenes shown in Figure 10 all correspond to Scene P. Since the first user's real-time image 151A is cropped along the contour of the first user, the display size of the first user superimposed on the slide changes depending on the size of the first user captured in the camera image.

[0064] For example, when the distance from camera 15 (see Figure 1) to the first user is far, the display size of the first user's real-time image 151A will be smaller than when the distance from camera 15 to the first user is close. For example, the second row of the first user's real-time image 151A from the top in Figure 10 is smaller than the first row of the first user's real-time image 151A from the top in Figure 10. If the camera image captured by camera 15 does not include the first user, only images of educational content such as the background will be displayed in the designated area. For example, the third image from the top in Figure 10 does not display the first user's real-time image 151A. However, as indicated by the dotted line, the designated area is still reserved.

[0065] <Processing Sequence> The following describes an example of a processing sequence performed by the educational system 1 (see Figure 1). The following describes the processing sequences performed by the educational system 1 according to their purpose.

[0066] <Role setting sequence> Figure 11 illustrates an example of a role assignment sequence between users. In Figure 11, each step is indicated by the symbol S. In this embodiment, a user's (or user account's) role is determined by prior settings, and the set role is maintained until a new role is set. In this embodiment, a user (or user account) can only be assigned one role. In other words, a user (or user account) cannot have multiple roles simultaneously.

[0067] The processing of user terminals 10A and 10B shown in Figure 11 is realized through the execution of programs by the control unit 101 (see Figure 2) of each terminal. The role setting sequence starts, for example, from user terminal 10A operated by a grandchild's parent. First, the user terminal 10A accepts the request to launch the educational app (step 101). Next, user terminal 10A accepts the designation of the user to be invited (step 102). In this embodiment, potential invitees are presented based on information about family users, etc., managed by a server (not shown) that manages the provision of educational services. If there is no registered information about family users, for example, a screen for registering users to invite will be displayed on the operation screen.

[0068] Upon receiving the request from the other user, user terminal 10A sends a message containing the invitation URL (=Uniform Resource Locator) to the other user (step 103). The user terminal 10B operated by the other user (for example, the second user) receives a message containing the invitation URL (step 104). User terminal 10B accepts an operation on the invitation URL and launches the educational app (step 105).

[0069] If a user who has received an invitation URL already has other users linked to it, the user terminal 10B will be notified that there are other users already linked (users different from the first user), and that in order to link with the first user who sent the invitation, the user must first unlink their existing users. Furthermore, if the inviting first user already has other users linked to it, the user terminal 10B will be informed that the inviting first user already has other users linked to it, and that the first user will need to disconnect their existing connections in order to accept the invitation. In either presentation, an error screen, for example, might be used.

[0070] If the second user accepts the invitation from the first user and there are no issues with the communication between the first and second users, user terminal 10B sends a reply to user terminal 10A confirming acceptance of the invitation (step 106). After this, the user terminal 10B operated by the second user begins coordinating with the user terminal 10A operated by the first user (step 107), and sets the role of the terminal to be used when communicating with the partner (i.e., the first user) with the terminal it is coordinating with (step 108). In the case of Figure 11, the educational app running on user terminal 10B sets the terminal to indicate that it is an invited terminal. In other words, the educational app running on user terminal 10B sets the terminal to be a so-called grandparent terminal.

[0071] Meanwhile, user terminal 10A, which sent the message containing the invitation URL, receives an acceptance of the invitation from user terminal 10B (step 109). However, user terminal 10A may also receive a rejection of the invitation from user terminal 10B. Rejection of an invitation includes not only an explicit rejection, but also a request to terminate the existing connection with user terminal 10A, the sender of the invitation.

[0072] If the invitation is accepted, the user terminal 10A operated by the first user begins coordinating with the user terminal 10B operated by the second user (step 110), and sets the role of the user terminal to be used for communication with the partner (i.e., the second user) that it is coordinating with (step 111). In the case of Figure 11, the educational app running on user terminal 10A sets its own terminal to indicate that it is the inviting terminal. In other words, the educational app running on user terminal 10A sets its own terminal to be a so-called grandchild terminal.

[0073] <Connection Sequence> Figure 12 illustrates an example of a connection sequence after role configuration. In Figure 12, the symbol S also indicates a step. In the case of Figure 12, the user terminal 10A operated by the grandchild launches the educational app (step 121). Note that the launch of the educational app by the user terminal 10B operated by the grandparents may occur before the launch of the educational app by the user terminal 10A operated by the grandchild. The user terminal 10A shown in Figure 12 accepts a call operation (step 122). As will be described later, the call operation is accepted through the home screen of the educational app.

[0074] The user terminal 10A, upon receiving the call operation, requests a connection to a pre-configured recipient (step 123). The call operation is accepted, for example, by pressing the call button provided on the home screen. In this case, the recipient is the grandparents, who are set as the connection partner. In this embodiment, there is only one user account that defines the connection partner. For example, a VoIP (Voice over Internet Protocol) push notification is used to request a connection to the user terminal 10B.

[0075] In the case of Figure 12, the user terminal 10B operated by the grandparents displays a connection request notification screen on the touch panel 14 (see Figure 1) (step 124). Next, the user terminal 10B launches the educational app after being operated by the grandparents (step 125). If the educational app is already running when the connection request is received, step 125 is skipped. When user terminal 10B receives a response from the grandparents, a connection is established between user terminal 10A and user terminal 10B (step 126). Subsequently, user terminal 10A and user terminal 10B each display their home screens (steps 127 and 128).

[0076] Figure 13 illustrates examples of the home screens 140A and 140B displayed before connection is initiated. Home screen 140A is displayed on user terminal 10A operated by the grandchild. Home screen 140B is displayed on user terminal 10B operated by the grandparents. After the educational app is launched, the processor 11 (see Figure 1) acquires the role of its own device and determines the content of the messages to be displayed on the home screens 140A and 140B, the available function buttons, etc., according to the acquired role.

[0077] The home screen 140A shown in the upper part of Figure 13 includes the invited username 140A1, an image 140A2 of the user operating the user terminal 10A, a message field 140A3, a "Schedule" button 140A4, a "Play" button 140A5, and a "Call" button 140A6. In Figure 13, the invited username 140A1 displays the display name of the user account registered with the educational service. In Figure 13, "Mago A" is displayed.

[0078] The user's image 140A2 may display, for example, an image of a grandchild that has been pre-registered in the educational app. However, the user's image 140A2 may also display a real-time image captured by the camera 15 (see Figure 1) of the user terminal 10A. The message field 140A3 displays a message for the user operating the user terminal 10A. In Figure 13, "Good morning, Grandchild A!" is displayed in the message field 140A3.

[0079] The "Schedule" button 140A4, the "Play" button 140A5, and the "Call" button 140A6 are all in an operable state. Therefore, each button is displayed in color, for example. The "Schedule" button (140A4), the "Play" button (140A5), and the "Call" button (140A6) are intended for use by children (including infants). Therefore, they use hiragana characters, which are taught in elementary education. If the characters on each button are kanji, they may also have phonetic readings. Additionally, each button may use child-friendly language. Furthermore, each button may be accompanied by pictograms or gestures that are easy for infants to understand.

[0080] In this embodiment, the only button that is not restricted in terms of operation by children is the "Call" button 140A6. On the other hand, when the "Schedule" button 140A4 or the "Play" button 140A5 is operated, a screen requesting adult operation will be displayed, for example. This display is realized by a function that displays a screen on the user terminal 10A operated by the first user to confirm adult operation in response to a predetermined operation by the first user. The operation of the "Schedule" button 140A4 and the operation of the "Play" button 140A5 are examples of "predetermined operations". In this embodiment, the "Schedule" button 140A4 and the "Play" button 140A5 are assigned functions related to user settings.

[0081] Figure 14 illustrates an example of how the invitation screen 141A and the confirmation screen 142A are displayed. The invitation screen 141A is displayed when the "Schedule" button 140A4 (see Figure 13) is clicked, but there is no linked partner associated with the "Call" button 140A6 (see Figure 13). The invitation screen 141A contains a message 141A1, the message body 141A2, a "Share invitation link" button 141A3, and a "Back" button 141A4.

[0082] Message 141A1 and the "Back" button 141A4 are written entirely in hiragana so that even children can read them. On the other hand, the message body 141A2 and the "Share Invitation Link" button 141A3 are intended for adults to read. Therefore, they contain text that includes kanji characters. Message body 141A2 displays, for example, "No linked user information is found," and "Let's invite your family to [App Name]."

[0083] The confirmation screen 142A is displayed when the "Play" button 140A5 (see Figure 13) is pressed and there is a linked partner associated with the "Call" button 140A6 (see Figure 13). The confirmation screen 142A contains a message 142A1, a question 142A2, a progress gauge 142A3, and an answer button 142A4. Message 142A1 is written entirely in hiragana so that even children can read it.

[0084] On the other hand, question 142A2 is intended for adult readers. Therefore, the instructions for the user's actions in question 142A2 are written in a sentence that includes kanji characters. In the case of Figure 14, it says, "Please press the smallest kanji number." Progress gauge 142A3 shows the progress of the review process by the length of the gauge. This is because reviewing only one question may result in a child's answer being correct by chance. The answer button 142A4 displays the answer choices for the question. In Figure 14, the labels of the five buttons display Chinese numerals.

[0085] Figure 15 illustrates another example of the display of confirmation screen 143A. The confirmation screen 143A is displayed when the "Play" button 140A5 (see Figure 13) is pressed, but there is no linked partner associated with the "Call" button 140A6 (see Figure 13). The confirmation screen 143A contains a message 143A1, the message body 143A2, a warning 143A3, and a "Continue" button 143A4. Of these, message 143A1 and the "Continue" button 143A4 are written entirely in hiragana so that even children can read them.

[0086] On the other hand, the main message text 143A2 and the warning text 143A3 are intended for adults to read. For this reason, they contain sentences that include kanji characters. Message body 143A2 displays, for example, "No linked user information found," and "Do you wish to continue?" Note 143A3 states, "We recommend that you share the invitation URL via Options → Issue Invitation URL and play with your family." If a user already has a partner (for example, grandparents), the game selection screen or picture book selection screen will be displayed on the user terminal 10A.

[0087] Returning to the explanation of Figure 13. The home screen 140B shown in the lower part of Figure 13 includes the invited username 140B1, an image 140B2 of the user operating the user terminal 10B, a message field 140B3, a "Schedule" button 140B4, a "Play" button 140B5, and a "Call" button 140B6. In Figure 13, the invited username 140B1 displays the display name of the user account registered with the educational service. In Figure 13, "Mago A" is displayed.

[0088] The user's image 140B2 may display, for example, an image of grandparents pre-registered in the educational app. However, the user's image 140B2 may also display a real-time image captured by the camera 15 (see Figure 1) of the user terminal 10B. On the home screen, the user of the second account may also be displayed in a form that appears to be extracted along the contours of the human body. The message field 140B3 displays a message for the user operating the user terminal 10B. In Figure 13, "Good morning, Grandma Y!" is displayed in the message field 140B3.

[0089] The "Schedule" button 140B4 and the "Play" button 140B5 are in an operable state. For this reason, each button is displayed in color, for example. However, the "Call" button 140B6 is in an unselectable state. For this reason, in Figure 13, the "Call" button 140B6 is grayed out. This is because only the grandchild terminal has the role set to request a connection. In other words, the grandparent terminal can only be a recipient of connection requests. The "Schedule" button (140B4), the "Play" button (140B5), and the "Call" button (140B6) are operated by adults. Therefore, they use notation that includes kanji characters.

[0090] <Setup sequence immediately after connection> Figure 16 illustrates the configuration sequence that is performed immediately after connection is initiated. In Figure 16, the symbol S also indicates a step. The processing operations shown in Figure 16 are executed on user terminals 10A and 10B (see Figure 1) respectively through the execution of the educational application. Hereafter, user terminals 10A and 10B will not be distinguished and will be referred to simply as user terminal 10. The processor 11 of the user terminal 10 (see Figure 1) determines whether its own terminal is a grandchild terminal or not (step 131).

[0091] In this embodiment, the grandchild terminal refers to the terminal operated by the first user who requested a connection to the grandparents through the educational app. As mentioned above, the role of the user terminal 10 on the educational app is set in advance. If the user's own device is a grandchild device, a positive result is obtained in step 131. In this case, the user device 10 sets the display of buttons on the home screen to that of a child (step 132A). On the other hand, if the user's device is the grandparent's device, a negative result is obtained in step 131. In this case, the user device 10 sets the display of buttons on the home screen to adult settings (step 132B).

[0092] Figure 17 illustrates examples of the home screens 140C and 140D displayed after connection is established. Figure 17 is denoted with corresponding reference numerals for parts that correspond to those in Figure 13. In the case of home screen 140C, which is compatible with the grandchild's device, only the "Play" button 140A5 is displayed as active (operable), while the "Schedule" button 140A4 and the "End" button 140A7 are displayed as inoperable. Specifically, the "Schedule" button 140A4 and the "End" button 140A7 are displayed in a grayed-out state. This indicates that the only button that the grandchild, as the first user, can operate is the "Play" button 140A5.

[0093] On the home screen 140D, which is compatible with the grandparents' device, the "Play" button 140B5 and the "End" button 140B7 are displayed as active (operable), while the "Schedule" button 140A4 is displayed as inoperable. Specifically, the "Schedule" button 140B4 is displayed in a grayed-out state. This indicates that the grandparents, as the second user, can only operate the "Play" button 140B5 and the "End" button 140B7, and the "Schedule" button 140B4 cannot be operated.

[0094] The state of these buttons is due to the fact that, after a P2P connection is established between the grandchild's terminal (user terminal 10A) and the grandparent's terminal (user terminal 10B) and communication becomes possible, the control over the progress of the educational app is granted to the adult grandparent. In fact, when the grandparents tap the "Disconnect" button 140B7 (perform the disconnection operation), information instructing the grandchild to disconnect the connection is sent to the grandchild's device, and the connection between the grandchild's device and the grandparent's device is severed. On the other hand, the "Yotei" button 140A4 and the "Yotei" button 140B4 are inoperable on both terminals. This is because these buttons are used when not making a call.

[0095] Returning to the explanation of Figure 16. Next, the grandchild terminal (user terminal 10A) sets a specific area of ​​the educational content as the output area for the user image operating another terminal (i.e., the grandparent terminal) (step 133A). Similarly, the grandparent terminal (user terminal 10B) sets a specific area of ​​the educational content as the output area for the user image operating its own terminal (step 133B). In this embodiment, as explained in Figure 4, a predetermined size in the upper left corner of each slide constituting the educational content is set as a specific area for displaying the grandparent's image.

[0096] Next, the grandchild terminal (user terminal 10A) assigns a user image of itself operating its terminal to a predetermined area that is set for each scene of the educational content (step 134A). Similarly, the grandparent terminal (user terminal 10B) assigns a user image of itself operating another terminal (i.e., the grandchild terminal) to a predetermined area that is set for each scene of the educational content (step 134B). The predetermined areas within each slide are set in advance for each piece of educational content. Therefore, in this processing step, the assignment of user images to be displayed in the predetermined areas is set according to the role of the terminal. For example, as shown in Figure 9, the predetermined areas change for each scene.

[0097] Furthermore, the grandparent terminal (user terminal 10B) sets the placement of buttons for controlling the progress within the educational content (step 135). Specifically, the processor 11 (see Figure 1) of the grandparent terminal (user terminal 10B) displays a first operation object that controls the progress of the educational content in each scene of the educational content displayed on its terminal. Incidentally, the grandchild terminal (user terminal 10A) does not have the processing corresponding to step 135. For this reason, the processor 11 (see Figure 1) of the grandchild terminal (user terminal 10A) does not display the first operation object that controls the progress of the educational content in each scene of the educational content displayed on its own terminal.

[0098] Figure 18 illustrates an example of a picture book content selection screen. Picture books are an example of educational content. Screen 140E is the screen displayed on the grandchild terminal (user terminal 10A). Screen 140F is the screen displayed on the grandparent terminal (user terminal 10B). The basic screen layout is the same for both the grandchild's device and the grandparent's device. For example, real-time images of the grandparents, 140E1 and 140F1, are displayed in the upper left corner (i.e., a specific area) of screens 140E and 140F. This display is performed based on the settings in steps 133A and 133B (see Figure 16).

[0099] Furthermore, the real-time images of the grandchild, 140E2 and 140F2, are displayed in the center (i.e., a predetermined area) of screens 140E and 140F. This display is performed based on the settings in steps 134A and 134B (see Figure 16). As explained using Figure 5, the real-time images of the grandchild, 140E2 and 140F2, are displayed in a way that they appear to be extracted along the contours of the human body. Furthermore, as explained using Figures 7 and 8, the grandchild's real-time images 140E2 and 140F2 are positioned in front of the background, but behind the "picture book" objects 140E3 and 140F3.

[0100] On the grandparents' terminal (user terminal 10B), screen 140F displays a message field 140F4, a "Confirm" button 140F5, and a "Back" button 140F6. These are information or controls that manage the progress of the educational app, presented only to the grandparents. This display is executed based on the settings in step 135 (see Figure 16). The "Confirm" button 140F5 and the "Back" button 140F6 are examples of the first operation object. Note that screen 140F of the grandparents' terminal (user terminal 10B) shown in Figure 18 is the screen after one picture book content has been selected from the available options. Therefore, the "Confirm" button 140F5, which instructs the user to confirm their selection, is displayed.

[0101] Therefore, when no picture book content is selected to play, the screen may display multiple picture book content objects and a "Select" button. The select button is also an example of a first operation object. In Figure 18, the message box 140F4 reads "Please choose a picture book," indicating that the selection of a picture book has not yet been finalized. The message box 140F4 displays progress advice for the grandparents viewing the screen.

[0102] The "Confirm" button 140F5 is used to confirm the selection of a picture book and to instruct playback to begin. When the "Confirm" button 140F5 is pressed, playback of the selected picture book content will begin on both the grandchild's device and the grandparent's device. The "Back" button 140F6 is used to instruct the user to switch to a screen where they can select another picture book. When the "Back" button 140F6 is pressed, both the grandchild's device and the grandparent's device switch to the picture book selection screen. The "Back" button 140F4 and the "Confirm" button 140F5 here are both examples of the first operation object.

[0103] Figure 19 illustrates an example of how a scene from picture book content is displayed. Screen 140G is the screen displayed on the grandchild terminal (user terminal 10A). Screen 140H is the screen displayed on the grandparent terminal (user terminal 10B). The basic screen layout is the same for both the grandchild's device and the grandparent's device. For example, real-time images of the grandparents, 140G1 and 140H1, are displayed in the upper left corner (i.e., a specific area) of screens 140G and 140H. This display is performed based on the settings in steps 133A and 133B (see Figure 16).

[0104] To the right of the "grandfather" object on screens 140G and 140H (i.e., in a predetermined area), the grandchild's real-time images 140G2 and 140H2 are displayed. This display is performed based on the settings in steps 134A and 134B (see Figure 16). As explained using Figure 5, the real-time images of the grandchild, 140G2 and 140H2, are displayed in a way that they appear to be extracted along the contours of the human body. In Figure 19, only the screen 140H of the grandparent's terminal (user terminal 10B) displays the "back" button 140H3 and the "forward" button 140H4.

[0105] This display is performed based on the settings in step 135 (see Figure 16). In Figure 19, the "back" button 140H3 is located in the lower left corner of the screen, and the "forward" button 140H4 is located in the lower right corner of the screen. These are controls that manage the progress of an educational app presented only to grandparents. The "Back" button 140H3 and the "Forward" button 140H4 are examples of the first operation object. These buttons need to be positioned so as not to interfere with viewing the picture book content. Furthermore, these buttons need to be displayed in an operable manner. Therefore, in Figure 19, the "Next" button 140H4 is displayed in front of the grandchild's real-time image 140H2. The first manipulation object may be displayed in front of the object to which the visual effect is associated, that is, it may be placed at the very front of the superimposed images in the content slide. Alternatively, it may be placed in essentially the same position across multiple slides that make up a single piece of content.

[0106] Figure 20 illustrates an example of how other scenes from the picture book content are displayed. Screen 140I is the screen displayed on the grandchild terminal (user terminal 10A). Screen 140J is the screen displayed on the grandparent terminal (user terminal 10B). In the case of screens 140I and 140J shown in Figure 20, the real-time images of the grandparents 140I1 and 140J1 are displayed in the upper left corner area of ​​the screen (i.e., a specific area). However, the grandchild's real-time images 140I2 and 140J2 are displayed in the left-hand area of ​​the screen (i.e., a designated area).

[0107] In Figure 20, the "Back" button 140J3 and the "Forward" button 140J4 are displayed only on the grandparent's terminal. The "Back" button 140J3 and the "Forward" button 140J4 are examples of first operation objects. These buttons need to be displayed in an operable manner. For this reason, in Figure 20, the "Back" button 140J3 is displayed in front of the grandchild's real-time image 140J2.

[0108] <Control sequence based on screen operation> Figure 21 illustrates the control sequence according to the user's role. In Figure 21, the symbol S also indicates a step. The control sequence shown in Figure 21 is executed as one of the functions of the educational application. The control sequence shown in Figure 21 is executed by the grandchild terminal (user terminal 10A). In this embodiment, the educational application executed on the grandchild terminal centrally controls responses to screen operations, etc.

[0109] With this system in place, the results of screen operations received by the grandchild terminal are first reflected on the grandchild terminal's touch panel 14 (see Figure 1). Subsequently, the content of the screen operations is notified to the grandparent terminal, and the results of the screen operations are also reflected on the grandparent terminal's touch panel 14. On the other hand, screen operations received by the grandparent terminal are notified to the grandchild terminal via the network, and the results of the screen operations are reflected on the grandchild terminal's touch panel 14. Next, instructions from the grandchild terminal are sent to the grandparent terminal via the network, and the results of the screen operations are also reflected on the grandparent terminal's touch panel. Thus, in this embodiment, the time from receiving a screen operation to the reflection of the result is shorter for the grandchild terminal than for the grandparent terminal.

[0110] When the grandchild's device accepts a screen operation, only one communication is required for the result of the screen operation to be reflected on the grandparent's device's touch panel 14. On the other hand, when the grandparent's device accepts a screen operation, two communications are required for the result of the screen operation to be reflected on the grandparent's device's touch panel 14. Therefore, screen changes on the grandparent's device are more susceptible to the network environment than those on the grandchild's device. In particular, when the grandparent's device accepts a screen operation, the screen changes on the grandparent's device tend to be delayed because two communications are required.

[0111] However, grandparents, being adults, can understand, for example, that screen operation is affected by communication delays, and it is also possible to gain their understanding by displaying a message about the communication delay on their devices. On the other hand, grandchildren may not be able to wait for the delay from screen input. As a result, they may not only repeatedly touch the screen, but they may also think it's broken and stop playing the educational content. If their input doesn't register as intended, their concentration on playing the educational content may be broken. Therefore, in this embodiment, screen operation input information is aggregated on the grandchild terminal, which is the main player of the educational content, and control over screen operations is executed collectively.

[0112] First, the grandchild terminal determines whether or not it has detected a screen input (step 141). This determination does not depend on whether the terminal that received the screen input is the grandchild terminal or the parent terminal. In other words, only whether or not a screen input was received is determined. If no screen operation is detected, a negative result is obtained in step 141. In this case, the grandchild terminal repeats the determination in step 141. On the other hand, if a screen operation is detected, a positive result is obtained in step 141. In this case, the grandchild terminal determines whether the screen operation was received by its own terminal or not (step 142). Here, "own terminal" refers to the grandchild terminal. The grandchild terminal is the terminal operated by the first user who requested a connection to the grandparents through the educational app.

[0113] If the local terminal receives a screen input, a positive result is obtained in step 142. In this case, the grandchild terminal further determines whether or not it has received an input on the object to which the visual effect has been assigned (step 143). If no visual effect is assigned to the manipulated object (for example, if the background is tapped), a negative result is obtained in step 143. In this case, the grandchild terminal does not show any reaction to the screen operation and terminates processing for the detected screen operation.

[0114] If a special effect is assigned to the manipulated object (for example, if the "grandfather" object is tapped), a positive result is obtained in step 143. In this case, the grandchild terminal instructs the output of special effects, etc., according to the combination of each terminal's role and the object being manipulated (step 144). The visual effects are output to both the grandchild's device and the grandparent's device, depending on the content of the effect. However, it is technically possible to output the visual effects only to the grandchild's device or only to the grandparent's device.

[0115] For example, even with the same object, a predetermined visual effect may be executed when the operation is performed from a grandchild terminal, while a specific visual effect may be executed when the operation is performed from a grandparent terminal. Differences between a predetermined effect and a specific effect include, for example, differences in the magnitude of movement, differences in the speed of movement, differences in the intensity of the effect (for example, the same type or content but different in appearance, etc.), differences in the type or content of the effect, and the presence or absence of an effect. Furthermore, the output of visual effects, etc., is notified to both the user's device (grandchild's device) and the grandparent's device. As a result, the predetermined visual effects appear on both the user's device (grandchild's device) and the grandparent's device.

[0116] Return to the judgment process in step 142. If the grandparent's terminal accepts a screen operation, a negative result is obtained in step 142. In this case, the grandchild's terminal determines whether or not the operation is on an object (step 145). This is because the grandparent's terminal also displays objects that control the progress of the scene, etc. For example, if the target of the operation is the background, a negative result is obtained in step 145. In this case, the grandchild terminal does not show any reaction to the detected screen operation and terminates processing for the detected screen operation.

[0117] If the target of the operation is an object, a positive result is obtained in step 145. In this case, the grandchild terminal further determines whether or not the operation is performed on the operation object (step 146). For example, if the target of the operation is an operation object, a positive result is obtained in step 146. In this case, the grandchild terminal instructs the scene to proceed according to the operation object (step 147).

[0118] Instructions are sent to both the grandchild's device (the user's own device) and the grandparent's device. This allows the scene to progress on both the grandchild's device and the grandparent's device. Scene progression includes switching between scenes (returning to the first slide of the previous scene or moving to the first slide of the next scene), as well as progression within the same scene. If the target of the screen operation received by the grandparent terminal is not an operation object, a negative result is obtained in step 146. In this case, the grandchild terminal proceeds to, for example, step 143 and performs the same determination and processing as the grandchild terminal described above.

[0119] For example, if an object is operated on by grandparents and the output of a special effect is assigned to that operation, the output of the special effect corresponding to the grandparents' terminal will be output to both the user's terminal (grandchild's terminal) and the grandparents' terminal. In contrast, for an object where the output of a visual effect is assigned only to operations performed by the grandchild, even if operations performed by the grandparents are detected on the object, a negative result is obtained in step 143. As a result, no visual effect appears on either the grandchild's terminal or the grandparents' terminal.

[0120] <If the grandparent's device accepts the command> Figure 22 illustrates the control sequence when the grandparent terminal accepts a scene switching operation. The control sequence shown in Figure 22 is a reorganization of the control sequence shown in Figure 21 as communication between the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). First, the grandparent terminal (i.e., the user terminal 10B) accepts an operation related to progress (step 151). This operation is accepted as an operation on an operation object such as the "back" button 140H3 (see Figure 19) or the "forward" button 140H4 (see Figure 19).

[0121] The grandparent terminal that receives the operation notifies the grandchild terminal (i.e., the user terminal 10A) that the operation regarding the progress has been received. Upon receiving the notification, the grandchild device switches the scene of the educational content displayed on its own device (i.e., the grandchild device) (Step 152). Next, the grandchild's device instructs the grandparent's device to switch scenes (step 153). Upon receiving this instruction, the grandparent's device switches the scene of the educational content displayed on its own device (step 154).

[0122] In other words, when the grandparent terminal receives an operation on the first operation object, the grandchild terminal and the grandparent terminal cooperate (i.e., the processor 11 on the grandparent terminal side (see Figure 1) and the processor 11 on the grandchild terminal side (see Figure 1) cooperate) to send scene switching information to the terminal operated by the user of the first account, and then the scene switching on the grandparent terminal is executed via the grandchild terminal.

[0123] Figure 23 illustrates an example of scene switching. Figure 23 shows the screen changes before and after operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). As shown in Figure 23, the "Forward" and "Back" buttons are displayed only on the grandparent's device and not on the grandchild's device. The "Forward" and "Back" buttons, which are the first operation objects, are just examples of objects. For this reason, the "Forward" and "Back" buttons are positioned in front of the grandchild's real-time image, as explained in Figure 7.

[0124] In Figure 23, the "Next" button was pressed by the grandparents, resulting in the educational content displaying the next scene. However, the screen could also switch to the next slide within the same scene. In the case of Figure 23, immediately after switching, the first slide of the next scene is displayed. When the "Next" button is used to switch to the next scene, even if the current scene contains multiple slides and there are still undisplayed slides, pressing the "Next" button will switch to the first slide of the next scene.

[0125] Figure 24 illustrates another example of scene transitions. In Figure 24, the screen changes before and after operation acceptance are shown on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). In Figure 24, the "Back" button was pressed by the grandparents, resulting in the educational content display switching to the previous scene. In this case, immediately after the switch, the first slide of the previous scene is displayed. As will be explained later, you may also want the screen to switch to the previous slide within the same scene when the "back" button is pressed.

[0126] Figure 25 illustrates an example of switching slides within the same scene. In Figure 25, the screen changes before and after operation are accepted on both the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). In Figure 25, the "Next" button is pressed by the grandparents, resulting in a switch to the next slide within the same scene. In Figure 25, the text in the background changes, and both the "Grandfather" and "Grandmother" objects begin waving.

[0127] Figure 26 illustrates another example of slide transitions within the same scene. In Figure 26, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are shown. In Figure 26, the "Back" button has been operated by the grandparents, resulting in a switch to the previous slide within the same scene. In Figure 26, the text content in the background has switched to the previous content, and the effects that were running on both the "Grandfather" and "Grandmother" objects have reverted to their state before the effects started.

[0128] Figure 27 illustrates another example of slide transitions within the same scene. In Figure 27, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are shown. In Figure 27, the "Next" button was pressed by the grandparents, resulting in a switch to the next slide within the same scene.

[0129] In Figure 27, the basic image is the same, but the background text displayed at the top of the slide has disappeared, and a quiz button object 140K1 has appeared on the right side of the slide. Also, the sleeping sound (zzz) object 140K2 displayed on the mouse on the left has disappeared. The newly appeared quiz button object 140K1 is also an example of a first-operation object. For example, when the quiz button object 140K1 is operated, a quiz is displayed on the grandparent's device. The grandparent can enjoy a conversation with their grandchild using the displayed quiz as a reference. Furthermore, the educational effect is enhanced through conversation between grandchildren and grandparents.

[0130] <When the grandchild terminal accepts the command> Here, we assume that the visual effects of the object being manipulated are linked only to the operation of the sub-device. Figure 28 illustrates the control sequence when a grandchild terminal receives an operation on an object. The control sequence shown in Figure 28 is a reorganization of the control sequence shown in Figure 21 as communication between the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B).

[0131] In the case of Figure 28, the grandchild terminal (i.e., the user terminal 10A) accepts operations on the first object, to which the visual effects are linked only to the operations of the grandchild terminal (step 161). The first object is included in the objects to which the visual effects are associated. However, there are restrictions on the display of the visual effects; that is, the operators are limited. Since the educational app on the grandchild's device performs the process described in Figure 21, there is no need to notify the grandparent's device of the detection of an operation on the first object.

[0132] The sub-terminal determines whether the operation was received by its own terminal or not (step 162). If the operation was received by the grandparent's terminal, a negative result is obtained in step 162. In this case, the grandchild's terminal terminates processing for the operation received without displaying any special effects. In contrast, if the operation was received by the user's own device (i.e., the grandchild device), a positive result is obtained in step 162. In this case, the grandchild device displays the corresponding visual effect on the first object of the educational content displayed on the user's own device (step 163). Subsequently, the grandchild terminal instructs the grandparent terminal to display the visual effects for the first object (step 164).

[0133] Upon receiving instructions from the grandchild's device, the grandparent's device displays the corresponding visual effect on the first object of the educational content displayed on its own device (step 164). In other words, the grandchild terminal activates a predetermined visual effect on the first object to which the visual effect is associated, in conjunction with the grandchild's operation on the first object (i.e., a predetermined operation). On the other hand, the grandchild terminal does not activate the predetermined visual effect even if the grandparents perform an operation on the first object (i.e., a predetermined operation). The first object here is also an example of the second manipulation object. Furthermore, the specified visual effect is also an example of the second visual effect.

[0134] Figure 29 illustrates an example of a visual effect that appears when the grandchild manipulates the "rice ball" object 140L1 within the slide. The "rice ball" object 140L1 in Figure 29 is an example of both a first object and a second manipulated object. In the case of Figure 29, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown.

[0135] In Figure 29, the grandchild taps an "onigiri" object 140L1, which has a visual effect associated with the grandchild's actions. In this case, a visual effect (e.g., animation playback) is executed, in which the tapped "onigiri" object 140L1 flies into the air and then falls into a "hole" object 140L2 from which music is playing. Along with the execution of the visual effect on the grandchild's terminal (user terminal 10A) screen, the grandchild's terminal sends information to the grandparent's terminal (user terminal 10B) that the "onigiri" object 140L1 was tapped or information about the screen where the visual effect is being executed as a result of the tap. As a result, the visual effect related to the "onigiri" object 140L1 is also displayed on the grandparent's terminal screen. The same applies when another "onigiri" object 140L1 is tapped by the grandchild.

[0136] Figure 30 illustrates an example of a visual effect that appears when grandparents interact with the "rice ball" object 140L1 in the slide. Figure 30 is indicated with corresponding reference numerals for parts that correspond to those in Figure 29. As mentioned above, the "rice ball" object 140L1 in Figure 30 is an example of a first object whose performance effects are linked only to the grandchild's actions, and is also an example of a second action object.

[0137] In the case of Figure 30, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown. In Figure 30, the "rice ball" object, whose visual effects are linked to the grandchild's actions, is tapped by the grandparents. As mentioned earlier, the visual effects of the "rice ball" object are linked to the grandchild's actions, not to the grandparents'. Therefore, the visual effects are not activated for the tapped "rice ball" object 140L1.

[0138] Figure 31 illustrates an example of a visual effect that appears when the grandchild manipulates the "grandfather" object 140M1 in the slide. Incidentally, the slide in Figure 31 corresponds to a scene where the grandfather falls into a hole and rolls down it. The "grandfather" object 140M1 in Figure 31 is an example of both a first object and a second operation object.

[0139] In the case of Figure 31, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are shown. In Figure 31, the "grandfather" object 140M1 is tapped by the grandchild. In this case, the animation of the "grandfather" object 140M1 stops at the tapped position. However, when the grandchild removes their hand from the screen, the animation of the "grandfather" object 140M1 rolling down may resume.

[0140] Figure 32 illustrates an example of a visual effect that appears when grandparents interact with the "grandfather" object 140M1 in the slide. Figure 32 is denoted with corresponding symbols for parts that correspond to those in Figure 31. The "grandfather" object 140M1 in Figure 32 is an example of both a first object and a second operation object.

[0141] In the case of Figure 32, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown. In Figure 32, the "grandfather" object 140M1, whose effects are linked to the grandchild's actions, is being tapped by the grandparents. As mentioned earlier, the effects of the "grandfather" object 140M1 are not linked to the grandparents. Therefore, the animation of the "grandfather" object 140M1 rolling down the hole continues despite the grandparents' touch operation.

[0142] Figure 33 illustrates an example of a visual effect that appears when the grandchild manipulates the "dust cloud" object 140N1 in the slide. Figure 33 is denoted with corresponding symbols for parts that correspond to Figure 32. The slide in Figure 33 corresponds to the scene where the grandfather falls to the bottom of a hole and dust clouds up. The "dust cloud" object 140N1 in Figure 33 is an example of both a first object and a second operation object. In the case of Figure 33, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are shown.

[0143] In Figure 33, the grandchild taps the "dust cloud" object 140N1, which has a visual effect linked to the grandchild's actions. In this case, the tapped "dust cloud" object 140N1 disappears from the screen, and the hidden "grandfather" object 140M1 appears. The same effect of the tapped "dust cloud" object 140N1 disappearing is also performed when another "dust cloud" object 140N1 is tapped. The "grandfather" object 140M1 appears only when the "dust cloud" object 140N1 that was hiding the "grandfather" object 140M1 disappears.

[0144] Figure 34 illustrates an example of a visual effect that appears when the grandparents manipulate the "dust cloud" object 140N1 in the slide. Figure 34 is denoted with corresponding symbols for parts that correspond to those in Figure 33. The "dust cloud" object 140N1 in Figure 34 is an example of both a first object and a second operation object.

[0145] In the case of Figure 34, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are shown. In Figure 34, the "dust cloud" object 140N1, whose effects are linked to the grandchild's actions, is tapped by the grandparents. As mentioned earlier, the effects of the "dust cloud" object 140N1 are not linked to the grandparents. Therefore, no change occurs in the "dust cloud" object 140N1 when it is tapped by the grandparents.

[0146] Figure 35 illustrates an example of a visual effect that appears when the grandchild manipulates the "mouse" object in the slide. Incidentally, the slide in Figure 35 corresponds to the scene where "Grandpa" and the "mouse" are making mochi at the bottom of a hole. The "mouse" object 140O1 in Figure 35 is an example of both a first object and a second operation object. In the case of Figure 35, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown.

[0147] In Figure 35, the grandchild taps a mouse object 140O1, which has a visual effect linked to the grandchild's actions. In this example, two musical note objects 140O2 appear around the tapped mouse object 140O1. This effect may be accompanied by sound effects or melodies, or it may be sound only. The output of sound effects or melodies enhances the playability and increases immersion in the educational content. The same effect will be executed if another mouse object 140O1, which has the same visual effect linked to it, is tapped. Although not shown in the diagram, even if the same "mouse" object 140O1 is tapped on the grandparents' device, the "musical note" object 140O2 shown in Figure 35 will not appear, as in the other examples mentioned above.

[0148] Figure 36 illustrates an example of a visual effect that appears when the grandchild manipulates the "gold coin" object 140P1 within the slide. Incidentally, the slide in Figure 36 corresponds to a scene where the "grandfather" returns home and, together with the "grandmother," opens a box of souvenirs they received and is surprised. The object 140P1 of "Oban" in Figure 36 is an example of both a first object and a second operation object.

[0149] In the case of Figure 36, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown. In Figure 36, the grandchild taps the "gold coin" object 140P1, which has a visual effect linked to the grandchild's actions. In this example, a visual effect is activated in which a new "gold coin" object 140P2 erupts on the screen. In this case, a sound effect may also be output. Although not shown in the illustration, even if the "gold coin" object 140P1 is tapped on the grandparents' terminal, the "gold coin" object 140P2 shown in Figure 36 will not appear, as in the other examples mentioned above.

[0150] <When the special effects are linked to the grandparents' actions (Part 1)> This section explains how special effects are linked to actions taken by grandparents. Figure 37 illustrates the control sequence when an operation object (i.e., the first operation object) displayed only on the grandparent terminal is operated. The control sequence shown in Figure 37 is a reorganization of the control sequence shown in Figure 21 as communication between the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). In the case of Figure 37, the grandparent terminal (i.e., the user terminal 10B) accepts an operation on the first operation object (step 171). The educational app on the grandparent terminal does not perform the process described in Figure 21, so it notifies the grandchild terminal of the detection of the operation on the first operation object.

[0151] The grandchild terminal, upon receiving notification of a screen operation, determines whether the received operation is an operation on an operation object (step 172). This determination process corresponds to step 146 (see Figure 21). Incidentally, if the target of the operation is not an operation object, a negative result is obtained in step 172. In this case, the grandchild terminal terminates the processing operation that was initiated by detecting an operation on the first operation object. On the other hand, if the target of the operation is an operation object, a positive result is obtained in step 172. In this case, the grandchild terminal reads the effect associated with the first operation object (step 173).

[0152] Next, the grandchild's device instructs the grandparent's device to display the visual effect only if it is a hint / quiz (step 174). Hints and quizzes are examples of information that assists in the progression of educational content. Unlike objects that control the progression of educational content, such as "forward" and "back" buttons, hints and quizzes are information that assists the grandparent's actions toward these objects. Upon receiving instructions from the grandchild's device, the grandparent's device displays visual effects corresponding to the educational content shown on its own device (Step 175). In other words, the grandchild's device displays information on the grandparent's device that supports the progression of educational content according to the displayed scene. On the other hand, the visual effects are not displayed on the grandchild's device.

[0153] Figure 38 illustrates an example of the display of support information that appears when the first operation object, which is displayed only on the grandparent's terminal, is operated. In Figure 38, button 140Q1 is displayed as the first operation object. In the case of Figure 38, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown.

[0154] In Figure 38, button 140Q1, which is displayed only on the grandparent's device, is being operated by the grandparent. In Figure 38, a speech bubble object 140Q2 appears at the top of the slide with the message "Hint for talking to your grandchild: Find Grandma." The grandparent can use this hint to talk to their grandchild. By providing support for talking to grandchildren that is appropriate to the scene and slide being displayed, communication between grandparents and grandchildren can be stimulated. Furthermore, by stimulating communication, it is possible to increase the grandchild's interest in the picture book content and improve the learning effect.

[0155] Figure 39 illustrates another example of the display of support information that appears when the first operation object, which is displayed only on the grandparent's terminal, is operated. In Figure 39, corresponding parts with those in Figure 38 are indicated with corresponding reference numerals. In the case of Figure 39, the screen changes before and after the operation is accepted on the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B) are also shown. In Figure 39, the first operational object is the house object 140Q3, which is one of the components of the slide.

[0156] Therefore, when the grandparents tap the house object 140Q3 on the screen, a speech bubble object 140Q2 appears at the top of the slide with the message "Hint for talking to your grandchild: What color is the house?". The grandparents can then use this hint to talk to their grandchild. In the case of Figure 39, the content of the hint is, for example, related to the current scene or the slide being displayed. However, when using an object within a slide as the primary action object, helpful information such as hints about the specific object (e.g., a house) will be displayed.

[0157] Therefore, grandparents can receive hints on how to address specific objects. Note that the visual effects of the house object 140Q3 are tied only to the grandparents' actions. Therefore, even if the grandchild taps the house object 140Q3, the speech bubble object 140K2 will not appear. However, if the house object 140Q3 has a visual effect tied to the grandchild's actions, that visual effect will appear on both the grandchild's and the grandparents' devices.

[0158] <When the special effects are linked to the grandparents' actions (Part 2)> Next, we will explain the case where the special effects are linked to the actions of the grandparents. In the previous example, hints and other visual effects were displayed only on the grandparents' device, but in this example, the visual effects appear on both the grandchild's device and the grandparents' device. Figure 40 illustrates another example of a control sequence when an operation object, which is displayed only on the grandparent terminal, is operated. The control sequence shown in Figure 40 is also a reinterpretation of the control sequence shown in Figure 21, but as communication between the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B).

[0159] The control sequence shown in Figure 40 can be executed independently, but it may also be executed in combination with the control sequence described in Figure 37. In the case of Figure 40, the grandchild terminal (i.e., the user terminal 10A) accepts an operation on the first object (step 181). For example, an object on the slide on the screen is operated as an answer to a quiz. The first object is displayed in common to both the grandchild terminal and the grandparent terminal as one of the components of the slide. In addition, if an answer button for a quiz or similar question is displayed on the grandchild terminal, that answer button will be considered the first object.

[0160] Next, the grandchild terminal determines whether the received operation is an answer to a question (step 182). In this embodiment, if an operation on some object is received while a hint or prompt is displayed on the grandparent terminal due to the grandparent's previous operation, it is considered an answer to a question. If voice responses are allowed, step 181 may be "receiving the grandchild's voice". If the response is not an answer to the question, a negative result is obtained in step 182. In this case, the grandchild terminal proceeds to step 143, for example (see Figure 21).

[0161] On the other hand, if it is an answer to the question, a positive result is obtained in step 182. In this case, the grandchild terminal instructs only the grandparent terminal to display the first operation object (step 183). Since displaying the first operation object is only permitted for the grandparent terminal, the grandchild terminal is not instructed to display it. Upon receiving instructions from the grandchild's device, the grandparent's device displays the first operation object (step 184).

[0162] The first operation object here may be a general-purpose object, or a dedicated object prepared according to the content of the verbal hint being displayed. The general-purpose object may display keywords for verbal responses to input on the grandchild's terminal, such as "Correct," "Too bad," "Try again," "Good luck," or "Praise." The dedicated object may display a string that reflects the content of the question, for example. For example, if the question is "What color is your house?", an example answer such as "Yellow is correct" may be displayed. In Figure 40, the first operation object is linked to the visual effects that are output to both the grandchild's terminal and the grandparent's terminal.

[0163] Next, the grandparent terminal (i.e., the user terminal 10B) accepts an operation on the first operation object (step 185). The educational app on the grandparent terminal does not perform the process described in Figure 21, so it notifies the grandchild terminal of the detection of an operation on the first operation object. The grandchild terminal, upon receiving notification of a screen operation, determines whether the received operation is an operation on an operation object (step 186). This determination process corresponds to step 146 (see Figure 21).

[0164] Incidentally, if the target of the operation is not an operation object, a negative result is obtained in step 186. In this case, the grandchild terminal proceeds to step 143 (see Figure 21). On the other hand, if the target of the operation is an operation object, a positive result is obtained in step 186. In this case, the grandchild terminal reads the effect associated with the first operation object (step 187).

[0165] Next, the grandchild's terminal displays the corresponding visual effect on its own terminal if the effect is output to both its own terminal and the grandparent's terminal (step 188). This type of visual effect includes, for example, "praising," "commendating," "supporting," and "encouraging" the grandchild's answers to quizzes, etc. Subsequently, the grandchild's device instructs the grandparent's device to display a special effect (step 189). Meanwhile, the grandparent's device, having received the instruction from the grandchild's device, displays a special effect corresponding to the educational content displayed on its own device (step 190). In other words, the grandchild terminal displays the first performance effect on its own terminal, and also displays the first performance effect on the grandparent terminal.

[0166] Figure 41 illustrates an example of the display of a first operation object on the grandparent's terminal resulting from the grandchild's response to a question. Figure 41 is denoted with corresponding reference numerals for parts corresponding to those in Figure 39. In the upper part of Figure 41, the grandparents tapped the "Grandma" object 140Q4 on the screen, resulting in the appearance of a speech bubble object 140Q2 that reads, "Hint for talking to your grandchild: Find Grandma." The grandparents then use the hint in speech bubble object 140Q2 to ask their grandchild a question.

[0167] The lower part of Figure 41 shows the scene where the grandchild taps the "Grandma" object 140Q4 on the screen as an answer to the question. The location where the grandchild tapped is displayed as the "Target" object 140Q5 on the grandparents' device screen. In addition, the "Praise!" button object 140Q6 appears inside the speech bubble object 140Q2. The display shown in the lower part of Figure 41 is achieved through steps 181 to 184 (see Figure 40).

[0168] In Figure 41, the object 140Q6, which is the "Praise!" button, is displayed regardless of whether the answer is correct or incorrect. The decision of whether or not to praise is left to the grandparents. The visual effect will not appear unless the grandparents interact with the "Praise!" button object 140Q6, so there is no problem in displaying the "Praise!" button object 140Q6 without checking whether the grandchild's answer is correct or incorrect.

[0169] In Figure 41, the "Praise!" button object 140Q6 is displayed a predetermined time after the speech bubble object 140Q2 is displayed, but it may also be displayed simultaneously with the speech bubble object 140Q2. Furthermore, the "Praise!" button object 140Q6 may be displayed independently of the speech bubble object 140Q2.

[0170] The processor 11 can determine whether the answer from the grandchild terminal is correct or incorrect, and may switch the object displayed on the grandparent terminal according to the content of the answer from the grandchild terminal. For example, if the grandchild's answer is correct, a mechanism may be adopted to display the "Praise!" button object 140Q6 on the grandparent terminal. On the other hand, if the answer is incorrect, the processor 11 may employ a mechanism to display an object (not shown) such as a "Too bad!" button on the grandparent's terminal.

[0171] Figure 42 illustrates an example of the display of visual effects when grandparents operate the first operation object (for example, object 140Q6, the "Praise!" button). Figure 42 is denoted with corresponding reference numerals for parts corresponding to Figure 41. The slides shown in Figure 42 are an example of a continuation of the slides shown in Figure 41. In Figure 42, the grandparents are tapping object 140Q6, which is the "Praise!" button.

[0172] As a result, two "cracker with confetti flying out" objects 140Q7 are displayed on the screens of both the grandchild's device and the grandparent's device. The effects shown in Figure 42 are just examples; text such as "Correct!" could be displayed on both the grandchild's and grandparent's devices, or the tapped object (in this case, the "grandmother" object 140Q4) could be made to perform special movements (for example, a dancing animation). These effects are expected to promote communication between grandchildren and grandparents, and to enhance their enjoyment of playing the picture book content.

[0173] <When the special effects are linked to the grandparents' actions (Part 3)> Next, we will explain the case where the special effects are linked to the actions of the grandparents. This example corresponds to a combination of the two control examples mentioned earlier. Specifically, it shares the first control example in that it is initiated by the grandparents' actions, and the second control example in that the visual effects appear on both the grandchild's and grandparents' devices. Figure 43 illustrates another example of a control sequence when an operation object, which is displayed only on the grandparent's terminal, is operated. Figure 43 is denoted with corresponding reference numerals for parts corresponding to those in Figure 37.

[0174] The control sequence shown in Figure 43 is also a reorganization of the control sequence shown in Figure 21 as communication between a grandchild terminal (i.e., user terminal 10A) and a grandparent terminal (i.e., user terminal 10B). The control from step 171 to step 173 in FIG. 43 is common with FIG. 37. Therefore, the explanation will start from the point when step 173 is executed. When the performance effect is output to both the grandchild terminal and the grandparent terminal, the grandchild terminal outputs the corresponding performance effect to its own terminal (step 191).

[0175] Subsequently, the grandchild terminal instructs the grandparent terminal to output the performance effect (step 192). On the other hand, the grandparent terminal that has received the instruction from the grandchild terminal displays the performance effect corresponding to the early childhood education content displayed on its own terminal (step 175). That is, the grandchild terminal causes the first performance effect to appear on its own terminal and also causes the first performance effect to appear on the grandparent terminal.

[0176] FIG. 44 is a diagram for explaining an example of displaying other performance effects when the grandparent operates the first operation object. In FIG. 44, the parts corresponding to those in FIG. 41 are marked with corresponding reference numerals. In the case of FIG. 44, the speech bubble object 140Q2 shows "Hint for greeting. What is grandpa doing?" The answer to this question is limited to the grandchild's verbal response. Therefore, the object 140Q8 of the "Wave your hand more!" button is initially displayed on the speech bubble object 140Q2 shown in FIG. 44.

[0177] The object 140Q8 of this "Wave your hand more!" button may be used, for example, as a hint to the grandchild or for instructing the output of the performance effect at the time of a correct answer. When the grandparent operates the object 140Q8 of the "Wave your hand more!" button, a performance effect 140Q10 appears in which the object 140Q9 of "grandpa" moves its hand greatly. This performance effect 140Q10 includes, for example, an effect of expanding the range of hand movement and an effect of accelerating the hand movement. By outputting this performance effect, the grandchild and the grandparent can enjoy playing the early childhood education content.

[0178] Figure 45 illustrates an example of the display of other visual effects when the grandparents manipulate the first control object. Figure 45 is denoted with corresponding reference numerals for parts that correspond to those in Figure 41. In Figure 45, the "Quiz Button" object 140Q11 is displayed only on the grandparents' terminal. The "Quiz Button" object 140Q11 shown in Figure 45 is an example of a first operation object. When the "quiz button" object 140Q11 is tapped, the speech bubble object 140Q12 appears. The control sequence from the operation of the "quiz button" object 140Q11 to the display of the speech bubble object 140Q12 is the same as in Figure 37.

[0179] In Figure 45, the speech bubble object 140Q12 displays the question, "Puzzle: Find the sleeping mouse." Upon seeing this question, the grandparents call out to their grandchild, asking them to find the sleeping mouse. In Figure 45, the grandparents tap the "sleeping mouse" object 140Q13, located on the left side of the slide, as a hint for their grandchild and an instruction for the visual effect when the answer is correct. The "sleeping mouse" object 140Q13 here is also an example of a first action object.

[0180] As a result of this operation, in Figure 45, the object 140Q14 representing breathing sounds ("zzz") appears near the object 140Q13 representing a "sleeping mouse." This visual effect allows grandchildren and grandparents to enjoy playing educational content. Furthermore, if the grandchild taps the sleeping mouse, the visual effects associated with the "sleeping mouse" object 140Q13 may be displayed on both the grandchild's device and the grandparent's device, as described in steps 164 and 165 above (see Figure 28).

[0181] <Linkage between grandchild's movement and object movement> The educational app described in this embodiment includes a function that detects the grandchild's movements from the grandchild's real-time image and links the movement of objects within the educational content to the detected movements of the grandchild. This function is implemented by changing at least one of a predetermined object within the educational content (for example, a "grandfather" object) and a predetermined object (for example, a "hat" object, a "beard" object) that is superimposed on the real-time image of the first user (for example, the grandchild), in conjunction with the movements detected for the first user (for example, the grandchild). The predetermined object that is linked to the first user's movements is specified, for example, on the educational content side.

[0182] Please note that motion detection is limited to the first user (e.g., grandchild), and does not apply to the second user (e.g., grandparents). Therefore, even if motion is detected from the grandparents' real-time image, the detected motion will not be reflected in the display of educational content on both the grandchild's and grandparents' devices. One of the purposes of this function is to increase the first user's interest in playing the educational content.

[0183] Figure 46 is a sequence diagram illustrating the activation of special effects in response to the actions of the first user (e.g., grandchild). The control sequence shown in Figure 46 is also a reorganization of the control sequence shown in Figure 21 as communication between the grandchild terminal (i.e., user terminal 10A) and the grandparent terminal (i.e., user terminal 10B). The control sequence shown in Figure 46 may be adopted for all educational content, or it may be adopted for only some of the educational content. Depending on the content of the educational content, it may not be appropriate to link the movements of the first user (e.g., grandchild) to objects within the educational content.

[0184] Furthermore, even if there are no issues in linking the movements of the first user (e.g., a grandchild) to objects within the educational content, it is possible to provide educational content that employs the control sequence shown in Figure 46 and educational content that does not. This allows for differentiation between educational content by adopting or not adopting certain features. Furthermore, the control sequence shown in Figure 46 may be executed on all slides, or it may be executed only on specific slides predetermined in advance. As with educational content, some slides may not be suitable for the use of the function, and even if there is no impediment to the use of the function itself, it is up to the content provider whether or not to use it.

[0185] When the control sequence shown in Figure 46 is enabled, the grandchild terminal detects the movement of the user (e.g., the grandchild) included in the camera image of the terminal (step 201). Next, the secondary device applies the detected movement-based visual effects to the objects in the educational content displayed on its own device (step 202). The relationship between detected movement and visual effects is defined, for example, on a per-educational content or per-slide basis. Therefore, even if the detected movement is the same, different visual effects may be activated. Also, as mentioned above, if a movement-based visual effect is not defined, or if a definition exists but it is not activated for some educational content or slides, it may not be activated.

[0186] Next, the grandchild terminal instructs the grandparent terminal to display visual effects for the object (step 203). Furthermore, the grandparent's device, upon receiving instructions from the grandchild's device, reflects the visual effects onto the objects in the educational content displayed on its own device (Step 204). In the case of Figure 46, the grandchild terminal instructs the grandparent terminal to display visual effects for the "grandfather" object, the "hat" object, and the "beard" object. However, the visual effects for each object may also be executed autonomously (i.e., without instructions from the grandchild terminal) based on the grandchild's real-time image 151A (see Figure 5) received by the grandparent terminal, provided, for example, a predetermined function is enabled.

[0187] Figure 47 illustrates an example of an object following the movements of a first user (e.g., a grandchild). In Figure 47, the "hat" object 162 and the "beard" object 163 are objects that are composited as a visual effect onto the grandchild's real-time image 161. The "grandfather" object 164 is an example of the first object.

[0188] Figure 47 shows three postures: ST0, STL, and STR. Pose ST0 is the basic pose, and there is almost no tilt in the grandchild's real-time image 161. The "hat" object 162 is superimposed on the grandchild's head, and the "beard" object 163 is superimposed on the grandchild's chin. In the pose STL, the grandchild's real-time image 161 is tilted to the left relative to the base pose. In this case, the "hat" object 162 superimposed on the grandchild's head is also tilted to follow the head's movement. Similarly, the "beard" object 163 superimposed on the grandchild's chin is also tilted to follow the chin's movement. Likewise, the "grandfather" object 164 is also tilted to the left. The angle of tilt is linked to the detected angle of tilt of the grandchild.

[0189] Furthermore, the tilt of the grandchild's real-time image 161 can be detected as the tilt of the "grandchild's" real-time image 161 within the camera image when the user terminal 10A (see Figure 1) capturing the grandchild is stationary. On the other hand, if the grandchild tilts the user terminal 10A, the tilt angle detected by the user terminal 10A's attitude sensor (not shown) may be considered as the tilt of the grandchild's real-time image 161. The same applies to other operations. Furthermore, the movements may include the grandchild's gestures (e.g., waving, raising and lowering hands, hand signs). In addition, the grandchild's facial expressions (e.g., widening eyes, squinting, showing teeth, closing mouth) may be reflected in the object's visual effects. By using these individually or in combination with movements, a wide variety of visual expressions become possible.

[0190] By having object 162 ("hat") and object 163 ("chin") track each other, the unnatural appearance caused by the grandchild's movement causing a large shift in the composite position is reduced. In addition, the fact that the inclination of the "grandfather" object 164 within the slide is linked to the movement of the grandchild himself / herself will be a fun experience for young children. As a result, the immersion of the grandchild in the educational content can be enhanced. Note that the display position and the follow-up of the inclination of the "hat" object 162 and the "jaw" object 163 depend on the image processing ability of the grandchild terminal. As a result, there may be a delay in the follow-up operation. On the other hand, in the grandparents' terminal, the follow-up processing of the object to be displayed on the own terminal is started after an instruction is received from the grandchild terminal. When the communication delay is large, there is a possibility that a further delay will occur before the change is reflected.

[0191] FIG. 48 is a diagram for explaining another follow-up example of an object with respect to the movement of the first user (for example, the grandchild). In FIG. 48, the reference numerals corresponding to the corresponding parts in FIG. 47 are shown. In the state STU of FIG. 48, the real-time image 161 of the grandchild is jumping up within the screen. As a result, the display positions of the "hat" object 162 synthesized on the head of the grandchild and the "mustache" object 163 synthesized on the jaw of the grandchild are also changing so as to follow the display positions of the corresponding parts of the head of the grandchild. In addition, the "grandfather" object 164 also follows the movement of the grandchild and is jumping up within the slide.

[0192] <Snapshot function> The educational app described in this embodiment is provided with a function of synthesizing and displaying the snapshots of each user (for example, the grandchild and the grandparents) automatically taken during the display of the educational content or taken in response to a predetermined operation on the slide corresponding to the imaging time point. This function is also assumed to be utilized as a record of the play of the educational content and as a memory of the grandchild and the grandparents.

[0193] FIG. 49 is a diagram for explaining the snapshot function. The symbol S in FIG. 49 also indicates a step. This function is also managed by the educational app operating on the grandchild terminal. First, the grandchild device determines whether or not the educational content playback has finished (step 211). If the educational content playback has not finished, a negative result is obtained in step 211. In this case, the grandchild device determines whether or not the snapshot capture timing has been detected (step 212). The capture timing here is assumed to be a timing predetermined in the educational content.

[0194] If it is not the timing for capturing a snapshot, a negative result is obtained in step 212. In this case, the grandchild terminal determines whether or not it has detected a snapshot capture operation (step 213). The capture operation here is assumed to be an instruction from the grandparent terminal. However, this is only the case if the operation object used to instruct the capture is displayed on the grandparent terminal. If no snapshot capture operation is detected, a negative result is obtained in step 213. In this case, the grandchild terminal returns to the process in step 211.

[0195] On the other hand, if the snapshot is captured at the appropriate time, a positive result is obtained in step 212. Also, if the snapshot capture operation is detected, a positive result is obtained in step 213. In any of these cases, the grandchild terminal combines the image displayed on the user terminal, the user image at the time of detection, and the corresponding educational content image and records it as a snapshot (step 214). The grandchild terminal then returns to the process in step 211.

[0196] The grandchild terminal repeats either a loop process via step 213 or a loop process via step 214 while the educational content is playing. Eventually, when the end of playback of the educational content is detected, a positive result is obtained in step 211. In this case, the grandchild terminal presents the recorded snapshot (step 215) and terminates the series of processes.

[0197] Figure 50 is a diagram illustrating the snapshot recording process. The horizontal axis in Figure 50 represents the time axis. In Figure 50, four imaging timings are provided. Also, in Figure 50, one imaging operation is recorded. In either case, the slide corresponding to the imaging timing or imaging operation is recorded in association with the real-time image of the grandchild and the real-time image of the grandparents.

[0198] Figure 51 illustrates an example of a snapshot presentation. In snapshot image 300, snapshot 302 and snapshot 303, recorded during playback of the educational content, are combined with snapshot 301, the final slide of the educational content, as the background. Since snapshot 301 is the final slide, the word "The End" is displayed at the top of the screen.

[0199] In snapshot image 310, a snapshot 312 recorded during playback of the educational content is superimposed on a snapshot 311 of the first slide of the educational content as the background. In the case of snapshot image 310, it is composed of multiple screens. For this reason, snapshot image 310 is provided with an "Another Photo" button 313 and an "End" button 314. For example, when the "Another Photo" button 313 is pressed, the screens of snapshot image 310 switch sequentially.

[0200] When the "End" button 314 is pressed, the snapshot image 310 will be closed. Snapshot images 300 and 310 generated on the grandchild's device are displayed on both the grandchild's device and the grandparent's device. However, only the grandparent operating the grandparent's device may be able to operate the "Another Photo" button 313, etc. In this case, the "Another Photo" button 313, etc. will not be displayed on the grandchild's device, or will be grayed out.

[0201] As a function of the educational app, a predetermined number of images may be selected from multiple recorded snapshot images 310 to automatically generate a photo movie. The generated photo movie is recorded in association with the first or second account and can be played back at the user's discretion as a MY picture book for each user of the educational service. The timing for acquiring snapshot images for each educational content may be predetermined for the images used in the photo movie. In addition, the user's facial expressions may be detected from the camera image of the user while the educational content such as a picture book is being played back. Furthermore, the timing for acquiring snapshot images may be determined by detecting the user's excitement level from the user's voice captured by the microphone.

[0202] <Summary> According to the educational system 1 described in this embodiment (see Figure 1), a first operation object that controls the progress of the educational content is displayed only on the terminal operated by the second user (e.g., grandparents) who receives a connection request from the first user (e.g., grandchild). Therefore, it is possible to advance the educational content in accordance with the reactions and other characteristics of the first user.

[0203] Furthermore, in the educational system 1 described in this embodiment, the display area for the real-time image of the second user (e.g., grandparents) playing the educational content is fixed, but the display area for the real-time image of the first user (e.g., grandchild) changes according to the scene and slide. Therefore, compared to the case where the display areas for both the first and second users' real-time images are fixed, the sense of immersion in the educational content for the first user can be enhanced.

[0204] <Embodiment 2> In the aforementioned Embodiment 1, operations on the grandchild terminal and the grandparent terminal are consolidated on the grandchild terminal, and the progress of educational content and the appearance of special effects are managed under the management of the educational application executed on the grandchild terminal. However, in this embodiment, operations on the grandchild terminal and the grandparent terminal may be centralized on a management server, and the progress of educational content and the appearance of special effects may be controlled between the grandchild terminal and the grandparent terminal under the management of the management server.

[0205] Figure 52 is a diagram showing an example configuration of the educational system 1A assumed in Embodiment 2. Figure 52 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 1. The educational content 1A shown in Figure 52 has an additional management server 20. The management server 20 has a processor 21, semiconductor memory 22, auxiliary storage device 23, and communication interface 24.

[0206] The processor 21 is, for example, a CPU. The semiconductor memory 22 includes ROM (=Read Only Memory) on which UEFI etc. is recorded, and RAM used as the work area of ​​the processor 21. The auxiliary storage device 23 is, for example, a hard disk drive or semiconductor storage. The auxiliary storage device 23 stores an application that controls the progress of educational content on user terminals 10A and 10B. The communication interface 24 is a device that enables communication with the user terminal 10 connected to the network N.

[0207] <Management Server Functional Configuration> Figure 53 is a diagram illustrating an example of the functional configuration of the management server 20 in Embodiment 2. Figure 53 is denoted with reference numerals corresponding to the parts that correspond to those in Figure 2. The functional configuration shown in Figure 53 is an example of a functional unit that is realized through program execution. The management server 20 includes a control unit 201, a presentation unit 202, a synthesis unit 203, a detection unit 204, a verification unit 205, and a storage unit 210. The storage unit 210 is implemented by a semiconductor memory 22 (see Figure 52) and an auxiliary storage device 23 (see Figure 52). Of these, the control unit 201 works in conjunction with the other functional units to handle the various functions already described.

[0208] The display unit 202 performs functions such as displaying educational content, displaying real-time images of the grandchild on both the grandchild's terminal and the grandparent's terminal, displaying real-time images of the grandparent on both the grandchild's terminal and the grandparent's terminal, displaying buttons to control the progress of the educational content on the grandparent's terminal, displaying visual effects for objects on both the grandchild's terminal and the grandparent's terminal, and displaying snapshots on both the grandchild's terminal and the grandparent's terminal.

[0209] The synthesis unit 203 performs functions such as compositing real-time images of grandchildren onto slides or the like for educational content, and compositing real-time images of grandparents onto slides or the like for educational content. The detection unit 204 provides functions such as detecting the movements of the grandchild operating the grandchild terminal and detecting the acceptance of an operation. The verification unit 205 provides a function to verify that the operation is being performed by an adult, for example.

[0210] The memory unit 210 shown in Figure 53 stores, for example, user information 211, educational content 212, and streaming images 213. The information illustrated in Figure 53 is only a part of the total, and other information may be stored in the memory unit 210. When educational content is played on the user terminal 10, the streaming image 213 corresponds to, for example, the real-time image of each user. In contrast, if educational content is also streamed, the streaming image 213 will include the educational content in addition to each user's real-time image. In this case, the user terminal 10 is used as an input / output device to the management server 20.

[0211] User information 211 is information about a user registered with the educational service. Figure 54 illustrates an example of user information data 211. User information 211 includes user account 211A, display name 2112B, administrative user 211C, relationship 211D, connected partner account 211E, connection information 211F, and schedule 211G.

[0212] User account 211A is the information used to log in to the educational service. User account 211A consists of, for example, an administrator ID, account name, and username. The administrator ID is assigned by, for example, the educational service. Display name 211B is the name used to display the user on educational content. Display name 211B can be registered by each user, for example, or by an administrator.

[0213] Administrator user 211C is information indicating a user with administrative attributes. In Figure 54, the display name "Mago" has administrative attributes, while the display name "Baaba" does not. Relationship 211D is information about the relationship to the child. In Figure 54, the user corresponding to the display name "Grandchild" is listed as "Mother," and the user corresponding to the display name "Grandma" is listed as "Grandmother."

[0214] The connection partner account 211E is information indicating a pre-configured connection partner. In Figure 54, "1020" is stored as the connection partner account for the display name "Mago". This account is the account for the display name "Baaba". On the other hand, "1010" is listed as the connection partner account for the display name "Baaba". This account is the account for the display name "Mago".

[0215] Connection information 211F indicates whether the connection is initiated by the sender or the recipient. The connection information 211F stores whether the connection is initiated by the sender or recipient at the start of the connection. This information is deleted at the end of the connection. Schedule 211G contains information about the time periods when grandparents, etc., are available to respond to connection requests from their grandchildren. In Figure 54, the display name "Grandma" has "10:00-11:30" and "14:00-16:00" recorded. Schedule 211G is used to control the forwarding of connection requests.

[0216] <Summary> In the case of the educational system 1A described in this embodiment (see Figure 52), it is also possible to control the progress of the educational content as described in Embodiment 1, and to arrange the real-time images of the grandchild according to the scenes of the educational content. Furthermore, it is not necessary for the management server 20 to perform all controls and other operations related to the provision of educational content. Some functions may be performed on the user terminal 10, and this can be achieved through coordinated operation between the management server 20 and the user terminal 10.

[0217] <Other Embodiments> The embodiments of the invention are not limited to those described above. For example, elements of each embodiment can be combined as appropriate. Combinations here also include the deletion of elements from each embodiment.

[0218] (1) For example, in the above-described embodiment 1, it is assumed that one user (one user account) can only register one other user (one user account) as a connection partner. However, it is also possible to register multiple users for one user. For example, it is possible to register multiple users as grandparent terminals that can connect to one grandchild terminal. If multiple grandparent terminals are registered for one grandchild terminal, a connection partner selection screen may be displayed on the grandchild terminal when the operation of the "Call" button 140A6 is received. In this case, a connection request is notified to the selected grandparent terminal.

[0219] (2) Embodiment 1 described above assumes that one user (one user account) can only connect to one other user (one user account) at a time. In other words, it assumes a P2P connection. However, it is also possible to allow three or more users to connect simultaneously, as in so-called video conferencing. Three or more simultaneous connections can include cases where there is one grandchild terminal and multiple grandparent terminals, cases where there are multiple grandchild terminals and one grandparent terminal, or cases where there are multiple grandchild terminals and multiple grandparent terminals.

[0220] Figure 55 is a diagram illustrating an example of assigning control rights when there are multiple grandparent terminals connected to one or more grandchild terminals. For example, in the case of "Number 1," control rights are set to the grandparent designated by the grandchild from among the multiple connected grandparent terminals. In this case, the first control object that controls the progress of the educational content will be displayed only for the scene display on the grandparent terminal that has control rights. Incidentally, if there are multiple grandchild terminals, the designation from the grandchild terminal registered as the representative may be given priority, or the designation of the grandchild terminal that entered the grandparent's designation first may be given priority.

[0221] For example, in the case of "Number 2," the control rights are set to the grandparent who requested the connection earliest or higher in rank among the multiple connected grandparent devices. For example, if the grandchild's device is connected to both the device operated by the grandmother and the device operated by the grandfather, and the grandchild requests to connect to the grandmother first, the first control object that controls the progress of the educational content will be displayed on the device operated by the grandmother, and the first control object that controls the progress of the educational content will not be displayed on the device operated by the grandfather.

[0222] Furthermore, if the grandmother's connection priority (or priority) is set higher than the grandfather's, even if a connection request is sent to the grandfather first, and then to the grandmother after the connection with the grandfather is established, the first operation object controlling the progress of the educational content may be displayed on the terminal operated by the grandmother, while the first operation object controlling the progress of the educational content may not be displayed on the terminal operated by the grandfather. However, until a connection request is sent to the grandmother, only the grandfather is the second user to whom a connection request has been made. Therefore, until a connection request is also made to the grandmother, the first operation object controlling the progress of the educational content will be displayed on the terminal operated by the grandfather.

[0223] For example, in the case of "number 3," the control rights are set to the grandparent terminal that responds fastest to the connection request from the grandchild among the multiple connected grandparent terminals. For example, when the "Call" button 140A6 (see Figure 13) is received, a connection request is multicast to multiple registered grandparent terminals simultaneously. By granting control rights to the grandparent that responds fastest to the connection request, the grandparent with control rights will be determined each time a connection request is made.

[0224] In other words, multiple registered grandparents are given a chance to acquire control each time a connection request is received. As a result, the grandparents who have control are not fixed. Furthermore, grandparents who wish to have control are motivated to respond quickly to connection requests. Furthermore, even if connection requests are sent one by one in sequence, rather than via multicast, it is possible to grant control rights to the user who responds first.

[0225] For example, in the case of "number 4," the control rights are set to one grandparent decided upon among multiple connected grandparents. For example, even if the grandmother and grandfather are connected to the grandchild's device through separate devices, the grandmother and grandfather can discuss and decide which device will display the first control object that controls the progress of the educational content. The setting of control rights may also be transferable between multiple grandparents while connected. For example, even if the grandmother initially has control rights, they can be transferred to the grandfather afterward. By enabling this setting, multiple grandparents connected simultaneously can control the progress of the educational content.

[0226] For example, in the case of "number 5," control rights are set to one of the multiple connected grandparents whose speech is detected. For example, if both the grandmother and grandfather are connected to the grandchild's device, control rights may be given to the grandmother if she is talking to the grandchild, and to the grandfather if he is talking to the grandchild. This is because it is expected that the user talking to the grandchild will be able to control the progress of the educational content and the appearance of special effects, which is expected to lead to a smoother experience.

[0227] (3) In the first embodiment described above, the operation of the first operation object is permitted only to the grandparents, who are users of the second account (i.e., the second user). In other words, in the first embodiment, only the grandparents can control the switching of scenes and slides within the educational content (e.g., forward, backward), control the start and end of playing the educational content, and control the acquisition of snapshots. However, some or all of the authority to manipulate the first operation object may be granted to a child who is a user of the first account (i.e., the first user).

[0228] For example, the user of the first account may be allowed to operate the first operation object that controls the switching of scenes and slides. In other words, the first object that controls the switching of scenes and slides may be displayed on the screen of the grandchild terminal. However, even when the first operation object is displayed on the grandchild's terminal, the first operation object is also displayed on the grandparent's terminal screen, as in Embodiment 1. This is because if the switching of scenes, etc., is left solely to the child's operation, there is a concern that it may hinder the progress of the educational content.

[0229] As an alternative, a mechanism could be adopted that allows the grandparents (second users) to switch whether or not to display the first operation object on the grandchild's device (first user). For example, if operating the first operation object itself is difficult or if the grandchild cannot manage the progress of the educational content, even if the first operation object is functionally possible to display, it can be configured not to display it on the grandchild's device. This allows for display screen settings to be adjusted according to the grandchild's age and intellectual level. A dedicated control or similar tool used to instruct the switching should be displayed on the grandparents' device.

[0230] Alternatively, the ability to enable and disable the operation of the first operation object displayed on the grandchild's terminal (the first user) can be controlled from the grandparent's terminal (the second user). For example, if the grandparents have disabled the grandchild's operation, even if the grandchild operates the first operation object on the screen, scene or slide switching (e.g., forward, backward) will not be performed. In this case, a dedicated control element used to toggle the operation on and off can be displayed on the grandparent's terminal.

[0231] (4) In the first embodiment described above, the real-time image of the grandparents, who are users of the second account (i.e., the second user), is displayed in an area having a rectangular outer border. However, the real-time image of the second user may also be defined by the outline of the second user (i.e., the outline of the human body) captured in the camera image.

[0232] (5) In the first embodiment described above, the real-time image of the grandparents, who are users of the second account (i.e., the second user), is displayed in a position (specific area) common to multiple slides. However, the real-time image of the second user may also be displayed superimposed on a background area that does not affect the progress of the educational content, similar to the real-time image of the user of the first account (i.e., the first user). However, the real-time image of the second user should be displayed in an area that does not affect the display of the real-time image of the first user.

[0233] (6) In the case of Embodiment 1 described above, the story development and ending scenes of the educational content are predetermined. However, it is also possible to prepare multiple sets of story developments and ending scenes for a single educational content, and to adopt a system where the story development and ending scenes change depending on user input during gameplay, the number of times special effects are activated, and the results of quizzes and mini-games. Furthermore, if educational content is played multiple times with the same story progression or the same ending scene is played multiple times, a mechanism may be adopted to select scenes that differ from the previous playback in terms of story progression or ending scene. Incidentally, when selecting the story progression and ending scenes, it would be a good idea to adopt a system that prioritizes the play history of the first account user over that of a second account user playing the educational content together. By changing the story progression and ending scenes, it is possible to reduce boredom and loss of concentration that can occur when repeatedly playing educational content.

[0234] (7) In the case of Embodiment 1 described above, it is assumed that there is one user for the first account and one user for the second account. Therefore, there is one user captured in the camera image of each terminal. However, if the camera image's facial recognition results in the determination that there are multiple people in the image, then real-time images of multiple people may be combined into the slides of the educational content. In this case, performance objects can be composited for each character. For example, performance objects (such as a hat or beard object) can be composited onto the real-time image of grandchild A and the real-time image of grandchild B. Note that if multiple people are visible in the image, they can be users of either the first account or the second account.

[0235] (8) In the above-described embodiment 1, it is assumed that the user inviting other users (e.g., the grandchild's parent, the grandchild themselves) and the user receiving the invitation (e.g., the grandchild's grandparents) live in different locations, and the physical distance between the terminals operated by each user is not taken into consideration. However, the distance between the two terminals can be determined from the location information of user terminal 10A and user terminal 10B. If the distance between the terminals is within a predetermined value (e.g., 5m), the call function using the microphone 16 and speaker 17 may be disabled. The predetermined value is, for example, a distance at which face-to-face conversation is possible in the real world. When direct conversation is possible, direct conversation is recommended rather than conversation through a screen. Location information may also be obtained from the detection signal of a GPS (Global Positioning System) receiver, information from a wireless LAN access point acquired via the communication interface 18 (see Figure 1), information from a BLE (Bluetooth Low Energy) beacon, etc.

[0236] (9) In the above-described embodiment 1, the case in which educational content is played in an educational app was explained. By the way, educational apps may require the use of in-app currency to play educational content. This in-app currency could be, for example, coins. In this case, the coins, as in-app currency, would be associated with and recorded in the user's account. By spending coins, users could play picture book content. They could also spend coins to add decorative frames to screenshots. Furthermore, they could spend coins to add visual effects (such as costumes or decorations within the content) used when playing educational content.

[0237] Coins, used as in-app currency, can be obtained as rewards for using educational content, as rewards for playing mini-games and quizzes within educational content, and through in-app purchases. When a user of the first account and a user of the second account play educational content together, the coins of the user of the second account may be consumed preferentially. Furthermore, regarding the distribution of coins, the distribution rate for users of the first account may be set higher than the distribution rate for users of the second account.

[0238] <Summary 1> [General tasks] One of the purposes of this disclosure is to provide a program, a content linking method, a terminal device, and an educational system that facilitate communication through educational content.

[0239] Issues corresponding to [Appendix 1] One of the purposes of this disclosure is to enable users of a second account to control the progress of educational content. [Note 1] The program relating to this disclosure causes one or more processors to control the execution of educational content consisting of multiple scenes, and when communication with a user of a second account is initiated by a connection request from a user of a first account, the scene on the terminal operated by the user of the first account does not display a first operation object that controls the progress of the educational content, while the scene on the terminal operated by the user of the second account displays a first operation object that controls the progress of the educational content. According to the aforementioned program, it is possible to control the progress of educational content by a user with a second account.

[0240] Issues corresponding to [Appendix 2] One of the purposes of this disclosure is to enable users of a second account to switch scenes. [Note 2] The first operation object is an object that instructs a scene switch, and when the program receives an operation on the first operation object from a user of the second account, it causes one or more processors to send scene switch information to the terminal operated by the user of the first account so that the scene on the terminal operated by the user of the first account can be switched, and then causes the scene switch on the terminal operated by the user of the second account to be executed, as described in Appendix 1. This allows a second account user to switch scenes.

[0241] Issues corresponding to [Appendix 3] One of the purposes of this disclosure is to support users of the second account in progressing through educational content. [Note 3] The first operation object is an object that instructs the display of information to support the progress of educational content by the user of the second account, and when the program receives an operation on the first operation object from the user of the second account, it instructs one or more processors to display information to support the progress of educational content corresponding to the scene displayed on the terminal operated by the user of the second account, as described in Appendix 1. This allows the user of the second account to support the progress of educational content.

[0242] Issues corresponding to [Appendix 4] One of the purposes of this disclosure is to enable the first special effect to be triggered by the actions of a user of the second account. [Note 4] The first operation object is an object that instructs the appearance of the first visual effect, and when the program receives an operation on the first operation object from a user of the second account, it instructs one or more processors to send information about the operation received on the first operation object to the terminal operated by the user of the first account, so that the first visual effect also appears in the scene display on the terminal operated by the user of the first account, and the program as described in Appendix 1 makes the first visual effect appear in the scene display on the terminal operated by the user of the second account. This allows the first visual effect to be triggered by the user of the second account.

[0243] Issues corresponding to [Appendix 5] One of the purposes of this disclosure is to include in educational content a first object that produces a second visual effect only when operated by a user of the first account. [Note 5] The educational content includes a second operation object that instructs the appearance of a second visual effect, and when an operation on the second operation object is received by a user of the first account, one or more processors cause the second visual effect to appear in the scene display on the terminal operated by the user of the first account, and also cause the second visual effect to appear in the scene display on the terminal operated by the user of the second account, by sending information about the operation received on the first object to the terminal operated by the user of the second account, and when an operation on the second operation object is received by a user of the second account, the program does not make the second visual effect appear, as described in Appendix 1. This allows the inclusion of a first object within educational content that only triggers a second visual effect when operated by the user of the first account.

[0244] Issues corresponding to [Appendix 6] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 6] The program described in Appendix 1, which causes one or more processors to display a real-time image of the user of the first account in a predetermined area within educational content determined for each scene, and to display a real-time image of the user of the second account in a specific area within the educational content that is different from the predetermined area. This can enhance the user's immersion in the first account.

[0245] Issues corresponding to [Appendix 7] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 7] The program described in Appendix 6 defines the outer edge of the real-time image of the user of the first account as being defined by the contour of the human body. This can enhance the user's immersion in the first account.

[0246] Issues corresponding to [Appendix 8] One of the purposes of this disclosure is to enable the smooth progress of educational content even when multiple users are using the second account. [Note 8] The program described in Appendix 1, which, when a user of the first account communicates with multiple users of the second account, causes one or more processors to display the first operation object in the scene display on a terminal operated by one of the multiple users of the second account, as specified by the user of the first account. This allows for smooth progression of educational content even when multiple users are using the second account.

[0247] Issues corresponding to [Appendix 9] One of the purposes of this disclosure is to enable the smooth progress of educational content even when multiple users are using the second account. [Note 9] The program described in Appendix 1, which, when a user of the first account communicates with multiple users of the second account, causes one or more processors to display the first operation object on the scene display of a terminal operated by one of the multiple users of the second account, as determined among the multiple users of the second account. This allows for smooth progression of educational content even when multiple users are using the second account.

[0248] Issues corresponding to [Appendix 10] One of the purposes of this disclosure is to enable the smooth progress of educational content even when multiple users are using the second account. [Note 10] The program described in Appendix 1, which, when a user of the first account communicates with multiple users of the second account, causes one or more processors to display the first operation object in the scene display on the terminal operated by one of the multiple users of the second account whose speech has been detected. This allows for smooth progression of educational content even when multiple users are using the second account.

[0249] Issues corresponding to [Appendix 11] One of the purposes of this disclosure is to allow only the user of the second account to disconnect from other devices. [Note 11] A program described in any one of the appendices 1 to 10, which causes one or more processors to send information to the terminal operated by the user of the first account instructing it to disconnect the connection in response to a disconnection operation by the user of the second account, thereby disconnecting the connection between the terminal operated by the user of the first account and the terminal operated by the user of the second account. This allows only the user of the second account to disconnect from other devices.

[0250] Issues corresponding to [Appendix 12] One of the purposes of this disclosure is to require adult operation for certain operations performed on the terminal operated by the user of the first account. [Note 12] A program described in any one of the appendices 1 to 10, which causes one or more processors to display a screen on the terminal operated by the user of the first account, confirming that the operation was performed by an adult, in response to a predetermined operation performed by the user of the first account. This allows for requiring adult operation for certain operations on the device operated by the user of the first account.

[0251] Issues corresponding to [Appendix 13] One of the purposes of this disclosure is to enable users of a second account to control the progress of educational content. [Note 13] The content linking method relating to this disclosure involves one or more processors performing the following: a process to control the execution of educational content composed of multiple scenes; and, when communication with a user of a second account is initiated by a connection request from a user of a first account, the first operation object that controls the progress of the educational content is not displayed in the scene on the terminal operated by the user of the first account, and the first operation object that controls the progress of the educational content is displayed in the scene on the terminal operated by the user of the second account. According to the content linking method described above, it becomes possible for a second account user to control the progress of educational content.

[0252] Issues corresponding to [Appendix 14] One of the purposes of this disclosure is to enable users of a second account to control the progress of educational content. [Note 14] The terminal device relating to this disclosure has a control unit, which controls the execution of educational content composed of multiple scenes, and when communication with a user of a second account is initiated by a connection request from a user of a first account, the scene on the terminal operated by the user of the first account does not display a first operation object that controls the progress of the educational content, while the scene on the terminal operated by the user of the second account displays a first operation object that controls the progress of the educational content. The above-mentioned terminal device makes it possible for a second account user to control the progress of educational content.

[0253] Issues corresponding to [Appendix 15] One of the purposes of this disclosure is to enable users of a second account to control the progress of educational content. [Note 15] The educational system disclosed herein comprises a first terminal device operated by a user of a first account and a second terminal device operated by a user of a second account. The first and second terminal devices control the execution of educational content composed of multiple scenes. When communication with the user of the second account is initiated by a connection request from the user of the first account, the first operation object that controls the progress of the educational content is not displayed in the scene on the first terminal device operated by the user of the first account, while the first operation object that controls the progress of the educational content is displayed in the scene on the second terminal device operated by the user of the second account. According to the above educational system, it is possible to allow a second account user to control the progress of educational content.

[0254] <Summary 2> The main features of the program, content linking method, terminal device, and educational system described in the embodiment are shown below. [General tasks] One of the purposes of this disclosure is to provide a program, a content linking method, a terminal device, and an educational system that can enhance immersion in educational content.

[0255] Issues corresponding to [Appendix 1] One of the purposes of this disclosure is to enhance the immersion of users of the first account into educational content. [Note 1] The program relating to this disclosure causes one or more processors to control the execution of educational content composed of multiple scenes, and when communication is initiated by a connection request from a user of the first account to a user of the second account, the program displays a real-time image of the user of the first account in a predetermined area within the educational content determined for each scene, and displays a real-time image of the user of the second account in a specific area within the educational content that is different from the predetermined area. According to the program described above, it is possible to enhance the immersion of users of the first account into educational content.

[0256] Issues corresponding to [Appendix 2] One of the purposes of this disclosure is to avoid interfering with the operation of the first object. [Note 2] The program described in Appendix 1, which causes one or more processors to arrange predetermined areas so as not to interfere with the display of a first object associated with a visual effect. This prevents interference with the operation of the first object.

[0257] Issues corresponding to [Appendix 3] One of the purposes of this disclosure is to display real-time images of the users of the first account in a manner that does not interfere with the display of the first object. [Note 3] The program described in Appendix 2, which causes one or more processors to position a predetermined area such that a portion of the outer edge of a real-time image displayed in the predetermined area overlaps with the display area of ​​a first object. This allows real-time images of the users of the first account to be displayed without interfering with the display of the first object.

[0258] Issues corresponding to [Appendix 4] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 4] The program described in Appendix 2 causes one or more processors to apply a blurring effect to the outer edges of a real-time image displayed in a predetermined area. This can enhance the user's immersion in the first account.

[0259] Issues corresponding to [Appendix 5] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 5] The program described in Appendix 2, which, when the display area of ​​a second object in educational content that does not have any associated effects overlaps with a predetermined area, displays the real-time image of the user of the first account in front of the second object, and when the display area of ​​the first object overlaps with a predetermined area, displays the real-time image of the user of the first account behind the first object. This can enhance the user's immersion in the first account.

[0260] Issues corresponding to [Appendix 6] One of the purposes of this disclosure is to display real-time images of the second account user in a fixed area of ​​the educational content. [Note 6] The program described in Appendix 1 causes one or more processors to place a specific area at a location common to multiple slides that make up educational content. This allows real-time images of the second account user to be displayed in a fixed area of ​​the educational content.

[0261] Issues corresponding to [Appendix 7] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 7] The program described in Appendix 1 defines the outer edge of the real-time image of the user of the first account by the outline of the human body. This can enhance the user's immersion in the first account.

[0262] Issues corresponding to [Appendix 8] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 8] The program described in Appendix 1, which, if one or more processors do not contain an image of the user of the first account within an image captured by a terminal operated by the user of the first account, displays the educational content in the predetermined area. This can enhance the user's immersion in the first account.

[0263] Issues corresponding to [Appendix 9] One of the purposes of this disclosure is to link the movement or synthesis of a predetermined object to the movement of a user of the first account. [Note 9] The program described in Appendix 1, which modifies at least one of a predetermined object within educational content and a predetermined object to be composited with a real-time image of the user of the first account, in conjunction with movements detected for the user of the first account, on one or more processors. This allows the movement or synthesis of a predetermined object to be linked to the movement of the user of the first account.

[0264] Issues corresponding to [Appendix 10] One of the purposes of this disclosure is to synchronize the movement of objects associated with special effects with the movement of the user of the first account. [Note 10] The specified object is the first object, the program described in Appendix 9. This allows the movement of objects associated with special effects to be linked to the movement of the user in the first account.

[0265] Issues corresponding to [Appendix 11] One of the purposes of this disclosure is to enable the activation of predetermined visual effects associated with the user's actions in the first account. [Note 11] An information processing device as described in Appendix 1, which uses one or more processors to activate a predetermined visual effect on a first object associated with a visual effect, in conjunction with a predetermined operation by a user of the first account. This makes it possible to activate predetermined visual effects associated with the user's actions on the first account.

[0266] Issues corresponding to [Appendix 12] One of the purposes of this disclosure is to activate a predetermined visual effect only in response to a predetermined action performed by a user of the first account. [Note 12] An information processing device as described in Appendix 11, wherein one or more processors do not activate predetermined visual effects for predetermined operations performed by a user of a second account. This allows for the activation of specific visual effects only in response to specific actions performed by the user of the first account.

[0267] Issues corresponding to [Appendix 13] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 13] An information processing device as described in Appendix 1, which causes one or more processors to synthesize specific visual effects corresponding to a scene, or specific visual effects related to specific objects appearing in educational content, onto the real-time image of a user of the first account. This can enhance the user's immersion in the first account.

[0268] Issues corresponding to [Appendix 14] One of the purposes of this disclosure is to enhance the user experience of the first account. [Note 14] A program described in Appendix 13 that, on one or more processors, changes the display size of a specific visual effect according to the display size of the real-time image of the user of the first account. This can enhance the user's immersion in the first account.

[0269] Issues corresponding to [Appendix 15] One of the purposes of this disclosure is to record screen sharing of educational content as a memory. [Note 15] A program described in any one of the appendices 1 to 14, which causes one or more processors to composite and display snapshots of each user, taken automatically while educational content is being displayed or taken in response to a predetermined operation, onto the slide corresponding to the time of capture. This allows you to record screen sharing of educational content as a memory.

[0270] Issues corresponding to [Appendix 16] One of the purposes of this disclosure is to enhance the immersion of users of the first account into educational content. [Note 16] The program described in Appendix 1, which, when communication is initiated by receiving a connection request from a user of the first account, causes one or more processors to display a real-time image of the user of the first account in a predetermined area and a real-time image of the user of the second account in a specific area. This can enhance the user's immersion in educational content for the first account.

[0271] Issues corresponding to [Appendix 17] One of the purposes of this disclosure is to enhance the immersion of users of the first account into educational content. [Note 17] The content linking method disclosed herein involves one or more processors performing the following: a process to control the execution of educational content composed of multiple scenes; a process to display a real-time image of the user of the first account in a predetermined area within the educational content, which is determined for each scene, when communication is initiated by a connection request from the user of the first account to the user of the second account; and a process to display a real-time image of the user of the second account in a specific area within the educational content, which is different from the predetermined area. The above content integration method can enhance the user's immersion in educational content for the first account.

[0272] Issues corresponding to [Appendix 18] One of the purposes of this disclosure is to enhance the immersion of users of the first account into educational content. [Note 18] The terminal device relating to this disclosure has a control unit, which controls the execution of educational content composed of multiple scenes, and when communication is initiated by a connection request sent from a user of the first account to a user of the second account, the control unit displays a real-time image of the user of the first account in a predetermined area within the educational content determined for each scene, and displays a real-time image of the user of the second account in a specific area within the educational content that is different from the predetermined area. According to the above-mentioned terminal device, it is possible to enhance the user's immersion in educational content for the first account.

[0273] Issues corresponding to [Appendix 19] One of the purposes of this disclosure is to enhance the immersion of users of the first account into educational content. [Note 19] The educational system disclosed herein comprises a first terminal device operated by a user of a first account and a second terminal device operated by a user of a second account. The first and second terminal devices control the execution of educational content composed of multiple scenes. When communication is initiated by a connection request from the first terminal device to the second terminal device, a real-time image of the user of the first account is displayed in a predetermined area within the educational content, which is determined for each scene, and a real-time image of the user of the second account is displayed in a specific area within the educational content, which is different from the predetermined area. According to the aforementioned educational system, it is possible to enhance the user's immersion in educational content for the first account. [Explanation of Symbols]

[0274] 1…Educational system, 10, 10A, 10B…User terminal, 20…Management server, 11, 21…Processor, 12, 22…Semiconductor memory, 13, 23…Auxiliary storage device, 14…Touch panel, 15…Camera, 16…Microphone, 17…Speaker, 18, 24…Communication interface, 101, 201…Control unit, 110, 210…Storage unit, 111…Educational app, 112, 212…Educational content, 202…Presentation unit, 203…Synthesis unit, 204…Detection unit, 205…Confirmation unit, 211…User information, 213…Streaming image

Claims

1. One or more processors, Control the execution of educational content composed of multiple scenes. When communication with the user of the second account is initiated by a connection request from the user of the first account, In the scene on the terminal operated by the user of the first account, the first object that controls the progress of the educational content is not displayed. The scene on the terminal operated by the user of the second account displays the first object that controls the progress of the educational content. A program that performs a process.

2. The first object is an object that instructs the scene to switch, When an operation on the first object is received by a user of the second account, In the above one or more processors, In order to enable the scene to be switched on the terminal operated by the user of the first account, scene switching information is sent to the terminal operated by the user of the first account. To cause the user of the second account to perform the scene switching on the terminal they operate, The program according to claim 1.

3. The first object is an object that instructs the display of information to support the user of the second account in progressing through the educational content, When an operation on the first object is received by a user of the second account, In the above one or more processors, The terminal operated by the user of the second account displays information that supports the progress of the educational content in accordance with the display of the scene. The program according to claim 1.

4. The first object is an object that instructs the appearance of the first special effect, When an operation on the first object is received by a user of the second account, In the above one or more processors, In order to make the first effect appear in the display of the scene on the terminal operated by the user of the first account, the terminal operated by the user of the first account is instructed to send information about the operation received for the first object. The first special effect is made to appear in the display of the scene on the terminal operated by the user of the second account. The program according to claim 1.

5. The aforementioned educational content includes a second object that instructs the appearance of a second special effect, When an operation on the second object is received by a user of the first account, The one or more processors mentioned above are: The second effect is made to appear in the display of the scene on the terminal operated by the user of the first account, and the second effect is also made to appear in the display of the scene on the terminal operated by the user of the second account, by transmitting information about the operation received for the second object to the terminal operated by the user of the second account. If an operation on the second object is received by a user of the second account, the second visual effect will not be displayed. The program according to claim 1.

6. In the above one or more processors, In a predetermined area within the educational content, which is determined for each scene, the real-time image of the user of the first account is displayed. In a specific area within the educational content that is different from the predetermined area, the real-time image of the user of the second account is displayed. The program according to claim 1.

7. The outer edge of the real-time image of the user of the first account is defined by the contour of the human body. The program according to claim 6.

8. When a user of the first account communicates with multiple users of the second account, In the above one or more processors, To display the first object in the scene on a terminal operated by one of the users of the first account, who is designated by a user of the first account, among the multiple users of the second account. The program according to claim 1.

9. When a user of the first account communicates with multiple users of the second account, In the above one or more processors, The first object is displayed on the scene on a terminal operated by one person determined among multiple users of the second account. The program according to claim 1.

10. When a user of the first account communicates with multiple users of the second account, In the above one or more processors, The first object is displayed in the scene on the terminal operated by one of the users of the second account whose speech was detected. The program according to claim 1.

11. In the above one or more processors, In response to a disconnection operation by the user of the second account, the system causes information instructing the user of the first account to disconnect the connection to be sent to the terminal operated by the user of the first account, thereby disconnecting the connection between the terminal operated by the user of the first account and the terminal operated by the user of the second account. The program according to any one of claims 1 to 10.

12. In the above one or more processors, In response to a predetermined operation performed by the user of the first account, a screen confirming that the operation is being performed by an adult is displayed on the terminal operated by the user of the first account. The program according to any one of claims 1 to 10.

13. One or more processors, A process that controls the execution of educational content consisting of multiple scenes, When communication with a user of the second account is initiated by a connection request from a user of the first account, the scene on the terminal operated by the user of the first account does not display the first object that controls the progress of the educational content, and the scene on the terminal operated by the user of the second account displays the first object that controls the progress of the educational content. A method for integrating content to perform this action.

14. It has a control unit, The control unit, Control the execution of educational content composed of multiple scenes. When communication with the user of the second account is initiated by a connection request from the user of the first account, In the scene on the terminal operated by the user of the first account, the first object that controls the progress of the educational content is not displayed. The scene on the terminal operated by the user of the second account displays the first object that controls the progress of the educational content. Terminal device.

15. The first terminal device operated by the user of the first account, A second terminal device operated by the user of the second account, It has, The first terminal device and the second terminal device are, Control the execution of educational content composed of multiple scenes. When communication with the user of the second account is initiated by a connection request from the user of the first account, In the scene on the first terminal device operated by the user of the first account, the first object that controls the progress of the educational content is not displayed. The scene in the second terminal device operated by the user of the second account displays the first object that controls the progress of the educational content. Educational system.

Images