Information processing device, information processing system, and program

Abstract

To easily provide moving images matching with a user.SOLUTION: An information processing device concerning one aspect of the present disclosure comprises: a playback control unit which controls playback of moving images displayed on a display unit according to a motion state of a user; a picture control unit which controls display by the display unit; and a script processing unit which commands control to the playback control unit and the picture control unit on the basis of a script in which a first command for commanding playback control according to playback time of the moving images to the playback control unit, and a second command for commanding display control according to the playback time to the picture control unit are described.SELECTED DRAWING: Figure 6

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing system, and a program. [Background technology]

[0002] 2. Description of the Related Art Conventionally, there is known an information processing device that changes a playback image displayed on a display device in accordance with a user's walking state, thereby allowing a user viewing the playback image to have a variety of outing experiences.

[0003] Also, a technique has been disclosed for outputting sound in accordance with the progress of video such as played images in order to allow the user to walk while experiencing a realistic sensation of taking a walk (see, for example, Patent Document 1). Summary of the Invention [Problem to be solved by the invention]

[0004] However, the technology described in Patent Document 1 does not provide a video that adds elements that attract the user's interest or that are effective for rehabilitation to the video being played, that is, a video that is tailored to the user. Also, it takes a lot of time and effort to create a video that is tailored to the user by editing the video being played.

[0005] The present disclosure aims to easily provide videos tailored to users. [Means for solving the problem]

[0006] An information processing device according to one embodiment of the present disclosure includes a playback control unit that controls playback of a video displayed on a display unit in accordance with a user's exercise state, a screen control unit that controls display by the display unit, and a script processing unit that instructs the playback control unit and the screen control unit to perform control based on a script that contains a first command to instruct the playback control unit to perform playback control in accordance with a playback time of the video, and a second command to instruct the screen control unit to perform display control in accordance with the playback time. Effect of the Invention

[0007] According to the present disclosure, videos tailored to users can be easily provided. [Brief description of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram illustrating an example of an information processing system according to a first embodiment of the present disclosure. [Diagram 2] FIG. 2 is a schematic diagram showing an information processing system according to a first modified example of the first embodiment of the present disclosure. [Diagram 3] FIG. 11 is a schematic diagram showing an information processing system according to a second modified example of the first embodiment of the present disclosure. [Figure 4] FIG. 13 is a schematic diagram showing an information processing system according to a third modified example of the first embodiment of the present disclosure. [Diagram 5] 1 is a block diagram showing an example of a hardware configuration of an information processing device according to a first embodiment of the present disclosure. [Figure 6] 2 is a block diagram showing an example of a functional configuration of an information processing device included in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 7] 2 is a diagram showing an example of a user information file in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 8] 2 is a diagram showing an example of a script group in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 9] 2 is a diagram showing an example of a group of thumbnails of scripts in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 10] 1 is a diagram showing a group of moving images in the information processing system according to the first embodiment of the present disclosure. [Figure 11] FIG. 2 is a diagram showing an example of a script in the information processing system according to the first embodiment of the present disclosure. [Figure 12] 5 is a flowchart showing an example of an overall operation of the information processing system according to the first embodiment of the present disclosure. [Figure 13]5 is a flowchart showing an example of a process performed by a playback control unit according to the first embodiment of the present disclosure. [Figure 14A] 4 is a first flowchart showing an example of processing by a script processing unit according to the first embodiment of the present disclosure. [Figure 14B] 11 is a second flowchart showing an example of processing by the script processing unit according to the first embodiment of the present disclosure. [Figure 14C] 13 is a third flowchart showing an example of processing by the script processing unit according to the first embodiment of the present disclosure. [Figure 14D] 4 is a fourth flowchart showing an example of processing by the script processing unit according to the first embodiment of the present disclosure. [Figure 15] 1 is a diagram showing a first example of a video screen displayed in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 16] 13 is a diagram showing a second example of a video screen displayed in the information processing system according to the first embodiment of the present disclosure. FIG. [Figure 17] FIG. 11 is a diagram showing a third example of a video screen displayed in the information processing system according to the first embodiment of the present disclosure. [Figure 18] FIG. 11 is a block diagram showing an example of a functional configuration of a computer according to a second embodiment of the present disclosure. [Figure 19] 13 is a flowchart illustrating an example of an overall process by a computer according to a second embodiment of the present disclosure. [Figure 20] 13 is a flowchart showing an example of a line editing process by a script editing unit according to a second embodiment of the present disclosure. [Figure 21] FIG. 11 is a diagram showing an example of a script creation screen displayed by a computer according to a second embodiment of the present disclosure. [Figure 22] FIG. 13 is a diagram illustrating an example of a configuration of an information processing system according to a third embodiment of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] An information processing device, an information processing system, and a program according to an embodiment of the present disclosure will be described in detail with reference to the drawings. However, the following embodiments are examples of an information processing device, an information processing system, and a program for realizing the technical ideas of the embodiments of the present disclosure, and are not limited to the following. In addition, in the following description, the same names and symbols indicate the same or similar components or functions, and detailed descriptions will be omitted as appropriate.

[0010] [First embodiment] <Configuration of Information Processing System 100> FIG. 1 is a schematic diagram illustrating an example of an information processing system 100 according to the first embodiment of the present disclosure.

[0011] The information processing system 100 is a system that promotes exercise of a user U who uses the information processing system 100. The information processing system 100 controls the playback of the video Mv displayed on the display unit 4 according to the exercise state of the user U. For example, the information processing system 100 speeds up the playback speed of the video Mv when the user U's walking speed is fast, and slows down the playback speed of the video Mv when the user U's walking speed is slow. Alternatively, the information processing system 100 plays the video Mv in the forward direction when the user U moves forward, and plays the video Mv in the reverse direction when the user U moves backward. On the other hand, the video Mv is, for example, a video captured of a landscape that comes into the field of view of a walker when taking a walk around a tourist spot.

[0012] The user U can have a simulated experience of strolling around a tourist spot by viewing the video Mv, whose playback speed changes according to the user U's walking speed and whose playback direction changes according to the user U's walking direction, through the display unit 4. In this way, the information processing system 100 can encourage the user U to walk while having fun and not get bored.

[0013] 1, the information processing system 100 includes an information processing device 1 and a motion sensor 2 that outputs motion state information of a user U. In the example shown in FIG. 1, the information processing system 100 also includes a gaze sensor 3 that detects the gaze of the user U.

[0014] The motion sensor 2 includes a pressure sensor 21 that detects pressure when stepped on by the soles of both feet of the user U, and a support member 22 that the user U can hold onto when exercising. The motion sensor 2 outputs a pressure detection signal by the pressure sensor 21 to the information processing device 1 as motion information related to the motion state of the user U. The motion sensor 2 may be an acceleration sensor that detects stepping walking, or an optical sensor.

[0015] The gaze sensor 3 detects the gaze of the user U and outputs gaze information related to the gaze of the user U to the information processing device 1. The gaze sensor 3 includes a camera and an image processing circuit. The gaze sensor 3 detects the gaze of the user U by specifying the position of the image area of ​​the iris included in the facial image of the user U captured by the camera using the image processing circuit. The gaze sensor 3 may be a directional key or a joystick of the operation unit 6 operated by the user.

[0016] The information processing device 1 acquires information on the walking speed of the user U from the motion information input from the motion sensor 2. The information processing device 1 determines the period in which the user U steps on the pressure sensor 21 based on the motion information, and acquires the walking speed of the user U by calculation from this period. The information processing device 1 also detects the walking direction of the user U by detecting whether the user U steps on the forward pressure sensor 21 or the backward pressure sensor 21. The information processing device 1 varies the playback speed or playback direction, etc. of the video Mv displayed on the display unit 4 according to the walking speed or walking direction, etc. of the user U. The display unit 4 is a display device such as a liquid crystal display, an organic EL (Electro Luminescence) display, or a plasma display.

[0017] 1, the information processing device 1 varies the image area of ​​the video Mv displayed on the display unit 4 in response to gaze information from the gaze sensor 3. For example, when the gaze of the user U is directed to the right, the information processing device 1 controls the playback of the video Mv so that the center of the right-side image area of ​​the entire image area of ​​the video Mv is displayed at the center of the screen of the display unit 4. By viewing the video Mv in which the image area displayed on the display unit 4 changes in response to the user U's gaze direction, the user U can feel a more realistic simulated experience of a walk.

[0018] In the example shown in FIG. 1, the information processing system 100 includes a speaker 5 that generates sound. The speaker 5 is incorporated in the information processing device 1. The information processing system 100 causes the speaker 5 to generate sound in accordance with the video Mv displayed on the display unit 4. For example, if the video Mv includes a scene that is visually recognized when walking on stone paving, the information processing system 100 causes the speaker 5 to generate the sound of footsteps walking on the stone paving. This allows the user U to have a more realistic simulated experience by utilizing his or her sense of hearing.

[0019] The user U receives motion stimuli, visual stimuli, auditory stimuli, etc. through the moving image Mv that changes depending on the motion information from the motion sensor 2 and the gaze information from the gaze sensor 3, and can walk while enjoying the scenery.

[0020] <Modification> The information processing system according to the first embodiment of the present disclosure is not limited to the example shown in Fig. 1, and various modifications are possible. Various modifications will be described below. Note that names and symbols that are the same as those in the embodiments of the present disclosure already described indicate the same or similar members or configurations, and detailed descriptions will be omitted as appropriate. This also applies to the modifications and embodiments of the present disclosure described below.

[0021] (First Modification) Fig. 2 is a schematic diagram showing an information processing system 100a according to a first modified example. The first modified example differs from the information processing system 100 shown in Fig. 1 in that a plurality of users U can use the information processing system 100a in parallel. Since a plurality of users U can use the information processing system 100a in parallel, a simulated experience such as a walk can be shared among the plurality of users U, and the user U can exercise while having more fun compared to when using the system alone.

[0022] 2, four users U use the information processing system 100a. Four motion sensors 2 are provided corresponding to the four users U. The information processing device 1 can acquire motion information of each of the four users U from each of the four motion sensors 2. Furthermore, in the information processing system 100a, a video Mv viewed by each of the four users U is displayed on one display unit 4, so that the video Mv viewed by each user U can be shared among the four users U.

[0023] By having multiple users U use the information processing system 100 in parallel, the multiple users U can recall and talk about scenes and share their impressions while performing exercises that lead to rehabilitation.

[0024] A plurality of display units 4 may be connected to the information processing device 1. This allows users U on different floors to share the video Mv and exercise together.

[0025] 2, one user U is walking while standing, and the other three users U are walking while sitting on chairs 23. In this way, the information processing system 100a can select walking while standing or while sitting, etc., according to the preferences and health condition of the user U.

[0026] (Second Modification) 3 is a schematic diagram showing an information processing system 100b according to a second modified example. The second modified example differs from the first modified example in that an operator 200 separate from the user U can operate the information processing system 100b. The operator 200 is, for example, a caregiver who cares for a care recipient as the user U. Since the operator 200 can operate the information processing system 100b, the user U can reduce the effort required for operation and can smoothly operate the information processing system 100b. This can increase the user U's willingness to use the information processing system 100b and encourage the user U to exercise.

[0027] The operator 200 uses the operation unit 6 to select a video to be displayed on the display unit 4, and to give instructions such as starting, stopping, or restarting playback of the video. The operation unit 6 may be a touch panel, an operation button, a keyboard, a joystick, or a combination of these. The operation unit 6 may be a remote control that is detachable from the information processing system 100 or the information processing device 1. The operation unit 6 may also be an information processing terminal such as a tablet or a smartphone.

[0028] (Third Modification) 4 is a schematic diagram showing an information processing system 100c according to a third modified example. The third modified example differs from the second modified example in that the display unit 4 is a VR (Virtual Reality) glass. The VR glass is an example of a head-mounted display device. By using the VR glass for the display unit 4, the user U can have a more realistic simulated experience, and the user U can be encouraged to exercise.

[0029] The VR glasses may be a head mounted display. The VR glasses are not limited to the goggle type shown in FIG. 4, but may be glasses.

[0030] <Configuration example of information processing device 1> (Hardware configuration example) 5 is a block diagram showing an example of a hardware configuration of the information processing device 1. The information processing device 1 is constructed by, for example, a computer. The information processing device 1 has a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, a HDD (Hard Disk Drive) / SSD (Solid State Drive) 104, and an I / F (Interface) 105. These are connected to each other via a system bus B so as to be able to communicate with each other.

[0031] The CPU 101 executes control processing including various kinds of arithmetic processing. The ROM 102 is a non-volatile memory that stores programs used to drive the CPU 101, such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory used as a work area for the CPU 101. The HDD / SSD 104 is a non-volatile memory that can store various information, programs, etc. used for control by the information processing device 1.

[0032] The I / F 105 is an interface for communicating between the information processing device 1 and a device or apparatus other than the information processing device 1. The I / F 105 can also communicate with a device or apparatus other than the information processing device 1 via a network or the like. The devices other than the information processing device 1 include the motion sensor 2, the gaze sensor 3, the display unit 4, the speaker 5, the operation unit 6, etc. The devices other than the information processing device 1 include a server 300 connected to be able to communicate via a network N, etc.

[0033] (Example of functional configuration) FIG. 6 is a block diagram showing an example of a functional configuration of the information processing device 1. As shown in FIG.

[0034] In the example shown in FIG. 6, the information processing device 1 has an operation receiving unit 11, an acquisition unit 12, a playback control unit 13, a screen control unit 14, a sound control unit 14-1, a script processing unit 15, a memory unit 16, and an output unit 17.

[0035] Each function of the operation reception unit 11, the acquisition unit 12, and the output unit 17 is realized by the I / F 105, etc. Note that some of the functions of the acquisition unit 12 and the output unit 17 may be realized by a processor such as the CPU 101 executing processing defined in a program stored in the ROM 102, etc. The function of the storage unit 16 is realized by the RAM 103, the HDD / SSD 104, etc. Each function of the playback control unit 13, the screen control unit 14, and the script processing unit 15 is realized by a processor such as the CPU 101 executing processing defined in a program stored in the ROM 102, etc.

[0036] Each function of the information processing device 1 can be realized by one or more processing circuits. The "processing circuit" here includes devices such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a DSP (digital signal processor), and a conventional circuit module designed to execute each function described above. Some of the above functions of the information processing device 1 can also be realized by an external device such as an external PC (Personal Computer) communicably connected to the information processing device 1, or a server. Furthermore, some of the above functions of the information processing device 1 can also be realized by distributed processing between the information processing device 1 and these external devices.

[0037] The operation reception unit 11 controls communication with the operation unit 6 to receive various operations performed by an operator using the operation unit 6. The operations in the example shown in Fig. 6 include selection of information about a user U who uses the information processing system 100, selection of a script Sc, selection of a video Mv, starting or stopping playback of the video Mv, and the like.

[0038] The acquisition unit 12 acquires motion information Mi of the user U from the motion sensor 2 by controlling communication with the motion sensor 2. The acquisition unit 12 also acquires gaze information Gi of the user U from the gaze sensor 3 by controlling communication with the gaze sensor 3. The acquisition unit 12 passes the acquired motion information Mi and gaze information Gi to the playback control unit 13.

[0039] The playback control unit 13 controls playback of the video Mv displayed on the display unit 4 in accordance with the motion state of the user U. In the example shown in Fig. 6, the playback control unit 13 causes the display unit 4 to display, via the screen control unit 14, the video Mv played at a playback speed or in a playback direction in accordance with the motion state of the user U obtained from the motion information Mi.

[0040] The screen control unit 14 controls the display by the display unit 4. In the example shown in Fig. 6, the screen control unit 14 controls the display by the display unit 4 by outputting a signal for controlling the display operation by the display unit 4 to the display unit 4 via the output unit 17. The screen control unit 14 also receives information about the moving image Mv selected by the operator via the operation receiving unit 11, and acquires the moving image Mv from the storage unit 16 based on this information. The screen control unit 14 causes the moving image Mv, whose playback is controlled by the playback control unit 13, to be displayed on the display unit 4.

[0041] The sound control unit 14-1 controls the output of sound from the speaker 5. In the example shown in Fig. 6, the sound control unit 14-1 controls the output of sound from the speaker 5 by outputting a signal for controlling the sound output operation of the speaker 5 to the speaker 5 via the output unit 17.

[0042] The script processing unit 15 issues control commands to the playback control unit 13 and the screen control unit 14 based on a script Sc in which a first command Cm1 for commanding playback control according to the playback time of the video Mv and a second command Cm2 for commanding the screen control unit to control display according to the playback time of the video Mv are written. A specific example of the script Sc will be described later with reference to FIG. 8.

[0043] Here, in this specification, "playback time of the video Mv" refers to the time calculated from the start of the video Mv. However, "playback time of the video Mv" does not mean the time that has actually elapsed since the start of playback of the video Mv, but represents the order (position) of the images included in the video Mv in the video Mv. For example, if the frame rate of the video Mv is Fs and the playback time is Rp, the playback time Rp represents the order (position) of the (Rp / Fs)th image among the multiple images included in the video Mv in order for each frame rate of the video Mv from the playback start time. Therefore, the playback time of the video Mv can also be expressed as the playback position. Meanwhile, in this specification, "elapsed playback time of the video Mv" refers to the time that has actually elapsed since the start of playback of the video Mv. Also, "video time" refers to the time required to play the entire video Mv.

[0044] 6, the storage unit 16 stores user information 161 in which various information related to each of a plurality of users U is recorded, a script group 162 including a plurality of scripts Sc associated with a plurality of videos Mv, and a video group 163 including a plurality of videos Mv.

[0045] 5 instead of the information processing device 1. In this case, the screen control unit 14 and the script processing unit 15 acquire the user information 161, the script Sc, and the video Mv via the network N. In addition, the information processing device 1 may store some of the user information 161, the script group 162, and the video group 163, and the server 300 may store the other parts.

[0046] The output unit 17 controls communication between the information processing device 1 and external devices in response to commands from the screen control unit 14. The external devices include the display unit 4, the speaker 5, the operation unit 6, the server 300, and the like.

[0047] Here, the user U may have different interests depending on his / her hometown, upbringing, living environment, work, family situation, etc. Furthermore, if an operator 200 is present, the user U may have different interests depending on the experience of the operator 200. Furthermore, the time the user U uses the information processing system 100 may differ depending on the physical strength and physical ability of the user U. Therefore, it is preferable to provide a video Mv that matches the interests of the user U and the operator 200. However, when editing a video to match the interests of the user U and the operator 200, tools such as editing software and editing skills are required for video editing, and editing according to requests may not be easily performed.

[0048] In the information processing device 1, the script processing unit 15 issues control commands to the playback control unit 13 and the screen control unit 14 based on the script Sc. By controlling the playback control unit 13 and the screen control unit 14 in response to the commands, it is possible to superimpose comments, quizzes, images used in quizzes, etc. on the video Mv, adjust the video time by deleting images other than the images that are of interest in the video Mv, and join different videos together, without editing the video Mv in advance. In addition, by using the script Sc, it is possible to create a script Sc for each of multiple users U and each of multiple operators 200. As a result, the information processing device 1 and the information processing system 100 can easily provide a video Mv that is tailored to the user U or the operator 200.

[0049] 6, the script processing unit 15 issues control commands to the playback control unit 13 and the screen control unit 14 based on the script Sc stored in the storage unit 16 included in the information processing device 1. By including the storage unit 16 in the information processing device 1, access to the script Sc becomes easier compared to a case where the storage unit 16 is provided in an external device such as the server 300.

[0050] 6, the storage unit 16 stores a script Sc corresponding to the user U or a group consisting of multiple users U. In response to a selection input of the user U or group, the script processing unit 15 refers to the storage unit 16 and acquires the script Sc corresponding to the user U or a group consisting of multiple users U. This makes it possible to easily provide a video Mv tailored to the user U or a group consisting of multiple users U.

[0051] The first embodiment of the present disclosure also includes a program. The program causes the computer to execute a process of controlling the playback of the video Mv displayed on the display unit 4 by the playback control unit 13 in accordance with the exercise state of the user U, and controlling the display by the display unit 4 by the screen control unit 14. The program also causes the computer to execute a process of instructing the playback control unit 13 and the screen control unit 14 to perform control based on a script Sc in which a first command Cm1 instructing the playback control unit 13 to perform playback control in accordance with the playback time of the video Mv and a second command Cm2 instructing the screen control unit 14 to perform display control in accordance with the playback time are described. Such a program can provide the same effects as those of the information processing device 1 and the information processing system 100 described above.

[0052] <Examples of information stored in the storage unit 16> The information stored in the storage unit 16 in the information processing system 100 will be described with reference to FIGS.

[0053] Fig. 7 is a diagram showing an example of user information 161. Fig. 8 is a diagram showing an example of a script group 162. Fig. 9 is a diagram showing an example of a thumbnail group 162s in the script group 162. Fig. 10 is a diagram showing an example of a moving image group 163. The moving image group 163 includes a plurality of electronic files corresponding to a plurality of moving images Mv.

[0054] The user information 161 is an electronic file in which information on each of a plurality of users U is recorded. The script group 162 includes a plurality of electronic files corresponding to a plurality of scripts Sc. The file extension of the script Sc is, for example, "scr." The thumbnails included in the thumbnail group 162s are electronic image files in which images used to allow the operator to easily recognize the contents of a video Mv when the operator is asked to select a video Mv are recorded. The image file extension of the thumbnails is, for example, "jpg." The file extension of the video Mv is, for example, "mp4."

[0055] 10, the moving image group 163 includes an image file "1.jpg" that is displayed on the display unit 4 before the start of moving image playback, an image file "end.jpg" that is displayed on the display unit 4 after the end of moving image playback, and "QR1.jgp" that displays a QR code. The moving image group 163 includes an audio file "audio1.mp3" that is output from the speaker 5 before the start of moving image playback, and an audio file "endsound2.jpg" that is output from the speaker 5 after the end of moving image playback.

[0056] <Example of script Sc> Fig. 11 is a diagram showing an example of a script Sc in the information processing system 100. In the example shown in Fig. 11, the script Sc is in a table format. In each row of the table, a first command Cm1 or a second command Cm2 is described. The row of the table represents a process including the first command Cm1 or the second command Cm2.

[0057] Each column of the table corresponds to an item to be specified. Among the columns of the table, "row number" is a number indicating each row in the table of the script Sc. "Processing type" indicates the type (in other words, classification) of processing to be performed according to the playback time of the video Mv. The content of the processing to be performed for each "processing type" is predetermined.

[0058] "Playback time (seconds)" indicates the playback time for performing a specified process on the video Mv. "Video file information" indicates the file name of the video Mv. "Text to display on screen" indicates the content of the text, such as letters or numbers, to be displayed on the screen of the display unit 4. "Display image" indicates the file name of the image to be displayed on the display unit 4 while the video Mv is being played. "Sound file" indicates the file name of the file in which sound information output from the speaker 5 is recorded. "Automatic display disappearance period" indicates the period until the text or image displayed on the display unit 4 during script processing is automatically erased after display.

[0059] "Video stop retention period" indicates the period during which text or images to be displayed on the display unit 4 are retained during script processing. "Jump destination time (frames / sec)" indicates the playback time of the jump destination when jumping playback time. "Video playback stop command ignoring period" indicates the period during which a playback stop command for video Mv is ignored. "Jump destination video file information" indicates the file name of another video file when another video file is interrupted and played.

[0060] In the "processing types" in FIG. 11, type T0 is a process for recording explanatory text for a thumbnail, specifying file information for a movie Mv, and specifying the time to start playing the movie Mv.

[0061] Type T1 is a process in which, when the elapsed playback time of the video Mv reaches the specified playback time, any text or image that should be displayed on the display unit 4 is displayed on the display unit 4, the video Mv is played according to the physical activity state of the user U, and the display on the display unit 4 is erased when the time specified in the "automatic display disappearance period" has elapsed.

[0062] Type T2 is a process in which, when the elapsed playback time of the video Mv reaches the specified playback time, any text or image to be displayed on the display unit 4 is displayed on the display unit 4, and playback of the video Mv is stopped regardless of the physical activity state of the user U, and when the time specified in the "video stop retention period" has elapsed, playback of the video Mv is resumed according to the physical activity state of the user U.

[0063] Type T3 is a process that stops playback of the current video Mv when the elapsed playback time of the video Mv reaches the specified playback time, and jumps playback of the video to the playback time specified in the "jump destination time (frames / second)".

[0064] Type T4 is a process that stops playback of the current video Mv when the elapsed playback time of the video Mv reaches the specified playback time, and jumps playback of the video to the "jump destination time (frames / second)" in the video file specified in the "jump destination video file information".

[0065] Type T5 is a process in which, when the elapsed playback time of the video Mv reaches a specified playback time, a command is issued to the playback control unit 13 to ignore a command to stop the video Mv, and once the "video playback stop command ignoring period" has elapsed, a command to stop playback of the video Mv according to the exercise state of the user U can be accepted.

[0066] Type T6 is a process for ending the playback of the moving image Mv when the elapsed playback time of the moving image Mv reaches a designated playback time.

[0067] In the process (lines) illustrated in Fig. 11, commands for causing the playback control unit 13 to execute playback control, such as a command to jump playback to "jump destination time (frames / second)" and a command to jump playback to a video specified in "jump destination video file information", correspond to a first command Cm1. Also, in the process illustrated in Fig. 11, commands for causing the screen control unit 14 to execute display control, such as a command to display text specified in "text to display on screen" on the screen of the display unit 4 and a command to display an image specified in "display image" on the screen of the display unit 4, correspond to a second command Cm2.

[0068] The items corresponding to each column of the script Sc may be increased or decreased according to the specifications of the information processing system 100. Furthermore, the description format of the script Sc is not limited to a table format, and may be other formats such as a code format in which code is described. The processing types are also not limited to the nine categories illustrated in FIG. 11, and may be increased or decreased according to the specifications of the information processing system 100.

[0069] <Example of operation of information processing system 100> (Overall operation) Fig. 12 is a flowchart showing an example of the overall operation of the information processing system 100. Fig. 12 shows an overall outline of the operation performed by the information processing system 100 when the user U uses the information processing system 100, in other words, the operation for allowing the user U to use the information processing system 100. For example, the information processing system 100 starts the operation of Fig. 12 when the information processing system 100 accepts an operation to start using the information processing system 100 by an operator using the operation unit 6.

[0070] First, in step S11, the information processing system 100 causes the display unit 4 to display a setting screen for user information.

[0071] Next, in step S12, the information processing system 100 receives, through the operation receiving unit 11 of the information processing device 1, a selection input of information on the number of users U by the operator via the operation unit 6.

[0072] Next, in step S13, the information processing system 100 receives input of the name information of the user U from an operator via the operation unit 6, through the operation reception unit 11 of the information processing device 1. Note that the operator here is the user U, the operator 200 shown in FIG. 3, or the like.

[0073] Next, in step S14, the information processing system 100 refers to the storage unit 16 based on the user U's name information to obtain the script Sc corresponding to the user U.

[0074] Next, in step S15, the information processing system 100 causes the screen control unit 14 of the information processing device 1 to display on the display unit 4 thumbnails and text for selecting moving images.

[0075] Next, in step S16, the information processing system 100 causes the operation receiving unit 11 of the information processing device 1 to receive, via the operation unit 6, a selection input of the moving image Mv from the operator.

[0076] Next, in step S17, the information processing system 100 starts playback of the selected video Mv by the playback control unit 13 of the information processing device 1. The screen control unit 14 displays the video Mv on the display unit 4 in response to an instruction from the playback control unit 13. After step S17, the information processing system 100 continues playing the video Mv until the playback of the video Mv ends.

[0077] Next, in step S18, the information processing system 100 executes script processing by the script processing unit 15 of the information processing device 1. The script processing unit 15 issues control commands to the playback control unit 13 and the screen control unit 14 in accordance with the description of each line of the script Sc illustrated in FIG.

[0078] Next, in step S19, the information processing system 100 determines whether or not to end the playback of the video Mv. For example, the information processing system 100 can determine whether or not to end the playback of the video Mv by determining, via the screen control unit 14 of the information processing device 1, whether or not the elapsed playback time of the video Mv matches the video time of the video Mv.

[0079] In step S19, when it is determined that the playback of the video Mv is not to be ended (step S19, NO), the information processing system 100 performs the operation of step S18 again. On the other hand, when it is determined that the playback of the video Mv is to be ended (step S19, YES), the information processing system 100 determines in step S20 whether or not to end the use of the information processing system 100. For example, the information processing system 100 can determine whether or not to end the use of the information processing system 100 in response to an operation input by the operator via the operation unit 6.

[0080] In step S20, if it is determined that the use of the information processing system 100 is not to be terminated (step S20, NO), the information processing system 100 performs the operations from step S11 onwards again. On the other hand, if it is determined that the use is to be terminated (step S20, YES), the information processing system 100 terminates the operation.

[0081] In this manner, the information processing system 100 allows the user U to use the information processing system 100. When multiple users U view a video Mv displayed on one display unit 4, the operator 200 can also select a video Mv or a script Sc that is common to each of the multiple users U.

[0082] (Recycling) Fig. 13 is a flowchart showing an example of processing by the playback control unit 13. Fig. 13 shows an example of playback control processing of a moving image Mv by the playback control unit 13. For example, the playback control unit 13 starts the operation of Fig. 13 when the processing shown in Fig. 12 reaches step S17 as a starting condition.

[0083] First, in step S31, the reproduction control unit 13 determines whether or not there is an input from the motion sensor 2.

[0084] In step S31, if it is determined that there is no input (step S31, NO), the playback control unit 13 performs the process of step S31 again. On the other hand, if it is determined that there is an input (step S31, YES), in step S32, the playback control unit 13 plays back the video Mv in response to the input from the motion sensor 2, and causes the video Mv to be displayed on the display unit 4 via the screen control unit 14.

[0085] Next, in step S33, the reproduction control unit 13 determines whether or not there is an input from the line-of-sight sensor 3.

[0086] In step S33, if it is determined that there is no input (step S33, NO), the playback control unit 13 proceeds to step S35. On the other hand, if it is determined that there is an input (step S33, YES), in step S34, the playback control unit 13 changes the image area of ​​the moving image Mv displayed on the display unit 4 in response to the input from the line of sight sensor 3, and causes the moving image Mv to be displayed on the display unit 4 via the screen control unit 14.

[0087] Next, in step S35, the playback control unit 13 obtains the elapsed playback time of the video Mv and passes it to the script processing unit 15.

[0088] Next, in step S36, the reproduction control unit 13 determines whether or not there is an input from the motion sensor 2 to indicate that motion has stopped.

[0089] In step S36, if it is determined that there is no input (step S36, NO), the playback control unit 13 performs the processes from step S31 onwards again. On the other hand, if it is determined that there is an input (step S36, YES), the playback control unit 13 stops the playback of the video Mv in step S37. After that, the playback control unit 13 ends the operation.

[0090] In this manner, the playback control unit 13 can perform playback processing of the moving image Mv.

[0091] (Script processing) 14A to 14D are flowcharts showing an example of processing by the script processing unit 15. FIG. 14A to 14D show a procedure in which the script processing unit 15 executes processing defined in each row of the table of the script Sc in order from the first row based on the script Sc. For example, the playback control unit 13 starts the processing of FIG. 14A to 14D when the processing shown in FIG. 12 reaches step S18 as a starting condition.

[0092] First, in step S41, the script processing unit 15 determines whether the processing type is type T0.

[0093] In step S41, if it is determined that the type is T0 (step S41, YES), the script processing unit 15 acquires the moving image Mv by referring to the memory unit 16 based on the file information of the moving image Mv from the operation receiving unit 11 in step S42.

[0094] Next, in step S43, the script processing unit 15 passes the time at which to start playing the video Mv to the playback control unit 13. After that, the script processing unit 15 moves the process to step S84.

[0095] On the other hand, if it is determined in step S41 that the processing type is not type T0 (step S41, NO), the script processing unit 15 determines in step S44 whether the processing type is type T1.

[0096] In step S45, if it is determined that the type is T1 (step S44, YES), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S45.

[0097] Next, in step S46, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0098] In step S46, if it is determined that the elapsed playback time is not equal to the playback time (step S46, NO), the script processing unit 15 performs the process of step S46 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S46, YES), the script processing unit 15 determines in step S47 whether or not the line of the script Sc has the specified text.

[0099] In step S47, if it is determined that there is no text (step S47, NO), the script processing unit 15 proceeds to step S49. On the other hand, if it is determined that there is text (step S47, YES), the script processing unit 15 passes the text to the screen control unit 14 in step S48. The screen control unit 14 causes the display unit 4 to display the text passed from the script processing unit 15.

[0100] Next, in step S49, the script processing unit 15 determines whether or not there is a file of the display image defined in the line of the script Sc.

[0101] In step S49, if it is determined that there is no display image file (step S49, NO), the script processing unit 15 proceeds to step S51. On the other hand, if it is determined that there is a display image file (step S49, YES), in step S50, the script processing unit 15 passes the display image file to the screen control unit 14. The screen control unit 14 causes the display unit 4 to display the image of the display image file passed from the script processing unit 15.

[0102] Next, in step S50-1, the script processing unit 15 determines whether or not there is a sound file defined in a line of the script Sc. If it is determined in step S50-1 that there is no sound file (step S50-1, NO), the script processing unit 15 proceeds to step S51. On the other hand, if it is determined in step S50-1 that there is a sound file (step S50-1, YES), the script processing unit 15 passes the sound file to the speaker 5 via the sound control unit 14-1, and causes the speaker 5 to output sound.

[0103] Next, in step S51, the script processing unit 15 acquires from the playback control unit 13 the current playback elapsed time of the video Mv.

[0104] Next, in step S52, the script processing unit 15 determines whether or not the current elapsed playback time has passed the display image disappearance period defined in the line of the script Sc.

[0105] In step S52, if it is determined that the elapsed playback time has not passed the display image disappearance period (step S52, NO), the script processing unit 15 performs the process of step S52 again. On the other hand, if it is determined that the elapsed playback time has passed the display image disappearance period (step S52, YES), in step S53, the script processing unit 15 erases the display of the text or display image on the display unit 4 by the screen control unit 14. Next, in step S53-1, the script processing unit 15 stops the playback of the sound file. After that, the script processing unit 15 proceeds to the process of step S84.

[0106] On the other hand, if it is determined in step S44 that the processing type is not type T1 (NO in step S44), the script processing unit 15 determines in step S54 whether the processing type is type T2 or not.

[0107] In step S54, when it is determined that the type is T2 (step S54, YES), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S55.

[0108] Next, in step S56, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0109] In step S56, if it is determined that the elapsed playback time is not equal to the playback time (step S56, NO), the script processing unit 15 performs the process of step S56 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S56, YES), the script processing unit 15 determines in step S57 whether or not the line of the script Sc contains the specified text.

[0110] In step S57, if it is determined that there is no text (step S57, NO), the script processing unit 15 proceeds to step S59. On the other hand, if it is determined that there is text (step S57, YES), the script processing unit 15 passes the text to the screen control unit 14 in step S58. The screen control unit 14 causes the display unit 4 to display the text passed from the script processing unit 15.

[0111] Next, in step S59, the script processing unit 15 acquires from the playback control unit 13 the current playback elapsed time of the video Mv.

[0112] Next, in step S60, the script processing unit 15 determines whether or not the current elapsed playback time has passed the display image disappearance period defined in a line of the script Sc.

[0113] In step S60, if it is determined that the elapsed playback time has not passed the display image disappearance period (step S60, NO), the script processing unit 15 performs the process of step S60 again. On the other hand, if it is determined that the elapsed playback time has passed the display image disappearance period (step S60, YES), in step S61, the script processing unit 15 causes the screen control unit 14 to erase the display of the text on the display unit 4. After that, the script processing unit 15 proceeds to the process of step S84.

[0114] On the other hand, if it is determined in step S54 that the processing type is not type T2 (NO in step S54), the script processing unit 15 determines in step S62 whether the processing type is type T3.

[0115] In step S62, when it is determined that the type is T3 (step S62, YES), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S63.

[0116] Next, in step S64, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0117] In step S64, if it is determined that the elapsed playback time is not equal to the playback time (step S64, NO), the script processing unit 15 performs the process of step S64 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S64, YES), the script processing unit 15 acquires the jump destination time defined in the line of the script Sc in step S65.

[0118] Next, in step S66, the script processing unit 15 causes the playback control unit 13 to shift the jump destination time of the video Mv to the acquired jump destination time. After that, the script processing unit 15 shifts the process to step S84.

[0119] On the other hand, if it is determined in step S62 that the processing type is not type T3 (NO in step S62), the script processing unit 15 determines in step S67 whether the processing type is type T4.

[0120] If it is determined in step S64 that the type is T4 (YES in step S67), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S68.

[0121] Next, in step S69, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0122] In step S69, if it is determined that the elapsed playback time is not equal to the playback time (step S69, NO), the script processing unit 15 performs the process of step S69 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S69, YES), the script processing unit 15 instructs the playback control unit 13 to stop playback of the video Mv in step S70.

[0123] Next, in step S71, the script processing unit 15 acquires the jump destination moving image file based on the information of the jump destination moving image file defined in the line of the script Sc, and also acquires the jump destination time defined in the line of the script Sc.

[0124] Next, in step S72, the script processing unit 15 switches the video playback target from the video Mv to the video in the video file, and shifts the playback of the video in the video file to the jump destination time, by the playback control unit 13. After that, the script processing unit 15 shifts the process to step S84.

[0125] On the other hand, if it is determined in step S67 that the processing type is not type T4 (NO in step S67), the script processing unit 15 determines in step S73 whether the processing type is type T5 or not.

[0126] In step S73, when it is determined that the type is T5 (step S73, YES), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S74.

[0127] Next, in step S75, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0128] In step S75, if it is determined that the elapsed playback time is not equal to the playback time (step S75, NO), the script processing unit 15 performs the process of step S75 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S75, YES), the script processing unit 15 instructs the playback control unit 13 to ignore the command to stop playback of the video Mv in step S76.

[0129] Next, in step S77, the script processing unit 15 acquires from the playback control unit 13 the current playback elapsed time of the video Mv.

[0130] Next, in step S78, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0131] If it is determined in step S78 that the elapsed playback time is not equal to the playback time (step S78, NO), the script processing unit 15 performs the process of step S78 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S78, YES), the script processing unit 15 instructs the playback control unit 13 to accept a command to stop playback of the video Mv in step S79. After that, the script processing unit 15 proceeds to the process of step S84.

[0132] On the other hand, if it is determined in step S73 that the processing type is not type T5 (step S73, NO), the script processing unit 15 determines in step S80 whether the processing type is type T6 or not.

[0133] In step S80, when it is determined that the type is T6 (step S80, YES), the script processing unit 15 acquires the current playback elapsed time of the video Mv from the playback control unit 13 in step S81.

[0134] Next, in step S82, the script processing unit 15 determines whether or not the current elapsed playback time is equal to the playback time defined in the line of the script Sc.

[0135] In step S82, if it is determined that the elapsed playback time is not equal to the playback time (step S82, NO), the script processing unit 15 performs the process of step S82 again. On the other hand, if it is determined that the elapsed playback time is equal to the playback time (step S82, YES), the script processing unit 15 instructs the playback control unit 13 to end playback of the video Mv in step S83.

[0136] On the other hand, if it is determined in step S80 that the type is not T6 (step S80, NO), the script processing unit 15 moves the process to step S84.

[0137] Next, in step S84, the script processing unit 15 determines whether or not there is a next line in the script Sc. If it is determined in step S84 that there is a next line (step S84, NO), the script processing unit 15 performs the processes from step S41 onwards again. On the other hand, if it is determined in step S84 that there is no next line (step S84, YES), the script processing unit 15 ends the process.

[0138] In this manner, the script processing unit 15 can execute the processes defined in each row of the table of the script Sc in order starting from the first row, based on the script Sc.

[0139] <Example of video playback> 15 to 17 are diagrams showing screens of a moving image Mv displayed on the display unit 4 in the information processing system 100. Fig. 15 is a first example of a screen of the moving image Mv, Fig. 16 is a second example of a screen of the moving image Mv, and Fig. 17 is a third example of a screen of the moving image Mv.

[0140] 15, a comment 151 saying "There are a lot of bars," a remaining comment time display 152, and a user display 150 are superimposed on the video Mv. The remaining comment time display 152 indicates the remaining time until the comment 151 is displayed. The remaining comment time display 152 can display a countdown of the remaining time. The user display 150 shows a user U who is using the information processing system 100. In response to a command from the script processing unit 15, the screen control unit 14 can superimpose the comment 151, the remaining comment time display 152, and the user display 150 on the video Mv.

[0141] In the second example shown in FIG. 16, a quiz 165 asking "What is this?", a quiz image 166 related to the quiz, a quiz remaining time display 167 indicating the remaining time until the quiz 165 is displayed, a code display 168, and a user display 150 are superimposed on the video Mv. The quiz remaining time display 167 can display a countdown of the remaining time. The code display 168 displays a QR code (registered trademark) for providing information related to the screen displayed by the video Mv. A user U can view information related to the screen displayed by the video Mv through the information terminal by photographing the QR code or the like of the code display 168 with a camera or the like provided on an information terminal such as a smartphone.

[0142] 16, the script processing unit 15 can cause the screen control unit 14 to display a quiz 165 for the user U on the display unit 4 during or after the playback of the video Mv, and can also cause the user U's answer to the quiz 165 to be displayed on the display unit 4 or stored in the storage unit 16. The information processing system 100 can record the results of what action the user U took in response to the quiz or the like, and analyze the user U's memory ability, judgment ability, and the like.

[0143] 17, a request 171 saying "Seafood! Name your favorite seafood," a request remaining time display 172 indicating the remaining time for which the request 171 is to be displayed, and a user display 150 indicating a user U who is using the information processing system 100 are superimposed on the video Mv. The request remaining time display 172 can display a countdown of the remaining time.

[0144] [Second embodiment] Next, a program according to a second embodiment of the present disclosure and a computer that executes processing by the program will be described.

[0145] The program according to the second embodiment of the present disclosure is a program for creating a script Sc. This program becomes available when, for example, a creator of the script Sc, such as a physical therapist, installs the program in a computer, or the creator downloads the program stored on the server 300 via the network N to the computer.

[0146] A computer that executes processing by a program according to the second embodiment of the present disclosure is, for example, the above-mentioned information processing device 1. Hereinafter, the computer is assumed to be the information processing device 1, and the description will be given assuming that the computer is replaced by the information processing device 1 in the information processing system 100. However, the computer is not limited to the information processing device 1, and may be an information processing device or an information processing terminal not included in the information processing system 100. The information processing device or the information processing terminal is, for example, a smartphone, a tablet, or a notebook PC.

[0147] <Example of functional configuration> Fig. 18 is a block diagram showing an example of a functional configuration of a computer 7 that causes a program according to the second embodiment of the present disclosure to execute processing. In the example shown in Fig. 18, the computer 7 has a script reading unit 71, a new script creating unit 72, a script writing unit 73, a video selecting unit 74, a line adding / deleting unit 75, a text input unit 76, a playback time acquiring unit 77, a script editing unit 78, a script display command unit 79, and a screen display direction acquiring unit 80.

[0148] The program according to the second embodiment of the present disclosure causes the computer 7 to execute a process of selecting, in response to a selection operation input by an operator, a moving image Mv to be the target of a script Sc in which a first command Cm1 commanding playback control according to the playback time of the moving image Mv and a second command Cm2 commanding display control according to the playback time of the moving image Mv are described by the moving image selection unit 74. The program also causes the computer 7 to execute a process of causing the playback time acquisition unit 77 to display the moving image Mv selected by the moving image selection unit 74 on the display unit 4 and to acquire information on the playback time in the script Sc that commands control according to at least one of the first command Cm1 and the second command Cm2. The program also causes the computer 7 to execute a process of describing, in the script Sc, the first command Cm1 and the second command Cm2 in association with the playback time acquired by the playback time acquisition unit 77.

[0149] The computer 7 can execute a process for creating the script Sc. In the embodiment of the present disclosure, by using the script Sc created by the computer 7, a video Mv tailored to the user U or the operator 200 can be easily provided.

[0150] The script reading unit 71 reads an existing script Sc file from the script group 162 stored in the storage unit 16 in response to an operation input by an operator received via the operation receiving unit 11.

[0151] In response to an operation input by an operator received via the operation receiving unit 11, the new script creating unit 72 opens a file for creating a new script Sc.

[0152] The script writing unit 73 writes out a file of an edited existing script Sc or a file of a newly created script Sc to an external device or an external apparatus in response to an operation input by an operator received via the operation receiving unit 11. Examples of the external device here include the storage unit 16 or the display unit 4. Examples of the external apparatus include the server 300 connected via the network N.

[0153] As described above, the moving image selection unit 74 selects a moving image Mv to be the target of the script Sc in response to a selection operation by the operator. In the example shown in Fig. 18, the moving image selection unit 74 accepts a selection operation input by the operator via the operation acceptance unit 11. The moving image selection unit 74 can select a moving image Mv corresponding to the accepted selection operation input from the moving image group 163 stored in the storage unit 16.

[0154] The line addition / deletion unit 75 adds or deletes a line corresponding to a process in the table of the script Sc in response to an operation input by an operator received via the operation receiving unit 11.

[0155] The text input unit 76 inputs text corresponding to a command in response to an operation input by an operator received via the operation receiving unit 11 into the table of the script Sc.

[0156] As described above, the playback time acquisition unit 77 displays the video Mv selected by the video selection unit 74 on the display unit 4, and acquires a playback time for instructing control according to at least one of the first command Cm1 and the second command Cm2 in the script Sc. In the example shown in FIG. 18, the playback time acquisition unit 77 displays the video Mv received from the video selection unit 74 on the display unit 4 via the screen control unit 14. Furthermore, the playback time acquisition unit 77 acquires information on the playback time for instructing control according to at least one of the first command Cm1 and the second command Cm2 in response to an operation input by the operator received by the video selection unit 74 via the operation reception unit 11, and passes it to the script editing unit 78.

[0157] As described above, the script editing unit 78 writes the first command Cm1 and the second command Cm2 in the script Sc in association with the playback time acquired by the playback time acquisition unit 77. In the example shown in Fig. 18, the script editing unit 78 generates information indicating the first command Cm1 and the second command Cm2 in response to an operation input by an operator received via the operation acceptance unit 11. The script editing unit 78 can write the information indicating the first command Cm1 and the second command Cm2 or a sound file in the script Sc in association with the playback time indicated by the information on the playback time received from the playback time acquisition unit 77.

[0158] The script display command unit 79, in accordance with a command from the script editing unit 78, extracts text, images, or sound files from the script Sc stored in the storage unit 16, and passes them to the screen control unit 14 or sound control unit 14-1.

[0159] The screen display direction acquisition unit 80 passes information on the screen direction seen by the user U when creating the script Sc using the computer 7 to the playback control unit 13 in response to an operation input by the operator received via the operation reception unit 11. The playback control unit 13 can change the image area of ​​the moving image Mv displayed on the display unit 4 by the screen control unit 14 in response to the received screen direction information.

[0160] <Example of processing by computer 7> (Overall processing example) Fig. 19 is a flowchart showing an example of the overall processing of the processing by the computer 7. Fig. 19 shows an overall overview of the processing executed by the computer 7 when the creator of the script Sc creates the script Sc using the computer 7. For example, the computer 7 starts the processing of Fig. 19 when the operation receiving unit 11 receives, as a start condition, an operation to start a program for creating the script Sc.

[0161] First, in step S91, the computer 7 determines whether or not to newly create a script Sc. The computer 7 can determine, in response to an operation received by the operation receiving unit 11, whether or not to newly create a script Sc.

[0162] When it is determined in step S91 that a new script Sc is to be created (YES in step S91), in step S92, the computer 7 causes the new script creation unit 72 to create a new script Sc, and causes the script display command unit 79 to display a table of the new script Sc on the display unit 4 via the screen control unit 14. The creator can create the script Sc while looking at the table of the script Sc displayed on the display unit 4.

[0163] Next, in step S93, the computer 7 selects a moving image Mv that is to be the target of the script Sc, in accordance with the operation accepted by the operation accepting unit 11, via the moving image selecting unit 74. The moving image selecting unit 74 passes the selected moving image Mv to the playback time acquiring unit 77.

[0164] Next, in step S94, the computer 7 causes the playback time acquisition section 77 to display the moving image Mv received from the moving image selection section 74 on the display section 4 via the screen control section .

[0165] Next, in step S95, the computer 7 starts playing the video Mv by the playback control unit 13 in response to the operation accepted by the operation accepting unit 11. For the playback here, fast-forward playback, jump playback in which the playback time is directly specified, or the like may be used.

[0166] Next, in step S96, the computer 7 stops the playback of the video Mv by the playback control unit 13 in response to the operation accepted by the operation accepting unit 11. For example, the creator plays the video Mv until a playback time at which the creator wants to give an instruction such as text display is reached, and stops the playback of the video Mv when the creator wants to give an instruction such as text display is reached.

[0167] Next, in step S97, the computer 7 acquires information on the playback time by the playback time acquisition unit 77. The playback time acquisition unit 77 passes the acquired information on the playback time to the script editing unit 78.

[0168] Next, in step S98, the row addition / deletion unit 75 of the computer 7 inserts a row into the table of the script Sc in accordance with the operation accepted by the operation acceptance unit 11.

[0169] Next, in step S99, the computer 7 causes the script editing unit 78 to edit the lines inserted into the table of the script Sc in accordance with the operation accepted by the operation accepting unit 11. The script editing unit 78 can input text or information other than text into the lines of the script Sc via the text input unit 76.

[0170] Next, in step S100, the computer 7 determines whether or not to end the editing. For example, the computer 7 can determine whether or not to end the editing in response to the operation accepted by the operation acceptance unit 11.

[0171] In step S100, if it is determined that the process should not end (step S100, NO), the computer 7 performs the processes from step S95 onward again. On the other hand, if it is determined that the process should end (step S100, YES), the computer 7 proceeds to step S110.

[0172] On the other hand, if it is determined in step S91 that a new script is not to be created (NO in step S91), the computer 7 causes the script reading unit 71 to read an existing script Sc stored in the storage unit 16 in step S101.

[0173] Next, in step S102, the computer 7 causes the script display command unit 79 to display a table of the existing script Sc on the display unit 4 via the screen control unit 14. The creator can create the script Sc while looking at the table of the script Sc displayed on the display unit 4.

[0174] Next, in step S103, the computer 7 selects a moving image Mv that is to be the target of the script Sc, in accordance with the operation accepted by the operation accepting unit 11, via the moving image selecting unit 74. The moving image selecting unit 74 passes the selected moving image Mv to the playback time acquiring unit 77.

[0175] Next, in step S104, the computer 7 causes the playback time acquisition unit 77 to display the moving image Mv received from the moving image selection unit 74 on the display unit 4 via the screen control unit .

[0176] Next, in step S105, the computer 7 starts playing the video Mv by the playback control unit 13 in response to the operation accepted by the operation accepting unit 11. For the playback here, fast-forward playback, jump playback in which the playback time is directly specified, or the like may be used.

[0177] Next, in step S106, the computer 7 stops the playback of the video Mv by the playback control unit 13 in response to the operation accepted by the operation accepting unit 11. For example, the creator plays the video Mv until a playback time at which the creator wants to give an instruction such as text display is reached, and stops the playback of the video Mv when the creator wants to give an instruction such as text display is reached.

[0178] Next, in step S107, the computer 7 causes the script editing unit 78 to select a line to be edited in the table of the script Sc in accordance with the operation accepted by the operation accepting unit 11.

[0179] Next, in step S108, computer 7 edits the selected line by script editing unit 78 in accordance with the operation accepted by operation accepting unit 11. Script editing unit 78 can input text or information other than text into each item of the line via text input unit 76.

[0180] Next, in step S109, the computer 7 determines whether or not to end the editing. For example, the computer 7 can determine whether or not to end the editing in response to the operation accepted by the operation acceptance unit 11.

[0181] In step S109, if it is determined that the process is not to be ended (step S109, NO), the computer 7 performs the process from step S105 onwards again. On the other hand, if it is determined that the process is to be ended (step S109, YES), the computer 7 shifts the process to step S110.

[0182] Next, in step S110, the computer 7 saves the script Sc by the script editing unit 78. The saved script Sc is stored in the storage unit 16.

[0183] In this manner, the computer 7 can execute the process of creating the script Sc.

[0184] (Example of line editing process) Fig. 20 is a flowchart showing an example of a line editing process by the script editing unit 78. Fig. 20 shows a process executed by the script editing unit 78 when the creator of the script Sc edits a line of the script Sc using the computer 7. For example, the script editing unit 78 starts the process of Fig. 20 when the process shown in Fig. 19 reaches either step S99 or step S108 as a starting condition.

[0185] First, in step S111, the script editing unit 78 determines whether the processing type is type T0.

[0186] If it is determined in step S111 that the processing type is type T0 (step S111, YES), then in step S112, the script editing unit 78 specifies the start time of the video Mv by inputting it into a line. After that, the script editing unit 78 moves the process to step S134. On the other hand, if it is determined in step S111 that the processing type is not type T0 (step S111, NO), then in step S113, the script editing unit 78 determines whether the processing type is type T1.

[0187] In step S113, if it is determined that the type is T1 (step S113, YES), the script editing unit 78 in step S114 specifies the start time of the process by inputting it into a line.

[0188] Next, in step S115, the script editing unit 78 inputs the text to be displayed.

[0189] Next, in step S116, the script editing unit 78 selects an image to be displayed.

[0190] In step S117, the script editing unit 78 inputs the end time of the process into a line to specify it, and then the process proceeds to step S134.

[0191] On the other hand, if it is determined in step S113 that the processing type is not type T1 (step S113, NO), the script editing unit 78 determines in step S118 whether the processing type is type T2 or not.

[0192] In step S118, if it is determined that the type is T2 (step S118, YES), the script editing unit 78 in step S119 specifies the start time of the process by inputting it into a line.

[0193] Next, in step S120, the script editing unit 78 inputs the text to be displayed.

[0194] Next, in step S121, the script editing unit 78 selects an image to be displayed.

[0195] Next, in script S121-1, the sound file to be played is selected.

[0196] In step S122, the script editing unit 78 inputs the playback stop period into the line to specify it, and then the process proceeds to step S134.

[0197] On the other hand, if it is determined in step S118 that the processing type is not type T2 (NO in step S118), the script editing unit 78 determines in step S123 whether the processing type is type T3.

[0198] In step S123, when it is determined that the type is T3 (step S123, YES), the script editing unit 78 in step S124 specifies the jump start time of the moving image Mv by inputting it in the line.

[0199] Next, in step S125, the script editing section 78 inputs the jump destination time of the video Mv into the line, thereby specifying it, and then the process proceeds to step SS134.

[0200] On the other hand, if it is determined in step S123 that the processing type is not type T3 (step S123, NO), the script editing unit 78 determines in step S126 whether the processing type is type T4.

[0201] In step S126, if it is determined that the type is T4 (YES in step S126), the script editing unit 78 selects the moving image Mv in step S127.

[0202] Next, in step S128, the script editing unit 78 displays the video Mv.

[0203] Next, in step S129, the script editing unit 78 specifies the start time of the jump destination video by inputting it into the line, and then proceeds to step S134.

[0204] On the other hand, if it is determined in step S126 that the processing type is not type T4 (NO in step S126), the script editing unit 78 determines in step S130 whether the processing type is type T5.

[0205] In step S130, if it is determined that the type is T5 (step S130, YES), the script editing unit 78 in step S131 specifies the start time of the process by inputting it into a line.

[0206] Next, in step S132, the script editing unit 78 inputs the stop ignore period into a line to specify it, and then the process proceeds to step S134.

[0207] On the other hand, if it is determined in step S130 that the type is not T5 (NO in step S130), the script editing unit 78 specifies the start time of the process by inputting it into a line in step S133. Then, the process proceeds to step S134.

[0208] Next, in step S134, the script editing unit 78 stores the moving image file name and start time in the script Sc.

[0209] In this manner, the script editing unit 78 can execute processing when the creator of the script Sc uses the computer 7 to edit lines of the script Sc.

[0210] <Script creation screen example> 21 is a diagram showing an example of a script creation screen 8 displayed by the computer 7. The script creation screen 8 is a screen that is displayed on the display unit 4 or the like when the computer 7 creates the script Sc.

[0211] In the example shown in FIG. 21, the script creation screen 8 includes a moving image display 81, a table display 82, an image area display 83, and a time table display 84.

[0212] The moving image display 81 is a screen area that displays the moving image Mv for which the script Sc is to be created. In the example shown in FIG. 21, the moving image display 81 includes a frame 811, a time indicator 812, a Pause / Resume button 813, and a moving image end button 814. The frame 811 indicates an area within an image of the moving image Mv that is displayed on the display unit 4 in the information processing system 100. The time indicator 812 is an indicator that indicates the playback time of the moving image Mv. The Pause / Resume button 813 is a button that is operated when pausing or resuming playback of the moving image Mv. The moving image end button 814 is a button that is operated when ending playback of the moving image Mv.

[0213] The table display 82 includes an add row above button 821, an add row below button 822, an edit row 823, and a selection box 824. The add row above button 821 is a button operated when adding a new row above a row in the table of the script Sc. The add row below button 822 is a button operated when adding a new row below a row in the table of the script Sc. The edit row 823 is the row being edited. The selection box 824 is a UI (User Interface) that displays predetermined processing types T1 to T6 and allows the creator to select one of the processing types.

[0214] The image area display 83 is used to set an area within the video Mv that is visible to a user U of the information processing system 100. An upper area 831 represents the case where the upper side of the video Mv is specified, a right area 832 represents the case where the right side of the video Mv is specified, a lower area 833 represents the case where the lower side of the video Mv is specified, and a left area 834 represents the case where the left side of the video Mv is specified.

[0215] The time table display 84 is a view for checking the displayed moving image Mv and script Sc. For example, the time table display 84 is used for checking the playback time at which there is a command within the moving image Mv.

[0216] In the script Sc illustrated in FIG. 21, three moving images are connected in time and played and displayed on the display unit 4. A first moving image mark 841 is a mark indicating the moving image to be displayed first on the display unit 4. A second moving image mark 842 is a mark indicating the moving image to be displayed second on the display unit 4. A third moving image mark 843 is a mark indicating the moving image to be displayed third on the display unit 4. In the example illustrated in FIG. 21, three moving images are played in order. The first moving image is displayed on the display unit 4 in a time domain displayed by the first moving image mark 841. The second moving image is displayed on the display unit 4 in a time domain displayed by the second moving image mark 842. The third moving image is displayed on the display unit 4 in a time domain displayed by the third moving image mark 843.

[0217] Each of the command marks 840 indicates the playback time at which a command is issued within the video Mv. The numbers shown on the command marks 840 indicate row numbers in the table. When each of the command marks 840 is selected by clicking or the like, the corresponding row in the table display 82 becomes editable.

[0218] The creator can create a script Sc using a script creation screen 8 as shown in FIG.

[0219] [Third embodiment] Next, an information processing system according to a third embodiment of the present disclosure will be described. Fig. 22 is a diagram illustrating an example of a configuration of an information processing system 100a according to the third embodiment of the present disclosure.

[0220] The information processing system 100a differs from the above-described embodiment and modified examples in that a plurality of users U who are remote from each other can walk while sharing the same video.

[0221] 22, the information processing device 1, the display units 4 used by each of the multiple users U, and the motion sensor 2 are connected to each other via a network so as to be able to communicate with each other. The display units 4 are, for example, a PC, a tablet, or a smartphone. The acquisition unit 12 of the information processing device 1 acquires motion status information of the user U via the network, and the output unit 17 can distribute the video data and data to be displayed on the display units 4 to each of the multiple display units 4 in streaming format.

[0222] The information processing system 100a can provide users U who are remote from each other with a sense of realism as if they are exercising in the same place. In the example shown in FIG. 22, the motion sensor 2 is connected to the display unit 4 by wire or wirelessly, and can transmit motion status information to the information processing device 1 via the display unit 4. However, the motion sensor 2 can also be directly connected to a network and transmit motion status information to the information processing device 1 without going through the display unit 4. Also, in the example shown in FIG. 22, the server 300 shown in FIG. 5 has a memory unit 16 and holds scripts, videos, and user information. However, the information processing device 1 may have the memory unit 16.

[0223] In addition, the data capacity of the video delivered in the information processing system 100a may be changed as appropriate depending on the device constituting the display unit 4. For example, the information processing device 1 can obtain information on the type of device of the display unit 4 used by the user U, and the performance of the CPU and memory from the display unit 4, and deliver the video with a data capacity suitable for each device. The data capacity can be adjusted by adjusting the resolution, frame rate, and bit rate. As a result, in the case of a device of the display unit 4 with a low processing speed, the data capacity is reduced to deliver the video, so that even when the user U uses devices with various processing speeds, everyone can participate in the exercise while watching a common video.

[0224] In addition, the relationship between the eye position of the user U and the height of the display unit 4 may differ depending on the usage mode of the user U. For example, the relationship between the eye position of the user U and the height of the display unit 4 differs between when the user U walks while standing and when the user U walks while sitting on a chair. In addition, the relationship may also differ depending on the installation environment of the device of the display unit 4 used by the user U. Therefore, in the information processing system 100a, control may be performed to adjust the eye position and the display height and inclination of the image displayed on the display unit 4 before the start of playback of the video. By controlling the eye position and the display height and inclination of the image displayed on the display unit 4, each of the multiple users U can easily view the video according to the usage mode.

[0225] Although the preferred embodiments have been described in detail above, the present disclosure is not limited to the above-described embodiments, and various modifications and substitutions can be made to the above-described embodiments of the present disclosure without departing from the scope of the claims.

[0226] All numbers such as ordinal numbers and quantities used in the description of the embodiments of the present disclosure are provided as examples to specifically explain the technology of the present disclosure, and the present disclosure is not limited to the illustrated numbers. In addition, the connection relationships between components are provided as examples to specifically explain the technology of the present disclosure, and do not limit the connection relationships that realize the functions of the present disclosure.

[0227] The division of blocks in the functional block diagram is an example, and multiple blocks may be realized as one block, one block may be divided into multiple blocks, or some functions may be moved to other blocks. Furthermore, the functions of multiple blocks having similar functions may be processed in parallel or in time-sharing by a single piece of hardware or software. Furthermore, some or all of the functions may be distributed to multiple computers.

[0228] Aspects of the present disclosure are, for example, as follows. <1> The information processing device has a playback control unit that controls playback of a video displayed on a display unit in accordance with a user's physical activity state, a screen control unit that controls the display by the display unit, and a script processing unit that instructs the playback control unit and the screen control unit to perform control based on a script that contains a first command to instruct the playback control unit to perform playback control in accordance with a playback time of the video, and a second command to instruct the screen control unit to perform display control in accordance with the playback time. <2> a storage unit that stores the script, and the script processing unit issues control commands to the playback control unit and the screen control unit based on the script stored in the storage unit; <1> The present invention relates to an information processing device. <3> the storage unit stores the script corresponding to the user or a group consisting of a plurality of the users, and the script processing unit refers to the storage unit in response to a selection input of the user or the group to acquire the script corresponding to the user or the group consisting of a plurality of the users; <2> The present invention relates to an information processing device. <4> the script processing unit causes the screen control unit to display a quiz for the user on the display unit during or after the playback of the video, and causes the display unit to display an answer to the quiz by the user or causes the storage unit to store the answer. <2> or the above <3> The present invention relates to an information processing device. <5> The above <1> From the above <4> and a motion sensor that outputs motion status information of the user. <6> a display unit that displays the video, the playback of which is controlled according to the user's exercise state, the display unit being a head-mounted or glasses-type display device; <5> The information processing system is as described above. <7> The above <1> From the above <4> and a server communicatively connected to the information processing device and configured to transmit the video and the script to the information processing device in response to a request from the information processing device. <8> The information processing device transmits the video to the display unit via a network. <7> The information processing system is as described above. <9> a motion sensor that transmits the motion status information of the user to the information processing device via a network; <7> The information processing system is as described above. <10> This is a program that causes a computer to execute a process of instructing the playback control unit and the screen control unit to perform control based on a script that includes a playback control unit to control the playback of a video displayed on a display unit in accordance with the user's exercise state, a screen control unit to control the display by the display unit, and a script processing unit to instruct the playback control unit to perform playback control in accordance with the playback time of the video and a second command to instruct the screen control unit to perform display control in accordance with the playback time. <11> a video selection unit selects a video to be the subject of a script in which a first command for instructing playback control according to a playback time of the video and a second command for instructing display control according to the playback time are described in response to a selection operation input by an operator; a playback time acquisition unit displays the video selected by the video selection unit on a display unit and acquires information on the playback time in the script instructing control according to at least one of the first command and the second command; and a script editing unit describes the first command and the second command in the script in association with the playback time acquired by the playback time acquisition unit. It is a program that causes a computer to execute processing. [Explanation of symbols]

[0229] 1. Information processing device 11 Operation reception section 12 Acquisition Department 13 Playback control section 14 Screen control section 15 Script processing section 16 Memory section 161 User information 162 Scripts 162s thumbnails 163 Videos 17 Output section 2. Motion Sensor 21 Pressure Sensor 22 Support member 23 Chair 3. Eye Tracking Sensor 4 Display section 5 Speakers 6 Control section 7. Computers 71 Script reading section 72 New Script Creation Section 73 Script writing section 74 Video Selection Section 75 Line addition / deletion section 76 Text input section 77 Playback time acquisition section 78 Script Editorial Department 79 Script display command section 80 Screen display direction acquisition unit 8 Script creation screen 81 Video display 811 frames 812 Hour Indicator 813 Pause / Resume button 814 Video end button 82 Table Display 821 Top row add button 822 Bottom row add button 823 Edited lines 824 Selection Box 83 Image area display 831 Upper area 832 Right area 833 Lower area 834 left area 84 Timetable display 840 Command Mark 841 1st video mark 842 2nd video mark 843 3rd video mark 100, 100a, 100b, 100c Information processing system 101 CPU 102 ROM 103 RAM 104 HDD / SSD 105 Interface 150 User display 151 Comments 152 Comment remaining time display 165 Quiz 166 Quiz Images 167 Quiz remaining time display 168 Code Display 171 Request 172 Request remaining time display 200 Operator 300 Servers B System Bus Cm1 1st command Cm2 2nd command Gi gaze information Mi exercise information Mv video N Network Sc Script U User [Prior art documents] [Patent documents]

[0230] [Patent Document 1] JP 2023-008123 A

Claims

1. a playback control unit that controls playback of the video displayed on the display unit in accordance with the user's exercise state; A screen control unit that controls a display by the display unit; an information processing device having: a script processing unit that instructs the playback control unit to perform playback control in accordance with a playback time of the video, and a script that instructs the screen control unit to perform display control in accordance with the playback time, based on a script that describes a first command to the playback control unit and a second command to the screen control unit to perform display control in accordance with the playback time.

2. A storage unit that stores the script, The information processing device according to claim 1 , wherein the script processing unit issues control commands to the playback control unit and the screen control unit based on the script stored in the storage unit.

3. The storage unit stores the script corresponding to the user or a group of the users, The information processing apparatus according to claim 2 , wherein the script processing unit refers to the storage unit in response to a selection input of the user or the group, and acquires the script corresponding to the user or the group consisting of a plurality of the users.

4. 3. The information processing device of claim 2, wherein the script processing unit causes the screen control unit to display a quiz for the user on the display unit during or after playback of the video, and causes the user's answer to the quiz to be displayed on the display unit or stored in the memory unit.

5. An information processing device according to any one of claims 1 to 4; and a motion sensor that outputs motion status information of the user.

6. a display unit for displaying the video, the playback of which is controlled in accordance with the user's exercise state; The information processing system according to claim 5 , wherein the display unit is a head-mounted or glasses-type display device.

7. An information processing device according to any one of claims 1 to 4; an information processing system comprising: a server communicatively connected to the information processing device, the server transmitting the video and the script to the information processing device in response to a request from the information processing device;

8. The information processing system according to claim 7 , wherein the information processing device transmits the video to the display unit via a network.

9. 8. The information processing system according to claim 7, further comprising an exercise sensor that transmits the exercise condition information of the user to the information processing device via a network.

10. The playback control unit controls playback of the video displayed on the display unit according to the user's exercise state; A screen control unit controls the display on the display unit; A script processing unit issues control commands to the playback control unit and the screen control unit based on a script in which a first command to instruct the playback control unit to perform playback control in accordance with a playback time of the video and a second command to instruct the screen control unit to perform display control in accordance with the playback time are written. A program that causes a computer to carry out processing.

11. a moving image selection unit selects, in response to a selection operation input by an operator, a moving image to be a target of a script in which a first command for instructing playback control according to a playback time of the moving image and a second command for instructing display control according to the playback time are described; a playback time acquisition unit causes a display unit to display the video selected by the video selection unit, and acquires information on the playback time that instructs control according to at least one of the first command and the second command in the script; A script editing unit writes the first command and the second command in the script in association with the playback time acquired by the playback time acquisition unit. A program that causes a computer to carry out processing.

Images