Footstep detection apparatus, information processing system, footstep detection method, and video playback method

Abstract

To provide a footstep detection apparatus.SOLUTION: A footstep detection apparatus 1000 comprises: a step board 1004; a load detection sensor 1022 configured to detect a load acting through the step board 1004, which varies depending on a position where the step board 1004 is stepped on; an amplifier 1030 for amplifying a component of a signal from the load detection sensor 1022 with a bias component of the signal removed; and a detection unit 1046 for detecting a footstep motion based on the amplified signal. The footstep detection apparatus 1000 further comprises a base board 1002, and a support member 1006 provided between the step board 1004 and the base board 1002 so as to create a gap between the step board 1004 and the base board 1002, and the load detection sensor 1022 is provided on the support member 1006.SELECTED DRAWING: Figure 5

Description

Technical Field

[0001] The present invention relates to a footstep detection device, an information processing system, a footstep detection method, and a video playback method.

Background Art

[0002] Conventionally, various input devices for detecting walking movements such as footsteps and performing signal analysis to operate a system for processing video such as an information device or an entertainment device have been known. As a device for detecting footsteps, a sheet-like or plate-like device installed on the floor surface is known.

[0003] In relation to the detection of the above-mentioned footstep movement, Japanese Unexamined Patent Application Publication No. 2023-008123 (Patent Document 1) is known. Patent Document 1 discloses a technique for making it possible to easily use an information processing system for assisting a user's movement. More specifically, Patent Document 1 discloses an information processing system for assisting a user's movement, which includes a sheet-like member including a predetermined pressurized region, a pressure detection unit that detects the pressure applied to the pressurized region when a user who places their body on the sheet-like member steps on it, and an output unit that outputs an image that is changed according to the walking state information of the user obtained based on the detection result by the pressure detection unit. The prior art of Patent Document 1 discloses a walking mat as a device for detecting footsteps. In such a walking mat, each walking detection pad detects the load of the user by a pressure sensor and uses the detection signal for various controls.

[0004] By the way, when considering applications for the elderly, in detecting footsteps, depending on the user's motor ability, there are cases where it cannot operate equivalently to a healthy person, and various operation patterns are exhibited depending on the user's motor ability. Therefore, it is required to correspond to various operation patterns of footstep movements. The prior art of Patent Document 1 was not sufficient from the viewpoint of corresponding to various operation patterns in footstep movements.

Summary of the Invention

Problems to be Solved by the Invention

[0005] The present disclosure has been made in view of the deficiencies in the above prior art, and an object of the present disclosure is to provide a footstep detection device capable of corresponding to a plurality of footstep operation patterns.

Means for Solving the Problems

[0006] In the present disclosure, in order to solve the above problems, a footstep detection device having the following features is provided. This footstep detection device includes a footrest, a load detection sensor configured to detect a load acting through the footrest that varies according to the position where the footrest is stepped on, an amplification unit that amplifies a component obtained by removing the bias component of the signal from the load detection sensor, and a detection unit that detects the operation of the footstep based on the amplified signal.

Effects of the Invention

[0007] With the above configuration, it becomes possible to correspond to a plurality of footstep operation patterns.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Mode for Carrying Out the Invention

[0009] The footstep detection device, information processing system, footstep detection method, and video playback method according to the embodiments of the present disclosure will be described in detail with reference to the drawings. However, the footstep detection device, information processing system, footstep detection method, and video playback method according to the embodiments of the present disclosure are not limited to those described below. In the following description, the same or similar members or functions are referred to by the same names and reference numerals, and detailed descriptions thereof are omitted as appropriate.

[0010] <Footstep detection pad> Hereinafter, with reference to FIGS. 1 to 9, first, the footstep detection pad 1000 as a footstep detection device according to an embodiment of the present disclosure will be described.

[0011] (Physical structure) Hereinafter, with reference to FIG. 1, the structural features of the footstep detection pad 1000 according to an embodiment of the present disclosure will be described. FIG. 1 shows the physical structure of the footstep detection pad 1000 according to an embodiment of the present disclosure. FIG. 1(A) shows a top view of the footstep detection pad 1000. FIG. 1(B) shows a side view of the footstep detection pad 1000 as viewed from the rear with the upward direction on the drawing surface of FIG. 1(A) as the front. FIG. 1(C) shows a side view of the footstep detection pad 1000 as viewed from the right with the upward direction on the drawing surface of FIG. 1(A) as the front.

[0012] As shown in FIG. 1, the footstep detection pad 1000 includes a bottom plate 1002 installed on the floor or the like, a tread plate 1004 disposed with a gap therebetween above the bottom plate 1002, a plurality of support members 1006, 1008 (shown by dotted lines in the top view shown in FIG. 1(A)) disposed such that the gap is formed between the tread plate 1004 and the bottom plate 1002, respectively, and a controller 1010. The footstep detection pad 1000 is a device having a plate-like structure composed of the bottom plate 1002 and the tread plate 1004 sandwiching the support members 1006, 1008.

[0013] In the embodiment shown in FIG. 1, as the tread plate 1004, two mutually independent tread plates 1004L and 1004R for the left foot and the right foot are provided side by side in a left-right symmetric manner with respect to a single bottom plate 1002. The left-foot tread plate 1004L is a tread plate assumed to be stepped on by the left foot when the user performs a footstep movement during use. Similarly, the right-foot tread plate 1004R is a tread plate assumed to be stepped on by the right foot when the user performs a footstep movement during use.

[0014] In addition, in the embodiment shown in FIG. 1, the side where the controller 1010 is disposed will be described as being used with the toe of the foot facing that side. However, it is not limited to such a usage mode, and it does not prevent the use with the toe of the foot facing the side opposite to the side where the controller 1010 is provided.

[0015] Further, in other embodiments, the footstep detection pad 1000 may be configured assuming that the side opposite to the side where the controller 1010 is disposed is used with the toe of the foot facing that side. Hereinafter, one of the tread plates 1004 assumed to be stepped on by the left foot may be referred to as the left-foot tread plate 1004L. Similarly, the other tread plate 1004 assumed to be stepped on by the right foot may be referred to as the right-foot tread plate 1004R.

[0016] The bottom plate 1002 is made of an arbitrary material having a predetermined rigidity. The material of the bottom plate 1002 is not particularly limited, and examples thereof include resins such as polycarbonate (PC), ABS resin, polypropylene (PP), PC-ABS resin, PMMA resin, or metals such as stainless steel, aluminum, aluminum alloy, and titanium alloy. The shape of the bottom plate 1002 is not particularly limited, but as shown in FIG. 1, it may have a substantially flat plate shape having a width that accommodates the range where the tread plates 1004L and 1004R for both left and right feet are provided.

[0017] The tread plate 1004 is made of an arbitrary material. The material of the tread plate 1004 is not particularly limited, and similarly, it may be a resin or a metal. As shown in FIG. 1, the tread plate 1004 has a substantially flat plate shape having a predetermined thickness. In such a substantially flat plate shape, preferably, it is configured to have so-called flexibility that elastically deforms moderately when the upper surface of the tread plate 1004 is stepped on by the user. As the tread plate 1004, resins having excellent impact resistance and heat resistance, such as polycarbonate (PC) and PC-ABS resin, are preferably used. Note that the tread plate 1004 may be a composite material in which a carpet, anti-slip rubber, etc. are laminated on the tread plate 1004, rather than a single material of resin or metal, in consideration of the comfort underfoot.

[0018] The support members 1006 and 1008 are made of an arbitrary elastic material. The material of the support members 1006 and 1008 is not particularly limited, but may be rubber or an elastomer. The support members 1006 and 1008 are excellent from the viewpoints of weather resistance, abrasion resistance, and mechanical properties, and are members that are difficult to hydrolyze and deteriorate. It is desirable to select those having a moderate thickness and elastic characteristics that provide a comfortable feeling underfoot. However, the feeling underfoot is a qualitative characteristic, and appropriate design may be made in consideration of the number, installation location, and combination of the sizes of the support members 1006 and 1008, and in addition, the assumed preferences of the users. More specifically, as the support members 1006 and 1008, a rubber sponge (EPDM sponge rubber) using ethylene propylene diene rubber (EPDM) or urethane rubber can be used. Further, as shown in FIG. 1, the shape of the support members 1006 and 1008 may be a quadrangular prism, but is not limited thereto, and may be a cylindrical shape, a triangular prism, or the shape is not limited.

[0019] In the embodiment shown in FIG. 1, four support members 1006L, 1008LA, 1008LB, and 1008LC are sandwiched between the bottom plate 1002 and the left foot pedal 1004L. Similarly, four support members 1006R, 1008RA, 1008RB, and 1008RC are sandwiched between the bottom plate 1002 and the right foot pedal 1004R.

[0020] Although the support members are referred to using different reference numbers, for each foot, the support member 1006 is a support member provided with a load detection sensor, and the support member 1008 is a support member not provided with a load detection sensor, as will be described later.

[0021] In the embodiment shown in FIG. 1, the support members 1008LA and 1008RA disposed on the left or right side of the entire footstep detection pad 1000 have an elongated shape and have a length that crosses one side thereof. On the other hand, at the center of the entire footstep detection pad 1000, that is, on the right side of the left foot pedal 1004L and the left side of the right foot pedal 1004R, a plurality of support members, three support members 1006L, 1008LB, 1008LC and support members 1006R, 1008RB, 1008LC in the example of FIG. 1 are provided respectively. The support members 1006L and 1006R, the support members 1008LB and 1008RB, and the support members 1008LC and 1008RC are respectively arranged at the front, center, and rear positions in the front-rear direction of the footstep detection pad 1000.

[0022] The controller 1010 is disposed on the front side of the footstep detection pad 1000. As shown by the broken line in FIG. 1, the controller 1010 includes a battery 1012 that supplies power to the footstep detection pad 1000 and a substrate 1020 that executes footstep detection logic using a load detection sensor described later.

[0023] Note that FIG. 1 shows the basic structure of the footstep detection pad 1000, and there may be additional elements not described in FIG. 1. For example, it does not prevent additional members such as other internal components, decorative materials, and exteriors from being provided.

[0024] (Sensor Configuration) Hereinafter, the sensor configuration in the footstep detection pad 1000 will be described with reference to FIGS. 2 and 3.

[0025] FIG. 2 is a diagram for explaining an installation example of a load detection sensor in the footstep detection pad 1000 according to one or more embodiments of the present disclosure. As shown in FIG. 2, the load detection sensor 1022 is provided on the support member 1006 and is connected to a connector 1026 on the substrate 1020 via a wiring 1024.

[0026] Figs. 2(A) to 2(C) respectively show the first to third installation examples of the load detection sensor 1022. In the embodiment shown in Fig. 2(A), the load detection sensor 1022 is inserted between the bottom plate 1002 and the support member 1006 on which the tread plate 1004 is provided. In the embodiment shown in Fig. 2(B), the load detection sensor 1022 is inserted between the support member 1006 provided with the bottom plate 1002 thereunder and the tread plate 1004. In the embodiment shown in Fig. 2(C), the load detection sensor 1022 divides the support member 1006 between the bottom plate 1002 and the tread plate 1004 and is inserted therebetween.

[0027] In any of the installation examples shown in Figs. 2(A) to 2(C), the load detection sensor 1022 is configured to detect the load acting on itself via the tread plate 1004 and the support member 1006 when a user steps on the tread plate 1004.

[0028] In the embodiment to be described, the load detection sensor 1022 is a pressure-sensitive sensor whose electrode contact area increases or decreases in response to a force applied in the thickness direction, and accordingly, the resistance changes. As the pressure-sensitive sensor, it is desirable to use a thin-film type pressure-sensitive sensor.

[0029] In other embodiments, a strain gauge may be used as the load detection sensor 1022. On the other hand, when using a strain gauge, it is necessary that the member on the side where the strain gauge is attached has sufficient rigidity to distort the strain gauge itself, and it is necessary to strengthen the rigidity of the structure. And the strengthening of the rigidity of the structure affects the comfort. Therefore, from the viewpoint of making the device easy for the user to handle without making the comfort hard, making it light and as thin as possible, it is optimal to provide the pressure-sensitive sensor on the support member 1006 which is an elastic material.

[0030] FIG. 3 is a diagram for explaining the planar arrangement of the support members in the footstep detection pad 1000 according to an embodiment of the present disclosure. In FIG. 3, locations where the support members 1006 and 1008 can be provided are indicated by dotted circles. More specifically, nine locations L1 to L9 for the left-foot pedal 1004L and nine locations R1 to R9 for the right-foot pedal 1004R are shown as candidates. The arrangement of the support members 1006 and 1008 is preferably devised according to the location where the load detection sensor 1022 is laid.

[0031] Here, in order to represent the planar arrangement, the following terms are defined. The direction in which the pedals 1004L and 1004R are arranged is referred to as the left-right direction. The direction perpendicular to the defined left-right direction is referred to as the front-back direction. Also, when representing the arrangement of the left and right foot pedals 1004L and 1004R and the entire bottom plate 1002, "pad" is appended with "terms indicating the arrangement" for reference. For example, "pad left-right direction center" means the central position in the left-right direction of the entire footstep detection pad 1000 including the pedals 1004L and 1004R. When representing the arrangement in an individual pedal 1004, "pedal" is appended with "terms indicating the arrangement" for reference. For example, "pedal front-back left-right direction center" means the central position in the front-back direction and the central position in the left-right direction in one pedal 1004.

[0032] For example, as described with reference to FIG. 1, when the support members 1006L and 1006R are arranged at locations L3 and R1 closer to the front of the pad left-right direction center and the load detection sensors 1022L and 1022R are provided there, it is preferable to satisfy the following two conditions (1) and (2). Condition (1): Not providing a support member at the location L5 and R5 at the pedal front-back left-right direction center. Condition (2): For the locations L1, L4, L7 corresponding to the left side of the pad of the pedal and the locations R3, R6, R9 corresponding to the right side of the pad, integrating the support members across each location.

[0033] By arranging the support members so as to satisfy the above conditions (1) and (2), even when a user steps on any location of the pedal 1004, the load is dispersed to each support member due to the deflection of the pedal 1004, and sufficient pressure is applied to locations L3 and R1 of the support member 1006. As a result, it becomes possible to detect foot stepping in a wider range using the load detection sensor 1022. Also, the load detected by the load detection sensor 1022 changes according to the distance between the positions of the support members 1006 and 1008 on the pedal 1004 and the position where the user steps. Thereby, it becomes possible to obtain a waveform for distinguishing differences in various foot stepping operation patterns described later.

[0034] In the arrangement example shown in FIG. 1, in each pedal 1004, the support member 1006 where the load detection sensor 1022 is provided is provided at the outer peripheral portion closer to the center in the left - right direction of the pad, and the other support members 1008 are provided at the outer peripheral portions excluding the center in the front - rear and left - right directions of each pedal 1004. Also, the support members 1008LA and 1008RA arranged on the left or right side of the pad of the pedal 1004 have an elongated shape over one side and are integrally formed. Therefore, in the arrangement example shown in FIG. 1, the above conditions (1) and (2) are satisfied.

[0035] Note that in the foot - stepping detection pad 1000 shown in FIG. 1, support members are not provided at locations L2, L8, R2, and R8 in the left - right direction center (excluding the front - rear and left - right direction center L5 and R5) of each pedal 1004 shown in FIG. 3, but support members may be provided at these locations.

[0036] Here, the reason for providing the support members 1006L, 1006R and the load detection sensor 1022 at locations L3 and R1 in front of and closer to the center in the left - right direction of the pad is that, as depicted in FIGS. 1 and 3, the controller 1010 is arranged outside the front of the pad. That is, it is to minimize the wiring length required for the connection from the load detection sensor 1022 to the substrate 1020 (connector 1026) in the controller 1010.

[0037] Here, it is assumed that the user steps on the pedal 1004 around the center in the front-rear, left-right directions. From the perspective of load balance, sensors can preferably be installed at locations L4 and R6 on the outer peripheral portions at the center in the front-rear direction of the pad. However, as shown in FIG. 1, when provided at locations L3 and R1, the physical distance between the sensor and the substrate becomes shorter than when provided at locations L4 and R6.

[0038] Alternatively, when the controller 1010 can be arranged within the pad surface, it is preferable to provide sensors at locations L6 and R4 at the center in the front-rear, left-right directions of the pad. However, in that case, it is necessary to arrange the controller 1010 under the bottom plate 1002. Then, the thickness of the entire pad increases, and the step difference from the periphery becomes large, resulting in a device unsuitable for the elderly.

[0039] Based on the above, when the substrate 1020 is provided on the outer peripheral portion of the pedal 1004, the load detection sensor 1022 is preferably provided at the outer peripheral portion closer to the center in the left-right direction of the pad (when the substrate 1020 is provided in the front, locations L3 and R1; when the substrate 1020 is provided in the rear, locations L9 and R7). As a result, the load detection sensor 1022 and the support member 1006 are arranged on the outer peripheral portion of the pedal 1004 to have a shorter physical distance from the substrate 1020 than the other support member 1008. In this way, by shortening the signal line of the sensor, the component cost can be reduced.

[0040] (The footstep motion pattern to be detected) Above, the physical structure and sensor arrangement of the footstep detection pad 1000 according to the embodiment of the present disclosure have been described. As described above, when considering a system for the elderly, it should be noted that depending on the user's motor ability, cases where they cannot operate equivalently to healthy people are observed, and it is necessary to accommodate various footstep motion patterns.

[0041] FIG. 4 is a diagram for explaining a stepping motion pattern to be detected by the stepping detection pad 1000 according to an embodiment of the present disclosure. FIG. 4(A) shows a normal stepping motion pattern of raising and lowering the foot. In the motion pattern shown in FIG. 4(A), both the heel and the toe repeat the motions of firmly leaving and contacting the ground. On the other hand, when stepping, a case where the foot does not completely leave the floor surface is assumed. FIG. 4(B) exemplifies a motion pattern in which only the toe moves up and down while the heel remains on the floor, and FIG. 4(C) exemplifies a motion pattern in which only the heel moves up and down while the toe remains on the floor. It is preferable to cope with the stepping motion patterns as shown in FIGS. 4(A) to 4(C).

[0042] (Circuit Configuration) The stepping detection pad 1000 according to the embodiment of the present disclosure adopts the circuit configuration described below in combination with the physical configuration and sensor configuration described above in order to correspond to the various stepping motion patterns described above.

[0043] FIG. 5 shows the internal circuit configuration of the stepping detection pad 1000 according to the embodiment of the present disclosure. As shown in FIG. 5, the stepping detection pad 1000 includes single load detection sensors 1022L and 1022R for the left foot and the right foot, a battery 1012, and a substrate 1020. The substrate 1020 is provided with a main switch 1014 for starting and stopping the stepping detection pad 1000. The power supplied from the battery 1012 is energized or cut off in response to the on / off of the main switch 1014.

[0044] The substrate 1020 further includes low-pass filters (LPFs) 1028L and 1028R and AC coupling amplifiers 1030L and 1030R corresponding to the load detection sensors 1022L and 1022R for both the left and right feet, respectively.

[0045] The load detection sensor 1022 is configured to detect a load acting through the tread plate 1004 that varies according to the position where the tread plate 1004 is stepped on. The signal output by the load detection sensor 1022 is an analog signal. An analog signal from the load detection sensor 1022 is input to the LPF 1028. The LPF 1028 is a filter that allows low-frequency components lower than the cut-off frequency of the input analog signal to pass through with little attenuation and attenuates high-frequency components higher than the cut-off frequency. The LPF 1028 has a function of removing high-frequency noise components included in the signal from the load detection sensor 1022.

[0046] The AC coupling amplifier 1030 receives the input of the analog signal from the load detection sensor 1022 (which has passed through the LPF 1028) and amplifies the component with the bias component removed. The AC coupling amplifier 1030 is an amplifier that amplifies a signal input via AC coupling, and the bias component to be removed is a DC component. The AC coupling amplifier 1030 amplifies, with a predetermined gain, a signal obtained by removing a bias component that can be regarded as DC from a relatively low-frequency signal from which high frequencies have been removed by the LPF 1028. The AC coupling amplifier 1030 constitutes the amplification unit in the present embodiment.

[0047] The AC coupling amplifier 1030 includes an AC coupling circuit that extracts an AC component from the input analog signal and an amplification circuit that amplifies the signal of the extracted AC component. FIG. 6(A) shows the circuit configuration of the AC coupling amplifier 1030 included in the footstep detection pad 1000 according to the embodiment of the present disclosure. As shown in FIG. 6(A), the AC coupling amplifier 1030 includes a coupling capacitor 1031 as the AC coupling circuit, an operational amplifier 1032 that amplifies the difference between two inputs as the amplification circuit and outputs it, and a variable resistor 1033 connected to the output of the operational amplifier 1032. Feedback is provided such that a signal returns to the - input side of the operational amplifier 1032 via the variable resistor 1033. The capacitance of the coupling capacitor 1031 may be a value considering the bias component to be removed as described above in combination with the resistor R2.

[0048] Here, the resistance value of the variable resistor 1033 changes the gain of the AC coupling amplifier 1030. The resistance value of the variable resistor 1033 may be set at the time of shipment of the footstep detection pad 1000, or may be changeable by the user. The variable resistor 1033 constitutes an adjustment unit that varies the gain for amplifying the components of the signal in the present embodiment. By making it possible to adjust the gain with the variable resistor 1033, it is possible to compensate for variations in device components and assembly variations. Alternatively, it is possible to adjust the sensitivity of footstep detection due to differences in the way of stepping for each user.

[0049] Here, the purpose of amplifying the component obtained by removing the bias component of the signal from the load detection sensor 1022 is to remove unnecessary analog signal information (DC component, bias component, offset component) when detecting a footstep based on the magnitude of the change in the value of the signal per unit time. Particularly when amplifying a weak vibration, the DC component of the signal hinders the detection of the footstep. For example, if amplification is performed without removing the DC component, the DC component is also amplified simultaneously, and the signal is clipped at the power supply voltage of the operational amplifier 1032, losing the desired amplitude information and making it impossible to detect the intended footstep.

[0050] Referring to FIG. 5 again, the substrate 1020 further includes a detection processing chip 1040. The detection processing chip 1040 includes analog / digital converters (ADCs) 1042L and 1042R for each load detection sensor 1022, digital filters (DFs) 1044R and 1044L for each load detection sensor 1022, and a footstep detection unit 1046. The detection processing chip 1040 can be provided as a system-on-chip in which these multiple functions are mounted on a single chip. The ADC 1042 constitutes the conversion unit in the present embodiment.

[0051] The ADC1042 converts an analog signal that has passed through the LPF1028 and been amplified by the AC coupling amplifier 1030 into a digital signal at a predetermined sampling frequency. The DF1044 is a filter that passes and blocks only predetermined components of the signal in the digital domain.

[0052] The digital signal that has passed through the LPF1028 and the AC coupling amplifier 1030 and been sampled by the ADC1042 is input to the footstep detection unit 1046 from the load detection sensor 1022 via the DF1044. The footstep detection unit 1046 detects the operation of stepping based on the amplified signal. Preferably, the footstep detection unit 1046 detects the operation of stepping based on the magnitude of the change in the value of the signal per unit time rather than the value of the signal of the load detection sensor 1022 itself. The footstep detection unit 1046 constitutes the detection unit according to the present embodiment.

[0053] The substrate 1020 further includes a transmitter 1016 connected to the detection processing chip 1040. The transmitter 1016 is configured to transmit information regarding the operation of stepping detected by the footstep detection unit 1046 to an external destination. More specifically, the transmitter 1016 transmits the motion information Mi of the user to an external device (such as the information processing device 1 described later) that utilizes the output of the footstep detection pad 1000. The transmitter 1016 is preferably a wireless communication module for communicating with an external device, although it does not prevent being for a wired connection.

[0054] (Footstep Detection Processing) Hereinafter, with reference to FIGS. 7 and 8, the footstep detection processing in the footstep detection pad 1000 according to the embodiment of the present disclosure will be described more specifically.

[0055] FIG. 7 is a diagram for explaining waveforms of voltage signals of load detection sensor 1022 (voltage signals before amplification by AC coupling amplifier 1030) obtained by various footstep operation patterns. FIG. 7(A) is a side view (showing components on the left foot side transparently) when looking at the portion of the right foot pedal 1004R of the footstep detection pad 1000 from the left direction.

[0056] When the right foot TR is placed at the center of the pedal 1004R with the toe tip facing the substrate 1020 side and the user applies weight, as shown in FIG. 7(A), the pedal 1004R deflects, and the support members 1006R, 1008RB, and 1008RC are deformed respectively, and a load is applied to the load detection sensor 1022R via the support member 1006R.

[0057] FIGS. 7(B) and 7(C) show the normal footstep operation pattern of lifting and lowering the foot including the heel and toe tip and the obtained voltage waveform respectively. As depicted in FIG. 7(B), since the sensor is lifted off and grounded at both the near toe tip and the heel, a relatively large voltage signal is obtained as shown in FIG. 7(C). In FIG. 7, it is described that a high voltage value is shown in the lifted-off state (OFF) where the foot is floating, and a low voltage value is shown in the grounded state (ON) where the foot is touching.

[0058] FIGS. 7(D) and 7(E) show the footstep operation pattern and the obtained voltage waveform of the way of stepping where only the toe tip moves up and down with the heel on the floor. In this case, as depicted in FIG. 7(D), since the sensor is lifted off and grounded only at the proximal toe tip, a voltage signal smaller than that in FIG. 7(C) but of a certain magnitude is obtained as shown in FIG. 7(E).

[0059] On the other hand, FIGS. 7(F) and 7(G) show the footstep operation pattern and the obtained voltage waveform of the way of stepping where only the heel moves up and down with the toe tip on the floor. In this case, as depicted in FIG. 7(F), since the sensor is lifted off and grounded only at the distal heel, a small and weak voltage signal is obtained as shown in FIG. 7(G), even when compared with FIG. 7(E).

[0060] In particular, in any case, since the user is stepping on the treadle 1004, a certain load is applied, including a bias component.

[0061] FIG. 8 is a diagram for explaining the footstep detection process in the footstep detection pad 1000 according to an embodiment of the present disclosure. FIG. 8(A) shows the signal waveform after amplification by the AC coupling amplifier 1030 when a relatively large signal such as that in FIG. 7(C) or FIG. 7(E) is obtained. On the other hand, FIG. 8(B) shows the signal waveform after amplification by the AC coupling amplifier 1030 when a weak signal such as that in FIG. 7(G) is obtained.

[0062] When a relatively large signal such as that in FIG. 7(C) or FIG. 7(E) is obtained, there is a possibility of saturation depending on the magnitude of the original signal. However, as shown in FIG. 8(A), by amplifying the signal by the AC coupling amplifier 1030, the vibration component with the bias component removed is amplified according to the gain. On the other hand, even when a weak signal such as that in FIG. 7(G) is obtained, as shown in FIG. 8(B), by amplifying the signal by the AC coupling amplifier 1030, the component with the bias component removed is amplified according to the gain. Thus, by passing the sensor signal through the AC coupling amplifier 1030, even in an operation pattern where only the heel distal to the sensor repeatedly leaves and touches the ground as shown in FIG. 7(F), the weak signal can be detected.

[0063] FIGS. 8(C) and 8(D) are enlarged diagrams of the signal waveforms during the lift-off operation and the touchdown operation in the footstep motion. As described above, the footstep detection unit 1046 detects the footstep operation based on the magnitude of the change amount (acceleration) of the signal value per unit time. FIGS. 8(C) and 8(D) depict a predetermined time unit ΔT and the change amount ΔV of the voltage value. By simply based on the magnitude of the change amount (acceleration) per unit time, regardless of the magnitude of the load itself, any change in the load can be detected. For example, when the threshold value is θ, the determination can be made as shown in Table 1 below.

[0064]

Table 1

[0065] As shown in Table 1, instead of making a binary determination of the voltage using a threshold value, by making a determination based on the magnitude of the change amount per unit time, it is possible to suppress the influence on low-frequency fluctuations caused by the gradual load movement that changes moment by moment and the interference of the load between the left and right, and it becomes possible to detect the footstep well.

[0066] As shown in Table 1, when ΔV>0, that is, when the load is decreasing, it is determined to be a liftoff operation (R↑ / L↑), and when ΔV<0, when the load is increasing, it is determined to be a grounding operation (R↓ / L↓).

[0067] Hereinafter, with reference to FIG. 9, the footstep detection process for both the left and right feet by the footstep detection pad 1000 will be described. FIG. 9 is a diagram for explaining the footstep detection process for both the left and right feet by the footstep detection pad 1000 according to an embodiment of the present disclosure. FIG. 9(A) is a diagram showing the state of the user's footstep movement with the left and right feet side by side. FIG. 9(A) shows a normal footstep movement pattern in which both the heel and the toe leave and touch the ground. FIG. 9(B) shows a part of the footstep movement pattern in which only the toe leaves and touches the ground. FIG. 9(C) shows a part of the footstep movement pattern in which only the heel leaves and touches the ground. Note that in the illustrations of FIGS. 9(A) to 9(C), since the state of the user's footstep is drawn from the front, it should be noted that the left and right are reversed. FIG. 9(D) shows the amplified signal waveforms of the load detection sensors 1022 for both the left and right feet, and further corresponds to each state (1) to (5) shown in FIG. 9(A). Also, in FIG. 9(A), the solid line indicates the waveform of the right foot, and the dashed line indicates the waveform of the left foot. In FIG. 9(A), further, the timing of the liftoff or grounding operation is shown by notations such as R↑.

[0068] In FIG. 9(A), (1) is a state in which both feet are on the ground before the start of the footstep. When the footstep starts, the states (2) to (5) described below are repeated: (2) shows a state where the right foot has left the ground while the left foot remains in contact with the ground. At the timing when this state occurs, in the determination of footstep detection, for the right foot, ΔV > 0 and |ΔV| / Δt > θ hold, and the take-off motion (R↑) of the right foot is detected. (3) shows a state where the right foot is in contact with the ground and both feet are in contact with the ground again. At the timing when this state occurs, in the determination of footstep detection, for the right foot, ΔV < 0 and |ΔV| / Δt > θ hold, and the landing motion (R↓) of the right foot is detected. (4) shows a state where the left foot has left the ground while the right foot remains in contact with the ground. At the timing when this state occurs, in the determination of footstep detection, for the left foot, ΔV > 0 and |ΔV| / Δt > θ hold, and the take-off motion (L↑) of the left foot is detected. (5) shows a state where the left foot is in contact with the ground and both feet are in contact with the ground again. In the determination of footstep detection, for the left foot, ΔV < 0 and |ΔV| / Δt > θ hold, and the landing motion (L↓) of the left foot is detected.

[0069] In the above description, the explanation was based on a normal footstep motion. However, even for the motion pattern where only the toe leaves and lands as shown in Fig. 9(B), or for the motion pattern where only the heel leaves and lands, due to the amplification by the AC coupling amplifier 1030, it is amplified into a larger signal, so it can be detected in the same way.

[0070] Next, in the footstep detection process, the signal waveform in the case where |ΔV| / Δt < θ and non-detection occurs is shown in Fig. 9(E). In this case, in (3) of Fig. 9(E), the take-off of the left foot (L↑) and the landing of the left foot (L↓) are not detected. On the other hand, even for a trapezoidal wave as shown in (4) and (5), for example, even if the amplitude of the voltage is small, if the voltage signal after passing through the AC coupling amplifier 1030 satisfies |ΔV| / Δt > θ, the take-off motion (L↑) and the landing motion (L↓) are detected. That is, by ignoring low-frequency voltage fluctuations and reacting to the steep changes at the moment when the foot touches the ground or leaves the ground, false detection can be suppressed and footsteps can be detected well.

[0071] In a specific embodiment, the number of times of foot tapping may be incremented at the timing of the above-described grounding operations (L↓ / R↓) and liftoff operations (L↑ / R↑) of the left and right feet, or both, and further, a foot tapping signal indicating that foot tapping has occurred externally can be output. When outputting the foot tapping signal, for example, it is transmitted to a predetermined transmission destination using the wireless or wired communication function provided in the detection processing chip shown in FIG. 5 or a separately prepared wireless or wired communication module.

[0072] In the above description, it has been described that the circuit configuration is such that the voltage decreases when the foot is lowered and a load is applied, and the voltage increases when the foot is raised and the load decreases. However, the relationship between the level of the voltage signal and foot tapping varies depending on the connection method of the sensor 1022.

[0073] FIGS. 6(B) and 6(C) are diagrams showing the circuit configuration around the load detection sensor 1022. FIG. 6(B) shows a circuit configuration in which the voltage decreases when a load is applied and the voltage increases when the load decreases. FIG. 6(C) shows the reverse configuration, in which the voltage increases when a load is applied and the voltage decreases when the load decreases. It is desirable to determine which circuit configuration to adopt in consideration of the characteristics of the sensor and the responsiveness of the acquired signal.

[0074] The result of the stepping detection process by the above-described stepping detection unit 1046 is transmitted as the motion information Mi of the user to an external device (such as the information processing device 1 described later) that uses the output of the stepping detection pad 1000 via, for example, the above-described transmitter 1016. Here, the motion information Mi may be information indicating that the above-described grounding or liftoff operation has been detected. Alternatively, the stepping detection unit 1046 may further determine whether the user is in a walking motion in response to detecting the above-described grounding or liftoff operation. In that case, the transmitter 1016 may transmit information indicating that the user is in a walking motion as the motion information Mi in response to detecting the above-described grounding or liftoff operation. In addition to this, when the stepping detection unit 1046 does not detect the above-described grounding or liftoff operation for a predetermined period, it determines that the user is stopped, and the transmitter 1016 may transmit information indicating that the user is stopped as the motion information Mi.

[0075] (Summary) According to the embodiment of the present disclosure described above, it is possible to provide a stepping detection device capable of corresponding to a plurality of stepping operation patterns.

[0076] The stepping detection pad 1000 of the embodiment of the present disclosure includes an AC coupling amplifier 1030 that amplifies a component obtained by removing the bias component of the signal from the load detection sensor 1022 configured to detect different loads according to the position where the tread plate 1004 is stepped on. With such a configuration, it becomes possible to detect a wide range of stepping patterns from a stepping method that generates a large signal to a stepping method that generates a small signal, and it becomes possible to respond to various stepping methods.

[0077] Further, by providing the load detection sensor 1022 on the support member 1006 provided so as to create a gap between the tread plate 1004 and the bottom plate 1002, the load changes according to the distance between the positions of the support members 1006 and 1008 on the tread plate 1004 and the stepping position. As a result, it is possible to generate waveforms corresponding to various stepping methods and accurately detect the stepping of various stepping methods.

[0078] In particular, although it is not essential for the footstep detection pad 1000 according to the embodiment of the present disclosure to have all of the following advantages, it may have the following advantages. That is, according to the footstep detection pad 1000, it is possible to detect various footstep patterns at low cost and with high accuracy in response to the footstep operation pattern performed by the user, without the trouble of changing the installation direction of the device. Regardless of which part of the tread plate 1004 is stepped on or how it is stepped on, it is possible to reliably detect the footstep with a small number of sensors. The footstep detection pad 1000 is also composed of a bottom plate 1002, a gap, and a tread plate 1004, and the step can be reduced. Therefore, even the elderly can exercise safely. The footstep detection pad 1000 is particularly suitable for use as a device for footstep detection in a system used for recreation or rehabilitation for the elderly.

[0079] (Modification example of footstep detection processing) In the above-described embodiment, it has been described that the grounding operation (L↓ / R↓) or liftoff operation (L↑ / R↑) of the left and right feet is detected from the amount of change in the signal value per unit time (ΔV / ΔT), and the footstep is counted at the detected timing. On the other hand, when adopting the physical structure as shown in FIG. 1, the left-foot tread plate 1004L and the right-foot tread plate 1004 cannot be completely independent, and there is a possibility of interference, for example, due to the load being transmitted through the bottom plate 1002.

[0080] Hereinafter, with reference to FIG. 10, the footstep detection processing in a preferred embodiment that can prevent false detection due to interference between a plurality of sensors will be described.

[0081] FIG. 10 is a state transition diagram showing control related to footstep detection in the footstep detection pad 1000 according to a preferred embodiment of the present disclosure. More specifically, the state transition diagram shown in FIG. 10 represents the determination of the start of the footstep movement, the determination of the stop of the footstep movement, the determination of the footstep, and the control of the count of the number of footsteps. The control of the state transition shown in FIG. 10 is the responsibility of the footstep detection unit 1046.

[0082] The state transition diagram shown in FIG. 10 shows six internal states (State = 0, 1, …, 5). In FIG. 10, the change in which the right foot rises is expressed as “R↑”, the change in which the right foot falls is expressed as “R↓”, the change in which the left foot rises is expressed as “L↑”, and the change in which the left foot falls is expressed as “L↓”. The liftoff operation “L↑” and touchdown operation “L↓” of the left foot are detected from the change in the signal of the load detection sensor 1022L for the left foot, and the liftoff operation “R↑” and touchdown operation “R↓” of the right foot are detected from the change in the signal of the load detection sensor 1022R for the right foot. The footstep detection unit 1046 holds and manages the internal state based on the detection of these operations, and controls the transition between internal states in response to these operations detected based on the change in the above signal.

[0083] The internal state 1110 shown in FIG. 10 represents three initial internal states (State = 0, 1, 2) that transition after the start of processing. The internal state 1110 relates to the processing at the start of the system.

[0084] On the other hand, the internal states 1120 to 1140 represent internal states for footstep detection that transition after the initial processing. In FIG. 10, a table having columns corresponding to the left foot (L) and the right foot (R) is shown within the frames of the internal states 1120 to 1140, and holds values of “ON” or “OFF” for each foot. In response to the above-described touchdown operation (R↓ / L↓) or liftoff operation (R↑ / L↑), the state in which it is determined that the foot is on the pedal is expressed as “ON”, and the state in which it is determined that the foot is off is expressed as “OFF”.

[0085] The internal state 1120 represents a state where both foot pedals 1004L and 1004R are being stepped on (ON&ON), or a state where neither of the foot pedals 1004L and 1004R is being stepped on (OFF&OFF), that is, an internal state (State = 3) representing a state where the user has not yet stepped on the footstep detection pad 1000. The internal state 1130 represents a state where only one of the foot pedals 1004L and 1004R is being stepped on (ON&OFF or OFF&ON), that is, an internal state (State = 4) representing a state where the user starts a footstep on the footstep detection pad 1000 and raises one foot. The internal state 1150 is similar to the internal state 1120 and represents a state where both foot pedals 1004L and 1004R are being stepped on (ON&ON), but it is an internal state (State = 5) during the footstep.

[0086] In the footstep detection process, in the state transition diagram shown in FIG. 10, starting from the internal state 1120 representing the stop state, the footstep is detected by detecting the back-and-forth movement between the internal state 1130 representing the state where one foot during the footstep is stepped on and the internal state 1140 representing the state of supporting with both feet during the footstep.

[0087] In FIG. 10, when transitioning from the initial state (State = 2) to the internal state (State = 3), a state where one foot is "ON" and the other foot is "OFF" may exist in the actual usage scenario, but since it does not affect the subsequent operations, it is omitted in FIG. 10. Similarly, in the internal states (State = 4, 5) that move back and forth during the footstep, there may also be a case where both feet become "OFF", which is also omitted. When both feet become "OFF", this means that the user has stepped off the footstep detection pad 1000 midway. In the embodiment shown in FIG. 10, for the case where the user steps off midway, it is handled by transitioning back to the internal state (State = 3) 1120 due to a timeout from the internal states (State = 4, 5) during the footstep.

[0088] The following provides a more detailed explanation of footstep detection using the state transition control shown in FIG. 10. The control starts from the initial state (State = 0). At T1, the state value is incremented (State++). At T2, it transitions from the last internal state of the initial state (State = 2) to the internal state (State = 3) 1120 that serves as the starting point for footstep detection.

[0089] While in the internal state (State = 3) 1120, it is in the "pseudo footstep stop state". In response to either the right foot or the left foot leaving the pedal 1004 (lifting-off motion), at T4, it transitions to the internal state (State = 4) 1130. While in the internal state (State = 3) 1120, if either pedal 1004 is stepped on, it remains in the internal state at T3.

[0090] In the internal state (State = 4) 1130, one of the feet is in the lifted state and is waiting for the lifted foot to touch the pedal 1004. In response to detecting that the lifted foot has touched the ground, that is, detecting either the right or left grounding motion (R↓ / L↓) in the internal state (State = 4) 1130, at T7, it transitions to the internal state (State = 5) 1140. At the time of this transition T7, it is regarded that one footstep has been taken, and the count of the number of footsteps is incremented (Step Count). At the time of transition T7, simultaneously, a footstep signal may be sent to an external device (for example, the information processing device described later).

[0091] On the other hand, in response to the transition to the internal state (State = 4) 1130, a timer that starts timing for a predetermined time is activated. If no foot grounding motion (R↓ / L↓) is detected until the timer expires and the transition to the internal state (State = 5) 1140 does not occur, in response to this, as a timeout, the state transitions to the reference state (State = 3) 1120 at T6.

[0092] The internal state (State = 5) 1140 is in the state of "pretending to be stepping" just like the internal state 1120 (State = 3), where the grounded state remains unchanged, and it is waiting for the liftoff motion (R↑ / L↑) of either the left or right foot to rise again. Then, in the internal state (State = 5) 1140, in response to detecting the liftoff motion (R↑ / L↑), it transitions to the internal state 1130 (State = 4) at T8.

[0093] On the other hand, in response to transitioning to the internal state (State = 5) 1140, a timer that starts counting for a predetermined time is activated. If the liftoff motion (R↑ / L↑) of the foot is not detected until the timer expires and the transition to the internal state (State = 4) 1140 does not occur, then in response to that, as a timeout, the state transitions to the reference state (State = 3) 1120 at T10.

[0094] In this way, by using the determination results of the foot rising and falling, starting from the internal state (State = 3) 1120, detecting the transitions back and forth between the two internal states (State = 4, 5), it is possible to realize the determination of the start of the stepping motion, the determination of the stop of the stepping motion, the determination of the stepping, and the control of the count of the number of steps.

[0095] When organizing the state transition in Fig. 10 as a condition for detecting that a step has been taken, it is as follows.

[0096] The footstep detection unit 1046 detects that a footstep has been taken on the condition that, in response to detecting the grounding motion (R↓ / L↓) on the left - foot or right - foot pedal 1004L, 1004R, one of the left - foot or right - foot pedals 1004L, 1004R was in a grounded state and the other was in a non - grounded state before the detection of the grounding motion (R↓ / L↓) (that is, it was in the internal state (State = 4) 1130).

[0097] The detection of the footstep being stepped on is further conditional on the detection of the liftoff operation (R↑ / L↑) from the left or right foot pedals 1004L, 1004R (accompanied by the transition T4 / T8 to the internal state (State = 4) 1130) before the detection of the grounding operation (R↓ / L↓).

[0098] The detection of the footstep being stepped on is further conditional on the detection of the grounding operation (R↓ / L↓) being performed within a predetermined time (before the timer expires after the transition to the internal state (State = 4) 1130) since the detection of the above liftoff operation (R↑ / L↑) from the left or right foot pedals 1004L, 1004R.

[0099] In the control shown in FIG. 10, even when only one foot, for example, only the right foot is moved, it is possible to determine the start of the footstep movement, determine the stop of the footstep movement, perform the footstep determination, and count the number of footsteps. Therefore, it is also possible to suitably detect the footstep pattern of a user in a physical state where only one foot can be moved.

[0100] The above-described state transition control is borne by the above footstep detection unit 1046. In the present embodiment, the footstep detection unit 1046 includes a change detection unit, a state holding unit, and a transition control unit.

[0101] By using the above-described state transition, even when an operation that mainly reacts to one sensor also affects the other sensor, that is, when interference occurs between the sensors, it is possible to cancel such an influence and correctly process the operation determination result. In particular, it is possible to compensate for the physical structure characteristics such as interference between the above-described left and right pedals 1004L, 1004R. Also, there is an advantage that even a user who can move only one foot can detect the footstep only by the reaction of one sensor. Furthermore, with the configuration using a timer, it is possible to reliably execute the determination of the start and stop of the footstep and the control of counting the number of footsteps.

[0102] The state transition diagram shown in FIG. 10 counted the foot pedals and sent a foot pedal signal in response to the grounding operation (R↓L↓) of the foot pedals 1004L and 1004R for the left or right foot. Hereinafter, another preferred embodiment that counts the foot pedals and sends a foot pedal signal in response to the liftoff operation (R↑ / L↑) from the foot pedals 1004L and 1004R for the left or right foot will be described with reference to FIG. 11.

[0103] FIG. 11 is a state transition diagram showing control related to footstep detection in the footstep detection pad 1000 according to another preferred embodiment of the present disclosure. Compared with FIG. 10, the internal states are the same. On the other hand, it is different in that a determination is made that a footstep has been made in response to the liftoff operation (R↑ / L↑) from the internal state (State = 5) 1140.

[0104] When the state transition in FIG. 11 is organized as a condition for detecting that a footstep has been made, it is as follows.

[0105] The footstep detection unit 1046, in response to the detection of the liftoff operation (R↑ / L↑) from the foot pedals 1004L and 1004R for the left or right foot, before the detection of the liftoff operation (R↑ / L↑), while one of the foot pedals 1004L and 1004R for the left or right foot is in the grounded state and the other is in the non-grounded state where it is not stepped on, detects that the grounding operation (R↓ / L↓) to the foot pedals 1004L and 1004R for the left or right foot has been detected (that is, the transition T7 from the internal state (State = 4) 1130 has occurred), and detects that a footstep has been made.

[0106] The detection of the footstep being made further requires that the detection of the liftoff operation (R↑ / L↑) be made within a predetermined time (before the timer expires after the transition to the internal state (State = 5) 1140) after the detection of the above-mentioned grounding operation (R↓ / L↓) to the foot pedals 1004L and 1004R for the left or right foot.

[0107] Based on the state transition shown in FIG. 11, it is also possible to obtain the same advantages as those described in FIG. 10.

[0108] Hereinafter, with reference to FIGS. 12 to 21, an information processing system that controls video playback based on the movement of a user using a footstep detection pad 1000, which is a footstep detection device according to the present embodiment, will be described.

[0109] <Configuration of Information Processing System 100> FIG. 12 is a schematic diagram showing an example of an information processing system 100 according to an embodiment of the present disclosure. The information processing system 100 is a system that promotes the movement of a user U who uses the information processing system 100. The information processing system 100 controls the playback of a video Mv displayed on the display unit 4 according to the movement state of the user U. For example, the information processing system 100 can advance the playback of the video Mv while the user U is stepping, and can stop the playback of the video Mv when the user U stops stepping. Alternatively, the information processing system 100 can increase the playback speed of the video Mv when the walking speed (converted from the stepping speed) of the user U is high, and can decrease the playback speed of the video Mv when the walking speed of the user U is low. The video Mv is not particularly limited, but is, for example, a video obtained by photographing a scenery that comes into the view of a walker when walking in a tourist destination.

[0110] The user U can have a pseudo-experience as if walking in a tourist destination by visually recognizing the video Mv reproduced according to the stepping or the video Mv whose playback speed changes according to the user U's own walking speed through the display unit 4. Thereby, in the information processing system 100, it is possible to make the user U perform a stepping motion while enjoying without getting bored, and to promote the movement of the user U.

[0111] As shown in FIG. 12, the information processing system 100 includes an information processing device 1 and a footstep detection pad 1000 on which the user U performs a stepping motion and outputs the movement information of the user U. Also, in the example shown in FIG. 12, the information processing system 100 includes a gaze sensor 3 that detects the gaze of the user U. The movement information of the user U described above is a signal of the stepping motion described above.

[0112] The environment of the user U is provided with a support member 22 that the user U can grasp during exercise. The footstep detection pad 1000 may be formed with a printed indication 21 serving as a guide for the stepping position. The footstep detection pad 1000 detects the states of the left and right feet of the user U placed thereon, and the grounding or liftoff operation of the user U's feet (including not only the operation where the entire foot leaves the pedal 1004 but also the operation where only the heel or only the toe leaves the pedal 1004), or the stepping or stopping of the user's footstep, or operation information regarding both of these (indicating the grounding operation, liftoff operation, footstep, footstep stop, etc.) is output as exercise information to the information processing device 1. The footstep detection pad 1000 is a footstep detection device that detects the footstep motion described with reference to FIGS. 1 to 11. On the pedal 1004 of the footstep detection pad 1000, surface members with various textures and softness can be prepared according to the preferences of the user U, such as flocking, carpets, and rugs.

[0113] The line-of-sight sensor 3 detects the line of sight of the user U and outputs line-of-sight information regarding the line of sight of the user U to the information processing device 1. The line-of-sight sensor 3 includes a camera and an image processing circuit. The line-of-sight sensor 3 detects the line of sight of the user U by specifying, with the image processing circuit, the position of the image region of the black eyes included in the face image of the user U captured by the camera. The line-of-sight sensor 3 may be a direction key or a joystick of the operation unit 6 operated by the user.

[0114] The information processing device 1 acquires information regarding the walking state and walking speed of the user U from the exercise information input from the footstep detection pad 1000. The information processing device 1 may obtain the cycle in which the foot of the user U steps on the pedal 1004 based on the exercise information, and obtain the walking speed of the user U by calculation from this cycle. Alternatively, instead of the stepped cycle, the pitch of the footstep (the time from stepping on one foot until stepping on the other foot) may be obtained, and the walking speed may be obtained by calculation from the pitch. The information processing device 1 may vary the playback speed of the moving image Mv displayed on the display unit 4 according to the walking speed of the user U. The display unit 4 is a display device such as a liquid crystal display, an organic EL (ElectroLuminescence) display, or a plasma display.

[0115] In the example shown in FIG. 12, the information processing apparatus 1 may make the image area of the moving image Mv displayed on the display unit 4 variable according to the line-of-sight information from the line-of-sight sensor 3. For example, when the line of sight of the user U is directed to the right, the information processing apparatus 1 controls the reproduction of the moving image Mv so that the center of the right image area among all the image areas of the moving image Mv is displayed at the center of the screen of the display unit 4. By visually recognizing the moving image Mv in which the image area displayed on the display unit 4 changes according to his / her own walking and line-of-sight direction, the user U can more realistically feel a virtual experience of going out as if walking in the place shown in the moving image Mv.

[0116] In the example shown in FIG. 12, the information processing system 100 includes a speaker 5 that generates sound. The speaker 5 is incorporated in the information processing apparatus 1. The information processing system 100 causes the speaker 5 to generate sound in accordance with the moving image Mv displayed on the display unit 4. For example, when the moving image Mv includes a scenery visually recognized when walking on cobblestones, the information processing system 100 causes the speaker 5 to generate the sound of footsteps walking on the cobblestones. Thereby, a more realistic virtual experience can be given to the user U by utilizing hearing.

[0117] The user U can receive exercise stimuli, visual stimuli, auditory stimuli, etc. through the moving image Mv that changes according to the exercise information from the footstep detection pad 1000 and the line-of-sight information from the line-of-sight sensor 3, and can perform a stepping exercise while enjoying the scenery.

[0118] Note that the stimuli given to the user U are not limited to those of the above-described embodiment. In other embodiments, in addition to the above-described video and audio, other sensory stimuli such as tactile stimuli and olfactory stimuli may be provided. For example, in addition to the display unit 4 and the speaker 5, other external devices such as a lighting device, an air conditioner, an odor generating device, and a blower device may be provided. The lighting device, the air conditioner, the odor generating device, and the blower device perform operations that act on (affect) the senses of the user U. A sense is a function or awareness of feeling external stimuli, and is, for example, at least one of a person's vision, hearing, smell, and touch. The operations that act on the user's senses may include operations related to at least one of the illuminance, wind, odor, water droplets, smoke, and temperature around the user U. The illuminance can be controlled by the lighting device, the wind can be controlled by the blower device, the odor can be controlled by the odor generating device, and the temperature can be controlled by the air conditioner, respectively.

[0119] As shown in FIG. 12, when using the information processing system alone, it is possible to obtain the feeling of actually walking by playing and stopping the video according to the continuation of the foot tapping, or by varying the speed of the video according to the walking speed. In actual walking, the speed changes even for walking at the same pitch depending on the stride of the foot. However, what the foot tapping detection pad 1000 detects is the foot tapping motion at a fixed position. Therefore, regarding the stride of the foot, it may be input as user information or a setting item may be provided so that it can be set.

[0120] <Modification Example of Information Processing System> The information processing system according to the embodiment of the present disclosure is not limited to the example shown in FIG. 12, and various modifications are possible. Hereinafter, various modification examples will be described. Note that the same names and reference numerals as those of the embodiment of the present disclosure already described indicate the same or similar members or configurations, and the detailed description will be omitted as appropriate.

[0121] (First Modification Example of Information Processing System) FIG. 13 is a schematic diagram showing an information processing system 100a according to a first modified example. In the first modified example, the difference from the information processing system 100 shown in FIG. 12 is that a plurality of users U can use the information processing system 100a in parallel. Since a plurality of users U can use the information processing system 100a in parallel, pseudo-experiences such as walking can be shared among the plurality of users U, so that the user U can exercise more enjoyably compared to when using it alone.

[0122] In the example shown in FIG. 13, four users U are using the information processing system 100a. Four footstep detection pads 1000 are prepared corresponding to the four users U. The information processing apparatus 1 can acquire the motion information of each of the four users U from each of the four footstep detection pads 1000. Further, in the information processing system 100a, by displaying the video Mv visually recognized by each of the four users U on a single display unit 4, the video Mv visually recognized by each user U can be shared among the four users U.

[0123] By a plurality of users U using the information processing system 100 in parallel, it is possible to perform exercises leading to rehabilitation while recalling and talking about a scene or sharing impressions among the plurality of users U.

[0124] A plurality of display units 4 may be connected to the information processing apparatus 1. Thereby, it is also possible for users U on different floors to share the video Mv and exercise together.

[0125] In the example shown in FIG. 13, one user U is performing a footstep exercise in a standing state, and the other three users U are performing a footstep exercise while sitting on the chair 23. Thus, in the information processing system 100a, it is also possible to select a footstep exercise in a standing state or a footstep exercise in a sitting state according to the preferences and health conditions of the user U.

[0126] When multiple people use an information processing system, it is difficult to play a video according to the speed of each individual because there is only one screen. Also, the user U may get tired halfway and stop the stepping motion. Therefore, the playback speed of the video may be set to a constant, and a stepping sound and a display unit 4 indicating the stepping pitch may be displayed to draw the user's speed to a standard speed in order to draw the person's stepping motion to a constant speed. When all of the multiple users U stop the stepping motion, the video playback may be paused. Then, in response to even one of the multiple users U resuming the stepping motion, the video playback may be resumed. Also, a foot mark indicating the stepping state of the multiple users U may be displayed on the display unit 4 and changed according to the stepping of the feet, so that the stepping states of the multiple users can be confirmed with each other, and they can talk to each other and encourage each other.

[0127] Alternatively, the playback may be advanced when a majority of the multiple users are performing the stepping motion, and the playback may be stopped when a majority of the multiple users stop stepping. Alternatively, the playback speed of the video may be variable according to the average speed across the users of the walking speed of the multiple users.

[0128] (Second Modified Example of Information Processing System) FIG. 14 is a schematic diagram showing an information processing system 100b according to the second modified example. The second modified example is different from the first modified example in that an operator 200 different from the user U can operate the information processing system 100b. The operator 200 is, for example, a caregiver who cares for a care recipient as the user U. Since the operator 200 can operate the information processing system 100b, the labor of the operation by the user U can be reduced, and the operation of the information processing system 100b can be performed smoothly. As a result, the willingness of the user U to use the information processing system 100b can be increased, and the movement of the user U can be promoted.

[0129] The operator 200 uses the operation unit 6 to give instructions such as selecting a video to be displayed on the display unit 4, starting, stopping, or resuming video playback. As the operation unit 6, a touch panel, operation buttons, a keyboard, a joystick, or a combination thereof can be used. The operation unit 6 may be a detachable remote control from the information processing system 100b or the information processing apparatus 1. Alternatively, the operation unit 6 may be provided integrally with the information processing apparatus 1. Further, the operation unit 6 may be an information processing terminal such as a tablet or a smartphone.

[0130] In the above-described embodiment, the image area of the video Mv displayed on the display unit 4 can be made variable according to the line-of-sight information from the line-of-sight sensor 3. In the second modification, further, instead of the line-of-sight information from the line-of-sight sensor 3, the line-of-sight direction of the designated user U may be detected by a button operation using the operation unit 6.

[0131] (Third Modification of the Information Processing System) FIG. 15 is a schematic diagram showing an information processing system 100c according to the third modification. In the third modification, the display unit 4 being a VR (Virtual Reality) glass is different from the second modification where the display unit 4 was a flat device. The VR glass is an example of a head-mounted display device. By using the VR glass for the display unit 4, a more realistic virtual experience can be given to the user U, and the movement of the user U can be promoted. The VR glass may be a head-mounted display. Further, the VR glass is not limited to the goggle type illustrated in FIG. 15 and may be a glasses type. Also, the image displayed on the display unit 4 may be a normal flat image or a 360-degree image. For example, when using the VR glass for the display unit 4, an image obtained by cutting out a part of the 360-degree image may be displayed according to the orientation of the VR glass.

[0132] (Further Modification of the Third Modification of the Information Processing System) The information processing system 100c according to the above-described third modification example uses VR glasses for the display unit 4. However, further modifications as described below are also contemplated. The information processing system of a further modification example of the third modification example may be a system that displays a virtual space on the display unit 4 which is VR glasses. The information processing system is configured to be able to display the user's avatar on the virtual space displayed on the display unit 4 which is VR glasses, and may include an information processing device as an image processing device that performs image processing so that the avatar's feet move according to the movement of the user's feet in response to an input from a footstep detection pad 1000 that detects the movement of the user's feet. Note that the display unit 4 may be not limited to VR glasses but a display of a flat device, and it may be configured to display a virtual space and an image of the avatar within the virtual space on the flat device.

[0133] (Fourth Modification Example of Information Processing System) Next, the information processing system according to the fourth modification example will be described. FIG. 16 is a diagram showing an example of the configuration of the information processing system 100d according to the fourth modification example. The information processing system 100d is different from the above-described embodiments and modification examples in that a plurality of users U who are remotely located from each other can perform a footstep exercise while sharing the same video.

[0134] In the example shown in FIG. 16, the information processing device 1 and the display unit 4 used by each of the plurality of users U are connected to be communicable with each other via a network. The display unit 4 is a PC, a tablet, a smartphone, or the like. The information processing device 1 can acquire the movement information of the user U via the network and distribute the video data and the data to be displayed on the display unit 4 in a streaming format to each of the plurality of display units 4.

[0135] The information processing system 100d can provide users U who are remotely located from each other with a sense of presence as if they are exercising in the same place. In the example shown in FIG. 16, the footstep detection pad 1000 is wirelessly connected to the display unit 4 and can transmit motion information to the information processing apparatus 1 via the display unit 4. However, the footstep detection pad 1000 can also be directly connected to the network and transmit motion state information to the information processing apparatus 1 without going through the display unit 4.

[0136] Also, the data capacity of the video distributed in the information processing system 100d may be appropriately changed according to the device constituting the display unit 4. For example, the information processing apparatus 1 can obtain information on the type of device of the display unit 4 used by the user U and the performance of the CPU and memory from the display unit 4, and distribute the video with a data capacity suitable for each device. The adjustment of the data capacity can be performed by adjusting the resolution, frame rate, and bit rate. As a result, even when the user U uses devices with various processing speeds, in a device of the display unit 4 with a slow processing speed, the data capacity is reduced to distribute the video, and all users can participate in the exercise while watching the same video.

[0137] Also, the relationship between the eye line height of the user U and the height of the display unit 4 may vary depending on the usage form of the user U. For example, the relationship between the eye line position of the user U and the height of the display unit 4 is different when the user U performs a footstep exercise while standing and when the user U performs a footstep exercise while sitting on a chair. Also, it may vary depending on the arrangement environment of the device of the display unit 4 used by the user U. Therefore, in the information processing system 100d, before starting the reproduction of the video, control for adjusting the eye line position and the display height and inclination of the image displayed on the display unit 4 may be performed. By performing control to adjust the eye line position and the display height and inclination of the image displayed on the display unit 4, each of the plurality of users U can easily view the video according to the usage form.

[0138] <Configuration example of the information processing apparatus 1> (Hardware configuration example) FIG. 17 is a block diagram showing an example of the hardware configuration of the information processing apparatus 1. The information processing apparatus 1 is constructed by, for example, a computer. The information processing apparatus 1 includes a CPU (Central Processing Unit) 101, a ROM (Read Only Memory) 102, a RAM (Random Access Memory) 103, an HDD (Hard Disk Drive) / SSD (Solid State Drive) 104, and an I / F (Interface) 105. These are communicably connected to each other via a system bus B.

[0139] The CPU 101 executes control processing including various arithmetic processes. The ROM 102 is a non-volatile memory that stores programs used for driving the CPU 101 such as an IPL (Initial Program Loader). The RAM 103 is a volatile memory used as a work area for the CPU 101. The HDD / SSD 104 is a non-volatile memory capable of storing various information and programs used for control by the information processing apparatus 1.

[0140] The I / F 105 is an interface for performing communication between the information processing apparatus 1 and devices or apparatuses other than the information processing apparatus 1. The I / F 105 can also perform communication with external devices other than the information processing apparatus 1 via a network or the like. The external devices are a footstep detection pad 1000, a gaze sensor 3, a display unit 4, a speaker 5, an operation unit 6, a lighting device (not shown), an air conditioner, an odor generating device, a blower device, etc., and control signals can be output to each of them. The external device may be a server 300 communicably connected via a network or the like.

[0141] (Functional configuration example) FIG. 18 is a block diagram showing an example of the functional configuration of the information processing apparatus 1. In the example shown in FIG. 18, the information processing apparatus 1 includes an operation reception unit 11, an acquisition unit 12, a reproduction control unit 13, a screen control unit 14, a sound control unit 14-1, a script processing unit 15, a storage unit 16, and an output unit 17.

[0142] The functions of the operation reception unit 11, the acquisition unit 12, and the output unit 17 are realized by the I / F 105 or the like. Note that a part of the functions of the acquisition unit 12 and the output unit 17 may be realized by a processor such as the CPU 101 executing processes defined in a program stored in the ROM 102. The function of the storage unit 16 is realized by the RAM 103, the HDD / SSD 104, or the like. The functions of the reproduction control unit 13, the screen control unit 14, and the script processing unit 15 are realized by a processor such as the CPU 101 executing processes defined in a program stored in the ROM 102.

[0143] Each function provided in the information processing apparatus 1 can also be realized by one or a plurality of processing circuits. Here, the "processing circuit" includes devices such as an ASIC (Application Specific Integrated Circuit), an FPGA (Field-Programmable Gate Array), a DSP (Digital Signal Processor), and a conventional circuit module designed to execute each function described above. In addition, a part of the functions provided in the information processing apparatus 1 can also be realized by an external device such as an external PC (Personal Computer) communicably connected to the information processing apparatus 1 or a server 300. Furthermore, a part of the functions provided in the information processing apparatus 1 can also be realized by distributed processing between the information processing apparatus 1 and these external devices.

[0144] The operation reception unit 11 controls communication with the operation unit 6 to receive various operations performed by the operator using the operation unit 6. Examples of operations in the example shown in FIG. 18 include selection of information of the user U using the information processing system 100, selection of the script Sc, selection of the video Mv, start or stop of reproduction of the video Mv, and the like.

[0145] The acquisition unit 12 acquires the motion information Mi of the user U from the footstep detection pad 1000 by controlling the communication with the footstep detection pad 1000. Further, the acquisition unit 12 acquires the gaze information Gi of the user U from the gaze sensor 3 by controlling the communication with the gaze sensor 3. The acquisition unit 12 passes the acquired motion information Mi and gaze information Gi to the playback control unit 13.

[0146] The playback control unit 13 controls the playback of the video Mv displayed on the display unit 4 according to the motion state of the user U. In the example shown in FIG. 18, the playback control unit 13 causes the video Mv that progresses according to the motion state of the user U obtained from the motion information Mi or the video Mv that is played back at a playback speed or in a playback direction according to the motion state of the user U to be displayed on the display unit 4 via the screen control unit 14.

[0147] The screen control unit 14 controls the display by the display unit 4. In the example shown in FIG. 18, the screen control unit 14 controls the display by the display unit 4 by outputting a signal for controlling the display operation by the display unit 4 to the display unit 4 via the output unit 17. Further, the screen control unit 14 receives information regarding the video Mv selected by the operator via the operation reception unit 11, and acquires the video Mv from the storage unit 16 based on this information. The screen control unit 14 causes the video Mv whose playback is controlled by the playback control unit 13 to be displayed on the display unit 4.

[0148] The sound control unit 14-1 controls the output of sound from the speaker 5. In the example shown in FIG. 18, the sound control unit 14-1 controls the output of sound by the speaker 5 by outputting a signal for controlling the sound output operation by the speaker 5 to the speaker 5 via the output unit 17.

[0149] The script processing unit 15 commands the playback control unit 13 and the screen control unit 14 based on a script Sc in which a first command Cm1 for commanding playback control according to the playback time of the video Mv and a second command Cm2 for commanding display control according to the playback time of the video Mv to the screen control unit are described. The script Sc may further include commands for controlling the speaker 5, the lighting device, the air conditioner, the odor generating device, or the blower device.

[0150] Here, in this specification, the "playback time of video Mv" refers to the time counted from the start of video Mv. However, the "playback time of video Mv" does not mean the time that has actually elapsed since the start of playback of video Mv, but represents the order (position) of the images included in video Mv within video Mv. For example, if the frame rate of video Mv is Fs and the playback time is Rp, then the playback time Rp represents the order (position) of the (Rp / Fs)-th image among the plurality of images included in video Mv in order at the frame rate of video Mv from the playback start time. Therefore, the playback time of video Mv can also be expressed as a playback position. On the other hand, in this specification, the "elapsed playback time of video Mv" refers to the time that has actually elapsed since the start of playback of video Mv. Also, the "video time" refers to the time required to play back the entire video Mv.

[0151] The storage unit 16 stores the script Sc. In the example shown in FIG. 18, the storage unit 16 stores user information 161 in which various information regarding each of the plurality of users U is recorded, a script group 162 including a plurality of scripts Sc associated with the plurality of videos Mv, and a video group 163 including the plurality of videos Mv.

[0152] The storage unit 16 may be provided in the server 300 shown in FIG. 17 instead of the information processing apparatus 1. In this case, the screen control unit 14 and the script processing unit 15 acquire the user information 161, the script Sc, and the video Mv via the network N. Also, a part of the user information 161, the script group 162, and the video group 163 may be stored in the information processing apparatus 1, and the other part may be stored in the server 300.

[0153] The output unit 17 controls the communication between the information processing apparatus 1 and the external device in response to a command from the screen control unit 14. This external device includes a display unit 4, a speaker 5, an operation unit 6, a server 300, a lighting device (not shown), an air conditioner, an odor generating device, and a blower device, etc.

[0154] Here, the user U may have different objects of interest depending on their place of origin, life story, living environment, job, family situation, etc. Also, when the operator 200 exists, the objects of interest also differ depending on the experience of the operator 200. Furthermore, depending on the physical strength and physical ability of the user U, the time for using the information processing system 100 may also differ. Therefore, it is preferable that the video Mv can be provided according to the objects of interest of the user U and the operator 200.

[0155] In the information processing apparatus 1, the script processing unit 15 issues commands for controlling the playback control unit 13 and the screen control unit 14 based on the script Sc. By causing the playback control unit 13 and the screen control unit 14 to be controlled according to the commands, comments, quizzes, images used for quizzes, etc. can be superimposed and displayed on the video Mv, images other than the images to be noted in the video Mv can be deleted to adjust the video time, or different videos can be joined together, all without pre-editing the video Mv. In this way, not only can the uniformly created video be enjoyed just by walking, but also by incorporating adding comments, quizzes, pictures into the video, deleting points other than the points of interest in the video to adjust the time, or combining different videos, it is possible to perform usage according to the user and the operator. Also, by using the script Sc, it becomes possible to create the script Sc individually for each of a plurality of users U and each of a plurality of operators 200. As a result, in the information processing apparatus 1 and the information processing system 100, it is possible to easily provide the video Mv according to the user U and the operator.

[0156] Also, in the example shown in FIG. 18, the script processing unit 15 issues commands for controlling the playback control unit 13 and the screen control unit 14 based on the script Sc stored in the storage unit 16 provided in the information processing apparatus 1.

[0157] In the example shown in FIG. 18, the storage unit 16 stores a script Sc corresponding to the user U or a group consisting of a plurality of users U. The script processing unit 15 refers to the storage unit 16 and acquires a script Sc corresponding to the user U or a group consisting of a plurality of users U in response to the selection input of the user U or the group. As a result, it becomes possible to easily provide a video Mv suitable for the user U or a group consisting of a plurality of users U.

[0158] Further, the embodiment of the present disclosure includes a program. The program causes a computer to execute a process of controlling the reproduction of the video Mv displayed on the display unit 4 according to the motion state of the user U by the reproduction control unit 13, and controlling the display by the display unit 4 by the screen control unit 14. Further, the program causes a computer to execute a process of instructing the reproduction control unit 13 to perform reproduction control according to the reproduction time of the video Mv and instructing the screen control unit 14 to perform display control according to the reproduction time, based on a script Sc in which a first command Cm1 and a second command Cm2 are described, and instructing the reproduction control unit 13 and the screen control unit 14 to perform control. By such a program, the same operational effects as those of the information processing apparatus 1 and the information processing system 100 described above can be obtained.

[0159] As described above, in the embodiment of the present disclosure, without editing the video, by obtaining an editing effect when using it on the information processing system 100, the effects of reducing the burden on the storage device and the burden of arrangement, and adding effects such as text insertion into the video and pause / resume of the video without advanced editing operations can be achieved. Further, it is also possible to add the effect of combining scenes of a plurality of videos or adjusting the length of the video itself. In addition, by being able to set those effects for each user (personal information) and each caregiver, the same video can be used according to each user and caregiver.

[0160] <Operation example of information processing system 100> (Overall operation) FIG. 19 is a flowchart showing an example of the overall operation of the information processing system 100. FIG. 19 shows the operations performed by the information processing system 100 when the user U uses the information processing system 100, in other words, the overall outline of the operations for causing the user U to use the information processing system 100. For example, the information processing system 100 starts the operations in FIG. 19 on the condition that the information processing system 100 has received a start operation of using the information processing system 100 by an operator using the operation unit 6.

[0161] First, in step S11, the information processing system 100 causes the display unit 4 to display a user information setting screen. Subsequently, in step S12, the information processing system 100 receives, via the operation unit 6 by the operator, a selection input of the number of users U by the operation reception unit 11 of the information processing apparatus 1. Subsequently, in step S13, the information processing system 100 receives, from the operator via the operation unit 6, an input of the name information of the user U by the operation reception unit 11 of the information processing apparatus 1. Here, the operator is the user U or the operator 200 shown in FIG. 14 or the like.

[0162] Subsequently, in step S14, the information processing system 100 refers to the storage unit 16 based on the name information of the user U and acquires a script Sc corresponding to the user U. Subsequently, in step S15, the information processing system 100 causes the display unit 4 to display a thumbnail and text for video selection by the screen control unit 14 of the information processing apparatus 1. Subsequently, in step S16, the information processing system 100 receives, from the operator via the operation unit 6, a selection input of the video Mv by the operation reception unit 11 of the information processing apparatus 1.

[0163] Subsequently, in step S17, the information processing system 100 starts playing the selected video Mv by the playback control unit 13 of the information processing apparatus 1. The screen control unit 14 causes the video Mv to be displayed on the display unit 4 in response to a command from the playback control unit 13. The information processing system 100 continues to play the video Mv until the video Mv finishes playing after step S17.

[0164] Subsequently, in step S18, the information processing system 100 executes script processing by the script processing unit 15 of the information processing apparatus 1. The script processing unit 15 commands the playback control unit 13 and the screen control unit 14 according to the description of each line of the script Sc. In step S18, in parallel with the execution of the script, playback control processing is executed. The playback control processing starts playing the video specified in the script from the specified position, and makes settings such as display and stop for the video according to the processing type and its setting content in the order of the table written in the script. Also as parallel processing, the progress of the video is controlled by the input of motion information from the footstep detection pad 1000. For example, when a footstep is detected, the video is played, and when no footstep is detected, the playback of the video is stopped.

[0165] Subsequently, in step S19, the information processing system 100 determines whether to end the playback of the video Mv. For example, the information processing system 100 can determine whether to end the playback of the video Mv by determining whether the playback elapsed time of the video Mv matches the video time of the video Mv by the screen control unit 14 of the information processing apparatus 1.

[0166] If it is determined in step S19 that the playback of the video Mv is not ended (step S19, NO), the information processing system 100 performs the operation of step S18 again. On the other hand, if it is determined that the playback of the video Mv is ended (step S19, YES), the information processing system 100 determines whether to end the use of the information processing system 100 in step S20. For example, the information processing system 100 can determine whether to end the use of the information processing system 100 according to the operation input by the operator via the operation unit 6.

[0167] In step S20, if it is determined that the use of the information processing system 100 is not to be terminated (step S20, NO), the information processing system 100 repeats the operations after step S11. On the other hand, if it is determined that the use is to be terminated (step S20, YES), the information processing system 100 ends the operation.

[0168] As described above, the information processing system 100 can allow the user U to use the information processing system 100. When a plurality of users U view the video Mv displayed on one display unit 4, the operator 200 can also select a video Mv or a script Sc common to each of the plurality of users U.

[0169] (Playback processing) FIG. 20 is a flowchart showing an example of the processing by the playback control unit 13. FIG. 20 shows an example of the video Mv playback control process by the playback control unit 13. For example, the playback control unit 13 starts the operation of FIG. 20 with the condition that it has reached step S17 in the process shown in FIG. 19. In the following description, it is assumed that a footstep signal is input from the footstep detection pad 1000. Here, information indicating that a footstep (for example, once) has been made is referred to as a footstep input.

[0170] First, in step S31, the playback control unit 13 determines whether there is a footstep input from the footstep detection pad 1000. If it is determined in step S31 that there is no footstep input (step S31, NO), the playback control unit 13 proceeds to the process of step S34. On the other hand, if it is determined that there is a footstep input (step S31, YES), the playback control unit 13 plays the video Mv in step S32 and causes the video Mv to be displayed on the display unit 4 via the screen control unit 14. In step S33, the playback control unit 13 passes the playback time (playback position) of the video to the script processing unit 15 and returns to step S31.

[0171] Next, in step S34, the playback control unit 13 determines whether there is no footstep input from the footstep detection pad 1000 for a certain period. Here, if there is no footstep for a certain time and the footstep input has stopped for a certain period, it is regarded that the footstep has stopped. If it is determined in step S34 that there is a footstep input (step S34, NO), the operation ends. On the other hand, if it is determined that there is no footstep input for a certain period (step S34, YES), in step S35, the playback control unit 13 stops the playback of the video Mv and ends the operation.

[0172] As described above, the playback control unit 13 can perform the playback process of the video Mv. When a footstep is detected by a footstep input from the footstep detection pad 1000, the video is played. When no footstep is detected, the playback of the video is stopped based on the fact that the footstep signal from the footstep detection pad 1000 has stopped for a predetermined period, and the next footstep input from the footstep detection pad 1000 is awaited.

[0173] In the above-described embodiment, it has been described that a footstep signal is input from the footstep detection pad 1000. However, the present invention is not limited to such an embodiment. In other embodiments, a footstep stop signal explicitly indicating that the footstep has stopped may be input from the footstep detection pad 1000, or information indicating the state of the grounding operation or the liftoff operation of each foot may be input, and on the side of the information processing apparatus 1, a determination as to whether it is a footstep or a footstep stop may be made from the information on the grounding or liftoff operation of each foot.

[0174] (Modification example of playback process) FIG. 21 is a flowchart showing another example of the process by the playback control unit 13. FIG. 21 shows an example of the playback control process of the video Mv by the playback control unit 13. For example, the playback control unit 13 starts the operation of FIG. 21 with the condition that it has reached step S17 in the process shown in FIG. 19 as the start condition.

[0175] First, in step S41, the playback control unit 13 determines whether there is a footstep input from the footstep detection pad 1000. If it is determined in step S41 that there is no footstep input (step S41, NO), the playback control unit 13 loops step S41. On the other hand, if it is determined that there is a footstep input (step S41, YES), in step S42, the playback control unit 13 plays the video Mv and causes the display unit 4 to display the video Mv via the screen control unit 14. In step S43, the playback control unit 13 records the input time. In step S44, the playback control unit 13 reads out the previously input time and calculates the elapsed time from the previous time. In step S45, the playback control unit 13 calculates the playback speed from the elapsed time. In step S46, the playback control unit 13 passes the playback time (playback position) and playback speed of the video to the script processing unit 15 and returns to step S41.

[0176] When using the present information processing system 100 alone as in the example shown in FIG. 12, the playback speed of the video can be made variable according to the walking pitch of the feet (the time from the previous footstep to the next footstep) according to the flow of FIG. 21. By varying the playback position and playback speed of the video according to the pitch of the footstep input, it is possible to perform playback of the video in accordance with the speed of the footstep.

[0177] Note that, in the above-described embodiment of the information processing system, the information processing apparatus 1 is configured separately from the footstep detection pad 1000, and has been described as performing reproduction control of the video displayed on the display unit 4 in response to the input of the motion information from the footstep detection pad 1000. That is, in the above-described embodiments, the footstep detection pad 1000 detects the footstep motion based on the amplified signal, while the information processing apparatus 1, which is another processing apparatus, receives the input from the footstep detection pad 1000 and controls the reproduction of the video data displayed on the display unit 4 in response to the detection of the footstep motion. However, in other embodiments, the functions of the information processing apparatus 1 may be integrally implemented in the footstep detection pad 1000. In such other embodiments, the footstep detection pad 1000 detects the footstep motion based on the amplified signal and controls the reproduction of the video data displayed on the display unit 4 in response to the detection of the footstep motion.

[0178] As described above in detail with respect to the preferred embodiments, the present disclosure is not limited to the above-described embodiments of the present disclosure, and various modifications and substitutions can be made to the above-described embodiments of the present disclosure without departing from the scope described in the claims.

[0179] The numbers such as ordinal numbers and quantities used in the description of the embodiments of the present disclosure are all examples for specifically explaining the technology of the present disclosure, and the present disclosure is not limited to the exemplified numbers. In addition, the connection relationships between the components are examples for specifically explaining the technology of the present disclosure, and the present disclosure is not limited to the connection relationships that realize the functions of the present disclosure.

[0180] The division of the blocks in the functional block diagram is an example, and a plurality of blocks may be realized as one block, one block may be divided into a plurality, or some functions may be transferred to other blocks. Also, the functions of a plurality of blocks having similar functions may be processed by a single piece of hardware or software in parallel or in time division. Further, some or all of the functions may be distributed among a plurality of computers.

[0181] Aspects of the present disclosure are as follows, for example. <1> A footstep detection device, comprising a treadle and a load detection sensor configured to detect a load acting through the treadle, which varies according to the position where the treadle is stepped on; an amplification unit that amplifies a component obtained by removing a bias component of a signal from the load detection sensor; a detection unit that detects a footstep operation based on the amplified signal. The footstep detection device includes the above components. <2> A bottom plate and a support member provided between the treadle and the bottom plate so as to create a gap between the treadle and the bottom plate. The load detection sensor is provided on the support member. The footstep detection device according to <1>. <3> Including a plurality of other support members provided between the treadle and the bottom plate, a substrate having the amplification unit and the detection unit is provided adjacent to the outer peripheral portion of the treadle, the support member on which the load detection sensor is provided is arranged on the outer peripheral portion of the treadle so as to have a shorter physical distance from the substrate than the plurality of other support members. The footstep detection device according to <2>. <4> The amplification unit has an adjustment unit that varies a gain for amplifying the component of the signal. The footstep detection device according to <2> or <3>. <5> The detection unit detects the footstep operation based on the magnitude of the change amount of the signal value per unit time. The footstep detection device according to any one of <2> to <4>. <6> The signal from the load detection sensor is an analog signal, and the footstep detection device includes a low-pass filter that passes a low-frequency component of the analog signal, a conversion unit that converts the analog signal that has passed through the low-pass filter and has been amplified by the amplification unit into a digital signal. Further includes, and the detection unit acquires the digital signal as the amplified signal, and is the footstep detection device according to any one of <1> to <5>. <7> The load detection sensor is a pressure sensor, and is the footstep detection device according to any one of <1> to <6>. <8> The pedal includes a first pedal for one foot and a second pedal for the other foot, the support member includes a first support member provided between the first pedal and the bottom plate, and a second support member provided between the second pedal and the bottom plate, and the load detection sensor includes a first sensor provided on the first support member and a second sensor provided on the second support member, and is the footstep detection device according to any one of <2> to <7>. <9> The first support member and the second support member are provided at an outer peripheral portion closer to the center of the entire pedal including the first pedal and the second pedal, and a plurality of other support members provided between the first pedal and the second pedal and the bottom plate are provided at an outer peripheral portion excluding the central portions of the first pedal and the second pedal, and is the footstep detection device according to <8>. <10> The pedal includes a first pedal and a second pedal. The detection unit, in response to detection of a grounding operation on the first pedal or the second pedal by the load detection sensor, detects that a footstep has been made on the condition that one of the first pedal and the second pedal was in a grounded state where it was stepped on and the other was in a non-grounded state where it was not stepped on before the detection of the grounding operation, and is the footstep detection device according to any one of <1> to <9>. <11> The detection of the footstep being made is further conditional on the detection of a liftoff operation from the first pedal or the second pedal before the detection of the grounding operation, and is the footstep detection device according to <10>. <12> The detection of the footstep being made is further conditional on the detection of the grounding operation being made within a predetermined time after the detection of the liftoff operation from the first pedal or the second pedal, and is the footstep detection device according to <11>. <13> The pedal includes a first pedal and a second pedal, The detection unit, in response to detection of a take-off operation from the first pedal or the second pedal by the load detection sensor, before the detection of the take-off operation, one of the first pedal and the second pedal is in a grounded state where it is stepped on, and the other is in a non-grounded state where it is not stepped on. The footstep detection device according to any one of <1> to <9>, wherein it is detected that a footstep has been made on the first pedal or the second pedal on the condition that a grounding operation on the first pedal or the second pedal has been detected. <14> The detection of the footstep being made is further conditional on the detection of the take-off operation being made within a predetermined time after the grounding operation on the first pedal or the second pedal is detected. The footstep detection device according to <13>. <15> The footstep detection device according to any one of <1> to <14>, A reproduction control unit that controls reproduction of video data to be displayed on a display unit in response to a footstep of a user detected by the footstep detection device An information processing system comprising: <16> The display unit is a head-mounted or glasses-type display device. The information processing system according to <15>. <17> An information processing system according to <15> or <16>, having an information processing device that transmits the video data to the display unit via a network. <18> The footstep detection device transmits information indicating the footstep movement of the user to the information processing device via a network. The information processing system according to <17>. <19> A footstep detection method executed by a footstep detection device including a load detection sensor and an amplification unit, Detecting, by the load detection sensor, a load acting through the pedal that varies according to a position where the pedal is stepped on; Amplifying, by the amplification unit, a component obtained by removing a bias component of a signal from the load detection sensor; Detecting a footstep operation based on the amplified signal A footstep detection method including <20> A video playback method of playing a video on a display unit using a load detection sensor and an amplifier unit, a step of detecting, by the load detection sensor, a load acting through the footrest that varies according to a position where the footrest is stepped on; a step of amplifying, by the amplifier unit, a component obtained by removing a bias component of a signal from the load detection sensor; a step of a processing device detecting a footstep operation based on the amplified signal; a step of the processing device or another processing device controlling reproduction of video data to be displayed on the display unit in response to detection of the footstep operation A video playback method including <22> An information input device that performs information input using a plurality of sensors, a change detection unit that detects a first change and a second change in signals of each of the plurality of sensors; a state holding unit that holds a first internal state and a second internal state based on the first change and the second change of each of the plurality of sensors; a transition control unit that controls to transition to the second internal state on the condition of being in the first internal state in response to detection of the first change by at least one of the plurality of sensors, and controls to transition to the first internal state on the condition of being in the second internal state in response to detection of the second change by at least one of the plurality of sensors An information input device including, wherein information is input in response to a transition to the second internal state or a transition to the first internal state. <23> The state holding unit holds a third internal state, and the information input device further includes a timer that starts timing in response to a transition between the first internal state and the second internal state, The transition control unit transitions to the third internal state in response to the fact that no transition has occurred between the first internal state and the second internal state even when the timer has expired, and, in response to detecting the first change or the second change by at least one of the plurality of sensors, performs control to transition to either the first internal state or the second internal state on the condition that it is in the third internal state. The information input device according to <22>.

Explanation of Signs

[0182] 1000… Footstep detection pad, 1002… Bottom plate, 1004… Pedal, 1006, 1008… Support members, 1010… Controller, 1012… Battery, 1020… Substrate, 1022… Load detection sensor, 1024… Wiring, 1026… Connector, 1028… Low-pass filter (LPF), 1030… AC coupling amplifier, 1031… Coupling capacitor, 1032… Operational amplifier, 1033… Variable resistor, 1040… Footstep detection chip, 1042… Analog / digital converter (ADC), 1044… Digital filter (DF), 1046… Footstep detection unit, 1110, 1120, 1130, 1140… Internal states, T1 to T10… Transitions, 1… Information processing device, 11… Operation reception unit, 12… Acquisition unit, 13… Reproduction control unit, 14… Screen control unit, 15… Script processing unit, 16… Storage unit, 161… User information, 162… Script group, 17… Output unit, 21… Print display, 22… Support member, 23… Chair, 3… Line-of-sight sensor, 4… Display unit, 5… Speaker, 6… Operation unit, 100, 100a, 100b, 100c, 110d… Information processing system, 101… CPU, 102… ROM, 103… RAM, 104… HDD / SSD, 105… I / F, 300… Server, B… System bus, Cm1… First command, Cm2… Second command, Gi… Line-of-sight information, Mi… Motion information, Mv… Video, N… Network, Sc… Script, U… User

Prior Art Documents

Patent Documents

[0183]

Patent Document 1

Claims

1. A footstep detection device, a treadle, a load detection sensor configured to detect a load acting through the treadle that varies according to the position where the treadle is stepped on, an amplification unit that amplifies a component obtained by removing a bias component of a signal from the load detection sensor, a detection unit that detects a footstep operation based on the amplified signal and includes a footstep detection device.

2. a bottom plate, a support member provided between the treadle and the bottom plate so that a gap is formed between the treadle and the bottom plate and includes a footstep detection device according to claim 1, wherein the load detection sensor is provided on the support member.

3. including a plurality of other support members provided between the treadle and the bottom plate, a substrate having the amplification unit and the detection unit is provided adjacent to an outer peripheral portion of the treadle, the support member on which the load detection sensor is provided is arranged at an outer peripheral portion of the treadle to have a shorter physical distance from the substrate than the plurality of other support members, the footstep detection device according to claim 2.

4. The amplification unit has an adjustment unit that varies a gain for amplifying the component of the signal The footstep detection device according to claim 2.

5. The detection unit performs detection of the footstep operation based on a magnitude of a change amount of a value of the signal per unit time, the footstep detection device according to claim 2.

6. The signal from the load detection sensor is an analog signal, and the footstep detection device a low-pass filter that passes a low-frequency component of the analog signal, a conversion unit that passes through the low-pass filter and converts the analog signal amplified by the amplification unit into a digital signal and further includes, and the detection unit acquires the digital signal as the amplified signal, the footstep detection device according to claim 1.

7. The load detection sensor is a pressure-sensitive sensor, the footstep detection device according to claim 1.

8. The treadle includes a first treadle for one foot and a second treadle for the other foot, the support member includes a first support member provided between the first treadle and the bottom plate and a second support member provided between the second treadle and the bottom plate, and the load detection sensor includes a first sensor provided on the first support member and a second sensor provided on the second support member, the footstep detection device according to claim 2.

9. The first support member and the second support member are provided at an outer peripheral portion closer to the center of the entire pedal including the first pedal and the second pedal, and a plurality of other support members provided between the first pedal and the second pedal and the bottom plate are provided at an outer peripheral portion excluding the central portions of the first pedal and the second pedal. The footstep detection device according to claim 8.

10. The pedal includes a first pedal and a second pedal. The detection unit, in response to detection of a grounding operation on the first pedal or the second pedal by the load detection sensor, on the condition that one of the first pedal and the second pedal was in a grounded state where it was stepped on and the other was in a non-grounded state where it was not stepped on before the detection of the grounding operation, detects that a footstep has been made. The footstep detection device according to claim 1.

11. The detection of the footstep being made is further conditional on the detection of a liftoff operation from the first pedal or the second pedal before the detection of the grounding operation. The footstep detection device according to claim 10.

12. The detection of the footstep being made is further conditional on the detection of the grounding operation being made within a predetermined time after the detection of the liftoff operation from the first pedal or the second pedal. The footstep detection device according to claim 11.

13. The pedal includes a first pedal and a second pedal. The detection unit, in response to detection of a liftoff operation from the first pedal or the second pedal by the load detection sensor, on the condition that a grounding operation on the first pedal or the second pedal was detected while one of the first pedal and the second pedal was in a grounded state where it was stepped on and the other was in a non-grounded state where it was not stepped on before the detection of the liftoff operation, detects that a footstep has been made. The footstep detection device according to claim 1.

14. The detection of the footstep being made is further conditional on the detection of the liftoff operation being made within a predetermined time after the detection of the grounding operation on the first pedal or the second pedal. The footstep detection device according to claim 13.

15. A footstep detection device according to any one of claims 1 to 14, and a playback control unit that controls the playback of video data displayed on a display unit in response to a user's footstep detected by the footstep detection device. An information processing system comprising.

16. The information processing system according to claim 15, wherein the display unit is a head-mounted or glasses-type display device.

17. The information processing system according to claim 15, comprising an information processing device that transmits the video data to the display unit via a network.

18. The information processing system according to claim 17, wherein the footstep detection device transmits information indicating the user's footstep movement to the information processing device via a network.

19. A footstep detection method executed by a footstep detection device including a load detection sensor and an amplification unit, detecting, by the load detection sensor, a load acting through the footrest that varies according to the position where the footrest is stepped on; amplifying, by the amplification unit, a component obtained by removing a bias component of a signal from the load detection sensor; detecting a footstep operation based on the amplified signal and including a footstep detection method.

20. A video playback method for playing back a video on a display unit using a load detection sensor and an amplification unit, detecting, by the load detection sensor, a load acting through the footrest that varies according to the position where the footrest is stepped on; amplifying, by the amplification unit, a component obtained by removing a bias component of a signal from the load detection sensor; detecting, by a processing device, a footstep operation based on the amplified signal; and controlling, by the processing device or another processing device, playback of video data to be displayed on the display unit in response to detection of the footstep operation and including a video playback method.

Images