Posture training support device, method, and program

Abstract

It reduces the impact of eye movement on body balance, effectively supporting posture training. [Solution] Multiple posture measurement data related to the user's posture are acquired, and information indicating the presentation priority is assigned to the acquired multiple posture measurement data. At the same time, multiple presentation areas are set within the user's field of view to present visual information representing the posture measurement data, and a presentation priority is set for each set presentation area based on the user's visual characteristics. Then, the visual information representing the posture measurement data is presented in the presentation areas in order from the area with the highest presentation priority, according to the presentation priority.

Description

Technical Field

[0001] One aspect of this invention relates to a posture training support device, method, and program used to assist in training a user's posture, for example, in the fields of art and sports.

Background Art

[0002] For example, when training standing balance and various poses in ballet or dance, the training effect can be enhanced if the user can check their own posture. Therefore, in Non-Patent Document 1, by using virtual reality technology or a plurality of displays, the user's posture imaged from various angles can be displayed, a part of the body part can be enlarged and displayed, or the locus of the center of gravity movement represented by the center of pressure (COP) can be displayed in real time. By using this method, the user can obtain a lot of information related to the posture of their own body as visual feedback information and can use it to improve the posture.

Prior Art Documents

Non-Patent Documents

[0003]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, the method described in Non-Patent Document 1 does not take into account the effect of eye movement when the user views the displayed information when multiple pieces of posture information are displayed simultaneously. Therefore, for example, if information representing the posture that the user wants to check is displayed in a direction that is within the user's field of view, it may be difficult to move the eyes, which can change the body's balance and disrupt posture, potentially negatively impacting posture training.

[0005] This invention was made in view of the above circumstances and aims to provide a technology that can more effectively support posture training by reducing the influence of eye movement on body balance. [Means for solving the problem]

[0006] To solve the above problems, one embodiment of the posture training support device or posture training support method according to the present invention acquires a plurality of posture measurement data related to the user's posture and assigns information representing the presentation priority to the acquired plurality of posture measurement data. At the same time, a plurality of presentation areas are set within the user's field of view for presenting visual information representing the posture measurement data, and a presentation priority is set for each set presentation area based on the user's visual characteristics. The visual information representing the posture measurement data is then presented in the presentation areas in order from the area with the highest presentation priority, according to the presentation priority.

[0007] According to one aspect of this invention, a presentation priority is set for a plurality of presentation areas set within the user's field of view, based on the user's visual characteristics. Visual information representing the acquired posture measurement data is presented in order from the area with the highest presentation priority, according to that presentation priority. Therefore, for example, among the visual information representing posture measurement data, information with a high presentation priority is preferentially presented to presentation areas that require less eye movement, taking into account the user's visual characteristics. As a result, when the user views the visual information with a high presentation priority, they do not need to move their eyes significantly, which makes it less likely for changes in body balance due to eye movement to occur, and thus makes it possible to prevent posture collapse. [Effects of the Invention]

[0008] In other words, according to one aspect of this invention, it is possible to provide a technology that reduces the influence of eye movement on body balance, thereby more effectively supporting posture training. [Brief explanation of the drawing]

[0009] [Figure 1] Figure 1 shows an example of a training support system using a posture training support device according to one embodiment of the present invention. [Figure 2] Figure 2 is a block diagram showing an example of the hardware configuration of a posture training support device according to one embodiment of the present invention. [Figure 3] Figure 3 is a block diagram showing an example of the software configuration of a posture training support device according to one embodiment of the present invention. [Figure 4] Figure 4 is a flowchart showing an example of the processing procedure and processing content of the posture training support control performed by the control unit of the posture training support device shown in Figure 3. [Figure 5] Figure 5 shows an example of a divided visual information presentation area set in the display area of ​​a head-mounted display, and the presentation priority set for these presentation areas. [Figure 6] Figure 6 shows an example of a display medium according to another embodiment of the present invention. [Modes for carrying out the invention]

[0010] Embodiments of this invention will be described below with reference to the drawings.

[0011] [One embodiment] (Example configuration) (1) System Figure 1 shows an example of a training support system using a posture training support device according to one embodiment of the present invention.

[0012] In one embodiment of the training support system, a head-mounted display (HMD) HD is attached to the head of the user US, and posture sensors CM are placed around the user US. These head-mounted display HD and posture sensors CM are connected to a posture training support device CS, for example, via a signal cable.

[0013] The head-mounted display (HD) is used to present visual information to the user (US) to train their standing balance.

[0014] The posture sensor CM, for example, acquires information about the user's (US) standing balance and consists of a force plate and an image sensor using multiple cameras.

[0015] The force plate measures the trajectory of postural sway, represented by the user's center of pressure (COP), and outputs the measurement data. The image sensor captures the user's standing posture from different angles and outputs the image data. Alternatively, a three-dimensional image sensor such as a Kinekt® sensor may be used as the image sensor, and multiple image data from different angles may be obtained from the three-dimensional image data captured by this three-dimensional image sensor.

[0016] (2) Posture training support device CS FIG. 2 and FIG. 3 are block diagrams showing an example of the hardware configuration and software configuration of a posture training support device CS according to an embodiment of the present invention.

[0017] The posture training support device CS is configured by, for example, a personal computer and includes a control unit 1 that uses a hardware processor such as a Central Processing Unit (CPU). The control unit 1 is connected via a bus 5 to a storage medium including a program storage unit 2 and a data storage unit 3, and an input / output interface (hereinafter, the interface is referred to as I / F) unit 4.

[0018] The input / output I / F unit 4 is connected to the above-described posture sensor CM and head-mounted display HD. The input / output I / F unit 4 receives a plurality of measurement data output from the posture sensor CM and outputs visual information generated by the control unit 1 to the head-mounted display HD.

[0019] Note that the input / output I / F unit 4 and the posture sensor CM and the head-mounted display HD may be connected using a wireless interface that adopts a low-power wireless data communication standard such as Wi-Fi (registered trademark) or Bluetooth (registered trademark).

[0020] The program storage unit 2 is configured by, for example, combining a non-volatile memory such as a Solid State Drive (SSD) that can be written and read at any time as a storage medium and a non-volatile memory such as a Read Only Memory (ROM), and stores application programs necessary for executing various controls according to an embodiment in addition to middleware such as an Operating System (OS). Hereinafter, the OS and each application program are collectively referred to as a program.

[0021] The data storage unit 3 combines, for example, a non-volatile memory such as an SSD that can be written to and read at any time, and a volatile memory such as RAM (Random Access Memory) as a storage medium. In its storage area, as the main storage unit according to one embodiment of this invention, a measurement data storage unit 31, a presentation priority storage unit 32, and a presentation priority storage unit 33 are provided.

[0022] The measurement data storage unit 31 stores multiple measurement data related to the user US's standing balance, which are acquired from the posture sensor CM.

[0023] The presentation priority storage unit 32 stores the measurement data selected as the target for presentation from among the multiple measurement data stored in the measurement data storage unit 31, along with information representing the presentation priority assigned to these measurement data.

[0024] The presentation priority storage unit 33 stores, for example, position coordinate information representing multiple divided presentation areas set in the display area of ​​the head-mounted display HD, along with information representing the presentation priority set for these divided presentation areas.

[0025] The control unit 1 includes, as processing functions necessary to realize one embodiment of this invention, a measurement data acquisition processing unit 11, a presentation target data selection processing unit 12, a presentation priority assignment processing unit 13, a visual information presentation area setting processing unit 14, a presentation priority setting processing unit 15, and a visual information output processing unit 16.

[0026] Each of the above-mentioned processing units 11 to 16 is implemented by having the hardware processor of the control unit 1 execute the application program stored in the program storage unit 2. Note that some or all of the above-mentioned processing units 11 to 16 may be implemented using hardware such as LSI (Large Scale Integration) or ASIC (Application Specific Integrated Circuit).

[0027] The measurement data acquisition processing unit 11 acquires multiple measurement data related to the user US's standing balance, output from the posture sensor CM, via the input / output I / F unit 4, and stores the acquired multiple measurement data in the measurement data storage unit 31.

[0028] The data selection processing unit 12 selects, for example, multiple measurement data (hereinafter referred to as "data to be presented") that the user has specified as data to be presented, from among the multiple measurement data stored in the measurement data storage unit 31.

[0029] The presentation priority assignment processing unit 13 sets a presentation priority for each of the selected multiple presentation target data, for example, according to the importance specified by the user US. The presentation priority assignment processing unit 13 then stores the multiple presentation target data in the presentation priority storage unit 32, with information representing the presentation priority attached to them.

[0030] The visual information presentation area setting processing unit 14 sets multiple divided presentation areas in the display area of ​​the head-mounted display HD.

[0031] The presentation priority setting processing unit 15 sets a presentation priority for the set multiple divided presentation areas based on the user's (US) visual characteristics. The presentation priority setting processing unit 15 then stores the position coordinate information of the multiple divided presentation areas in the presentation priority storage unit 33, associating it with the information representing the presentation priority.

[0032] The visual information output processing unit 16 assigns the data to be presented, stored in the presentation priority storage unit 32, to a plurality of divided presentation regions stored in the presentation priority storage unit 33, in order from the region with the highest presentation priority, according to the presentation priority assigned to the data. The visual information output processing unit 16 then synthesizes the plurality of data to be presented, assigned to the divided presentation regions, to generate visual information, and outputs the generated visual information from the input / output I / F unit 4 to the head-mounted display HD.

[0033] (Example of operation) Next, we will explain an example of the operation of the posture training support device CS configured as described above.

[0034] Figure 4 is a flowchart showing an example of the processing procedure and processing content of the training support control performed by the control unit 1 of the posture training support device CS.

[0035] In this example, we will explain the case where the user (US) is assisting with standing balance training, for example, while practicing ballet.

[0036] (1) Acquisition of measurement data Suppose that user US, while wearing the head-mounted display HD, inputs a request to start training support by, for example, operating a button (not shown) provided on the head-mounted display HD. When the control unit 1 of the posture training support device CS detects the above training support start request in step S11, under the control of the measurement data acquisition processing unit 11, in step S12, it executes the following process to acquire multiple measurement data related to user US's standing balance.

[0037] In other words, the measurement data acquisition processing unit 11 acquires multiple measurement data output in real time from the posture sensor CM via the input / output I / F unit 4, and stores the acquired multiple measurement data in the measurement data storage unit 31.

[0038] For example, measurement data representing the user's center of pressure (COP) is acquired from the force plate, and this acquired measurement data representing the COP is stored in the measurement data storage unit 31 as data representing the trajectory of the user's center of gravity sway. At the same time, image data of the user captured from multiple different angles is acquired from the camera, and this acquired image data from multiple angles is stored in the measurement data storage unit 31.

[0039] (2) Selection of data to be presented and assignment of presentation priority The control unit 1 of the posture training support device CS first selects the measurement data to be presented from the multiple measurement data acquired in step S13, under the control of the data selection processing unit 12.

[0040] The selection of measurement data to be presented is performed, for example, by having the user US pre-specify the angles necessary for their training, and then the data selection processing unit 12 selects the measurement data to be presented according to that specified information. In this example, measurement data representing COP and eight types of image data with different angles are selected.

[0041] Furthermore, the selection of measurement data to be presented may be configured so that, for each pose of the user's US (Ultrasound) display, multiple measurement data necessary for training are selected based on pre-specified information.

[0042] The control unit 1 of the posture training support device CS then assigns a presentation priority to the data to be presented selected by the data to be presented by the data to be presented selection processing unit 12 in step S14, under the control of the presentation priority assignment processing unit 13.

[0043] For example, the presentation priority assignment processing unit 13 assigns a priority to the data to be presented according to the priority information specified in advance by the user US. In this example, the measurement data representing COP is assigned a priority of 1, and presentation priorities are set for eight types of image data with different angles according to the user US's wishes.

[0044] The presentation priority assignment processing unit 13 stores each of the above-mentioned presentation target data in the presentation priority storage unit 32 with information representing the set presentation priority. Note that the priority information may be set in advance as a default value for the type of measurement data.

[0045] (3) Setting of visual information presentation area and presentation priority The control unit 1 of the posture training support device CS first sets a display area for displaying measurement data on the head-mounted display HD in step S15, under the control of the visual information display area setting processing unit 14. Then, in step S16, sets multiple divided display areas within the above display area.

[0046] Figure 5 shows an example of the settings for the divided presentation areas. In this example, nine divided presentation areas E1 to E9, which are equally divided into rectangles, are set in the display area of ​​the head-mounted display HD.

[0047] Furthermore, the number of divisions in the presentation area is not limited to the nine divisions described above; it can be set arbitrarily. For example, the number of divisions can be 2 to 8, or even 10 or more. In addition, the shape of each division can be set so that the central area is larger and the surrounding areas are smaller, and the shape of each area is not limited to rectangles; it can also be set to circles, ellipses, or polygons.

[0048] The control unit 1 of the posture training support device CS then, under the control of the presentation priority setting processing unit 15, sets the presentation priority for presenting multiple presentation target data to the divided presentation areas E1 to E9 in step S17, according to the visual characteristics of the user US.

[0049] In general, human visual characteristics include a large field of view in the left-right direction, followed by a large field of view downwards, while the field of view upwards is small. Therefore, in one embodiment, under the control of the presentation priority setting processing unit 15, as shown in Figure 5, for example, the first priority P1 is set for the central region E1 among the divided presentation regions E1 to E9. Next, the second priority P2 and third priority P3 are set for the left and right regions E2 and E3, and then the fourth priority P4, fifth priority P5, and sixth priority P6 are set for the three regions E4 to E6 located downwards. Finally, the seventh priority P7, eighth priority P8, and ninth priority P9 are set for the three regions E7 to E9 located upwards.

[0050] In step S18, the presentation priority setting processing unit 15 stores the position coordinate information of the divided presentation regions E1 to E9 set as described above, and the information representing their presentation priority P1 to P9, in the presentation priority storage unit 33 in a corresponding manner.

[0051] (4) Output of visual information The control unit 1 of the posture training support device CS then, under the control of the visual information output processing unit 16, reads measurement data of the presented object and information representing its presentation priority from the presentation priority storage unit 32 in step S19, and also reads position coordinate information of the divided presentation area and information representing its presentation priority from the presentation priority storage unit 33.

[0052] The visual information output processing unit 16 first assigns the measurement data with the highest presentation priority among the measurement data of the presented targets, in this example the COP measurement data, to the central region E1, which is set to the highest presentation priority P1. The visual information output processing unit 16 then assigns the next highest presentation target data, for example, image data of the user US pose captured from the front, to either the left or right region E2 or E3, which is set to the second highest presentation priority P2. Similarly, the remaining presentation target data, for example, image data of the user US captured from different angles, are sequentially assigned to the divided presentation regions E3 to E9 in order of presentation priority, according to their respective presentation priorities.

[0053] The visual information output processing unit 16 synthesizes the display information of each presentation target data assigned to each divided presentation area E1 to E9 to generate visual information, and outputs the generated visual information from the input / output I / F unit 4 to the head-mounted display HD.

[0054] As a result, the head-mounted display HD displays COP measurement data in the central area, and multiple image data representing the user's standing posture, captured from different angles, are displayed in the peripheral areas in order of their presentation priority. Therefore, the user can check the COP data, which is the most important for confirming standing balance, in the central area E1, while simultaneously checking each image data representing their standing posture, captured from different angles.

[0055] Furthermore, in this case, each image data presented in the peripheral area is shown in order of presentation priority, in the left, right, or lower peripheral areas E2-E6, which have larger field of view angles. As a result, the user (US) can relatively easily view images at angles that are of high priority for confirmation without having to make unnatural eye movements. This prevents changes in standing balance caused by eye movements.

[0056] (5) Repeated control The control unit 1 of the posture training support device CS determines in step S20 whether a request to end posture training support has been received after each of the above-described series of visual information presentation processes has been completed. If no training end request has been received, the control unit 1 returns to step S12 and repeatedly executes the series of processing procedures related to training support, from acquiring measurement data to outputting visual information, as described in steps S12 to S19.

[0057] Therefore, the user (US) is presented with real-time visual information, where COP data related to the standing balance of the pose they performed, and multiple image data showing their standing posture captured from different angles, are combined on a single screen.

[0058] Then, when step S20 detects that a training termination request has been received, the control unit 1 of the posture training support device CS terminates the above series of processing steps and returns to the standby state.

[0059] (effect) As described above, in one embodiment, during the training support period, multiple measurement data related to the user's standing balance are acquired from the posture sensor CM, and presentation priority is assigned to the data selected as presentation targets from the acquired multiple measurement data. Multiple divided presentation areas are set in the display area of ​​the head-mounted display HD, and presentation priority is set for these divided presentation areas according to human visual characteristics, such as the size of the field of view. The presentation targets are then assigned to the multiple divided presentation areas in order from the area with the highest presentation priority, according to the presentation priority assigned to the presentation targets, and the assigned presentation targets are combined to generate visual information, which is then output to the head-mounted display HD.

[0060] Therefore, the head-mounted display HD displays visual information that is assigned to multiple divided display regions according to the presentation priority of the user's (US) standing balance, starting with the region with the highest presentation priority. For example, the COP data, which is most important for checking standing balance, is placed in the central region E1, and multiple image data of the standing posture taken at different angles are displayed in the peripheral regions E2 to E according to their presentation priority.

[0061] As a result, user US can comprehensively view multiple data points representing their standing balance within the display area of ​​the head-mounted display HD. Moreover, the most important COP data can be viewed in the central region E1 without shifting their gaze.

[0062] On the other hand, among the multiple image data of the standing posture captured from different angles, the image data from the angle with a relatively high presentation priority is presented in the peripheral areas E2-E6 on the left, right, or below, where the user's field of view is relatively large. Therefore, the user can easily review it without making unnatural eye movements. As a result, the user can continue training without compromising their standing balance due to unnatural eye movements.

[0063] [Other embodiments] (1) In one embodiment, the case in which a head-mounted display HD is used as the display medium was described as an example, but it is also possible to use a smart mirror that can display digital information on its mirror surface.

[0064] Figure 6 shows an example of the display results of visual information on a smart mirror. In this example, the central area of ​​the smart mirror MR is set as the area that displays the user's standing figure, and the areas to the left and right of the central area and the area below the central area of ​​the smart mirror MR are set as areas for displaying measurement data. A display priority is then set for the central area, the left and right areas, and the lower area, with the priority increasing in that order.

[0065] In this state, when the user begins standing balance training, the optical image of their standing posture UI is projected onto the central area of ​​the Smart Mirror MR, and three types of measurement data WI1, WI2, and WI3, which represent changes in the user's standing balance obtained from the posture sensor, are displayed in the left and right areas and the lower area of ​​the Smart Mirror MR in order of their presentation priority.

[0066] According to this embodiment, the user can view their own standing posture UI reflected in the central area of ​​the Smart Mirror MR, and further check changes in their standing balance using the measurement data WI1, WI2, and WI3 displayed in the left, right, and lower areas of the Smart Mirror MR, thereby enabling them to correct their standing balance. In addition, measurement data is not displayed in the upper area, where the field of view is generally small and eye movement is difficult for humans. Therefore, when viewing the measurement data, the user is less likely to lose their balance due to the influence of eye movement, thereby enabling them to train their standing balance more efficiently.

[0067] (2) In one embodiment, a series of training support control procedures are described, which are performed each time measurement data for a single pose of the user US is acquired, from assigning a presentation priority to the data to be presented, to setting the divided presentation area and its presentation priority, and outputting visual information.

[0068] However, it is also possible to pre-configure the divisional presentation areas and their presentation priority for the display area of ​​the head-mounted display HD before acquiring measurement data, acquire the measurement data in this state, select the data to be presented from the acquired measurement data, set the presentation priority, and then execute the process of assigning this data to the divisional presentation areas according to its presentation priority to generate visual information. This reduces the load on the control unit 1 during the posture training support control period and improves the responsiveness of visual information presentation.

[0069] (3) In addition to head-mounted displays and smart mirrors, other display media may include, for example, walls or screens, and projectors or other projectors may be used to project multiple visual pieces of information related to the user's standing balance onto these walls or screens.

[0070] (4) In addition, the multiple processing functions provided by the posture training support device CS may be installed and executed on a server computer located on the Web or in the cloud, or the multiple processing functions may be distributed across multiple server computers or personal computers.

[0071] (5) Furthermore, the application of this invention may extend beyond arts such as ballet and dance, and sports such as gymnastics and figure skating, to include exercise support for the elderly and rehabilitation for patients.

[0072] (6) In addition, the configuration of each processing function of the posture training support device, its processing procedure and content, the type of posture to be trained, the type and number of data to be presented, etc., can be modified in various ways without departing from the spirit of this invention.

[0073] Although embodiments of this invention have been described in detail above, the above description is merely illustrative in all respects. It goes without saying that various improvements and modifications can be made without departing from the scope of this invention. In other words, when implementing this invention, specific configurations may be adopted as appropriate depending on the embodiment.

[0074] In short, this invention is not limited to the embodiments described above, and in the implementation stage, the components can be modified and materialized without departing from the gist of the invention. Furthermore, various inventions can be formed by appropriately combining the multiple components disclosed in the embodiments. For example, some components may be deleted from all the components shown in the embodiments. Moreover, components from different embodiments may be appropriately combined. [Explanation of Symbols]

[0075] CS…Posture training support device CM...Posture Sensor HD…Head-mounted display 1…Control Unit 2…Program memory 3…Data storage unit 4…Input / Output I / F section 5... Bus 11...Measurement data acquisition processing unit 12…Data selection processing unit for presentation 13…Processing unit for assigning presentation priority 14…Visual Information Presentation Area Setting Processing Unit 15…Presentation priority setting processing unit 16…Visual Information Output Processing Unit 31...Measurement data storage unit 32...Presentation priority storage unit 33...Presentation priority storage unit

Claims

1. A first processing unit that acquires multiple posture measurement data related to the user's posture, A second processing unit that assigns information indicating presentation priority to multiple posture measurement data, A third processing unit sets up multiple display areas within the user's field of view for displaying visual information representing the posture measurement data, A fourth processing unit that sets a presentation priority for a plurality of presentation areas based on the user's visual characteristics, A fifth processing unit that presents visual information representing the posture measurement data in the presentation area in order from the area with the highest presentation priority, according to the presentation priority. A posture training support device equipped with the following features.

2. The posture training support device according to claim 1, wherein the second processing unit selects the posture measurement data to be presented from among a plurality of posture measurement data, and assigns information representing the presentation priority to the selected posture measurement data according to its importance.

3. The posture training support device according to claim 1, wherein the fourth processing unit sets the presentation priority order for a plurality of presentation areas, prioritizing the presentation area corresponding to the direction of the user's widest field of view.

4. The third processing unit divides the area corresponding to the user's field of view into a central area and a plurality of peripheral areas located around it, and sets the central area and the plurality of peripheral areas as the presentation area. The fourth processing unit sets the presentation priority order such that the central region, left and right peripheral regions, lower peripheral region, and upper peripheral region have increasing priority, according to the size of the user's field of view. The posture training support device according to claim 1.

5. The posture training support device according to claim 1, wherein the first processing unit acquires a plurality of measurement data relating to standing balance, including measurement data representing the user's center of gravity sway, as the posture measurement data.

6. A posture training support method in which an information processing device performs processing to support the training of a user's posture, The process of acquiring multiple posture measurement data related to the user's posture, A process of assigning information representing presentation priority to multiple posture measurement data, The process of setting up multiple display areas within the user's field of view to display visual information representing the posture measurement data, A process of setting a presentation priority for multiple presentation areas based on the user's visual characteristics, The process of presenting visual information representing the posture measurement data in the presentation area in order from the area with the highest presentation priority, according to the presentation priority, and A posture training support method equipped with the following features.

7. A program that causes a processor in a posture training support device to execute at least one of the processes performed by the first to fifth processing units in a posture training support device according to any one of claims 1 to 5.

Images