Presentation information generation device, information presentation system, and information presentation method

Abstract

In the present invention, in a state in which an overlay image is not displayed on see-through AR glasses worn by a user during running, a real space, such as the path 202 on which the user is running, is visible. A presentation information generation device generates an image of a virtual runner 204a, 204b, 204c who runs together with the user and displays the image on the AR glasses in an integral manner with an image of the real space. The appearance and movements of the virtual runner 204a, 204b, 204c can correspond to those of a real person. In addition to virtual runners, the presentation information generation device generates an image selected from among virtual objects corresponding to spectators, passers-by, coaches and the user him / herself.

Description

Information generation device, information presentation system, and information presentation method

[0001] The present invention relates to an information generation device, an information presentation system, and an information presentation method for generating information to be presented to people who are engaging in healthy behaviors.

[0002] Conventionally, various technologies have been developed to present information with the aim of improving performance or increasing motivation to continue health activities such as walking, running, cycling, and muscle training. For example, Non-Patent Document 1 discloses a technology that creates a simulated cycling environment by acquiring the movement of a user pedaling in an indoor environment using sensors and reflecting it in the movement of a bicycle in a virtual space displayed on a screen. Non-Patent Document 2 discloses a technology that creates a simulated running environment by acquiring the movement of a user running on a treadmill using sensors and reflecting it in the movement of the user's avatar in a virtual space displayed on a screen.

[0003] Non-patent document 3 discloses a technology that allows users to run while checking the degree of lead or delay by using a smartwatch worn by the user to acquire the user's own pace and displaying a virtual partner running at a set pace. Non-patent document 4 discloses that using smart glasses worn by the user while running to acquire the user's own pace and display a virtual character running ahead in conjunction with that pace, thereby providing motivation for running.

[0004] "No. 1 Indoor Cycling App", [online], Zwift Inc., [searched on January 8, 2025], Internet <URL:https: / / www.zwift.com / ja> "It's Easy to Start ZWIFT!", [online], Zwift Inc., [searched on January 8, 2025], Internet <URL:https: / / www.zwift.com / ja / run> "FORERUNNER 965 Operation Manual", [online], Garmin Ltd., [searched on January 8, 2025], Internet <https: / / static.garmin.com / pumac / forerunner965_OM_JA-JP.pdf> T Hamada, et al., "Solitary Jogging With A Virtual Runner using Smartglasses." 2022 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), March 2022, p. 644-654

[0005] Although the technologies disclosed in Non-Patent Documents 1 and 2 realize cycling and running visually, as healthy behaviors, they are limited to pedaling indoors or running on a treadmill. Therefore, it is difficult to experience elements important when actually driving or running on real-world roads, etc., such as changes in the center of gravity of a bicycle when driving on the road or the feeling of kicking the ground when running. According to the technologies disclosed in Non-Patent Documents 3 and 4, although running outdoors is possible as an actual exercise, since the information presented is limited, the effects are also limited, and it is difficult to say that it can meet various needs.

[0006] The present invention has been made in view of such problems, and an object thereof is to provide a technology capable of easily realizing maintenance of motivation and improvement of performance in healthy behaviors. Another object of the present invention is to provide a technology that can easily adapt the environment of healthy behaviors to the abilities and preferences of individual users.

[0007] One aspect of the present invention relates to a presentation information generation device. This presentation information generation device generates superimposed images to be displayed on a see-through wearable display worn by a user performing a health activity, and is characterized by comprising: a superimposed image control unit that virtually places selected virtual objects, from among virtual objects performing the same health activity and virtual objects observing the health activity, in the space where the user is located and controls their movement; a superimposed image generation unit that generates superimposed images representing how the virtual objects appear from the user's viewpoint; and a data transmission unit that transmits the superimposed image data to the wearable display.

[0008] Another aspect of the present invention relates to an information presentation system. This information presentation system comprises a see-through wearable display worn by a user performing a health activity, and a presentation information generation device that generates superimposed images to be displayed on the wearable display. The presentation information generation device is characterized by comprising: a superimposed image control unit that virtually places selected virtual objects, from among virtual objects performing the same health activity and virtual objects observing the health activity, in the space where the user is located and controls their movement; a superimposed image generation unit that generates superimposed images representing how the virtual objects appear from the user's viewpoint; and a data transmission unit that transmits superimposed image data and displays it on the wearable display.

[0009] Another aspect of the present invention relates to an information presentation method. This information presentation method is a method for presenting information to a user performing a health behavior, and is characterized by comprising the steps of: a presentation information generation device virtually arranging selected virtual objects from among virtual objects performing the same health behavior and virtual objects observing the health behavior in the space where the user is located, and controlling their movement; generating a superimposed image representing how the virtual objects appear from the user's viewpoint; and transmitting the superimposed image data to a see-through wearable display worn by the user for display.

[0010] Furthermore, any combination of the above components, as well as conversions of the expression of the present invention between methods, apparatus, systems, computer programs, recording media containing computer programs, etc., are also valid embodiments of the present invention.

[0011] According to the present invention, it is possible to easily maintain motivation and improve performance in healthy behaviors. Furthermore, the environment for healthy behaviors can be easily adapted to the individual user's abilities and preferences.

[0012] This figure shows an example configuration of an information presentation system to which this embodiment can be applied. This figure compares the state in which the superimposed image is not displayed on the AR glasses and the state in which it is displayed in this embodiment. This figure shows another example of a virtual runner to be displayed as a superimposed image in this embodiment. This figure shows an example in which a different superimposed image is displayed on the AR glasses in this embodiment. This figure shows an example in which a different superimposed image is displayed on the AR glasses in this embodiment. This figure shows an example in which a different superimposed image is displayed on the AR glasses in this embodiment. This figure lists examples of superimposed image options that can be represented as virtual objects in this embodiment. This figure shows an example in which a superimposed image other than a virtual object is displayed on the AR glasses in this embodiment. This figure shows the internal circuit configuration of the presentation information generation device in this embodiment. This figure shows the configuration of the functional blocks of the presentation information generation device in this embodiment. This figure illustrates the data structure of the situation information stored in the situation information storage unit in this embodiment. This figure illustrates the data structure of the registration information stored in the registration information storage unit in this embodiment. This figure illustrates the data structure of the information used for drawing virtual objects among the information stored in the image information storage unit in this embodiment. This figure shows the configuration of the functional blocks of a presentation information generation device having a superimposed image optimization function in this embodiment. This figure illustrates the input screen for a psychological evaluation scale presented to the user by the psychological evaluation scale acquisition unit in this embodiment. This figure illustrates an example of correlation analysis performed by the parameter correlation analysis unit in this embodiment. This flowchart shows the processing procedure by which the presentation information generation device in this embodiment generates superimposed images to be displayed on AR glasses.

[0013] This embodiment relates to a technology that presents diverse information to users who engage in health activities such as running, walking, cycling, and muscle training. The types of health activities performed by the user are not limited to this embodiment. This embodiment will primarily focus on the case of a user who engages in running as an example.

[0014] Figure 1 shows an example configuration of an information presentation system to which this embodiment can be applied. In this example, the information presentation system 1 includes AR (Augmented Reality) glasses 12 worn by the user and a presentation information generation device 10a that generates images to be displayed on the AR glasses 12. The information presentation system 1 may further include a smartwatch 17 worn by the user and an information providing server 18 that provides information used by the presentation information generation device 10a to generate images.

[0015] The AR glasses 12 are wearable displays that realize at least augmented reality by being worn by a user while running, showing the user the real-world environment in front of them through the glasses 14, and displaying images of virtual objects, etc., in a position that merges with the image of the real space. Generally, there are two methods for making the image of the real space visible in the AR glasses 12: the optical see-through method and the video see-through method.

[0016] The former method allows natural light from the real world to pass through the glasses portion 14 for direct viewing. The latter method converts real-time video captured by a camera 15 positioned in front of the AR glasses 12 into an image from the user's point of view and electronically displays it on a display inside the glasses portion 14. Either method may be adopted in this embodiment, but the following description will mainly assume the optical see-through method.

[0017] In this embodiment, the image superimposed on the AR glasses 12 is an image of a virtual object such as a person or character, depicted as if it existed in real space. Specifically, it is possible to select from various variations, including roles, number, and appearance, such as running partners, spectators, passersby, coaches, and the user themselves. The AR glasses 12 may also superimpose text information, figures, maps, etc. Hereafter, these images superimposed on the AR glasses 12 may be collectively referred to as "superimposed images."

[0018] Generally, methods for superimposing images onto AR glasses 12 include electronically displaying the image on a display inside the glasses 14, and projecting a laser onto the user's retina. In this embodiment, any of these commercially available methods may be used.

[0019] The AR glasses 12 establish communication with the presentation information generation device 10a to send and receive various data, and are equipped with a control mechanism 16 that controls the display of the received superimposed image. The control mechanism 16 implements these processes using a microprocessor or the like. The specific configuration of the control mechanism 16 can be directly adopted from the AR glasses 12 that are already in practical use, so a detailed explanation is omitted here.

[0020] The presentation information generation device 10a generates superimposed image data and transmits it to the AR glasses 12. In order to realize augmented reality with superimposed images, the presentation information generation device 10a acquires information about the real space that the user is looking at and draws images of virtual objects to match it. For example, the presentation information generation device 10a receives real-time video captured by the camera 15 of the AR glasses 12 and sequentially acquires information about the real space and information related to the user's viewpoint using technologies such as V-SLAM (Visual Simultaneous Localization and Mapping). V-SLAM is a common technology that generates an environmental map of the surroundings based on captured images and tracks the position and orientation of the camera.

[0021] The information display generator 10a places a virtual object, represented in three dimensions, at a suitable position in the virtual space corresponding to the real space, and renders an image of it as seen from the user's perspective. The AR glasses 12 immediately display the superimposed image transmitted from the information display generator 10a, so that the user sees the real space and the superimposed image merged. The information display generator 10a is, for example, a terminal such as a smartphone carried by a user while running.

[0022] However, depending on the environment in which the health behavior is performed, the information generation device 10a may be a personal computer or the like, and its type is not limited. In any case, the information generation device 10a may include input means such as a touchpad, hardware keys, or microphone, and output means such as a liquid crystal display or speaker.

[0023] The smartwatch 17 is worn by the user while running and acquires real-time information about the user's status. This information about the user's status can be any parameter that can be acquired by a typical smartwatch or activity tracker, such as vital data like heart rate, pace, speed, step count, vertical oscillation, stride length, cadence, time spent on the watch, calories burned, current location, and current altitude. The smartwatch 17 establishes communication with the information display device 10a and transmits the acquired information at a predetermined rate.

[0024] Real-time information regarding the user's status may be acquired by the AR glasses 12 or the information display generator 10a itself. For example, the information display generator 10a may acquire the user's location information using well-known technologies such as GPS (Global Positioning System) or A-GPS (Assisted-GPS), or acquire the user's speed, acceleration, posture, etc. using a motion sensor. The information display generator 10a may also acquire information regarding the movement of a predetermined part of the user's body, such as their hands, by applying image recognition to the real-time video captured by the camera 15 of the AR glasses 12. The information display generator 10a may recognize this movement information as input via gesture.

[0025] Alternatively, the AR glasses 12 may be equipped with a GPS receiving function and a motion sensor to acquire situational information similar to that described above and transmit it to the presentation information generation device 10a. The AR glasses 12 may also be equipped with a microphone (not shown) to acquire voice data such as user speech and transmit it to the presentation information generation device 10a. The presentation information generation device 10a may recognize the voice data as a voice command.

[0026] The information provision server 18 establishes communication with the presentation information generation device 10a via a network such as the Internet and provides various information necessary for generating superimposed images. For example, the information provision server 18 may acquire driving data of another user from the presentation information generation device 10b used by that user and transmit it to the presentation information generation device 10a. The information provision server 18 may also transmit driving data of top athletes or celebrities, data of 3D models of various virtual objects, course map data, etc., to the presentation information generation device 10a.

[0027] Communication between the AR glasses 12 and smartwatch 17 and the information display device 10a may be via a wireless connection such as Bluetooth® or a wired connection such as USB. The configuration of the information display system 1 shown in the figure is just one example and is not intended to limit the number or division of functions of the information display devices 10a, 10b, AR glasses 12, smartwatch 17, and information provision server 18.

[0028] For example, at least a portion of the functions of the information display generation device 10a described above may be implemented in the information provision server 18, AR glasses 12, or smartwatch 17. Also, at least a portion of the functions of the information provision server 18 or smartwatch 17 may be implemented in the information display generation device 10a. Furthermore, at least a portion of the functions of the information display generation device 10a or information provision server 18 may be realized by cloud computing. Hereafter, the information display generation devices 10a and 10b will be collectively referred to as the information display generation device 10.

[0029] Next, the information to be displayed on the AR glasses in this embodiment will be described. In this embodiment, superimposed images are displayed that produce remarkable effects in the following three aspects: 1. Social facilitation 2. Co-action effect 3. Audience effect

[0030] Social facilitation refers to the phenomenon where the effectiveness of an activity is enhanced by the presence of others. The co-behavior effect is the effect where social facilitation is accelerated by the presence of others performing the same action. The audience effect is the effect where social facilitation is accelerated by the presence of others watching in the vicinity. Conventional technologies, such as displaying a virtual partner on a smartwatch or a pacemaker on smart glasses, show little change in terms of social facilitation compared to not using them. To realize group running in the real world, each participant needs to set aside time and gather in the same place, which can be very time-consuming to coordinate.

[0031] In this embodiment, AR glasses are used to virtually place a variety of objects around the user, thereby inducing the three effects described above. Specifically, this achieves the effects of running in a group, the effects of the appearance of virtual objects, the effects of changes in the movement of virtual objects such as pace and relative distance, and the effects of the presence of spectators and coaches.

[0032] Figure 2 compares the state in which the superimposed image is not displayed on the AR glasses 12 and the state in which it is displayed. As shown in (a), when the superimposed image is not displayed on the AR glasses 12, the scenery in front of the eyes, such as the road 202 being driven on, can be seen. For example, with optical see-through type AR glasses 12, this state is the same as when wearing ordinary glasses. A similar state can be simulated with video see-through type AR glasses 12 as well.

[0033] When displaying an image on the AR glasses 12, a pair of images with parallax are displayed on the left and right glasses 14 to allow the user to perceive a three-dimensional space. In the figure, the image recognized by the user is schematically represented within a rectangle. The image examples described later are similar. (b) shows the state in which virtual objects are displayed as superimposed images. That is, the human-shaped virtual runners 204a, 204b, and 204c are superimposed on the real-space image shown in (a). The virtual runners 204a, 204b, and 204c are virtual objects that are intended to run together with the user.

[0034] Virtual runners 204a, 204b, and 204c can be running partners, pacemakers, or a combination of both. In the case of running partners, varying their speeds over time will create a more realistic representation. For example, virtual runners 204a, 204b, and 204c may be controlled to temporarily disappear from the user's field of view if they fall behind, or to move further away if they run at a faster pace than the user.

[0035] According to the AR glasses 12, images of virtual objects that should be in the new field of view are displayed in response to changes in the user's gaze. Therefore, if the user turns around, they can see their running partners running behind them. In the case of a pacemaker, by controlling it to run ahead of the user at a constant pace, it can give the user the feeling of actually running with a pacemaker.

[0036] There is no limit to the number of virtual runners 204a, 204b, and 204c that can be displayed. Furthermore, virtual runners 204a, 204b, and 204c may be avatars modeled after real people or fictional characters. For example, avatars of the user's acquaintances, top athletes, or celebrities could be displayed. In this case, the running pace of virtual runners 204a, 204b, and 204c may be based on actual measured values ​​for each individual, or a virtual value may be generated to suit the user.

[0037] Alternatively, one of the virtual runners 204a, 204b, or 204c may be the user's past self or future self. If the user's past self is used as the virtual runner, the virtual runner's pace is determined to match the user's saved running data. This allows the user to run with the motivation of surpassing their past self.

[0038] When a virtual runner is created representing the user's future self, the system predicts the running pace that the user will achieve after a specified period, based on the user's previously saved running data. The virtual runner's pace is then determined to match this predicted pace. If the user's running pace has increased from the past to the present, the virtual runner (the future self) will be ahead of the current self. Therefore, the user can be motivated to run by striving to catch up to their future self.

[0039] Figure 3 shows another example of a virtual runner displayed as a superimposed image. In this example, in addition to the human virtual runners 204b and 204c shown in Figure 2(b), a character is used as the virtual runner 206. Here, "character" refers to fictional creatures or robots that appear in games or animations, or are used in advertisements for companies or local governments. The running pace of the character virtual runner 206 may be determined by the same rules as the human virtual runners 204b and 204c. That is, a virtual pace may be generated to suit the user, or a pace actually measured for a real person may be given. On the other hand, expressing the unique movements and expressions of the character, or showing changes such as sweating as images, can provide a different kind of enjoyment and relaxation than realism.

[0040] Even when the virtual runners 204b and 204c are avatars of top athletes or celebrities, their appearance and performance can be stimulating, leading to improved performance in the user's own running. In any case, the special situation of running with friends that wouldn't be possible in reality makes running more enjoyable and less likely to become boring. Furthermore, it becomes easier to run together with friends who are not physically present, overcoming temporal and spatial barriers.

[0041] Furthermore, by increasing the number of virtual runners displayed, group running, such as in marathon races or running club practices, can be easily replicated. Having more companions allows users to experience the encouragement they wouldn't get running alone, further improving motivation and performance. Additionally, the attributes of the virtual runners—such as physique, clothing, and running pace—can be easily changed, allowing users to run in an environment tailored to their individual characteristics and preferences. As a result, even when running alone, the system can significantly enhance social facilitation and collaborative effects.

[0042] Figure 4 shows an example of displaying another superimposed image on the AR glasses 12. In this example, in addition to the virtual runners 204a and 204c shown in Figure 2(b), virtual spectators (for example, virtual spectators 208a, 208b, and 208c) are also superimposed. Each virtual spectator may be controlled to sequentially look at the virtual runners 204a and 204c or the user themselves, or to cheer by waving a flag, etc. Note that the number and arrangement of virtual spectators are not limited to those shown in the figure.

[0043] Instead of virtual spectators, or in addition to virtual spectators, virtual passersby may be superimposed. It is generally known that the mere presence of people in the surroundings during exercise promotes that exercise. Therefore, displaying virtual passersby, who are less likely to pay attention to the run, can also help improve user motivation. By superimposing images such as virtual spectators or virtual passersby, even if the runner is actually running alone, social promotion and the effect of having a spectator can be greatly enhanced.

[0044] Figure 5 shows an example of displaying another superimposed image on the AR glasses 12. In this example, in addition to the virtual runners 204a and 204c shown in Figure 2(b), a virtual coach 210 is also superimposed. In the example shown, the virtual coach 210 is controlled to run alongside the user and provide encouraging words through a megaphone. However, the operation and display mode of the virtual coach 210 are not limited to this. For example, the virtual coach 210 could simply watch over the user by displaying only a frontal view of its face in a fixed display area on the screen, like a picture-in-picture in a television program.

[0045] In any case, by creating a situation where the virtual coach 210 is looking at the user, the user can feel more tense and can run in an environment similar to actual coaching, enhancing the spectator effect. The appearance of the virtual coach 210 may be a real person, such as the user's actual coach, a professional running coach, a top athlete or a celebrity, or the user's friend or family member, or it may be a virtual person or character.

[0046] FIG. 6 shows an example in which another superimposed image is displayed on the AR glass 12. In this example, in addition to the virtual runners 204a, 204b, and 204c shown in (b) of FIG. 2, the avatar 212 of the user himself / herself is also superimposed and displayed. Here, the avatar 212 of the user himself / herself corresponds to the user who is currently running, rather than the past or future user described above, and is controlled to run at the real-time running pace. As a result, the user is in a state of observing himself / herself running from behind.

[0047] Here, the avatar 212 of the user himself / herself may mimic the actual user or may have an appearance different from the actual one. It is known that the impression the user has of the appearance of his / her avatar affects the actual behavior and performance (see, for example, Yuki Okubo, three others, "The Influence of Avatar Strength on Self-Perception Change on Weight Perception", Journal of the Society of Image Information and Television Engineers, 2023, Vol. 77, No. 3, p. 394-400). This is called the Proteus effect. For example, it is considered that the more powerful the avatar looks, the more the self-perception changes to increase the sense of power, and the performance improves.

[0048] Therefore, by making the avatar 212 of the user himself / herself have a tough appearance with more developed muscles than the actual one, it is easier to improve the actual running performance. Note that the strength of the appearance of the avatar 212 of the user himself / herself is not limited to being expressed by muscles, etc., and it may be expressed by replacing it with an appearance other than a person, such as a powerful monster or robot.

[0049] The virtual objects exemplified above, such as runners, spectators, passersby, coaches, and the user's own avatar, may be displayed as one type or as two or more types simultaneously, and various combinations are possible. Furthermore, as mentioned above, the virtual objects should be placed appropriately around the user, including behind them, not just in the user's direction of travel. This ensures that the presence of these objects is visible to the user wearing the AR glasses 12, regardless of which direction they look, enhancing the sense of realism.

[0050] Figure 7 lists examples of superimposed image options that can be represented as virtual objects. In this example, the types of virtual objects can be broadly categorized into "virtual runner," "virtual bystander," and "user themselves." For example, for the role of a "virtual runner," options such as "running partner," "pacemaker," and "past / future self" are prepared. For the appearance of a "running partner," options such as one or more acquaintances ("acquaintance A," "acquaintance B," ...), top athletes ("top athlete C," "top athlete D," ...), celebrities ("celebrity E," "celebrity F," ...), characters ("character G," "character H," ...), and fictional characters ("fictional character I," "fictional character J," ...) are prepared.

[0051] Here, "acquaintances" refers to real people the user knows, such as family, friends, members of the same club, or coaches. "Top athletes" are accomplished athletes, such as Olympic medalists or medalists from various competitions, and do not need to know the user personally. "Celebrities" are also entertainers or other people the user does not know personally. "Characters" are, as mentioned above, fictional creatures or robots. "Fictional characters" are fictional characters newly created as virtual runners.

[0052] Each virtual runner is assigned a running pace, and consequently, a running speed. In the diagram, the user's own real-time speed is represented as "V". The speed of the running partner is automatically generated so that it changes within the range of speed V ± ΔV. Here, ΔV is the speed difference relative to speed V that is acceptable for the running partner. If the running partner is a real person, the speed may be generated from that person's running data. For example, the running partner's speed may be the speed indicated by the running data itself, or it may be a speed that has been processed so that the difference in distance from the user does not exceed a predetermined value.

[0053] When using a friend running in a different location as a running partner, the user can instantly obtain the friend's real-time running speed and reflect it in the running partner's speed on the superimposed image. In this case, if the friend increases their pace, the user increases their pace to keep up, and if the friend takes a break, the user also takes a break, creating the feeling of running together even though they are actually running in different locations.

[0054] The appearance options for the "pacemaker" can be the same as for the "running partner," but its speed can be a fixed value, or a value that changes gradually depending on the distance run, and may be set in advance by the user. The appearance of the "past / future self" can be the "user themselves," or a fictional person ("fictional person K") or character ("character L") may be selected. If it is the "past self," its speed will be generated from the user's past running data. If it is the "future self," a speed predicted from the user's past running data will be generated.

[0055] Furthermore, the speed of "future self" is not limited to being predicted from the user's past running data; it may also be based on the user's target pace. For example, if a user sets a target pace of running at a pace of 4 minutes per kilometer, the speed of "future self" may be determined based on that target pace. In this case, the user's set value may be directly reflected in the speed of "future self," or a speed that conforms to the setting may be automatically generated, such as by converting it to an appropriate target speed at the present time based on the user's actual running data. For example, the current target speed may be determined by interpolating the currently achieved pace and the target pace onto the time axis, depending on the set target pace and target achievement date.

[0056] "Virtual bystanders" is a general term for virtual people who are not runners but are present nearby, and options for their role are provided, such as "spectator," "passerby," and "coach." The appearance options for these can be the same as those for "running partners." For "spectators," no speed setting is provided, and they are given random movements to appear as if they are cheering. For "passersby," speed and direction of movement are given randomly. The speed of the "coach" may be the same as the user's own real-time speed V, or as mentioned above, speed may be disabled and only a face may be displayed in a designated area on the screen.

[0057] "The user themselves" refers to "their avatar," whose appearance can be that of the user themselves, or it can be a fictional character ("fictional character M") or a character ("character N"), among other options. As mentioned above, performance can be expected to improve by making the appearance of one's avatar robust. Therefore, it is desirable to prepare such an object as an option in advance. The speed of one's avatar is naturally V.

[0058] The options shown in the illustrations are merely examples and are not intended to limit this embodiment. For example, the running pace of a virtual runner may be defined not only by speed but also by distance from the user. Furthermore, even if virtual objects look similar, differences in their roles may be made immediately apparent by adding shapes or differentiating their clothing. In addition, restrictions may be placed on the selection so that the same person or character does not appear in multiple roles, or in some cases, such a situation may be permitted.

[0059] Figure 8 shows an example of displaying superimposed images other than virtual objects on the AR glasses 12. In this example, in addition to the virtual runners 204a, 204b, and 204c shown in Figure 2(b), text information 214 and graphic information 216 indicating predetermined content are superimposed. In the illustrated example, the text information 214 displays the current pace, heart rate, distance to "runner a", and estimated finish time. Here, "runner a" is a runner that has been pre-selected from among the virtual runners. Since the values ​​of these parameters change moment by moment, the text information 214 is also updated at a predetermined interval.

[0060] As graphic information 216, in the illustrated example, a three-dimensional arrow pointing in the direction of travel is displayed with the words "Pace up! You're falling behind" on its top surface. This graphic information 216 functions as a means of notifying that the cyclist is more than a predetermined distance behind the pacemaker or target pace. In other words, graphic information 216 is displayed temporarily during the period in which the condition is met.

[0061] However, the illustrated text information 214 and graphic information 216 are merely examples, and the content, shape, position, and timing of display can vary. For example, the text information could display any of the various real-time information available, such as the distance traveled so far, the distance to the goal, the current position, and the altitude. The graphic information could display a three-dimensional object representing words of encouragement, or a course map or gradient information.

[0062] By visually presenting this information through AR glasses 12, it becomes easier to grasp the necessary information at a glance, compared to voice notifications from smartwatches and other devices. Furthermore, it enables robust information transmission even in the face of changes in surrounding conditions such as noise. As a result, users can run while reliably and easily understanding their current situation, which helps maintain motivation to push themselves a little further.

[0063] Figure 9 shows the internal circuit configuration of the information display device 10. The information display device 10 includes a CPU (Central Processing Unit) 20, a GPU (Graphics Processing Unit) 22, and a main memory 24. These components are interconnected via a bus 26. An input / output interface 28 is further connected to the bus 26. A communication unit 30, a storage unit 32, an output unit 34, an input unit 36, and a recording medium drive unit 38 are connected to the input / output interface 28.

[0064] The communication unit 30 includes peripheral device interfaces such as Bluetooth® and USB, and network interfaces such as wired LAN or wireless LAN. The storage unit 32 includes a hard disk drive and non-volatile memory. The output unit 34 outputs images and sounds necessary for user operation and notifications to the user via a display, speaker, etc. The input unit 36 ​​accepts user operations via a touchpad, keyboard, microphone, etc. The input unit 36 ​​may also acquire user movement and location information via a motion sensor or GPS function, etc. The recording medium drive unit 38 drives removable recording media such as magnetic disks, optical disks, or semiconductor memory.

[0065] The CPU 20 controls the entire information presentation device 10 by executing the operating system stored in the memory unit 32. The CPU 20 also executes various programs that are read from the memory unit 32 or removable recording medium and loaded into the main memory 24, or downloaded via the communication unit 30. The GPU 22 has the functions of a geometry engine and a rendering processor, performs drawing processing according to drawing commands from the CPU 20, and outputs the drawing results to the output unit 34 or the communication unit 30. The main memory 24 is composed of RAM (Random Access Memory) and stores programs and data necessary for processing.

[0066] Figure 10 shows the configuration of the functional blocks of the presentation information generation device 10. The presentation information generation device 10 may perform processes that are performed by general information processing devices, such as processes related to communication with external devices such as AR glasses 12 and application processing, but Figure 10 specifically shows the functional blocks related to the process of generating superimposed images to be displayed on the AR glasses 12. At least some of the illustrated functions may be implemented on a server such as an information provision server 18 or a cloud computer connected to the presentation information generation device 10 via a network, or they may be implemented on the AR glasses 12 or smartwatch 17.

[0067] Furthermore, the multiple functional blocks shown in Figure 10 can be realized in hardware terms using the various circuits shown in Figure 9, and in software terms using a computer program that implements the functions of the multiple functional blocks. Therefore, it will be understood by those skilled in the art that these functional blocks can be realized in various ways using hardware alone, software alone, or a combination thereof, and are not limited to any one of these.

[0068] The presentation information generation device 10 includes an operation information acquisition unit 50 that acquires the content of user operations, a status information acquisition unit 52 that acquires information related to the user's status during running, a status information storage unit 54 that stores information related to the user's status, a superimposed image control unit 56 that controls the superimposed image, an external information acquisition unit 66 that acquires external information via a network, a registration information storage unit 68 that stores registration information of people etc. to be represented as virtual objects, an image information storage unit 70 that stores image information necessary for generating the superimposed image, a superimposed image generation unit 72 that generates the superimposed image, and a data transmission unit 74 that transmits the superimposed image data to the AR glasses 12.

[0069] The operation information acquisition unit 50 receives user operations such as starting / ending running and selecting the superimposed image to be displayed. The status information acquisition unit 52 sequentially acquires information about the user's status during running from the AR glasses 12, smartwatch 17, and internal sensors. Here, information about the user's status refers to selected parameters such as vital data like heart rate, step count, vertical oscillation, stride length, cadence, running time, running distance, calories burned, speed, acceleration, location information, altitude, and head and hand position and posture.

[0070] The situation information storage unit 54 stores at least the information acquired by the situation information acquisition unit 52. For example, the situation information storage unit 54 stores the time changes of various parameters acquired during a single run, associating them with environmental information such as the date and time the run took place, the course, the weather, and the temperature. This makes it possible to extract past running data in an environment similar to the current environment. By reflecting the speed changes shown in the extracted running data in the speed changes of a virtual runner representing one's past or future self, a proper comparison with one's current self can be made.

[0071] The superimposed image control unit 56 selects the type of superimposed image to be generated and controls its movement. Specifically, the superimposed image control unit 56 comprises a virtual runner control unit 58, a virtual bystander control unit 60, a virtual user control unit 62, and a character / graphic information control unit 64. The superimposed image control unit 56 operates the control unit corresponding to the superimposed image to be displayed by acquiring the content of the user's selection operation from the operation information acquisition unit 50. When displaying virtual objects, each control unit may acquire the content of the user's selection operation related to their position and appearance from the operation information acquisition unit 50.

[0072] As shown in Figure 7, there are various options for the position and appearance of virtual objects. Therefore, each control unit may display options such as information of acquaintances that have been registered in advance, information of top athletes and celebrities provided by the information provision server 18, and information of pre-created characters and fictional figures, and allow the user to select one. The display means in this case may be a display included in the presentation information generation device 10, or it may be AR glasses 12 or a smartwatch 17.

[0073] The virtual runner control unit 58 determines the position and appearance of the object to be displayed and its speed when a virtual runner is selected for display. In other words, the virtual runner control unit 58 continuously generates the speed to be given to each virtual runner during the period the user is running. The speed generation rules are as described above. For example, the virtual runner control unit 58 sequentially acquires the user's real-time speed V from the status information acquisition unit 52 and generates a speed for each virtual runner that changes randomly within the range of V ± ΔV.

[0074] If a past version of the user is selected as the virtual runner, the virtual runner control unit 58 reads the user's past running data from the situation information storage unit 54 and determines the virtual runner's speed accordingly. Since speed usually changes over time during a running period, the virtual runner's speed may also change over time. If there are multiple past running data sets, the virtual runner control unit 58 may accept a selection of the running data to be used from the user based on the date, time, environment, etc.

[0075] If a future version of oneself is selected as the virtual runner, the virtual runner control unit 58 reads the user's past running data from the situation information storage unit 54 and extrapolates it to the time axis to estimate the running data that will be obtained in the future after a predetermined period. For example, by estimating the speed for each time step in the running period, an estimation result of running data in which the speed changes over time during the running period is obtained. The virtual runner control unit 58 then determines the speed of the virtual runner in accordance with the estimation result.

[0076] The virtual bystander control unit 60 determines the position and appearance of the object to be displayed when a virtual bystander is selected as the object to be displayed, and determines its position and movement. For example, the virtual bystander control unit 60 appropriately places virtual bystanders on or around the running course and continuously generates the movement of the virtual bystanders for at least the period during which they are expected to be displayed, that is, the period during which they are located within a predetermined range from the user. When displaying a coach as a virtual bystander, the user may be given the option to have the coach run alongside the user or display the coach in a predetermined area of ​​the screen.

[0077] The virtual user control unit 62 determines the appearance of the object to be displayed when the user's own avatar is selected as the display target. The virtual user control unit 62 then continuously generates the positioning and movement of the avatar so that it runs a predetermined distance ahead of the actual user while the user is running.

[0078] The character and graphic information control unit 64 controls the content and timing of display of character and graphic information to be displayed, according to the user's selection or pre-generated rules. For example, the character and graphic information control unit 64 decides to display the heart rate as an overlaid image while the user is running, and controls the display to be updated with the latest heart rate acquired from the status information acquisition unit 52. Alternatively, the character and graphic information control unit 64 controls the display to appear at a predetermined position when a situation that should notify the user has occurred.

[0079] The external information acquisition unit 66 acquires information provided via the network from an information provision server 18 or the like. This information may include real-time running data of acquaintances running in other locations, past running data of acquaintances, running data of top athletes and celebrities, and profiles of each person. The external information acquisition unit 66 may also acquire 3D model data used for rendering objects such as acquaintances, top athletes, celebrities, characters, and fictional characters, as well as data related to movement rules. Furthermore, the external information acquisition unit 66 may acquire geographical data such as course maps and gradients.

[0080] The external information acquisition unit 66 stores the acquired running data and profile in the registration information storage unit 68, associating them with identification information such as the object's name. The external information acquisition unit 66 also stores 3D model data and data related to movement rules in the image information storage unit 70, associating them with identification information such as the object's name. The external information acquisition unit 66 may also directly supply the acquaintance's real-time running data to the superimposed image control unit 56, and the virtual runner control unit 58 may immediately reflect this in the running of the virtual runner corresponding to that acquaintance. Furthermore, the external information acquisition unit 66 stores acquired geographical data such as course maps in the image information storage unit 70.

[0081] The registration information storage unit 68 stores data that associates identification information of virtual objects that are selections for display, corresponding driving data, rules for placement and movement, profiles, etc. The registration information storage unit 68 also stores the content of text information and graphic information of the selections for display, display conditions, etc. This data may be acquired by the external information acquisition unit 66 from an information provision server 18 or the like, or it may be generated by the user. Data that can be used by default may be stored in the registration information storage unit 68 in advance.

[0082] The image information storage unit 70 stores data that associates identification information of virtual objects that are selected as display targets with their 3D models. The image information storage unit 70 also stores font data, layout rules, and 3D model data necessary for generating text and graphic information. This data may be acquired by the external information acquisition unit 66 from an information provision server 18 or the like, or it may be generated by the user. Data that can be used by default may be stored in the image information storage unit 70 in advance.

[0083] The superimposed image generation unit 72, under the control of the superimposed image control unit 56, reads the corresponding data from the image information storage unit 70 and generates a superimposed image. For example, the superimposed image generation unit 72 places virtual objects to be displayed in a virtual three-dimensional space corresponding to the real space around the user and draws their images as seen from the user's viewpoint. The movement of the user's viewpoint is identified at a predetermined rate based on the position and orientation of the user's head acquired by the situation information acquisition unit 52.

[0084] The superimposed image generation unit 72 continuously generates superimposed images in accordance with the display rate of the AR glasses 12. This allows the movement of each virtual object to be represented at the speed and motion determined by the superimposed image control unit 56. The superimposed image generation unit 72 also generates images of characters and shapes that should be displayed on the AR glasses permanently or temporarily. The data transmission unit 74 sequentially transmits the superimposed image data generated by the superimposed image generation unit 72 to the AR glasses 12 for display.

[0085] Figure 11 illustrates the data structure of the status information stored in the status information storage unit 54. In this example, the status information 300 is data that associates the date and time field 302a, the running data field 302b, and the heart rate field 302c for each run performed. In the figure, "run(0)" etc. shown in the running data field 302b is the file name of the running data for the run that started at the date and time recorded in the date and time field 302a, and the actual running data is linked to it. The running data represents the time change of parameters that indicate the speed of movement, such as speed, pace, and lap time.

[0086] The "bpm(0)" etc. shown in the heart rate field 302c is the file name of the heart rate data for a run that started at the date and time specified in the date and time field 302a, and the actual heart rate data is associated with it. The heart rate data represents the change in heart rate over time. As mentioned above, the situation information is not limited to this, and various data related to the user's own performance and running environment may be associated with it.

[0087] Figure 12 illustrates the data structure of registration information stored in the registration information storage unit 68. The figure shows registration information 310 relating to real people, which are options for the appearance of virtual objects, and associates the object name field 312a with the running data field 312b. The object name field 312a may store the name of each person. "run(A)" etc. shown in the running data field 312b is the file name of the running data of the person described in the object name field 312a, and the actual running data is linked to it.

[0088] In the diagram, one running data entry is associated with each person, but similar to the situational information data shown in Figure 11, multiple running data entries may be stored for each person, associated with date and time, etc. Furthermore, various data necessary for selecting and controlling virtual objects, such as heart rate data, each person's profile, and movement characteristics, may also be associated. It is not necessary to associate measured running data with every person. Individuals without running data may be excluded from the virtual runner selection, or if they are designated as virtual runners, they may be given an automatically generated speed.

[0089] Figure 13 illustrates the data structure of information used for drawing virtual objects among the information stored in the image information storage unit 70. In this example, the image information 314 is data that associates the object name field 316a with the model data field 316b. The object name field 316a stores the name and identification information of a person or character, which are options for the appearance of the virtual object. "model(A)" etc. shown in the model data field 316b is the file name of the 3D model of the object described in the object name field 316a, and is linked to the actual model data.

[0090] As described above, in this embodiment, by displaying virtual objects with various positions and appearances, as well as text and graphic information, on the AR glasses 12, it is possible to significantly enhance the social facilitation, co-action effect, and spectator effect for users while running. On the other hand, it is conceivable that the optimal display object will differ depending on the individual characteristics of the user, such as athletic ability (running speed, muscle strength, etc.), personality, exercise history, strengths and weaknesses.

[0091] For example, users have various goals and priorities when it comes to running; some want to run 10km as fast as possible in preparation for a race they plan to participate in, while others prioritize enjoyment even if it means running slower. Therefore, in this embodiment, a function may be provided to optimize the superimposed image for each individual user based on user status information acquired during running, as well as additionally acquired psychological evaluation scales.

[0092] Figure 14 shows the configuration of the functional blocks of a presentation information generation device having an overlay image optimization function. In this figure, functional blocks having the same functions as the presentation information generation device 10 shown in Figure 10 are denoted by the same reference numerals, and their descriptions are omitted as appropriate. The overlay image control unit 56 in Figure 14 may include the control units shown in Figure 10, but these are omitted from the illustration. In addition to the functional blocks shown in Figure 10, the presentation information generation device 10c includes a psychological evaluation scale acquisition unit 80, a parameter correlation analysis unit 82, and a display target recommendation unit 84. At least some of these functional blocks may be implemented in an information provision server 18 or a cloud computer.

[0093] The psychological evaluation scale acquisition unit 80 acquires psychological evaluation scales related to running. Here, the psychological evaluation scale is data that expresses the user's own evaluation and impressions of running, such as the degree of load, fatigue level, and enjoyment level, using a predetermined scale. After the run is completed, the psychological evaluation scale acquisition unit 80 presents an input screen for the psychological evaluation scale via the display and acquires the evaluation content entered by the user via the operation information acquisition unit 50a.

[0094] The parameter correlation analysis unit 82 collects information on the user's situation, psychological evaluation scale, and the superimposed image displayed for multiple runs, and analyzes the trends in the impact on the user's mind and body when various superimposed images are displayed. For this reason, the situation information storage unit 54a stores not only the situation information shown in Figure 11, but also the superimposed image displayed and the results of the psychological evaluation scale for each run performed.

[0095] The parameter correlation analysis unit 82 performs correlation analysis, for example, to determine what kind of superimposed images improve performance or what users find enjoyable. The method of correlation analysis is not particularly limited, and any statistical processing method may be used. Parameters that represent performance include pace, distance run, and heart rate. Since these parameters change depending on the time and distance run during running, the parameter correlation analysis unit 82 may perform correlation analysis for each time period after the start of running.

[0096] The display target recommendation unit 84 recommends superimposed images suitable for the user based on the correlation relationships obtained by the parameter correlation analysis unit 82. The display target recommendation unit 84 presents candidate superimposed images that match the user's desired mental and physical state, such as how much importance the user wants to give to performance or enjoyment. The presentation means may be AR glasses 12 or the display of the presentation information generation device 10c.

[0097] The user may select the presented superimposed image as is, or they may choose a similar superimposed image of their own accord. The display target recommendation unit 84 may present multiple candidate superimposed images associated with multiple patterns of mental and physical states, allowing the user to select one of them. The operation information acquisition unit 50a receives such user selection input and notifies the superimposed image control unit 56 of the selection. As a result, the superimposed image control unit 56 starts controlling the selected superimposed image, as described in Figure 10.

[0098] Figure 15 illustrates an example of the input screen for a psychological evaluation scale presented to the user by the psychological evaluation scale acquisition unit 80. In this example, the psychological evaluation scale input screen 320 allows input of three types of scales 322: "degree of load," "degree of fatigue," and "degree of enjoyment." For each scale, a GUI (Graphical User Interface) of indicators 324a, 324b, and 324c is provided, with the left end representing the minimum and the right end representing the maximum. The user can input the degree they actually felt by moving a slider bar (for example, slider bar 326) left or right.

[0099] When the user presses the GUI for the confirmation button 328, the psychological evaluation scale acquisition unit 80 receives notification to that effect from the operation information acquisition unit 50a, assigns scores to each scale based on the position of the slider bar at that time, and stores them in the situation information storage unit 54a. The psychological evaluation scale input screen is not limited to the one shown, nor are the scales that require input particularly limited. However, it is desirable that the scales include emotions and items that the user can easily use as a reference when selecting superimposed images.

[0100] Figure 16 is a diagram illustrating an example of correlation analysis performed by the parameter correlation analysis unit 82. In the example shown, the binary position coordinates obtained during the actual run are indicated by circles in a two-dimensional space where enjoyment from the psychological evaluation scale is on the horizontal axis and running pace from the situational information is on the vertical axis. Each circle is linked to the superimposed image that was displayed during the corresponding run. As an example, white circles are used to represent running partners as characters, gray circles to represent friends, and black circles to represent top athletes.

[0101] The superimposed images that make running enjoyable and those that encourage a faster pace vary depending on the user's individual athletic ability and personality. In the example shown in the figure, it can be seen that when a character is used as a running partner, enjoyment increases but the pace tends to slow down, while when a top-class athlete is used as a running partner, enjoyment decreases but the pace tends to speed up. The parameter correlation analysis unit 82 can derive such correlations by plotting as shown in the figure. It is also thought that even for the same user, the trends may change due to physical and mental growth and improvement in athletic ability. Therefore, it is desirable for the parameter correlation analysis unit 82 to continuously perform correlation analysis to obtain the latest trends.

[0102] The display target recommendation unit 84 presents candidate superimposed images suitable for the user based on such tendencies. For example, for a user who doesn't particularly want to improve their pace but wants to run for fun, the unit suggests that region 330 in the two-dimensional space corresponds to that region, and recommends characters that were displayed during running within that region 330 as running companions.

[0103] In reality, the variations of the superimposed images displayed for each run are further subdivided, so it is also possible to recommend the same character or a similar character that was displayed for the run within region 330. For example, if the same character is repeatedly displayed as a running partner, the user may get bored. Therefore, the display target recommendation unit 84 may, based on the results of the correlation analysis, deliberately recommend a similar character.

[0104] For simplicity, only three patterns of running partners are shown in the diagram, but in reality, as shown in Figure 7, various combinations of roles and appearances are allowed for virtual objects. Therefore, the display target recommendation unit 84 determines the target to be selected based on the detailed combination of virtual objects that were displayed in the running within the area 330. For example, the display target recommendation unit 84 can recommend displaying character G and the past self as running partners, acquaintance B as a coach, but not spectators.

[0105] The parameter correlation analysis unit 82 may actually plot the running results on a multidimensional space corresponding to the number of parameters of the situational information and psychological evaluation scales whose relationships are being analyzed. Furthermore, the information linked to each run may include not only the combination of superimposed images displayed, but also the running course and distance. In other words, optimization and recommendations to users are possible using the same principle, as long as the target can be adjusted to suit individual users. In any case, the running environment desired by the user can be easily concretized through analysis similar to that shown in the diagram. However, as mentioned above, the correlation analysis method is not limited to that shown in the diagram.

[0106] Next, the operation of the presentation information generation device 10c realized by the configuration described above will be explained. Figure 17 is a flowchart showing the processing procedure by which the presentation information generation device 10c in this embodiment generates a superimposed image to be displayed on the AR glasses 12. This flowchart is started when the user puts on the AR glasses 12 and activates the presentation information generation device 10c, and the necessary communication is established.

[0107] First, the display target recommendation unit 84 presents candidate superimposed images suitable for the user based on a correlation analysis between performance and psychological evaluation scales and superimposed images using the running results obtained so far (S10). When the user selects a desired superimposed image, the operation information acquisition unit 50a receives it and supplies it to the superimposed image control unit 56 (S12). Next, the situation information acquisition unit 52 starts acquiring user situation information using various sensors (S14). The presentation information generation device 10a waits for the start of running based on the situation information, etc. (N in S16).

[0108] When the user starts running (Y in S16), the superimposed image control unit 56 starts controlling the superimposed image selected in S12 (S18). At this time, the superimposed image control unit 56 acquires, as necessary, the user's own status information acquired by the status information acquisition unit 52, as well as other people's running data acquired by the external information acquisition unit 66 from the information provision server 18, etc., and the user's own past running data read from the status information storage unit 54a, and reflects these in the movement of the virtual object to be displayed as a superimposed image.

[0109] The superimposed image generation unit 72 generates a superimposed image to be displayed under the control of the superimposed image control unit 56 and transmits it to the AR glasses 12 via the data transmission unit 74 (S20). During the period when running is ongoing, the superimposed image generation unit 72 repeats the generation and transmission of superimposed images at a predetermined rate (N in S22, S20). When it is determined that running has ended based on the status information (Y in S22), the psychological evaluation scale acquisition unit 80 presents the user with an input screen for the psychological evaluation scale and accepts the input (S24).

[0110] Then, the input results of the psychological evaluation scale, the situation information acquired by the situation information acquisition unit 52, and the identification information of the superimposed image to be displayed are stored in the situation information storage unit 54a in association with the date and time of the run, thereby completing the processing for one run (S26). Correlation analysis between performance, psychological evaluation scales, and superimposed images may be performed simultaneously with S26 in preparation for the next run. If optimization of the superimposed image is not performed, the processing in S10 and S24 may be omitted, and in S26, only the user's situation information should be stored in the situation information storage unit 54a.

[0111] According to the embodiment described above, users performing health activities are fitted with AR glasses, and the presence of various people and characters, such as companions and bystanders, is simulated. This makes it easy to enhance social facilitation, co-activity effects, and spectator effects in a real environment, even for health activities that involve outdoor movement. Furthermore, it becomes possible to engage in activities or compete with past or future versions of oneself, acquaintances who are not present, top athletes, celebrities, etc., which would be impossible in reality, increasing the enjoyment of the activity and boosting motivation. In addition, even when actually engaging in activities alone, it is possible to create a situation as if participating in a competition, increasing the sense of tension. As a result, it becomes easier to maintain motivation and improve performance.

[0112] Furthermore, by saving individual user performance and psychological evaluation scales for health behaviors in various environments thus realized, it becomes possible to analyze the impact of the environment on the user's physical and mental state. This makes it easy to identify and implement environments that suit the individual user's needs from a diverse range of options.

[0113] The present invention has been described above based on embodiments. The above embodiments are illustrative, and it will be understood by those skilled in the art that various modifications are possible in combinations of their respective components and processing processes, and that such modifications also fall within the scope of the present invention.

[0114] For example, this embodiment focuses on displaying images of virtual objects on AR glasses in a way that blends them with real space. This makes it possible to create the illusion that other runners and spectators are actually present, thereby enhancing effects such as social promotion. On the other hand, it is believed that similar effects can be obtained to some extent even if the blending with real space is not strictly accurate. Therefore, the information display device may display the same superimposed images as described in this embodiment on a see-through wearable display other than AR glasses, such as smart glasses or a head-mounted display.

[0115] Furthermore, while this embodiment primarily illustrates virtual objects intended for running, the options for the virtual object's role and appearance can vary depending on the type of health activity. For example, if walking is the health activity, a pet such as a dog could be displayed as a virtual object. In this case, its appearance could be an avatar of a real pet, or a fictional pet or character. The pet's movements could be similar to those of the "running companions" described in this embodiment, or pet-specific movements could be set. This allows for the addition of the enjoyment of walking a pet, even when walking alone, and can achieve the same effects as in this embodiment, such as maintaining motivation.

[0116] This invention can be used in a presentation information generation technology that generates information to be presented to people who are engaging in healthy behaviors.

[0117] 10 Display information generation device, 12 AR glasses, 17 Smartwatch, 18 Information provision server, 20 CPU, 22 GPU, 24 Main memory, 50 Operation information acquisition unit, 52 Status information acquisition unit, 54 Status information storage unit, 56 Superimposed image control unit, 58 Virtual runner control unit, 60 Virtual bystander control unit, 62 Virtual user control unit, 64 Character / graphic information control unit, 66 External information acquisition unit, 68 Registered information storage unit, 70 Image information storage unit, 72 Superimposed image generation unit, 74 Data transmission unit.

Claims

1. A presentation information generation device that generates superimposed images to be displayed on a see-through wearable display worn by a user performing a health activity, comprising: a superimposed image control unit that virtually places selected virtual objects, from among virtual objects performing the same health activity and virtual objects observing the health activity, in the space where the user is located and controls their movement; a superimposed image generation unit that generates superimposed images representing how the virtual objects appear from the user's viewpoint; and a data transmission unit that transmits the superimposed image data to the wearable display.

2. The presentation information generation device according to claim 1, further comprising: a parameter correlation analysis unit that analyzes the effect of the displayed superimposed image on the user's mind and body based on the results of past health behaviors; and a display target recommendation unit that presents candidate superimposed images suitable for the user based on the results of the analysis.

3. The presentation information generation device according to claim 2, further comprising: a situation information acquisition unit that acquires information relating to the performance of a user while performing the health behavior; and a psychological evaluation scale acquisition unit that receives input of a psychological evaluation scale for the health behavior from the user, wherein the parameter correlation analysis unit performs a correlation analysis between the displayed superimposed image, the performance information, and the psychological evaluation scale; and the display target recommendation unit presents candidates for the superimposed image that are suitable for the physical and mental state desired by the user, based on the results of the correlation analysis.

4. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image control unit provides the selected virtual object with movement in a selected position and a selected appearance.

5. The presentation information generation device according to claim 4, characterized in that the superimposed image control unit provides the selected virtual object with an appearance corresponding to a selected real person.

6. The presentation information generation device according to claim 4, further comprising an external information acquisition unit that acquires information relating to the performance when a real person performs the same health behavior, wherein the superimposed image control unit provides movement to a virtual object performing the same health behavior based on the performance information.

7. The presentation information generation device according to claim 6, characterized in that the external information acquisition unit immediately acquires information relating to the performance of the actual person performing the same health behavior, and the superimposed image control unit immediately reflects the performance information in the movement of the virtual object.

8. The presentation information generation device according to any one of claims 1 to 3, further comprising a situation information storage unit that stores information relating to the past performance of a user who performed the health behavior, wherein the superimposed image control unit provides movement to a virtual object performing the same health behavior based on the information relating to the past performance.

9. The presentation information generation device according to any one of claims 1 to 3, further comprising a situation information storage unit that stores information relating to the past performance of a user who performed the health behavior, wherein the superimposed image control unit uses the information relating to past performance to predict information relating to the user's performance in the future after a predetermined period, and then gives a virtual object that performs the same health behavior movements based on the predicted performance information.

10. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image control unit determines information relating to the performance of a virtual object that performs the same health behavior based on the user's settings relating to the health behavior goal, and then gives the virtual object movement based on the determined performance information.

11. The superimposed image control unit further arranges a virtual object corresponding to a user performing the health behavior, and immediately reflects real-time information related to the user's performance in the movement of the virtual object, as described in any one of claims 1 to 3.

12. The presentation information generation apparatus according to claim 11, characterized in that the superimposed image control unit gives a virtual object corresponding to the user a different appearance from that of the actual user.

13. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image control unit gives the virtual object movements corresponding to a selected position from among spectator, passerby, and coach, representing the position of a virtual object observing the health behavior.

14. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image control unit provides the virtual object with movements from the perspective of a running partner or a pacemaker, selected from the perspective of the virtual object performing running as a health activity.

15. The presentation information generation device according to any one of claims 1 to 3, further comprising a status information acquisition unit that acquires information relating to the performance of a user while performing the aforementioned health behavior, wherein the superimposed image generation unit further generates a superimposed image in which selected information from the performance-related information is represented as text information.

16. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image generation unit further generates a superimposed image representing a notification when it is necessary to notify the user.

17. The presentation information generation device according to any one of claims 1 to 3, characterized in that the superimposed image generation unit generates a superimposed image representing a front view of a predetermined virtual object among the selected virtual objects for display in a predetermined area of ​​the wearable display.

18. An information presentation system comprising: a see-through wearable display worn by a user performing a health activity; and a presentation information generation device that generates superimposed images to be displayed on the wearable display, wherein the presentation information generation device comprises: a superimposed image control unit that virtually places selected virtual objects, from among virtual objects performing the same health activity and virtual objects observing the health activity, in the space where the user is located and controls their movement; a superimposed image generation unit that generates superimposed images representing how the virtual objects appear from the user's viewpoint; and a data transmission unit that transmits the data of the superimposed images and displays them on the wearable display.

19. A method for presenting information to a user performing a health behavior, characterized in that an information presentation device includes the steps of: virtually placing selected virtual objects from among virtual objects performing the same health behavior and virtual objects observing the health behavior in the space where the user is located, and controlling their movements; generating a superimposed image representing how the virtual objects appear from the user's viewpoint; and transmitting the data of the superimposed image to a see-through wearable display worn by the user for display.

20. A computer program that generates superimposed images to be displayed on a see-through wearable display worn by a user performing a health activity, and that includes the following functions: a function to virtually place selected virtual objects, from among virtual objects performing the same health activity and virtual objects observing the health activity, in the space where the user is located and to control their movements; a function to generate a superimposed image representing the virtual objects as seen from the user's point of view; and a function to transmit the data of the superimposed image to the wearable display.

Images