Information display device and method

Abstract

This information display device includes a processor and a display. The processor, during execution of a virtual reality application, converts real world time information into time information of a virtual reality world realized by the virtual reality application to generate converted time information, and displays a display screen of the virtual reality application and the converted time information on the display.

Description

Information display device and method

[0001] The present invention relates to an information display device and method, and more particularly to improving the sense of immersion in a virtual reality world.

[0002] Patent Document 1 discloses a program and a game device, which states, "In order to have the player character achieve the clear goal displayed on the clear requirement display, the player sets a time schedule by operating a schedule setting operation icon and selecting in order a plurality of service actions including action elements and recovery elements. After setting the schedule, when an action start operation is input, the player character executes the scheduled service action, and the stamina value of the player character changes depending on the service action executed. The stamina value affects the judgment of the outcome (success / failure) of the player character's service action, and if service actions continue to fail, it becomes difficult to achieve the clear goal (abstract excerpt)."

[0003] Japanese Patent Application Laid-Open No. 2017-055998

[0004] If a game player wants to check their actual schedule while enjoying a game, they need to check real-world time information. Therefore, if the game player is enjoying a game during the day but the game space is set at night, the game player will have to check the daytime time information in the real world to check their schedule, which will ruin their immersion in the game world.

[0005] Although Patent Document 1 allows a schedule to be set within the game space and for the player character to act in accordance with that schedule, it does not address the issue that the sense of immersion in the game space is lost when the player checks the schedule set in the real world.

[0006] The present invention has been made in consideration of the above-mentioned circumstances, and aims to provide an information display device and method that allows a user to check time information in the real world without losing the sense of immersion in the virtual reality world.

[0007] In order to solve the above problems, the present invention has the configuration described in the claims. As an example, the present invention is an information display device comprising a processor and a display, wherein, while a virtual reality application is running, the processor converts time information in the real world into time information in a virtual reality world realized by the virtual reality application to generate converted time information, and displays a display screen of the virtual reality application and the converted time information on the display.

[0008] The present invention also provides an information display method, which includes the steps of executing a virtual reality application, converting real-world time information into virtual reality world time information realized by the virtual reality application while the virtual reality application is being executed to generate converted time information, and displaying the display screen of the virtual reality application and the converted time information on a display.

[0009] According to the present invention, it is possible to provide an information display device and method that allows users to check time information in the real world without losing the sense of immersion in the virtual reality world. Note that problems, configurations, and effects other than those described above will become clear from the description of the following embodiments.

[0010] 1 is a system configuration diagram of an information display device according to an embodiment of the present invention. FIG. 1 is a hardware configuration diagram of an information display device. A top view showing a state in which a user is wearing the information display device. A front view showing a state in which a user is wearing the information display device. A side view showing a state in which a user is wearing the information display device. A functional block diagram of an information display device. A hardware configuration diagram of an information terminal. A diagram showing an example of the configuration of scheduled activity information. A diagram showing an example of a scheduled activity display. A flowchart showing the processing flow of scheduled activity display process 1. A schematic diagram showing an example of a basic screen (main menu screen) in scheduled activity display process 1. A schematic diagram showing an example of a basic screen (scheduled activity notification image) in scheduled activity display process 1. A flowchart showing the processing flow of scheduled activity display process 2. A schematic diagram showing an example of a display screen (VR application execution screen) in scheduled activity display process 2. An explanatory diagram showing the contents of time information conversion processing. A schematic diagram showing a display screen (VR application execution screen + converted scheduled activity information) in scheduled activity display process 2. A flowchart showing the processing flow of scheduled activity display process 3. A diagram showing a display screen (VR application execution screen + converted action plan notification image: alarm) displayed in action plan display process 3. A diagram showing a modified example of the display screen (VR application execution screen + progress bar) displayed in action plan display process 3. A diagram showing a modified example of the display screen (VR application execution screen + warning display) displayed in action plan display process 3.

[0011] The information display device according to the present embodiment allows users to check time information and planned activities in the real world without losing the sense of immersion in the virtual reality world, thereby improving the convenience of the information display device that displays images of a virtual reality space. Therefore, the present invention can increase the commercial value of information display devices to which the present invention is applied, and is therefore expected to contribute to Goal 8.2 of the Sustainable Development Goals (SDGs) advocated by the United Nations (increasing economic productivity through diversification, technological improvement, and innovation, particularly in industries that increase the value of goods and services and labor-intensive industries).

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. The same components are designated by the same reference numerals throughout the drawings, and duplicated explanations will be omitted.

[0013] [First Embodiment] In the first embodiment, while an application displaying a virtual reality world is running, time information in the real world is converted into time information in the virtual reality world and an action schedule is displayed. In the following description, virtual reality will be referred to as "VR (Virtual Reality)," the virtual reality world will be referred to as "VR world," and the application displaying the virtual reality world will be referred to as "VR app."

[0014] (System Configuration) FIG. 1 is a diagram showing the system configuration of an information display device according to this embodiment.

[0015] 1, the information display device will be described using a head-mounted display (HMD) 100. The information display device 100 is a video see-through type VR goggle, which a user U1 wears on his / her head and runs a VR application to view an image of a virtual space.

[0016] The information terminal 200 is a terminal that stores the activity schedule of the user U1, and may be, for example, a mobile information terminal such as a Smart Phone or a Smart Watch, or may be a personal computer or a server on the cloud.

[0017] The server 300 cooperates with the information terminal 200 and stores the activity schedule of the user U1.

[0018] The information display device 100 , the information terminal 200 , and the server 300 are connected to each other via a network 400 .

[0019] (Configuration of Information Display Device) FIG. 2 is a diagram showing the hardware configuration of the information display device 100. As shown in FIG.

[0020] The information display device 100 includes a processor 101 using a CPU, MCU, etc., RAM 103, storage 110, an operation input device 120, an image processing unit 130, an audio processing unit 140, a group of sensors 160, a communication I / F 170, an expansion I / F 180, and a timer 190, which are connected to each other by a bus 102.

[0021] The storage 110 is configured using a non-volatile storage medium such as a flash ROM, an EEPROM, an SSD, or an HDD.

[0022] The operation input device 120 includes operation keys 121 and a touch sensor 122 .

[0023] The operation keys 121 include, for example, switches, a power key, and a volume key.

[0024] The touch sensor 122 is configured by, for example, a touch pad, and is used to operate the information display device 100, etc. When a separate controller (not shown) is used, the touch sensor 122 may be omitted.

[0025] The image processing unit 130 includes a display 131, an image signal processing unit 132, an outer camera 133, and an inner camera 134. The outer camera 133 is a camera for capturing images of the surroundings. The inner camera 134 is a camera for detecting the line of sight.

[0026] The display 131 may be a transmissive display or a non-transmissive display. In this embodiment, the information display device 100 is a video see-through type VR goggle, so a non-transmissive display is used.

[0027] The image signal processing unit 132 is configured using, for example, an image (video) signal processor.

[0028] The audio processing unit 140 includes a speaker 141 , an audio signal processing unit 142 , and a microphone 143 .

[0029] The audio signal processing unit 142 is configured using, for example, an audio signal processor.

[0030] The sensor group 160 includes a gyro sensor 161 , a geomagnetic sensor 162 , and an acceleration sensor 163 .

[0031] The communication I / F 170 corresponds to, for example, a LAN I / F 171 and a BT (BlueTooth (registered trademark)) I / F 172 .

[0032] The expansion I / F 180 is, for example, a USB device connection terminal, and is used for data transmission / reception, charging, and the like.

[0033] The timer 190 is a timer that holds the current time in the real world, and holds a time such as Coordinated Universal Time (UTC) as the current time. The timer 190 may be configured as software, or may be configured using an RTC (Real Time Clock).

[0034] The bus 102 forms a path for sending and receiving commands and data.

[0035] Figure 3A is a top view showing user U1 wearing information display device 100, Figure 3B is a front view showing user U1 wearing information display device 100, and Figure 3C is a side view showing user U1 wearing information display device 100.

[0036] 3A and 3B , in the information display device 100, the left display 131L is disposed inside the housing of the information display device 100 at a position visible to the left eye of the user U1. The right display 131R is disposed inside the housing of the information display device 100 at a position visible to the right eye of the user U1. In this embodiment, the left display 131L and the right display 131R may be collectively referred to as the display 131. As another configuration example, a single display 131 may divide its display area to display an image for the left eye and an image for the right eye side by side.

[0037] Additionally, an upper left outer camera 133LU, an upper right outer camera 133RU, a lower left outer camera 133LD, and a lower right outer camera 133RD are arranged on the outside of the housing of the information display device 100. These four cameras may be collectively referred to as outer cameras 133. Images captured by these four cameras are used to photograph and measure distances to objects around the user U1, i.e., around the information display device 100. As another configuration example, the outer camera 133 and a distance measurement sensor may be provided separately.

[0038] 3C , the left side of the housing of the information display device 100 is provided with a left touch sensor 122L and a left stereo speaker 141L. The outer camera 133 is not limited to a stereo camera and may be a single camera. Although the right side is not shown in FIG. 3 , the right side of the information display device 100 is provided with a right stereo speaker, a monaural microphone, and a charging and wired communication terminal, which is one of the extension I / F 180.

[0039] Instead of the left touch sensor 122L, a right touch sensor may be provided on the right side surface of the information display device 100. The touch sensor may be provided on either the left or right side, or on both sides.

[0040] FIG. 4 is a functional block diagram of the information display device 100.

[0041] The storage 110 of the information display device 100 includes a basic operation program 1001 including basic programs such as an OS, an application program 1002 for realizing the functions of the information display device 100 of this embodiment, and a various data storage area 1009 which serves as an area for storing various operation setting values ​​and various information (video, still images, audio, etc.).

[0042] The basic operation program 1001 and application program 1002 stored in these storages 110 are expanded (loaded) into RAM 103, and the processor 101 executes the code of the expanded basic operation program 1001 and application program 1002 to configure the functional parts and other functional parts of the information display device of this embodiment.

[0043] The RAM 103 includes a basic operation function unit 1101 , a VR application function unit 1102 , an in-app time related information confirmation unit 1111 , an action schedule information acquisition unit 1112 , a time information conversion unit 1113 , a display control unit 1114 , and a temporary storage area 1199 .

[0044] The basic operation function unit 1101 controls the basic operations of the information display device 100 .

[0045] The VR application function unit 1102 controls the operation of a VR application selected arbitrarily by the user U1, such as a VR game, a web conference, or a shopping site within a VR space.

[0046] The in-app time-related information confirmation unit 1111 confirms information regarding the time axis and degree of time passage set for each application (hereinafter referred to as "virtual reality world time definition information") during the operation of a specific VR application selected arbitrarily by user U1 and controlled by the VR application function unit 1102.

[0047] The degree of time passage here can be rephrased as the difference between a unit length of time in the real world (e.g., 1 second) and a unit length of time in the virtual reality world (e.g., 1 second). For example, this means that 1 second in the real world corresponds to 2 seconds in the virtual reality world, and the degree of time passage in the virtual reality world corresponds to twice the speed of the degree of time passage in the real world.

[0048] The degree of time passage in the virtual reality world may differ for each VR application. In this case, the degree of time passage defined in the running VR application is used as the virtual reality world time definition information.

[0049] Furthermore, even for the same VR application, if the rate of time passage differs depending on the logged-in user, the rate of time passage set in the running VR application by the user operating the information display device 100 is used as the virtual reality world time definition information. For example, in VR application game X, setting information is assumed to include setting information in which user U1 sets the rate of time passage in the virtual reality world to twice as fast as in the real world, and user U2 sets the rate of time passage in the virtual reality world to three times as fast as in the real world. In this case, the in-app time related information confirmation unit 1111 identifies the user currently logged in to the information display device 100 as user U1, and uses the rate of time passage in the virtual reality world at double the speed set by user U1 as the virtual reality world time definition information.

[0050] The activity schedule information acquisition unit 1112 acquires information about the activity schedule of user U1 stored in the information terminal 200 linked to the information display device 100 via the communication I / F 170. If the information terminal 200 stores information about the activity schedule of user U1 in a predetermined server 300, the information may be acquired from the server 300. If the information about the activity schedule of user U1 is stored in the various data storage area 1009 of the information display device 100, the information about the activity schedule of user U1 may be acquired from the various data storage area 1009.

[0051] The time information conversion unit 1113 converts the time information of the information related to the activity schedule of the user U1 acquired by the activity schedule information acquisition unit 1112, based on the virtual reality world time definition information of the running VR application confirmed by the in-app time related information confirmation unit 1111. The running VR application is a VR application whose operation is controlled by the VR application function unit 1102 and is currently running.

[0052] Furthermore, when the operation input device 120 of the information display device 100 receives an instruction from user U1 requesting to display the activity schedule, the time information conversion unit 1113 converts the activity schedule information of user U1 defined using real-world time information into virtual reality world time based on the virtual reality world time definition information in the VR application being executed by the time information conversion unit 1113, and generates the converted activity schedule information of user U1.

[0053] The display control unit 1114 displays the execution screen of the running VR application, i.e., the application image, on the display 131.

[0054] Furthermore, the display control unit 1114 displays the activity schedule information of user U1 converted into virtual reality world time together with the execution screen of the VR application. The display mode for displaying the activity schedule information converted into virtual reality world time may be superimposed on the execution screen, or a small screen may be opened on the display screen of the display 131 and the activity schedule information converted into virtual reality world time may be displayed therein. Alternatively, the execution screen may be switched to display the activity schedule information converted into virtual reality world time. In this case, the activity schedule information converted into virtual reality world time is also displayed, allowing user U1 to maintain a sense of immersion in the virtual reality world without losing its sense of immersion.

[0055] The temporary storage area 1199 is a temporary storage area for various pieces of information created or acquired by various applications.

[0056] (Configuration of Information Terminal 200) Fig. 5 is a hardware configuration diagram of the information terminal 200. Duplicate explanations of components with the same names as those in the information display device 100 will be omitted.

[0057] The information terminal 200 includes a processor 201, a RAM 203, a storage 210, an operation input device 220, an image processing unit 230, an audio processing unit 240, a positioning sensor 250, a group of sensors 260, a communication I / F 270, an expansion I / F 280, and a timer 290, which are connected to each other by a bus 202.

[0058] The storage 210 includes an activity schedule information DB 211. The activity schedule information DB 211 is a database that stores information related to the activity schedule of the user U1.

[0059] The positioning sensor 250 is a sensor that detects the current position of the information terminal 200, and may be a GPS signal receiver, a BlueTooth (registered trademark) beacon receiver, or the like.

[0060] The sensor group 260 includes a gyro sensor, a geomagnetic sensor, an acceleration sensor, an angle sensor, a distance measurement sensor, an illuminance sensor, and the like.

[0061] The timer 290 is a timer that holds the current time in the real world, and holds a time such as Coordinated Universal Time (UTC) as the current time. The timer 290 may be configured as software, or may be configured using an RTC (Real Time Clock).

[0062] FIG. 6 is a diagram showing an example of the configuration of the activity schedule information DB 211. As shown in FIG.

[0063] The schedule information DB 211 stores an identification number 2111, date information 2112, start time information 2113, end time information 2114, alarm flag 2115, alarm time information 2116, event information 2117, and location information 2118 for each schedule in association with each other.

[0064] The identification number 2111 is an identification number that identifies each scheduled activity.

[0065] The date information 2112 is information about the planned date when each scheduled activity is to be carried out.

[0066] The start time information 2113 is information on the start time of each scheduled activity.

[0067] The end time information 2114 is information about the end time of each scheduled activity.

[0068] The alarm flag 2115 is a flag indicating whether or not an alarm is to be issued a predetermined time before each scheduled activity.

[0069] Alarm time information 2116 is information about the alarm issuance time when the alarm flag is ON.

[0070] The task information 2117 is information relating to the task of each scheduled activity.

[0071] The location information 2118 is information relating to the location where each scheduled activity will be performed.

[0072] FIG. 7 is a diagram showing an example of an action schedule display.

[0073] When a calendar application or a scheduler application is launched on the information display device 100, information requesting acquisition of an activity schedule is sent to the information terminal 200. The information terminal 200 reads out the activity schedule information stored in the activity schedule information database 211 and sends it to the information display device 100. The display control unit 1114 of the information display device 100 displays the activity schedule of user U1 on the display 131.

[0074] In the example of Fig. 7, user U1 taps the date portion of August 15, 2022 in the calendar app or scheduler app to check his / her schedule for August 15, 2022, causing information about the event for August 15, 2022 (the planned activity No. 1 in Fig. 6) to be displayed at the bottom of the calendar. In the example of Fig. 7, the calendar app or scheduler app is launched when the VR app is not running, causing the calendar and activity information defined using real-world time information to be displayed. This embodiment is characterized in that, while the VR app is running, the VR app execution screen and the virtual reality world time are displayed after converting the time into virtual reality world time defined by the running VR app.

[0075] (Schedule Display Process 1: Basic Process for Schedule Display) First, a basic process for schedule display (referred to as "Schedule Display Process 1") will be described, in which the schedule stored in the information terminal 200 is displayed on the information display device 100. Fig. 8 is a flowchart showing the process flow of the schedule display process 1.

[0076] In S101, the basic operation function unit 1101 checks whether the operation input device 120 has received an instruction to request display of an action schedule from the user U1. If an instruction to request display of an action schedule has been received (S101; Yes), the process proceeds to S102. If an instruction to request display of an action schedule has not been received (S101; No), the process ends.

[0077] The activity schedule information acquisition unit 1112 requests the linked information terminal 200 to transmit information related to the activity schedule of the user U1 (activity schedule information) (S102). The activity schedule information acquisition unit 1112 measures the elapsed time T from the transmission of the activity schedule information request signal.

[0078] If the elapsed time T exceeds a predetermined time threshold, i.e., if information regarding the user U1's activity schedule cannot be received from the information terminal 200 within the predetermined time threshold, the activity schedule information acquisition unit 1112 displays an error message and terminates the process. When information regarding the activity schedule is received, the activity schedule information acquisition unit 1112 stops measuring the elapsed time T.

[0079] Meanwhile, when the activity schedule information acquisition unit 1112 receives information regarding user U1's activity schedule sent by the information terminal 200 in response to the processing of S102 (S103), the display control unit 1114 creates an activity schedule notification image to notify user U1 of the activity schedule based on the information regarding user U1's activity schedule acquired in the processing of S103 (S104).

[0080] The display control unit 1114 displays the activity schedule notification image created in the process of S104 on the display 131 (S105). When the VR application is not running, the processes of S104 to S105 may be performed by activating a calendar application or a scheduler application to display the activity schedule of the user U1, as shown in FIG.

[0081] FIG. 9 is a schematic diagram showing an example of a basic screen (main menu screen) in the first action schedule display process.

[0082] The basic screen 131A is a screen displayed on the display 131 when no apps are running on the information display device 100. The basic screen 131A includes a main menu display 131A1 and a date and time display 131A2. The date and time display 131A2 displays the current date and time displayed on the basic screen 131A of the information display device 100. This current date and time is displayed as the date and time in the real world.

[0083] FIG. 10 is a schematic diagram showing an example of a basic screen (action schedule notification image) in the action schedule display process 1. As shown in FIG.

[0084] 10 is a display screen in which the application screen of the calendar application or the scheduler application is superimposed on the basic screen 131 A. The activity schedule display screen 131B corresponds to the state in which the calendar application or the scheduler application is started in the processing of S104 to S105 and the activity schedule is displayed.

[0085] The activity schedule display screen 131B displays a main menu display 131B1 and a date and time display 131B2, with an activity schedule display 131B3 of user U1 superimposed on them. The main menu display 131B1 and the date and time display 131B2 are the same as the main menu display 131A1 and the date and time display 131A2 of the basic screen 131A in Fig. 9. The activity schedule display 131B3 of user U1 is a display of the activity schedule controlled by a calendar application or a scheduler application.

[0086] (Planned event display process 2: displaying a planned event while a VR application is running) Next, a process (referred to as "planned event display process 2") for displaying a planned event while a VR application is running on the information display device 100 will be described. Fig. 11 is a flowchart showing the flow of the planned event display process 2.

[0087] When the basic operation function unit 1101 receives a request to start a specific VR application (VR app) from the operation input device 120 (S201), the VR application function unit 1102 related to the specific VR app is expanded in RAM 103, thereby starting the specific VR app (S202).

[0088] The VR application function unit 1102 generates an execution screen of a predetermined VR application and displays it on the display 131 (S203).

[0089] The basic operation function unit 1101 checks whether the operation input device 120 has received an instruction to display an activity schedule from the user U1 (S204). If no instruction to display an activity schedule has been received (S204: No), the processing related to the display of the activity schedule ends. (The operation of the VR application started in the processing of S202 to S203 may continue.)

[0090] If an action plan display instruction operation is performed (S204: Yes), the action plan information acquisition unit 1112 requests the linked information terminal 200 to transmit information related to the action plan of the user U1 (S205). The action plan information acquisition unit 1112 measures the elapsed time T from the transmission of the action plan information request signal.

[0091] If the elapsed time T exceeds a predetermined time threshold, i.e., if information regarding the user U1's activity schedule cannot be received from the information terminal 200 within the predetermined time threshold, the activity schedule information acquisition unit 1112 displays an error message and terminates the process. When information regarding the activity schedule is received, the activity schedule information acquisition unit 1112 stops measuring the elapsed time T.

[0092] On the other hand, when the activity schedule information acquisition unit 1112 receives information regarding the activity schedule of user U1 sent by the information terminal 200 in response to the processing of S205 (S206), the in-app time related information confirmation unit 1111 confirms information regarding the time axis and degree of time passage within the app set within a specific VR app whose operation is controlled by the VR application function unit 1102 (S207).

[0093] The time information conversion unit 1113 converts the time information of the information regarding user U1's planned activities acquired in the processing of S206 based on the information regarding the time axis and the degree of time passage within the specified VR app confirmed in the processing of S207 (S208).

[0094] The display control unit 1114 creates a converted action plan notification image for notifying the (converted) time information of the action plan of the user U1 converted in the process of S208 (S209), and displays the converted action plan notification image superimposed on the execution screen of the predetermined VR application being displayed on the display 131 in the process of S203 (S210).The process then ends.

[0095] FIG. 12 is a schematic diagram showing an example of a display screen (VR application execution screen) in the activity schedule display process 2.

[0096] 12 is a display screen displayed when the execution screen of the VR application launched in the processing of S202 to S203 is displayed on the display 131 of the information display device 100. Date and time display 131C1 included in the VR application execution screen 131C displays the date and time within the VR application and the current position of the player character in the VR world. Therefore, the date and time displayed in the virtual reality world, which are different from the real date and time, are displayed.

[0097] In the example of Figure 12, the date and time in the real world is August 15, 2022, 16:30, and in VR world time it is "February 17, Year 815 of the Kingdom Calendar, 10:40." Also, 30 minutes in the real world is equivalent to 1 hour in VR world time, meaning that time passes twice as fast in the VR world.

[0098] Furthermore, user U1 is actually currently located in Tokyo at "latitude XX, longitude XX degrees east," but his current location in the VR world is "△△ Grassland," which is not defined by the above-mentioned real latitude and longitude.

[0099] FIG. 13 is an explanatory diagram showing the contents of the time information conversion process.

[0100] In the time information conversion process, the user U1's planned activity information 3001 obtained from the information terminal 200 in S206 is converted based on the real-world current date and time information 3002 obtained from the timer 190 built into the information display device 100 and the virtual reality world time definition information 3003 obtained in the process of S207, and converted planned activity information 3004 is generated.

[0101] 13, the "Birthday Party" starts 1 hour and 30 minutes after the current time in the real world. If the virtual reality world time definition information defines the rate of time passage as time progressing twice as fast in the virtual reality world as in real time, then 30 minutes in the real world corresponds to 1 hour in the VR app, and therefore the "Birthday Party" will start 3 hours later (i.e., 1:40 PM) on the timeline within the VR app.

[0102] FIG. 14 is a schematic diagram showing the display screen 2 in the action schedule display process 2. As shown in FIG.

[0103] The VR application execution screen 131D in FIG. 14 is a VR application execution screen for August 15, 2022, at 4:30 p.m. in the real world. The VR application execution screen 131D includes a date and time display 131D1 within the VR application and a converted action plan notification image 131D2. The date and time display 131D1 within the VR application displays the date, time, and current location within the VR application. This display displays the date and time in the virtual reality world, which differ from the date and time in the real world. The converted action plan notification image 131D2 displays the (converted) action plan information of user U1. The VR application execution screen 131D has the converted action plan notification image 131D2 created in the process of S209 superimposed on the VR application execution screen displayed in the process of S203. Therefore, by comparing the time display within the VR application with the time display within the VR application, the user can determine the amount of time remaining until the start time of the action plan in the real world without having to check the clock in the real world.

[0104] The title of the event in the schedule may be converted to something that does not create a sense of incongruity within the VR app and displayed. For example, in Figure 14, the title of the event in the real world, "Birthday Party," is displayed, but it may be converted to a term in the VR world, such as "Guild Meeting," and displayed.

[0105] According to this embodiment, when a planned activity is displayed while a VR app is running, the planned activity is converted to VR world time, a converted planned activity notification image is generated, and the image is displayed together with the execution screen of the VR app. Even if the current time and the progression of time differ between VR world time and the real world, real-world time information is converted to VR world time information using virtual reality world time definition information and displayed. This allows the time interval between the current time in the real world and the time interval between the current time in the VR world to be the same, so the user can grasp the time remaining until the planned activity in the real world simply by looking at the planned activity displayed in VR world time, and can check the planned activity without losing the sense of immersion in the VR world.

[0106] Second Embodiment In the second embodiment, while an application that displays a virtual reality world is being executed, time information in the real world is converted into time information in the virtual reality world and a reminder is displayed.

[0107] (Scheduled Action Display Process 3: Process for Displaying Reminders Using Scheduler Function While VR Application is Running) FIG. 15 is a flowchart showing the process flow of scheduled action display process 3.

[0108] (Information display device 100) In the information display device 100, when the basic operation function unit 1101 receives a request operation to start a specific application (VR app) from the operation input device 120 (S301), the VR application function unit 1102 related to the specific VR app is expanded in RAM 103, and the specific VR app is started (S302).

[0109] The VR application function unit 1102 generates an execution screen of a predetermined VR application and displays it on the display 131 (S303).

[0110] (Information Terminal 200) In the information terminal 200, the processor 201 checks the activity schedule of the user U1 stored in the activity schedule information database 211 of the storage 210 (S304).

[0111] The processor 201 acquires current time information from the timer 290 built in the information terminal 200, and checks whether there is a scheduled event that should be notified (reminded) to the user U1 based on the current time information (S305). If there is no scheduled event that should be notified (reminded) to the user U1 (S305: No), the process returns to S304, and the check of the scheduled event information database 211 is repeated.

[0112] If there is a planned activity to be notified (reminded) (S305: Yes), the processor 201 transmits information regarding the planned activity to be notified (reminded) to the associated information display device 100 via the communication I / F 270, which was confirmed in the processing of S305 (S306).

[0113] (Information display device 100) In the information display device 100, the activity schedule information acquisition unit 1112 receives information on the activity schedule to be notified (reminded) to the user U1, which information is transmitted from the information terminal 200, via the communication I / F 170 (S307).

[0114] The in-application time related information confirmation unit 1111 confirms information relating to the time axis and the degree of time passage within the application set within a specific VR application whose operation is controlled by the VR application function unit 1102 (S308).

[0115] The time information conversion unit 1113 converts the time information of the information regarding the planned activities to be notified (reminded) to user U1, obtained in the processing of S307, based on the information regarding the time axis and the degree of time passage within the specified VR app confirmed in the processing of S308 (S309).

[0116] The display control unit 1114 creates a converted action plan notification image (alarm) for notifying the user U1 of the (converted) time information of the action plan that should be notified (reminded) to the user U1 converted in the process of S309 (S310).

[0117] The display control unit 1114 superimposes and displays the converted action schedule notification image (alarm) created in the process of S310 on the execution screen of the predetermined VR application being displayed on the display 131 in the process of S303 (S311).

[0118] The activity plan information acquisition unit 1112 transmits to the information terminal 200 via the communication I / F 170 a notification that the information regarding the activity plan to be notified (reminded) to user U1, which was received in the processing of S307, has been notified to user U1 (S312).

[0119] FIG. 16 is a diagram showing a display screen (VR application execution screen+converted action schedule notification image: alarm) displayed in the action schedule display process 3.

[0120] 16 is displayed is 17:30, August 15, 2022. At this time, a VR application execution screen 131E is displayed on the display 131. The VR application execution screen 131E is a screen in which date and time information 131E1 in the VR world and a converted action schedule notification image (alarm) 131E2 created in the processing of S310 are superimposed on the VR application execution screen displayed in the processing of S303.

[0121] The VR application execution screen 131E displays the date and time in the VR application and the current location in the VR application. The date and time in the VR world and the current location display in the VR application are different from the date, time, and current location in the real world.

[0122] The converted action schedule notification image (alarm) 131E2 is an image that displays an alarm indicating the time remaining until the start time of the action schedule.

[0123] This display process allows you to understand how much time you have until the start of your planned activity in the real world by comparing it with the time displayed in the VR app without having to check the clock in the real world.

[0124] FIG. 17 is a diagram showing a modified example of the display screen (VR application execution screen+progress bar) displayed in the activity schedule display process 3.

[0125] 17 is displayed is August 15, 2022, 5:30 PM. At this time, a VR application execution screen 131F is displayed on the display 131. The VR application execution screen 131F is a screen in which date and time information 131F1 in the VR world and a converted action schedule notification image (progress bar) 131F2 created in the processing of S310 are superimposed on the VR application execution screen displayed in the processing of S303.

[0126] The VR application execution screen 131F displays the date and time in the VR application and the current location in the VR application. The date and time in the VR world and the current location display in the VR application are different from the date, time, and current location in the real world.

[0127] The converted action plan notification image (progress bar) 131F2 is an image that displays the remaining time until the converted action plan (birthday party) using a progress bar.

[0128] This display process allows you to know how much time you have until the start time of your planned activity in the real world by checking the progress of the progress bar without having to check the clock in the real-world VR app.

[0129] As a further modification, the progress bar may be replaced with a graphic display such as an hourglass.

[0130] FIG. 18 is a diagram showing a modified example of the display screen (VR application execution screen+warning display) displayed in the activity schedule display process 3.

[0131] 18 is displayed is 17:30, August 15, 2022. At this time, a VR application execution screen 131G is displayed on the display 131. The VR application execution screen 131G is a screen in which date and time information 131G1 in the VR world, a converted action schedule notification image (alarm) 131G2 created in the processing of S310, and a warning display 131G3 are superimposed on the VR application execution screen displayed in the processing of S303.

[0132] The VR application execution screen 131G displays the date and time in the VR application and the current location in the VR application. The date and time in the VR world and the current location display in the VR application are different from the date, time, and current location in the real world.

[0133] The converted action schedule notification image (alarm) 131G2 is an image that displays an alarm indicating the time remaining until the start time of the action schedule.

[0134] The warning display 131G3 is a display that warns the user U1 that the rate of time elapsed differs between the real world and within the VR app. In the example of Fig. 18, 30 minutes in the real world corresponds to 1 hour within the VR app, so even if the time display within the VR app makes the user think that there is an hour until the start of the event, the user is made aware that there is actually only 30 minutes.

[0135] A real-time passage display 131G4 showing the passage of time in the real world may also be displayed. The real-time passage display 131G4 is a display that allows the degree of time passage in the real world to be understood even within the VR app. In Fig. 18, an hourglass image is displayed as the real-time passage display 131G4, and the hourglass image in Fig. 18 is displayed so that, for example, all the sand falls out in one hour in the real world (two hours within the VR app).

[0136] According to this embodiment, the remaining time until a scheduled event set in the real world can be converted into virtual reality world time and displayed. This allows reminders based on scheduled events set in a scheduler app or the like to be converted into virtual reality world time, so that reminders can be sent without impairing the sense of immersion in the virtual reality space.

[0137] In the above embodiment, the action plan may be displayed using real-world time information when the information display device 100 switches from a state in which the VR application execution screen is displayed to another application screen or the basic screen. In other words, even when the VR application is being executed, if the screen displayed in the foreground is not the VR application execution screen, the display control unit 1114 displays the action plan based on real-world time information on the foreground screen, and if the VR application execution screen is displayed in the foreground, the converted action plan notification image is displayed on the VR application execution screen.

[0138] The present invention is not limited to the above-described embodiments, and it is possible to replace part of the configuration of one embodiment with another embodiment. It is also possible to add the configuration of another embodiment to the configuration of one embodiment. These all fall within the scope of the present invention, and the numerical values, messages, etc. appearing in the text and figures are merely examples, and the use of different ones does not impair the effects of the present invention.

[0139] Furthermore, some or all of the functions of the invention may be implemented in hardware, for example, by designing an integrated circuit. They may also be implemented in software by a microprocessor unit, processor, etc., interpreting and executing an operating program. Furthermore, the scope of software implementation is not limited, and both hardware and software may be used.

[0140] The above embodiment includes the following aspects.

[0141] (Supplementary Note 1) An information display device comprising: a processor; and a display, wherein, while a virtual reality application is being executed, the processor converts real-world time information into time information of a virtual reality world realized by the virtual reality application to generate converted time information, and displays a display screen of the virtual reality application and the converted time information on the display.

[0142] (Supplementary Note 2) An information display method comprising: a step of executing a virtual reality application; a step of converting real-world time information into time information of a virtual reality world realized by the virtual reality application while the virtual reality application is being executed to generate converted time information; and a step of displaying a display screen of the virtual reality application and the converted time information on a display.

[0143] DESCRIPTION OF SYMBOLS 100: Information display device 101: Processor 102: Bus 103: RAM 110: Storage 120: Operation input device 121: Operation keys 122: Touch sensor 122L: Left touch sensor 130: Image processing unit 131: Display 131A: Basic screen 131A1: Main menu display 131A2: Date and time display 131B: Action plan display screen 131B1: Main menu display 131B2: Date and time display 131B3: Action plan display 131C: VR application execution screen 131C1: Date and time display 131D: VR application execution screen 131D1: Date and time display 131D2: Converted action plan notification image 131E: VR application execution screen 131E1: Date and time display 131E2 : Converted action schedule notification image (alarm) 131F: VR application execution screen 131F1: Date and time display 131F2: Converted action schedule notification image (progress bar) 131G: VR application execution screen 131G1: Date and time display 131G2: Converted action schedule notification image (alarm) 131G3: Warning display 131G4: Real-time progress display 131L: Left display 131R: Right display 132: Image signal processing unit 133: Outer camera 133LD: Lower left outer camera 133LU: Upper left outer camera 133RD: Lower right outer camera 133RU: Upper right outer camera 134: Inner camera 140: Audio processing unit 141: Speaker 141L: Left stereo speaker 142: Audio signal processing unit 143: Microphone 160: Sensor group 161: Gyro sensor 162: Geomagnetic sensor 163: Acceleration sensor 170: Communication I / F 171: LAN I / F 180: Expansion I / F 190: Timer 200: Information terminal 201: Processor 202: Bus 203: RAM 210: Storage 211: Action schedule information database (DB) 220: Operation input device 230: Image processing unit240: Audio processing unit 250: Positioning sensor 260: Sensor group 270: Communication I / F 280: Expansion I / F 290: Timer 300: Server 400: Network 1001: Basic operation program 1002: Application program 1009: Data storage area 1101: Basic operation function unit 1102: VR application function unit 1111: In-application time-related information confirmation unit 1112: Action schedule information acquisition unit 1113: Time information conversion unit 1114: Display control unit 1199: Temporary storage area 2111: Identification number 2112: Date information 2113: Start time information 2114: End time information 2115: Alarm flag 2116: Alarm time information 2117: Message information 2118: Location information 3001: Action plan information 3002: Current date and time information 3003: Virtual reality world time definition information 3004: Converted action plan information

Claims

1. An information display device comprising: a processor; and a display, wherein while a virtual reality application is being executed, the processor converts real-world time information into time information of a virtual reality world realized by the virtual reality application to generate converted time information, and displays a display screen of the virtual reality application and the converted time information on the display.

2. An information display device as described in claim 1, wherein the processor, when the degree of time passage in the virtual reality world differs from the degree of time passage in the real world, converts the degree of time passage in the real world to the degree of time passage in the virtual reality world to generate the converted time information.

3. An information display device as described in claim 2, wherein the processor acquires planned action information including a subject and a start time of the subject in the real world, converts the real world time information included in the planned action information into time information in the virtual reality world to generate the converted time information, generates a converted planned action notification image including the subject and the converted time information, and displays the converted planned action notification image when a display request for the planned action information is received while the display screen of the virtual reality application is being displayed on the display.

4. An information display device as described in claim 3, wherein the processor calculates the remaining time in the real world from the current time in the real world to the start time of the message in the real world, converts the remaining time in the real world to a degree of time passage in the virtual reality world to generate the converted time information, and displays the converted action schedule notification image including the message and the converted time information.

5. An information display device as described in claim 2, wherein the processor: acquires planned action information including a subject and a start time of the subject in the real world; calculates a remaining time in the real world from a current time in the real world to the start time of the subject in the real world; converts the remaining time in the real world to a degree of time passage in the virtual reality world to generate the converted time information; generates a converted planned action notification image that displays the remaining time of the converted time information using a progress bar; and displays the converted planned action notification image when a display request for the planned action information is received while the display screen of the virtual reality application is being displayed on the display.

6. An information display device according to claim 2, wherein the processor further displays a warning indicating that the rate of time passage in the real world differs from the rate of time passage in the virtual reality world.

7. An information display device according to claim 6, wherein the processor uses a real-time progress display showing the degree of time passing in the real world as the warning display.

8. An information display device as described in claim 3, further comprising a communication interface for communicating with an information terminal, wherein the processor displays the converted action plan notification image based on the action plan information received from the information terminal.

9. An information display method comprising the steps of: executing a virtual reality application; converting real-world time information into time information of a virtual reality world realized by the virtual reality application while the virtual reality application is being executed, to generate converted time information; and displaying a display screen of the virtual reality application and the converted time information on a display.

Images