Video display device, video display method, and video display program

Abstract

To obtain a video display device that can instantly move to a movable location and view images from different viewpoints. [Solution] The system comprises an operation unit that generates various commands based on the user's manual operation, a pointer display unit that displays the desired location on the video screen as a pointer when a location identification command is received, a determination unit that determines whether the location displayed by the pointer is a movable location, a viewpoint movement control unit, and a head-mounted display. When the viewpoint movement control unit receives a viewpoint movement command while an immovable second trajectory is displayed, it maintains the user's current viewpoint position and viewpoint height without moving the user's viewpoint, and continues to accept manual operations by the user to identify the desired location on the video screen. This allows the user, having moved to various movable locations, to simulate a fire outbreak from various viewpoint positions and viewpoint heights.

Description

Technical Field

[0005]

[0001] The present disclosure relates to a video display device, a video display method, and a video display program for moving a user's viewpoint on a displayed video.

Background Art

[0002] Virtual Reality (VR) has become popular as a technology that can provide "a physical experience equivalent to reality," even though it is a "virtual space" created by wearing a dedicated "head-mounted display (HMD)" or goggle-type "VR goggles" on the face to stimulate vision and hearing with video and sound. For example, VR technology is used in games, simulation systems for performing virtual experiences, and the like.

[0003] As a specific example, there is a conventional device that can visually recognize the movement path of an object to be manipulated when a user performs a virtual experience of a crisis by moving the object to be manipulated within a virtual space (see, for example, Patent Document 1). According to this conventional device, after the user finishes the virtual experience of the crisis, the user can visually recognize the movement path of the object to be manipulated and determine whether the object to be manipulated was appropriately operated.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In virtual reality technologies, when moving a controlled object, it is common practice to sequentially move the controlled object along a traversable path based on user input, while displaying images corresponding to the new position. In other words, by moving a controlled object existing within a virtual space, the user simulates the feeling of actually moving within that virtual space.

[0006] However, in the case of a simulation system that uses virtual reality to provide a fire simulation experience, for example, it may be desirable not only to have participants simulate an actual evacuation route, but also to simulate a fire situation from different perspectives. Specifically, considering a fire occurring in an office, it would be important not only to simulate an evacuation route from a fixed location, but also to have participants quickly move to various locations and experience the fire from different viewpoints.

[0007] In other words, what is desired is not only a technology that allows users to simulate movement within a virtual space at real-world speeds, but also a technology that allows them to instantly move to a movable point and simulate the experience from different viewpoints within the virtual space. Furthermore, not limited to virtual spaces, a technology is desired that allows users to instantly move to a movable point and view the image from different viewpoints even in the space of actually captured video.

[0008] This disclosure is made to solve the above-mentioned problems and aims to provide a video display device, a video display method, and a video display program that can instantly move to a movable location and view the video from different viewpoints. [Means for solving the problem]

[0009] The video display device relating to this disclosure is a video display device that moves the user's viewpoint on a displayed video, and includes an operation unit that generates a movement point identification command to identify a point on the video to be moved to, and a viewpoint movement command to move the user's viewpoint on the video, based on the user's manual operation; a pointer display unit that, when it receives a movement point identification command from the operation unit, displays the point on the video to be moved to with a pointer; a determination unit that determines whether the point to be moved to, as indicated by the pointer, is a walkable location on the video, and determines whether the point to be moved is a walkable location or not; an identification display unit that, when the determination unit determines that the point to be moved is a walkable location, displays a first trajectory on the video from the user's viewpoint toward the walkable location, and when the determination unit determines that the point to be moved is an walkable location, displays a second trajectory on the video that is identifiable as the first trajectory from the user's viewpoint toward the walkable location; and when it receives a viewpoint movement command, it selects a walkable location based on the user's manual operation. The system comprises a viewpoint movement control unit that controls the user's viewpoint to move to a movable location or maintain the user's current viewpoint without moving the viewpoint, and a head-mounted display worn by the user that allows the user to view images from any viewpoint direction and height from their current position by moving their head to shift their line of sight direction and height. When a viewpoint movement command is received while a first trajectory is displayed, the viewpoint movement control unit moves the user's viewpoint to a movable location, and when a viewpoint movement command is received while a second trajectory is displayed, it maintains the user's current viewpoint position and height without moving the user's viewpoint, and continues to accept manual operations by the user to specify the desired location on the image. This allows a user who has moved to various movable locations to view the same fire from different viewpoints, thereby simulating the state of a fire from various viewpoint positions, as well as from various viewpoint heights at the same location.

[0010] Furthermore, the video display method relating to this disclosure is a video display method executed by a computer when the user moves their viewpoint on the displayed video, wherein when the computer receives a command to identify a point to move to on the video by the user's manual operation, it performs the steps of: displaying the point to move to on the video with a pointer; determining whether the point to move to, as indicated by the pointer, is a walkable location on the video, thereby determining whether the point is a moveable or immovable location; if the computer determines that the point to move to is a moveable location, it performs the steps of: displaying a first trajectory on the video from the user's viewpoint toward the moveable location; if the computer determines that the point to move to is an immovable location, it performs the steps of: displaying a second trajectory on the video that is identifiable as the first trajectory from the user's viewpoint toward the immovable location; and when the computer receives a viewpoint movement command by the user's manual operation, it moves the user's viewpoint to the moveable location selected based on the user's manual operation. The control process involves either moving the user's viewpoint or maintaining the user's current viewpoint without moving the user's viewpoint, and enabling the user, wearing a head-mounted display, to move their head to shift their gaze direction and height, thereby allowing the user to view images from any desired gaze direction and height from their current position. The control process involves, when a viewpoint movement command is received while the first trajectory is displayed, moving the user's viewpoint to a movable point; when a viewpoint movement command is received while the second trajectory is displayed, maintaining the user's current viewpoint position and height without moving the user's viewpoint, and continuously accepting manual input from the user to specify the desired location on the image. This allows the user, having moved to various movable points, to view the same fire from different viewpoints, thereby simulating the fire's state from various viewpoint positions, as well as from various viewpoint heights at the same location.

[0011] Furthermore, the video display program relating to this disclosure is a video display program that moves the user's viewpoint on a displayed video, comprising: an operation unit that generates a movement point identification command to identify a point on the video to be moved to, and a viewpoint movement command to move the user's viewpoint on the video, based on the user's manual operation; a pointer display unit that, upon receiving a movement point identification command from the operation unit, displays the point on the video to be moved to with a pointer; a determination unit that determines whether the point to be moved to, as indicated by the pointer, is a walkable location on the video, thereby determining whether the point to be moved to is a walkable location or not; an identification display unit that, if the determination unit determines that the point to be moved to is a walkable location, displays a first trajectory on the video from the user's viewpoint toward the walkable point, and if the determination unit determines that the point to be moved to is an walkable location, displays a second trajectory on the video that is identifiable as the first trajectory from the user's viewpoint toward the walkable point; and, upon receiving a viewpoint movement command, selects a walkable location based on the user's manual operation. The system includes a viewpoint movement control unit that controls the user's viewpoint to move to a specific location or to maintain the user's current viewpoint without moving the viewpoint, and a video display program that functions as a head-mounted display worn by the user, allowing the user to view images from any viewpoint direction and height from their current position by moving their head to shift their line of sight direction and height. The viewpoint movement control unit, when it receives a viewpoint movement command while a first trajectory is displayed, moves the user's viewpoint to a movable location, and when it receives a viewpoint movement command while a second trajectory is displayed, maintains the user's current viewpoint position and height without moving the user's viewpoint, and continues to accept manual operations by the user to specify the desired location on the video. This allows a user who has moved to various movable locations to simulate the state of a fire from various viewpoint positions by viewing the same fire from different viewpoints, and also allows them to simulate the state of a fire from various viewpoint heights at the same location. [Effects of the Invention]

[0012] According to this disclosure, it is possible to obtain a video display device, a video display method, and a video display program that can instantly move to a movable location and view the video from different viewpoints. [Brief explanation of the drawing]

[0013] [Figure 1] This figure shows the overall configuration of the video display device in Embodiment 1 of the present disclosure. [Figure 2] This is a flowchart illustrating a series of processes for the video processing method performed by the video display device in Embodiment 1 of this disclosure. [Figure 3] This figure shows the first display screen and the second display screen as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 4] This figure shows the third and fourth display screens as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 5] This figure shows the fifth and sixth display screens as examples of display screens that are displayed on the display unit when using the video display device in Embodiment 1 of this disclosure. [Figure 6] This figure shows the seventh and eighth display screens as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 7] This figure shows the 9th and 10th display screens as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 8] This figure shows the 11th and 12th display screens as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 9] This figure shows the 13th to 15th display screens as examples of display screens that are displayed on the display unit when using the video display device according to Embodiment 1 of this disclosure. [Figure 10]FIG. is a diagram showing display screen examples from the 16th display screen to the 18th display screen when using the video display device according to Embodiment 1 of the present disclosure on the display unit. [Figure 11] FIG. is a diagram showing display screen examples from the 19th display screen to the 21st display screen when using the video display device according to Embodiment 1 of the present disclosure on the display unit. [Figure 12] FIG. is a diagram showing display screen examples from the 22nd display screen to the 24th display screen when using the video display device according to Embodiment 1 of the present disclosure on the display unit.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, preferred embodiments of the video display device, video display method, and video display program of the present disclosure will be described with reference to the drawings. The video display device, video display method, and video display program according to the present disclosure identify and display the movement trajectory to a movable point and the movement trajectory to an immovable point for a movement point selected based on a manual operation of a user, and are configured to be able to instantaneously move to a movable point and view a video from a different viewpoint, which is a technical feature.

[0015] Embodiment 1. FIG. 1 is a diagram showing the overall configuration of the video display device according to Embodiment 1 of the present disclosure. The video display device according to Embodiment 1 includes an operation unit 10, a display control processing unit 20, and a display unit 30.

[0016] The operation unit 10 corresponds to a controller that is held by the user and gives various commands generated according to the manual operation of the user to the display control processing unit 20.

[0017] The display control processing unit 20 provides an appropriate video (image) corresponding to various commands to the user by executing display control of the video displayed on the display unit 30 based on the various commands received from the operation unit 10.

[0018] The display control processing unit 20 comprises a pointer display unit 21, a determination unit 22, an identification display unit 23, and a viewpoint movement control unit 24. The specific functions of each component will be described later with reference to Figures 2 to 12. The display control processing unit 20 is responsible for setting (placing) three image characteristics, described later, namely "immovable terrain," "no-movement zone," and "action." The determination unit 22 also determines whether a point specified by the user with the pointer on the operation unit 10 is a point that can be moved, based on the setting status of the "no-movement zone" and "movable zone" set in the image, as described later.

[0019] The display unit 30 provides the user with appropriate images in response to various commands, and this can be achieved, for example, by a head-mounted display worn by the user.

[0020] In the following, we will explain the functions of the video display device of this disclosure with reference to Figures 2 to 12, using a head-mounted display as the display unit 30, and a user wearing the head-mounted display performing a fire simulation based on manual operation using the operation unit 10 as a specific example.

[0021] Figure 2 is a flowchart showing a series of processes of the video processing method performed by the video display device in Embodiment 1 of this disclosure. Figures 3 to 12 are diagrams showing examples of display screens displayed on the display unit 30 when using the video display device in Embodiment 1 of this disclosure.

[0022] Before explaining the flowchart in Figure 2, we will first describe the actions that a user wearing a head-mounted display (display unit 30) can perform while holding the controller (operation unit 10) in their hand.

[0023] While wearing a head-mounted display, users can move their heads to shift the image of the virtual space displayed on the head-mounted display to the desired direction of their gaze. In other words, by moving their heads and shifting their gaze direction, users can view images from any direction of their gaze from their current position.

[0024] Furthermore, by holding down the trigger button on the controller, the user can project a laser pointer onto the image displayed on the display unit 30. The user can move the laser pointer by moving the direction of the controller held in their hand, and can select the desired location on the image.

[0025] Then, once the user has selected a point on the video where they want to move, they can release the trigger button, and if the selected point is a movable location, they will instantly be moved to that point.

[0026] As a result of such user actions using the trigger button, the controller, which is the operation unit 10, outputs the following two types of commands to the display control processing unit 20. • While the user continues to press the trigger button, a command to specify the destination is output to direct the laser pointer to the desired location. When the user stops pressing the trigger button, a viewpoint movement command is output to attempt to move the viewpoint to the selected desired location.

[0027] Assuming that such actions are taken by the user, the series of processes executed by the display control processing unit 20 will be explained in detail using Figure 2. Note that the explanation in Figure 2 omits the control that moves the image of the virtual space displayed on the head-mounted display to the image in the desired line of sight direction when the user moves their head, and focuses on the control operations for the "movement point identification command" and the "viewpoint movement command".

[0028] In step S201, the pointer display unit 21 determines whether or not it has received a command from the operation unit 10 to specify the location to be moved to on the video. If the pointer display unit 21 has received a command to specify the location to be moved to on the video, it proceeds to step S202 and displays the location to be moved to on the video using a pointer. In other words, the pointer display unit 21 can display a pointer at the location set as the location to be moved to in response to the user's manual operation.

[0029] Next, in step S203, the determination unit 22 determines whether the point indicated by the pointer is a point that the user can move to or a point that the user cannot move to. For example, if the location selected by the user as a point they want to move to is a walkable floor surface, corridor, road, etc. on the video, the determination unit 22 determines that the point indicated by the pointer is a point that can be moved to.

[0030] On the other hand, if the location selected by the user as a place to move to is a desk, wall, building, etc., on the video, the determination unit 22 determines that the location indicated by the pointer is an immovable location.

[0031] In other words, the determination unit 22 can determine from the video data of the virtual space displayed on the display unit 30 whether a location is movable or immovable. If it is determined to be YES in step S203, the process proceeds to step S204 and beyond. On the other hand, if it is determined to be NO in step S203, the process proceeds to step S207 and beyond.

[0032] If the determination unit 22 determines in step S203 that the point to be moved to is a movable point, and the user proceeds to step S204, the identification display unit 23 displays the trajectory of the movement path on the video in a first color (for example, blue) from the user's perspective toward the movable point.

[0033] Next, in step S205, the viewpoint movement control unit 24 determines whether or not it has received a viewpoint movement command from the operation unit 10 while the trajectory in the first color is displayed. If the viewpoint movement control unit 24 determines YES in step S205, it proceeds to step S206, displays the image on the display unit 30 after moving the user's viewpoint to a movable point, and then returns to step S201.

[0034] On the other hand, if the viewpoint movement control unit 24 determines NO in step S205, it returns to step S201. Therefore, by returning to step S201, the user can continue to perform the operation of identifying the point on the video that they want to move to.

[0035] If the determination unit 22 determines in step S203 that the point to be moved to is an immovable point, and the process proceeds to step S207, the determination unit 22 further determines whether the point to be moved, as indicated by the pointer, is a door portion that can be opened or closed. If the determination in step S207 is NO, the process proceeds to step S208 and beyond. On the other hand, if the determination in step S207 is YES, the process proceeds to step S211 and beyond.

[0036] If the determination unit 22 determines in step S207 that the point to be moved to is not a door that can be opened or closed, and the user proceeds to step S208, the identification display unit 23 displays the trajectory of the movement path on the video in a second color (for example, red) towards the point that is inaccessible from the user's perspective.

[0037] Next, in step S209, the viewpoint movement control unit 24 determines whether or not it has received a viewpoint movement command from the operation unit 10 while the trajectory in the second color is being displayed. If the viewpoint movement control unit 24 determines YES in step S209, it proceeds to step S210, where it displays an image on the display unit 30 that maintains the user's current viewpoint without moving the user's viewpoint, and then returns to step S201.

[0038] On the other hand, if the viewpoint movement control unit 24 determines NO in step S209, it returns to step S201. Therefore, by returning to step S201, the user can continue to perform the operation of identifying the point on the video that they want to move to.

[0039] If the determination unit 22 determines in step S207 that the point to be moved to is a door section that can be opened and closed, and the user proceeds to step S211, the identification display unit 23 displays the trajectory of the movement path on the video in a third color (for example, green) from the user's perspective toward the door section that can be opened and closed.

[0040] Next, in step S212, the viewpoint movement control unit 24 determines whether or not it has received a viewpoint movement command from the operation unit 10 while the trajectory in the third color is displayed. If the viewpoint movement control unit 24 determines YES in step S212, it proceeds to step S213, transitions the door to the open state, displays the image on the display unit 30 after moving the user's viewpoint to a movable point outside the door, and then returns to step S201 (see Figure 9).

[0041] On the other hand, if the viewpoint movement control unit 24 determines NO in step S212, it returns to step S201. Therefore, by returning to step S201, the user can continue to perform the operation of identifying the point on the video that they want to move to.

[0042] Furthermore, in the decision-making process in step S212, it is also possible to adopt a configuration in which a door open command is output from the operation unit 10 instead of a viewpoint movement command.

[0043] A specific example of the video display device of this embodiment 1 is a simulation device for learning how to evacuate during disasters such as fires and earthquakes. In the simulation device, a map of a building or city is provided, and in the event of a disaster such as a fire, the user can experience in a virtual space checking evacuation routes to a safe point on the map.

[0044] Next, I will explain in more detail what was described in the flowchart in Figure 2. Here, I will discuss the method for determining whether a point in an image is movable or immovable.

[0045] The image displayed by the display unit 30 has three designated areas, as shown below. These three areas are not visible on the image and are controlled by the display control processing unit 20 using a microcontroller or the like. The first is an area called "immovable terrain." This indicates terrain that does not move during gameplay once it is placed. For example, "ground," "clusters of buildings," and "floors, walls, and ceilings inside buildings" are all objects that look like objects but do not move from their location and are considered "immovable terrain."

[0046] The second type of area is called a "no-movement zone." This is an area where users playing the simulation are placed in locations where movement is prohibited, such as buildings, ceilings, and walls. While it is possible to define areas such as "immovable terrain" and "no-movement zones" in a two-dimensional plane within an image, in this embodiment, the image is defined by setting a rectangular box of arbitrary size, which represents a three-dimensional space, as the image, thereby setting the image's characteristics such as "immovable terrain" and "no-movement zones."

[0047] By stacking and arranging multiple rectangular boxes, it is possible to create complex shapes such as restricted areas. These "restricted areas" are visually transparent and cannot be seen.

[0048] The third area is called an "Action" area, which is an area placed to trigger special actions other than movement (for example, opening a door). Note that every area in the image is assigned at least one of these characteristics: "Immovable Terrain," "No Movement Zone," and "Action." Depending on the location in the image, multiple characteristics may be assigned, for example, two of them: "Immovable Terrain" and "No Movement Zone."

[0049] For example, in the area of ​​the building cluster shown in Figure 6, a fixed terrain consisting of slightly smaller blocks may be set, and a no-movement zone consisting of blocks covering the entire fixed terrain block may also be set. Even if such zones overlap, the system determines the zone where the user's pointer first touches, so multiple zones will not be determined simultaneously.

[0050] For "immovable terrain" that you want to be movable, you need to place "movable areas" on top of it using rectangular blocks, similar to "immovable areas." If no "movable areas" are set and only "immovable terrain" is placed, the determination unit 22 of the display control processing unit 20 will not recognize it as a movable location, and therefore, it will not be possible to move to that location.

[0051] Now, let's explain the judgment process during movement. The judgment process when the pointer specified by the controller, which is the operation unit 10, touches one of the following areas: "Immovable Terrain," "No Movement Zone," or "Action" is as follows (1) to (3).

[0052] (1) When the pointer from the control unit 10 hits "immovable terrain" If the tip of the parabola trajectory emanating from the control unit 10 hits an immovable terrain, the system checks whether that terrain is a no-movement zone or an action zone. If it is neither, the system proceeds to the next step. From the point where the immovable terrain was hit, the system checks whether there are any movable locations within a certain distance in each of the X, Y, and Z axes. If there are movable locations, those locations are recognized as movable locations. In other words, at the end of the movement detection process, a location is recognized as a movable location, and the pointer on the screen changes to blue.

[0053] Let me elaborate a bit on how movement is determined. The system checks if there are any movable locations within a certain distance range in each of the X, Y, and Z axes. In other words, it searches for coordinates that are determined to be movable within a range of, for example, 1 meter in each direction (up, down, left, right, forward, and backward) from where the user's pointer is located on the screen, and the closest movable coordinate is designated as the location to move to. Note that the aforementioned "movable area" must be set for the movement determination to be possible.

[0054] Furthermore, if the location is recognized as either a restricted area or an action zone, or if it is recognized that there are no movable locations within a certain distance in each of the XYZ axes, the pointed location will be recognized as an immovable location, and the screen pointer will change to red.

[0055] (2) When the pointer from the control unit 10 hits a "no movement zone" If the tip of the parabola trajectory emanating from the control unit 10 hits a no-movement zone, it is recognized as a location where movement is impossible. Therefore, at the end of the movement process, the pointer is changed to red to indicate that it is a location where movement is impossible.

[0056] (3) When the pointer from the control unit 10 hits "Action" When the tip of the parabola extending from the control unit 10 hits an action point, it is recognized as an immovable location and as an object capable of performing an action. At the end of the movement determination process, priority is given to whether the object capable of performing the action is available, rather than whether the location is immovable, and the pointer is changed to green. If the object capable of performing the action is unavailable, the color is determined by whether the location is movable, and the pointer is changed to red.

[0057] Examples of various display screens shown on the display unit 30 will be explained in detail using the first to 24th display screens summarized in Figures 3 to 12.

[0058] These display screen examples assume that the first color is blue, the second color is red, and the third color is green. While it may be difficult to distinguish them when viewed as a grayscale image as shown in Figures 3 to 12, on the actual screen, the first trajectory C1 is displayed in blue, the second trajectory C2 in red, and the third trajectory C3 in green, allowing the user to easily identify which area the designated destination belongs to based on the displayed color.

[0059] Figure 3 illustrates a case where the display unit 30 shows an image of an office with desks, chairs, and people arranged within it. In the first display screen, since the point selected by the user's operation unit 10 is a movable floor surface, the first trajectory C1 is displayed in blue, which is the first color.

[0060] The images shown in Figures 3 to 8, which will be explained later, are all diagrams consisting of two images, and basically show the same location, that is, the same image from the user's perspective. When the operation unit 10 is operated, a blue parabola (which may be a straight line in some cases) is displayed on one image to indicate that movement is possible. On the other image, a red parabola (which may be a straight line in some cases) is displayed to indicate that movement is impossible because the pointer operated by the operation unit 10 is in a restricted area.

[0061] On the other hand, in the second display screen, which shows an image of the same location as the first display screen, a second trajectory C2 is displayed in red, the second color, because the point selected by the user is an immovable part of the desk. Trajectories C1 and C2 are basically displayed as parabolas. The base end of the parabola starts from any point at the bottom of the screen, but depending on the operation of the control unit 10, the operator's hand may be displayed on the screen, as shown in the first display screen, and the parabola may be displayed from there.

[0062] Figure 4 illustrates a scenario where the display unit 30 shows an image of an office space divided into a foreground section and a background section by a partition. In the third display screen, since the point selected by the user is the first section in the foreground, a first trajectory C1 in the first color is displayed, indicating that it is movable.

[0063] On the other hand, in the fourth display screen, the point selected by the user is in the second section at the back, or more precisely, the selected point is in a "no-movement zone," so a second trajectory C2 in a second color is displayed as impossible to move to. When a blue trajectory C1 is displayed on the screen, it is possible to move (also called jumping or warping) to the point indicated by the tip of that parabola.

[0064] Here, the display screen is set so that even if you select a point quite far away with the pointer, you cannot move to it. This is to prevent the simulation's effectiveness from decreasing, as allowing instantaneous movement to distant locations would detract from the realism of the simulation and hinder the learning experience when simulating evacuation routes during a disaster.

[0065] Figure 5 illustrates the case where the display unit 30 shows an image of a corridor leading to an emergency staircase inside a building. In the fifth display screen, since the point selected by the user is a movable corridor surface, the first trajectory C1 in the first color is displayed. On the other hand, in the sixth display screen, since the point selected by the user is an immovable wall section, the second trajectory C2 in the second color is displayed.

[0066] Furthermore, the identification display unit 23 can also display guidance messages such as "Please head towards the emergency stairs" as appropriate, as shown in the fifth and sixth display screens.

[0067] Figure 6 illustrates a case where the display unit 30 shows an image with a sidewalk in the foreground and a cluster of buildings beyond a road in the background. In the seventh display screen, since the point selected by the user is a movable sidewalk, the first trajectory C1 in the first color is displayed. On the other hand, in the eighth display screen, since the point selected by the user is an immovable building, the second trajectory C2 in the second color is displayed.

[0068] As mentioned earlier, although not visible to the naked eye, the display control processing unit 20 has set a "no-movement zone" in the area where the buildings are displayed. On the other hand, the blue display on the seventh display screen, which indicates that movement is possible, is because the pointed point is "immovable terrain," and it has been determined that there is a movable area within a certain distance in each of the XYZ axes from that pointed location, so the pointer turns to the first color (blue).

[0069] Figure 7 illustrates the case where the display unit 30 shows video of the user moving along a sidewalk. In the ninth display screen, since the point selected by the user is a movable sidewalk, the first trajectory C1 in the first color is displayed. On the other hand, in the tenth display screen, since the point selected by the user is an immovable building, the second trajectory C2 in the second color is displayed.

[0070] Figure 8 illustrates the case where the display unit 30 shows video of a person walking across a pedestrian crossing. In the 11th display screen, the first trajectory C1 in the first color is displayed because the point selected by the user is a pedestrian crossing that is movable. On the other hand, in the 12th display screen, the second trajectory C2 in the second color is displayed because the point selected by the user is a part that interferes with a person that is immovable.

[0071] Figure 9 illustrates a case where the display unit 30 shows the video of the door transitioning from a closed state to an open state and moving to the outside of the door. In the 13th display screen, since the point selected by the user is the door portion which cannot be moved, a third trajectory C3 in the third color, green, is displayed.

[0072] Thus, when the tip of the parabola extending from the control unit 10 hits an object called a door, which is an "action," the system recognizes that it is a place where movement is impossible and that there is an object that can perform an action. At the end of the movement determination process, the system prioritizes recognizing the object that can perform an action as usable over the place where movement is impossible, and changes the pointer to green.

[0073] Meanwhile, on the 14th display screen, the third trajectory C3 in the third color is displayed, and the door is shown to be open because the user has outputted a command to open the door. Furthermore, on the 14th display screen, with the door open, a movable point outside the door becomes selectable, and the first trajectory C1 in the first color is displayed.

[0074] On the 15th display screen, the user is shown having moved to the movable location selected on the 14th display screen. Through this series of operations, the user can pass through the openable door section and move to the outside of the door. In this way, the user can escape from the room where the fire has occurred (see Figure 11).

[0075] Furthermore, the identification display unit 23 can also display guidance messages such as "Please head towards the emergency stairs" as appropriate, as shown in the 14th and 15th display screens. Once the user has reached the emergency stairs, the simulation ends, indicating that the evacuation is complete.

[0076] Figure 10 illustrates, in chronological order, the state after sequentially moving to movable locations based on manual user operation. The state after moving to the location selected in the 16th display screen is shown in the 17th display screen, and the state after moving to the location selected in the 17th display screen is shown in the 18th display screen.

[0077] In other words, the 17th display screen is the view from the user's perspective after moving to the location selected in the 16th display screen, and the 18th display screen is the view from the user's perspective after moving to the location selected in the 17th display screen (the tip of the blue trajectory C1). In this way, the user can instantly move to any desired location within the virtual space, have a simulated experience at each location, and instantly select the next location they want to move to.

[0078] In simulation devices for disasters such as fires and earthquakes, users are given a map of a building or city of a certain size, and they operate controls on this vast map to move to a safe location. Therefore, if it takes a long time to move from a room where a fire has broken out to the emergency staircase while wearing a head-mounted display, the user will be operating the controls for a long time, which can easily lead to so-called VR sickness.

[0079] In this embodiment, when moving on the map, the user can sequentially specify the desired location on the displayed screen and move by jumping (warping). This allows for quick movement on the given map and prevents the operation time from becoming unnecessarily long, thus reducing the likelihood of VR sickness.

[0080] Figure 11 illustrates the conditions of a fire occurring indoors, viewed from three different locations. After moving to various locations, the user can view the same fire from different perspectives, as shown in Display Screens 19 to 21, and experience the fire situation from various viewpoints.

[0081] Furthermore, the identification display unit 23 can also display guidance messages such as "Please head towards the corridor" as appropriate, as shown in the 19th and 20th display screens.

[0082] Figure 12 illustrates the state of a fire as seen from three different viewpoints at different heights. In particular, when smoke is generated indoors, the field of view differs completely depending on the height, even at the same location. Therefore, the display control processing unit 20 acquires information about the eye-line height of the user wearing the head-mounted display and performs processing to shift the image displayed on the display unit 30 according to the eye-line height. As a result, as shown in display screens 22 to 24, the user can experience a simulated fire from various viewpoint heights at the same location.

[0083] Furthermore, the identification display unit 23 can also display guidance messages such as "Please head towards the emergency stairs" as appropriate, as shown in the 14th and 15th display screens.

[0084] In the display screen examples shown in Figures 3 to 12 above, movable and immovable points are distinguished using different colors. However, it is also possible to distinguish them using different line types, different shapes, different thicknesses, etc., instead of different colors.

[0085] Furthermore, it is conceivable to differentiate the shape of the tip of the trajectory, which indicates the destination point, for movable and immovable points. For example, for movable points, as shown in the 14th display screen of Figure 9, an arrow can be added to the tip to further emphasize that it is movable.

[0086] In this regard, when movement is not possible, the tip of the pointer is displayed as just a flattened circle, and the trajectories C1 and C2 are made to differ not only in color but also in shape, so that the user can see at a glance whether movement is possible or not.

[0087] Furthermore, even in locations where movement is impossible, when displaying the third trajectory, which represents the openable / closable door section, the fact that it is openable / closable can be further emphasized by adding an ellipse or other indicator to the tip, as shown in the 13th display screen of Figure 9.

[0088] As described above, Embodiment 1 provides a configuration that allows the user to instantly move to a movable location and to an immovable location, and to view the image from a different viewpoint, for a movement trajectory to a movable location selected based on the user's manual operation.

[0089] As a result, it is possible to create a video display that not only simulates moving through a virtual space at real-world speeds, but also allows users to instantly move to accessible locations and experience the virtual space from different perspectives.

[0090] In the above-described Embodiment 1, an example was given of a simulation device using a head-mounted display to provide a realistic fire simulation experience. However, the video display technology described herein is not limited to this example. It can be applied to various applications that instantly move to a movable location based on user input and provide video displays from different viewpoints. Furthermore, it is possible to use display devices other than head-mounted displays as the display unit.

[0091] Furthermore, while Embodiment 1 described above explained a video display device having the configuration shown in Figure 1 and a video processing method that performs the series of processing steps shown in Figure 2, the disclosure is not limited thereto. This disclosure also includes providing a video display program to cause a computer to function with the configuration shown in Figure 1. [Explanation of Symbols]

[0092] 10 Operation unit, 20 Display control processing unit, 21 Pointer display unit, 22 Judgment unit, 23 Identification display unit, 24 Viewpoint movement control unit, 30 Display unit.

Claims

1. A video display device that allows the user to move their viewpoint on the displayed image, An operation unit that generates a movement point identification command to identify the point to be moved to on the video, and a viewpoint movement command to move the user's viewpoint on the video, based on the user's manual operation, When the operation unit receives the command to specify the move location, the pointer display unit displays the desired move location on the video as a pointer, A determination unit determines whether the point the user wants to move to, as indicated by the pointer, is a walkable location on the video, thereby determining whether the point is a place the user can move to or a place that is not. If the determination unit determines that the point to be moved is a movable point, the identification display unit displays a first trajectory on the video from the user's viewpoint toward the movable point, and if the determination unit determines that the point to be moved is an immovable point, the identification display unit displays a second trajectory on the video from the user's viewpoint toward the immovable point, which is distinguishable from the first trajectory. A viewpoint movement control unit, upon receiving the viewpoint movement command, performs control to move the user's viewpoint to the movable point selected based on the user's manual operation, or to maintain the user's current viewpoint without moving the viewpoint. A head-mounted display worn by the user allows the user to view images from any desired viewing direction and height by moving their head to shift their gaze direction and height. Equipped with, The viewpoint movement control unit, When the first trajectory is displayed and the viewpoint movement command is received, the user's viewpoint is moved to the movable point. When the second trajectory is displayed and the viewpoint movement command is received, the current viewpoint position and height of the user are maintained without moving the user's viewpoint, and the manual operation for the user to identify the point on the video to be moved to is continuously accepted. Users can move to various movable locations and view the same fire from different perspectives, thereby simulating the fire's state from various viewpoints, as well as from various viewpoint heights at the same location. Video display device.

2. On the displayed image, there are restricted areas where movement is prohibited and accessible areas where movement is permitted. These restricted and accessible areas are represented by stacking rectangular boxes of arbitrary size on the image, creating a three-dimensional space. The determination unit determines whether the location of the pointer specified by the user by the operation unit is either a restricted area where movement is prohibited or a movable area where movement is permitted. The video display device according to claim 1.

3. The aforementioned operating unit generates an open command to transition a door that can be opened or closed from a closed state to an open state based on the user's manual operation. The determination unit determines whether the point to be moved to, indicated by the pointer, is the location of a door that can be opened and closed. If the determination unit determines that the point to be moved to is the location of the openable door, the identification display unit displays a third trajectory on the image that is identifiable from the first and second trajectories toward the location of the openable door from the user's viewpoint. When the viewpoint movement control unit receives the open command while the third trajectory is displayed, it opens the openable door and moves the user's viewpoint to the point that becomes accessible as a result of the door being opened. The video display device according to claim 1 or 2.

4. A video display method performed by a computer when the user's viewpoint is moved on the displayed video, The aforementioned computer, When a command to specify a destination is received, which specifies a destination on the video screen through manual operation by the user, the process includes displaying the destination on the video screen with a pointer, The process involves determining whether the point to be moved to, as indicated by the pointer, is a walkable location on the video, thereby determining whether the point is a location the user can move to or not. If it is determined that the point to be moved to is a movable point, the process includes displaying a first trajectory on the video from the user's viewpoint toward the movable point, If it is determined that the point to be moved to is an immovable point, the process includes displaying a second trajectory on the video that is identifiable as the first trajectory, from the user's viewpoint toward the immovable point, The process of receiving a viewpoint movement command via manual operation by the user, and then either moving the user's viewpoint to the movable point selected based on the user's manual operation, or maintaining the user's current viewpoint without moving the viewpoint, The process involves enabling a user wearing a head-mounted display to view images from any desired viewing direction and height by moving their head to shift their gaze direction and height. Execute, The process of performing the aforementioned control is: When the first trajectory is displayed and the viewpoint movement command is received, the user's viewpoint is moved to the movable point. When the second trajectory is displayed and the viewpoint movement command is received, the current viewpoint position and height of the user are maintained without moving the user's viewpoint, and the manual operation for the user to identify the point on the video to be moved to is continuously accepted. Users can move to various movable locations and view the same fire from different perspectives, thereby simulating the fire's state from various viewpoints, as well as from various viewpoint heights at the same location. Video display method.

5. A video display program that moves the user's viewpoint on the displayed image, Computers, An operation unit that generates a movement point identification command to identify the point to be moved to on the video, and a viewpoint movement command to move the user's viewpoint on the video, based on the user's manual operation, When the operation unit receives the command to specify the move location, the pointer display unit displays the desired move location on the video as a pointer, A determination unit determines whether the point the user wants to move to, as indicated by the pointer, is a walkable location on the video, thereby determining whether the point is a place the user can move to or a place that is not. If the determination unit determines that the point to be moved is a movable point, the identification display unit displays a first trajectory on the video from the user's viewpoint toward the movable point, and if the determination unit determines that the point to be moved is an immovable point, the identification display unit displays a second trajectory on the video from the user's viewpoint toward the immovable point, which is distinguishable from the first trajectory. A viewpoint movement control unit, upon receiving the viewpoint movement command, performs control to move the user's viewpoint to the movable point selected based on the user's manual operation, or to maintain the user's current viewpoint without moving the viewpoint. A head-mounted display worn by the user allows the user to view images from any desired viewing direction and height by moving their head to shift their gaze direction and height. A video display program to enable the function, The viewpoint movement control unit, When the first trajectory is displayed and the viewpoint movement command is received, the user's viewpoint is moved to the movable point. When the second trajectory is displayed and the viewpoint movement command is received, the current viewpoint position and height of the user are maintained without moving the user's viewpoint, and the manual operation for the user to identify the point on the video to be moved to is continuously accepted. Users can move to various movable locations and view the same fire from different perspectives, thereby simulating the fire's state from various viewpoints, as well as from various viewpoint heights at the same location. Video display program.

Images