Virtual experience system and program

The virtual experience system using head-mounted displays and sensors allows users to simulate fire visibility with adjustable smoke density, addressing the limitations of large-scale fire training devices.

JP2024165708AActive Publication Date: 2024-11-28NOHMI BOSAI LTD
View PDF 6 Cites 0 Cited by

Patent Information

Application Number
JP2023082117
Authority / Receiving Office
JP · JP
Patent Type
Applications
Current Assignee / Owner
Filing Date
2023-05-18
Publication Date
2024-11-28
Estimated Expiration
2043-05-18

AI Technical Summary

Technical Problem

Conventional fire training devices are large-scale and not suitable for easy and flexible experience of a fire scene.

Method used

A virtual experience system using head-mounted displays, sensors, and a specifying unit to simulate smoke density and direction, allowing users to experience fire visibility through augmented reality.

Benefits of technology

Enables users to easily and flexibly experience fire visibility without the need for large-scale equipment, with adjustable smoke density and stereoscopic viewing.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 2024165708000001_ABST
    Figure 2024165708000001_ABST
Patent Text Reader

Abstract

To allow a user to easily virtually experience a field of vision during a fire.SOLUTION: A virtual experience system according to the present invention is to allow a user to virtually experience a field of vision during a fire, and the virtual experience system comprises: a head-mounted display; one or more sensors for detecting the direction of the head-mounted display; a specification unit that specifies the density of smoke selected by the user; and a display control unit that displays, on the head-mounted display, an image of the smoke according to the direction detected by the one or more sensors and the density specified by the specification unit.SELECTED DRAWING: Figure 1
Need to check novelty before this filing date? Find Prior Art

Description

[Technical field]

[0001] The present invention relates to a system and a program for allowing a user to virtually experience visibility during a fire. [Background technology]

[0002] Conventionally, devices for firefighting personnel and the like to conduct fire training have been known. For example, Patent Document 1 describes a real fire training device. This real fire training device is a device that allows trainees to experience thick smoke and hot air with a sense of realism equivalent to that of a fire scene, and includes a main body and a smoke treatment device. The main body is in the shape of a large casing, the inside of which serves as a training booth for trainees, and a combustion smoke generation corner that generates thick smoke and hot air is provided at the back side, and a container door with an outside air intake is provided at the opposite side. The smoke treatment device is provided above the main body and includes a burner, a smoke combustion area, a smoke blower, etc., and thick smoke is introduced from the main body as smoke, and is completely burned to be smokeless and harmless before being released into the atmosphere. This real fire training device allows trainees to fully experience the sense of realism of a fire scene. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-34024 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the above-mentioned real fire training equipment is a large-scale device and is not suited to the need to allow people to easily experience a fire scene. The present invention has been made in consideration of the above circumstances, and has an object to allow a user to easily virtually experience visibility during a fire. [Means for solving the problem]

[0005] In order to solve the above problems, the virtual experience system of the present invention is a virtual experience system for allowing a user to virtually experience the view during a fire, and includes a head-mounted display, one or more sensors for detecting the orientation of the head-mounted display, an identification unit for identifying the smoke concentration selected by the user, and a display control unit for causing the head-mounted display to display an image of smoke corresponding to the orientation detected by the one or more sensors and the concentration identified by the identification unit. Effect of the Invention

[0006] According to the present invention, it is possible to allow a user to easily virtually experience visibility during a fire. [Brief description of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of a virtual experience system 100 . [Diagram 2] FIG. 2 shows an example of the configuration of the smart glasses 101. [Diagram 3] FIG. 3 shows an example of a virtual space. [Figure 4] FIG. 4 shows an example of a virtual experience process. [Diagram 5] FIG. 5 shows an example of a smoke image with a smoke density of 5% / m. [Figure 6] FIG. 6 shows an example of a smoke image with a smoke density of 10% / m. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] 1. Example An embodiment of the present invention will now be described with reference to the drawings. 1-1.Configuration FIG. 1 illustrates an example of a virtual experience system 100 according to an embodiment of the present invention. The virtual experience system 100 is a system for allowing a user to virtually experience visibility during a fire. The virtual experience system 100 is used, for example, in evacuation drills.

[0009] This virtual experience system 100 consists of smart glasses 101 and a controller 102. The smart glasses 101 are head-mounted display devices, and more specifically, are glasses-type wearable terminals. The smart glasses 101 use augmented reality (AR) technology to display an image of smoke superimposed on a real landscape.

[0010] The controller 102 is a portable terminal device for operating the smart glasses 101. The controller 102 includes an operation input device such as a touch panel. The controller 102 is communicatively connected to the smart glasses 101 via wired or wireless communication and outputs an operation signal to the smart glasses 101. The smart glasses 101 are described in more detail below.

[0011] FIG. 2 is a block diagram showing an example configuration of the smart glasses 101. The smart glasses 101 include a main memory device 201 such as a RAM, an auxiliary memory device 202 such as a flash memory, a processor 203 such as a CPU, an input device 204 such as various sensors and operation buttons, an output device 205 such as a display and a speaker, and a communication control unit 206 such as a wireless communication module.

[0012] Among these, the input device 204 includes one or more sensors 207 for detecting the position and orientation of the smart glasses 101. The one or more sensors 207 are, for example, an acceleration sensor, a geomagnetic sensor, and an angular velocity sensor. If these sensors are used, the position of the smart glasses 101 is estimated by autonomous navigation based on the output values ​​of the acceleration sensor and the geomagnetic sensor. On the other hand, the orientation of the smart glasses 101 is estimated based on the output value of the angular velocity sensor.

[0013] The output device 205 includes an optically transmissive display 208. This optically transmissive display 208 is made up of a sub-display for the left eye and a sub-display for the right eye. The left-eye sub-display displays an image for the left eye, and the right-eye sub-display displays an image for the right eye. By displaying images with parallax on these sub-displays, the user can visually recognize a three-dimensional image.

[0014] Various programs are stored in the main memory device 201. These programs are programs that can be distributed via a non-transitory storage medium or a network such as the Internet. These programs include a virtual experience program. This virtual experience program is a program for allowing a user to virtually experience the field of view during a fire. Various functions are realized by the processor 203 executing this virtual experience program. The realized functions include a virtual space generation unit 211, an object control unit 212, a virtual camera control unit 213, a field of view image generation unit 214, a display control unit 215, and an operation reception unit 216. Each function will be described below.

[0015] The virtual space generation unit 211 generates a virtual space based on the virtual space information 221. Fig. 3 shows an example of the generated virtual space. The virtual space 301 shown in the figure is a hemispherical space.

[0016] The object control unit 212 places a smoke object in the virtual space based on the object information 222. The smoke object to be placed is a three-dimensional image made up of a plurality of particles. A smoke object 302 shown in FIG. 3 is an example of a smoke object to be placed in the virtual space 301. When a smoke density is selected by the user, the object control unit 212 specifies the selected smoke density. In this embodiment, the object control unit 212 specifies the smoke density based on an operation signal output from the controller 102. Then, the object control unit 212 places a smoke object of the specified smoke density in the virtual space.

[0017] The virtual camera control unit 213 places a virtual camera in the virtual space. The virtual camera 303 shown in FIG. The virtual camera control unit 213 controls the virtual camera in conjunction with the movement and tilt of the smart glasses 101. To this end, the virtual camera control unit 213 identifies the position and orientation of the smart glasses 101 based on the output value of the sensor 207. The virtual camera control unit 213 then changes the position and orientation of the virtual camera in the virtual space according to the identified position and orientation. For example, when the user moves forward and faces to the right, the virtual camera control unit 213 moves the virtual camera forward and faces to the right.

[0018] The field of view image generator 214 identifies the user's field of view based on the position and orientation of a virtual camera arranged in the virtual space. The field of view 304 shown in Fig. 3 is an example of the identified user's field of view. After identifying the user's field of view, the field of view image generating unit 214 generates an image of the smoke object included in the identified field of view. The image generated here is a field of view image. The generated field of view image is made up of two images, an image for the left eye and an image for the right eye. These two images are provided with parallax so that stereoscopic viewing is possible.

[0019] The display control unit 215 causes the display 208 to display the field of view image generated by the field of view image generation unit 214. If a smoke object is included in the user's field of view, the displayed field of view image is an image including smoke. The smoke included in this image corresponds to the position and orientation of the smart glasses 101 and the smoke density selected by the user. As a result of the field of view image being displayed, if a smoke object is included in the user's field of view, the user will see smoke superimposed on the scenery.

[0020] The operation reception unit 216 receives an operation signal output from the controller 102. The received operation signal includes a change instruction for changing the density of the smoke to be displayed and an end instruction for ending the virtual experience program.

[0021] Next, the auxiliary storage device 202 will be described. The auxiliary storage device 202 stores virtual space information 221 and object information 222. Of these, the virtual space information 221 is definition information of a virtual space. On the other hand, the object information 222 is information about a smoke object placed in a virtual space. Specifically, the information is image information of the smoke object and information indicating the position and orientation when placed in a virtual space.

[0022] The smoke object is a three-dimensional image made up of a plurality of particles. In this embodiment, two types of smoke objects are stored. The first smoke object is a smoke object representing smoke with a smoke density of 5% / m, and the second smoke object is a smoke object representing smoke with a smoke density of 10% / m. These two types of smoke objects are switched and displayed according to the user's selection.

[0023] The smoke represented by the smoke object may have uneven density. The smoke represented by the smoke object may become thicker as it approaches the source of the fire. When such a smoke object is placed in a virtual space, the density of the smoke will change depending on the position in the virtual space and the line of sight.

[0024] 1-2.Operation Next, the virtual experience processing executed by the smart glasses 101 will be described. FIG. 4 is a flow diagram showing an example of the virtual experience processing executed by the smart glasses 101. To start the virtual experience processing 400 shown in the figure, the user wears the smart glasses 101 on his / her head. Then, the user operates the controller 102 to instruct the smart glasses 101 to execute a virtual experience program. The processor 203 of the smart glasses 101 receives this execution instruction and executes the virtual experience program. As a result, the virtual experience processing 400 is started.

[0025] After starting this process, first, the virtual space generation unit 211 generates a virtual space based on the virtual space information 221 (step 401). The virtual space generated here is a hemispherical space, as exemplified in FIG.

[0026] Next, the object control unit 212 places a smoke object in the generated virtual space based on the object information 222 (step 402). The smoke object placed here is a smoke object when the smoke density is 5% / m.

[0027] Next, the virtual camera control unit 213 places a virtual camera facing a predetermined direction at an initial position in the generated virtual space (step 403). As an example, the virtual camera is placed at the center 305 of the virtual space as illustrated in FIG.

[0028] Next, the field of view image generator 214 identifies the user's field of view based on the position and orientation of the virtual camera. The field of view image generator 214 then generates an image of the smoke object included in the identified field of view (step 404). The image generated here is a field of view image. The generated field of view image is made up of two images, an image for the left eye and an image for the right eye. These two images are provided with parallax so that stereoscopic viewing is possible.

[0029] Next, the display control unit 215 causes the display 208 to display the generated field of view image (step 405). As a result, if a smoke object is included in the user's field of view, the user visually recognizes smoke with a smoke density of 5% / m superimposed on the scenery.

[0030] Figure 5 shows an example of a smoke image with a smoke density of 5% / m. The smoke image shown in the figure is superimposed on a landscape. The smoke image obscures the landscape.

[0031] Next, the virtual camera control unit 213 identifies the position and orientation of the smart glasses 101 based on the output value of the sensor 207 (step 406). Then, the virtual camera control unit 213 changes the position and orientation of the virtual camera in the virtual space according to the identified position and orientation (step 407). For example, when the user moves forward and faces right, the virtual camera control unit 213 moves the virtual camera forward and faces right.

[0032] Next, the field of view image generator 214 identifies the user's field of view based on the changed position and orientation of the virtual camera. The field of view image generator 214 then generates an image of the smoke object included in the identified field of view (step 408). The image generated here is a field of view image. The generated field of view image is made up of two images, an image for the left eye and an image for the right eye. These two images are provided with parallax so that stereoscopic viewing is possible.

[0033] Next, the display control unit 215 causes the display 208 to display the generated field of view image (step 409). As a result, if a smoke object is included in the user's field of view, the user visually recognizes smoke with a smoke density of 5% / m superimposed on the scenery.

[0034] Here, the user can change the smoke density to 10% / m by operating the controller 102. When the user performs an operation to change the smoke density to 10% / m, the controller 102 instructs the smart glasses 101 to change the smoke density.

[0035] The operation reception unit 216 of the smart glasses 101 receives the change instruction output from the controller 102 (YES in step 410). When the change instruction is received, the object control unit 212 changes the smoke object based on the object information 222 (step 411). Specifically, the object control unit 212 places a smoke object with a smoke density of 10% / m in the virtual space, instead of the smoke object with a smoke density of 5% / m.

[0036] Next, the field of view image generator 214 identifies the user's field of view based on the position and orientation of the virtual camera. The field of view image generator 214 then generates an image of the smoke object included in the identified field of view (step 412). The image generated here is a field of view image. The generated field of view image is made up of two images, an image for the left eye and an image for the right eye. These two images are provided with parallax so that stereoscopic viewing is possible.

[0037] Next, the display control unit 215 causes the display 208 to display the generated field of view image (step 413). As a result, if a smoke object is included in the user's field of view, the user visually recognizes smoke with a smoke density of 10% / m superimposed on the scenery.

[0038] Figure 6 shows an example of a smoke image with a smoke density of 10% / m. The smoke image shown is overlaid on a landscape. The smoke in this image is more dense than the 5% / m smoke, making it more difficult to see the landscape.

[0039] In addition, in the above step 410, if the operation receiving unit 216 does not receive a change instruction from the controller 102 (NO in step 410), steps 411 to 413 are skipped.

[0040] Here, the user can end the virtual experience program by operating the controller 102. When the user performs an operation to end the virtual experience program, the controller 102 instructs the smart glasses 101 to end the program.

[0041] The operation acceptance unit 216 of the smart glasses 101 accepts the end instruction output from the controller 102 (YES in step 414). When this end instruction is accepted, the virtual experience processing 400 ends.

[0042] On the other hand, in step 414, if the operation reception unit 216 does not receive an end instruction from the controller 102 (NO in step 414), step 406 is executed again. The above is a description of the virtual experience process 400.

[0043] According to the above-described virtual experience system 100, a user can virtually experience the view during a fire. Since the virtual experience system 100 is not a large-scale device like the real fire training device described in the above-mentioned Patent Document 1, a user can easily virtually experience the view during a fire.

[0044] Furthermore, the user can freely change the density of the smoke to be displayed by operating the controller 102. This allows the user to virtually experience how the scenery looks different depending on the density of the smoke.

[0045] 2. Variations The above embodiment may be modified as follows. The following modifications may be combined with each other. (1) In the above embodiment, the smart glasses 101 alone execute all of the virtual experience processing 400 shown in Fig. 4. Alternatively, a server for image processing may be prepared, and some or all of the steps of the virtual experience processing 400 may be executed by the server. That is, the smart glasses 101 may execute the virtual experience processing 400 in cooperation with the server for image processing.

[0046] (2) In the above embodiment, smart glasses are used as hardware for providing a virtual experience to a user. However, smart glasses are merely an example. Instead of smart glasses, a head mounted display (HMD) may be used. When an HMD is used, the display method may be any of a non-transparent type, an optically transparent type, and a video transparent type.

[0047] (3) In the above embodiment, as an example, it is assumed that the position and orientation of the smart glasses 101 are estimated based on the output value of an acceleration sensor or the like. However, this method is merely an example. As another method, position tracking may be adopted. In this case, an outside-in method (a method in which a camera is installed outside the smart glasses 101 and the smart glasses 101 are tracked by observing the smart glasses 101 with the camera) may be adopted, or an inside-out method (a method in which a camera is installed in the smart glasses 101 and the smart glasses 101 are tracked by observing the outside world with the camera) may be adopted.

[0048] (4) In the above embodiment, two types of smoke density, 5% / m and 10% / m, can be selected. However, these smoke densities are merely examples. Other smoke densities may be selectable. Also, three or more types of smoke densities may be selectable.

[0049] (5) In the above embodiment, the virtual camera is controlled according to the position and orientation of the smart glasses 101. However, position information is not necessarily required. In a scenario in which the user does not move, the virtual camera may be controlled according to only the orientation of the smart glasses 101.

[0050] (6) In the above embodiment, the smart glasses 101 are operated using the controller 102. Alternatively, the smart glasses 101 may be equipped with an operation means, and the smart glasses 101 may be operated using the operation means. For example, a slide switch may be equipped as the operation means, and the smoke density may be switched using the slide switch. In this case, the object control unit 212 identifies the smoke density based on the operation using the operation means.

[0051] As another example, the smart glasses 101 may be equipped with a microphone and may be capable of accepting voice operations. Alternatively, the smart glasses 101 may be equipped with a camera and may be capable of accepting gesture operations.

[0052] (7) The smoke object may be an animated image. That is, the smoke object may be an image that changes over time. For example, the smoke object may be an image of smoke that forms a layer at the top of the virtual space when a fire breaks out, descends downward over time, and finally spreads evenly throughout the virtual space.

[0053] (8) The smart glasses 101 according to the above embodiment are intended to be used during evacuation drills in peacetime. The smart glasses 101 may also be used for evacuation support in the event of a fire. Specifically, a scenic image of the interior of a building in peacetime is recorded in the smart glasses 101 in association with position information. Then, in the event of a fire, the scenic image of the interior of the building in peacetime is displayed on the display 208 according to the position and orientation of the smart glasses 101. This allows the user to evacuate from the building by relying on the scenic image in peacetime even if smoke obstructs the view during a fire.

[0054] In addition to the scenery images during normal times, audio evacuation guidance may be recorded in the smart glasses 101 in association with location information, so that evacuation guidance according to the current location can be output as audio.

[0055] (9) Note that the present invention is not limited to the above-described embodiments, and various modifications are included. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and the present invention is not necessarily limited to those having all of the configurations described. It is also possible to replace a part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. It is also possible to add, delete, or replace a part of the configuration of each embodiment with another configuration.

[0056] In addition, the above-mentioned configurations, functions, processing units, processing means, etc. may be realized in part or in whole by hardware, for example, by designing them as integrated circuits. In addition, the above-mentioned configurations, functions, etc. may be realized in software by a processor interpreting and executing a program that realizes each function. Information such as the program, table, file, etc. that realizes each function can be stored in a memory, a recording device such as a hard disk or SSD (Solid State Drive), or a recording medium such as an IC card, SD card, or DVD.

[0057] In addition, the control lines and information lines shown are those that are considered necessary for the explanation, and not all control lines and information lines in the product are necessarily shown. In reality, it can be considered that almost all components are connected to each other. The above-described embodiments disclose at least the configurations described in the claims. [Explanation of symbols]

[0058] 100...virtual experience system, 101...smart glasses, 102...controller, 211...virtual space generation unit, 212...object control unit, 213...virtual camera control unit, 214...field of view image generation unit, 215...display control unit, 216...operation reception unit, 221...virtual space information, 222...object information

Claims

1. A virtual experience system for allowing a user to virtually experience visibility during a fire, comprising: A head-mounted display; one or more sensors for detecting an orientation of the head mounted display; An identification unit that identifies a smoke concentration selected by the user; a display control unit that causes the head-mounted display to display an image of smoke corresponding to the direction detected by the one or more sensors and the concentration identified by the identification unit; A virtual experience system comprising:

2. The one or more sensors further detect a position of the head mounted display; The display control unit causes the display to display an image of smoke corresponding to the direction and position detected by the one or more sensors and the concentration identified by the identification unit.

2. The virtual experience system according to claim 1 .

3. The virtual experience system according to claim 1 , wherein the determination unit determines the concentration of the smoke based on an operation signal output from a controller communicatively connected to the head-mounted display.

4. The head mounted display further comprises an operating means for selecting a smoke concentration; The virtual experience system according to claim 1 , wherein the specifying unit specifies the density of the smoke based on an operation using the operating means.

5. 2. The virtual experience system according to claim 1, wherein the image of smoke is an image that changes over time.

6. A program for allowing a user to virtually experience visibility during a fire, On the computer, determining an orientation of a head mounted display; identifying a smoke density selected by the user; displaying an image of smoke corresponding to the identified direction and the identified concentration on the head mounted display; A program for executing.

Citation Information

Patent Citations

  • Image processing apparatus

    JP2018124624A

  • Information processing method, apparatus, and program for implementing that information processing method in computer

    JP2018192107A

  • Information processing device, display method, and computer program

    JP2020174329A

  • Augmented reality display integrated with self-contained breathing apparatus

    US20020039085A1

  • Virtual reality fire-fighting experience system

    US20200030651A1