Booting display system and server for booting display system

Abstract

The present disclosure relates to a guidance display system and a server of the guidance display system. The guidance display system includes an AR display device and a server. The AR display device includes a display unit and a display control unit capable of causing the display unit to display an augmented reality image obtained by superimposing an image of a virtual guidance object on a real-world landscape. The server is wirelessly connected to the display device. Furthermore, the server includes a theme park-specific character storage unit, a 3D model extraction unit, and a sending unit. The theme park-specific character storage unit stores information on characters set for the theme park. The 3D model extraction unit extracts the character of the theme park set as the destination as a virtual guidance object. The sending unit sends the extracted image data of the virtual guidance object to the display control unit of the AR display device.

Description

Technical Field

[0001] A boot display system and a server thereof are disclosed in this specification. Background Art

[0002] Conventionally, there is known a guidance display device using augmented reality (AR) technology. For example, in Japanese Patent Application Laid-Open No. 2014-20863, an image of a character traveling along a route to a destination is superimposed on an image of the real world in a navigation system. Summary of the invention

[0003] Furthermore, for example, in a comprehensive amusement facility having a plurality of theme parks, characters are sometimes set for each theme park, and when a guide display is provided with a predetermined theme park as the destination, a character image that does not match the world view of the theme park may be displayed.

[0004] Therefore, this specification discloses a guidance display system and a server thereof that can perform guidance display using augmented reality images without damaging the world view of a destination facility such as a theme park.

[0005] In this specification, a guidance display system is disclosed, which uses a facility constructed based on a specific theme as a destination and performs guidance display to the destination. The system includes a display device and a server. The display device includes a display unit and a display control unit, and the display control unit can cause the display unit to display an augmented reality image obtained by superimposing an image of a virtual object for guidance on a scenery in the real world. The server is wirelessly connected to the display device. Furthermore, the server includes a storage unit, an extraction unit, and a sending unit. Information on a character set for a facility is stored in the storage unit. The extraction unit extracts the character of the facility set as the destination as a virtual object for guidance. The sending unit sends the extracted image data of the virtual object for guidance to the display control unit.

[0006] According to the above configuration, since a character corresponding to a destination facility (eg, a theme park) is displayed as a guidance virtual object, it is possible to provide guidance display without damaging the world view of the destination facility.

[0007] In the above configuration, the server may also include a user calculation unit and a route search unit. The user calculation unit calculates the user density of each of the plurality of facilities configured based on different themes. The route search unit sets the facility with the smallest user density at the destination setting time point among the plurality of facilities as the destination, and creates a guidance route to the destination.

[0008] According to the above configuration, by setting a relatively empty facility (for example, a theme park) as a destination, it is possible to smooth the congestion rate of each facility (ie, relieve congestion).

[0009] In the above configuration, the extracting unit may extract a character of a facility closest to the display device at the destination setting time as the farewell virtual object. In this case, the transmitting unit transmits image data of the extracted farewell virtual object to the display control unit.

[0010] According to the above configuration, in addition to the virtual object for guidance, the virtual object for farewell can also be displayed in the augmented reality image, thereby enabling, for example, a performance such as having the virtual object for farewell accompany the person on the way from the facility they are leaving to the next facility.

[0011] In the above configuration, the display device may also include a photographing unit for photographing scenery in the real world and an image recognition unit for recognizing objects included in the photographed image. In this case, when the image recognition unit recognizes the warning target object in the photographed image, the display control unit highlights the warning target object on the display unit.

[0012] According to the above configuration, it is possible to prevent a user of the display device from contacting (colliding with) the warning target object.

[0013] In the above configuration, the route search unit may create the guidance route so as to be a route for avoiding the warning target object.

[0014] According to the above configuration, it is possible to cause the user of the display device to avoid (get away from) the warning target object.

[0015] In the above configuration, when a plurality of display devices come within a predetermined threshold distance, the route search unit may recreate each guide route so as to avoid each display device.

[0016] According to the above configuration, it is possible to prevent users of the display device from coming into contact with each other.

[0017] In the above configuration, the display device may be a light-transmitting display having a half mirror, which is a mirror through which a landscape image of the real world is transmitted and on which a guiding virtual object is projected.

[0018] In addition, in this specification, a server of a guidance display system is disclosed, which is wirelessly connected to a display device, and the display device can display an augmented reality image obtained by superimposing an image of a virtual object for guidance on a landscape in the real world. The server includes a storage unit, an extraction unit, and a transmission unit. The storage unit stores information on characters set for facilities constructed based on a specific theme. The extraction unit extracts the character of the facility set as the destination as a virtual object for guidance. The transmission unit sends the extracted image data of the virtual object for guidance to the display device.

[0019] According to the guidance display system and the server thereof disclosed in this specification, it is possible to perform guidance display using augmented reality technology without impairing the world view of the destination facility. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Hereinafter, features, advantages, and technical and industrial significance of exemplary embodiments of the present invention will be described with reference to the accompanying drawings, in which like reference numerals represent like elements, and wherein:

[0021] Figure 1 This is a diagram illustrating an integrated entertainment facility including the guidance display system according to the present embodiment.

[0022] Figure 2 This is a diagram illustrating the hardware configuration of a display device and a server in the guidance display system according to the present embodiment.

[0023] Figure 3 This is a diagram illustrating the functional blocks (frames) of a server.

[0024] Figure 4 This diagram takes a smartphone as an example of a display device and illustrates reference coordinate axes of the orientation and direction detected by the smartphone.

[0025] Figure 5 The diagram illustrates the functional blocks of the display device.

[0026] Figure 6 This is a diagram illustrating an initial setting flow of a guidance display process when a destination is set by a user.

[0027] Figure 7 This is a diagram illustrating a scene in the real world.

[0028] Figure 8 The diagram shows an example of an augmented reality image being displayed on an AR display device.

[0029] Fig. 9 This is a diagram illustrating an initial setting flow of a guidance display process when a destination is set by a server.

[0030] Fig.10 The diagram exemplifies a captured image when a warning target object is highlighted.

[0031] Fig.11 FIG. 1 is a diagram illustrating an example of a guidance path for an object to be avoided as a warning.

[0032] Fig.12 The diagram exemplifies a guide route for avoiding contact between users of the AR display device.

[0033] Fig.13 This is a diagram illustrating a head mounted display (HMD) as an example of an AR display device.

[0034] Fig.14 This is a diagram illustrating a head-up display (HUD) installed in a vehicle as an example of an AR display device. DETAILED DESCRIPTION

[0035] exist Figure 1 , a comprehensive amusement facility 10 is shown as an example. The guidance display system according to the present embodiment is used in the facility. The comprehensive amusement facility 10 is provided with a plurality of theme parks 12 to 18. A theme park refers to a facility that forms a world view based on a specific theme, and based on the world view, comprehensively forms and performs facilities, activities, landscapes, etc. The theme parks 12 to 18 are connected by a connecting passage 20, and users can travel between the theme parks 12 to 18 via the connecting passage 20.

[0036] The comprehensive amusement facility 10 is provided with theme parks with different themes. For example, in the comprehensive amusement facility 10, as theme parks, there are provided a sports park (athletic park) 12, an amusement park 14, an aquarium 16, and a zoo 18. The theme parks 12 to 18 are respectively provided with virtual object characters based on their respective themes. As described later, the virtual objects are used to guide the users (users) of the AR display device 30 to the respective theme parks 12 to 18.

[0037] The virtual object characters are set to characters that match the theme and world view of each theme park 12 to 18. For example, in the sports park 12, characters such as adventurers, commandos, and ninjas are set as virtual objects. For example, in the amusement park 14, characters such as clowns and karts are set as virtual objects. For example, in the aquarium 16, characters such as dolphins, goldfish, and sharks are set as virtual objects. Furthermore, in the zoo 18, characters such as elephants, pandas, and penguins are set as virtual objects. The information of these set virtual objects is stored in the theme park-specific character storage unit 82 of the server 70 (see Figure 3 ). The details of the stored virtual object information will be described later.

[0038] In addition, for example, each of these theme parks 12 to 18 is a paid facility, and the number of users of each theme park 12 to 18 is counted. This counting is performed, for example, by a gate device (not shown) installed in each theme park 12 to 18. The counted number of users of each theme park 12 to 18 is stored in the park user storage unit 80 of the server 70 (see Figure 3 ).

[0039] A beacon transmitter 22 is installed on the connecting passage 20 connecting the theme parks 12 to 18 (see Figure 1 ). A plurality of transmitters 22 are provided on the connection passage 20, for example, at equal intervals. For example, the transmitter 22 is mounted on the upper end of the guide display support 28. As described later, the beacon receiver 37 (see Figure 2 ) receives the signal from the transmitter 22, thereby being able to obtain the current position of the AR display device 30.

[0040] The guidance display system according to the present embodiment is configured to include an AR display device 30 and a server 70. As described in detail below, the guidance display system according to the present embodiment takes theme parks 12 to 18 as destinations and performs guidance display to the destinations.

[0041] <Server Configuration>

[0042] exist Figure 2 , the hardware structure of the AR display device 30 and the server 70 is illustrated. The server 70 is composed of a computer, for example. The server 70 is wirelessly connected to the AR display device 30 through a communication path such as a wireless LAN. The server 70 has an input unit 71 such as a keyboard and a mouse, a CPU 72 of a computing device, and a display unit 73 such as a display. In addition, the server 70 has a ROM 74, a RAM 75, and a hard disk drive (HDD) 76 as a storage device. Furthermore, the server 70 has an input / output controller 77 for managing the input and output of information. These components are connected to an internal bus 78.

[0043] exist Figure 3 2 shows an example of a functional block of the server 70. This functional block diagram is formed by, for example, the CPU 72 executing a program stored in the ROM 74, the HDD 76, or a non-transitory storage medium such as a DVD.

[0044] The server 70 includes a receiving unit 83 that receives signals from external devices such as the AR display device 30 and a door entry device (not shown). In addition, the server 70 includes a user calculation unit 84 that receives signals from the receiving unit 83, a path search unit 85, and a 3D (three-dimensional) model extraction unit 86. Furthermore, the server 70 includes a sending unit 87 that sends a signal generated in the server 70 to an external device such as the AR display device 30.

[0045] The server 70 also includes a park user storage unit 80, a facility map storage unit 81, and a theme park-specific character storage unit 82. The park user storage unit 80 stores the number of users and the user density of each of the theme parks 12 to 18.

[0046] The facility map storage unit 81 stores map information within the comprehensive amusement facility 10. For example, location information of passages and facilities within the comprehensive amusement facility 10 is stored. In addition, attribute information and location information of warning target objects such as fences set in the connecting passage 20 and pools set along the connecting passage 20 are also stored in the facility map storage unit 81. Furthermore, area information of each theme park 12 to 18 is also stored in the facility map storage unit 81.

[0047] The theme park-specific character storage unit 82 stores information about characters set for the theme parks 12 to 18. The character information may be, for example, 3D model data of each character. The 3D model data includes, for example, 3D image data of the character, and the image data includes shape data, texture data, and motion data.

[0048] The user calculation unit 84 calculates the number of users of each theme park 12 to 18 based on the in-and-out visitor information transmitted from the in-and-out gate devices (not shown) of each theme park 12 to 18. The calculated user number information is stored in the park user storage unit 80.

[0049] Furthermore, the user calculation unit 84 refers to the facility map storage unit 81 to calculate the user density (number of users / theme park area) of each of the theme parks 12 to 18. The calculated user density information is stored in the park user storage unit 80.

[0050] As described later, when the navigation function of the AR display device 30 is stopped (turned off), the user calculation unit 84 sets the destination of the AR display device 30 based on the user density of each of the theme parks 12 to 18 .

[0051] The route search unit 85 creates a route (guidance route) to the theme park set as the destination among the theme parks 12 to 18 by the navigation function of the AR display device 30 or the user calculation unit 84. For example, when the current position is transmitted from the AR display device 30, the route search unit 85 creates a route (guidance route) from the current position of the AR display device 30 to the theme park 12 to 18 as the destination based on the map information of the comprehensive entertainment facility 10 stored in the facility map storage unit 81. The guidance route data includes, for example, position coordinate information of the guidance route from the current position of the AR display device 30 as the departure point to the entrance of the theme park as the destination among the theme parks 12 to 18.

[0052] The 3D model extraction unit 86 extracts the characters of the theme parks 12 to 18 set as destinations by the navigation function of the AR display device 30 or the user calculation unit 84 as guidance virtual objects. The extracted virtual object data of the characters, that is, the 3D model data, is transmitted from the transmission unit 87 to the AR display device 30 together with the guidance route.

[0053] <AR display device configuration>

[0054] Reference Figure 1 The AR display device 30 is a display device used by a user of the comprehensive amusement facility 10. The AR display device 30 can display a virtual reality (AR: Augmented Reality) image in which an image of a virtual object is superimposed on a scene in the real world.

[0055] The AR display device 30 can be classified from various perspectives. For example, from the perspective of portability, the AR display device 30 can be classified into two types: portable and fixed.

[0056] As a portable AR display device 30, for example, a smartphone or a glasses-type head-mounted display (HMD) can be cited. In addition, as a fixed AR display device 30, for example, a head-up display (HUD) can be cited, which is installed in a vehicle (mobility) moving between the theme parks 12 to 18 to project virtual objects on the windshield of the vehicle.

[0057] In addition, from the perspective of the display method of the real world scenery, the AR display device 30 can be divided into a video see-through display (VST display) and an optical see-through display (OST display). The VST display is a camera or other camera that captures the scenery of the real world, and the display displays the captured image. On the other hand, the OST display is a perspective display unit that can recognize the scenery of the real world through a semi-transparent mirror, and the virtual object is projected on the display unit.

[0058] The above-mentioned smart phone etc. is equipped with a camera 35 (see Figure 2 ) is classified as a VST display. In addition, the above-mentioned head-mounted display (HMD) and head-up display (HUD) use the lenses of glasses and the windshield as display parts to visually recognize the scenery of the real world, so they are classified as OST displays.

[0059] exist Figure 4 , a portable smartphone with a VST display is shown as an example of the AR display device 30. The smartphone may be owned by a user of the comprehensive amusement facility 10, or may be a rental item such as a tablet terminal lent to a user of the comprehensive amusement facility 10.

[0060] exist Figure 2 , the hardware structure of the AR display device 30 and the hardware structure of the server 70 are illustrated at the same time. The AR display device 30 includes a CPU (central processing unit) 31, an acceleration sensor 32, a gyro sensor 33, a geomagnetic sensor 34, a camera 35, a GPS receiver 36, a beacon receiver 37, a vibrator 38, an input / output controller 39, a system memory 40, a storage 41, a GPU 42, a frame memory 43, a RAMDAC 44, a display control unit 45, a display unit 46, and an input unit 47.

[0061] The system memory 40 is a storage device used by an operating system (OS) executed by the CPU 31. The storage 41 is an external storage device, and stores, for example, a program for displaying a virtual reality image (AR image) described later.

[0062] The camera 35 is, for example, a camera device mounted on a smartphone, and can capture the scenery of the real world through static images and dynamic images. The camera 35 is, for example, configured to include an imaging device such as a CMOS and a CCD. Furthermore, the camera 35 may be a so-called RGB-D camera, which, in addition to capturing the real world, also has a function of measuring the separation distance from the camera 35. As a function of measuring the separation distance, for example, in the camera 35, in addition to the above-mentioned imaging device, a distance measuring mechanism using infrared rays is also provided.

[0063] The GPU (Graphics Processing Unit) 42 is a computing device for image processing, and mainly works when performing image recognition described later. The frame memory 43 is a storage device for storing images captured by the camera 35 and then processed by the GPU 42. The RAMDAC (Random Access Memory Digital-to-Analog Converter) 44 converts the image data stored in the frame memory 43 into an analog signal suitable for the display unit 46 as an analog display.

[0064] The acceleration sensor 32, the gyro sensor 33, and the geomagnetic sensor 34 can be used to estimate the direction and orientation of the AR display device 30. The acceleration sensor 32 measures the acceleration of the AR display device 30. Figure 4 As shown, the acceleration sensor 32 can measure accelerations in three orthogonal axial directions. That is, the accelerations in the axial directions of the X-axis and Y-axis which are parallel to the display surface of the display unit 46 and orthogonal to each other, and the Z-axis which is orthogonal to the display surface of the display unit 46 are measured by the acceleration sensor 32. The acceleration sensor 32, the gyro sensor 33, and the geomagnetic sensor 34 are composed of so-called micromachines such as MEMS (Micro Electro Mechanical Systems).

[0065] The gyro sensor 33 measures the angular velocity of the AR display device 30. Figure 4 As shown, the gyro sensor 33 measures rotations around three orthogonal axes. That is, the pitch angle as rotation around the X-axis, the roll angle as rotation around the Y-axis, and the azimuth angle (also called yaw angle) as rotation around the Z-axis are measured by the gyro sensor 33. In addition, the geomagnetic sensor 34 detects the tilt of the AR display device 30 from the magnetic north.

[0066] The GPS receiver 36 receives data from the GPS satellites 24 (see Figure 1) receives a positioning signal, namely a GPS signal. The GPS signal contains the location coordinate information of latitude, longitude and altitude. The beacon receiver 37 receives the location signal from the transmitter 22 of the beacon set in the comprehensive entertainment facility 10, including the connection channel 20.

[0067] Here, both the GPS receiver 36 and the beacon receiver 37 have a position estimation function. Therefore, only one of the GPS receiver 36 and the beacon receiver 37 may be provided in the AR display device 30 .

[0068] The input unit 47 can input characters and / or select a destination theme park 12 to 18 described later. For example, the input unit 47 may be a touch panel integrated with the display unit 46 .

[0069] The display control unit 45 can display an augmented reality image (AR image) obtained by superimposing an image of a virtual object for guidance on a scenery in the real world on the display unit 46. For example, the display control unit 45 extracts an image area corresponding to the guidance path (channel) from a captured image of the real world, performs image processing (rendering) to superimpose the image of the virtual object for guidance on the extracted image area, and displays it on the display unit 46. The display unit 46 may be, for example, a liquid crystal display or an organic EL display.

[0070] exist Figure 5 2 shows a functional block diagram of the AR display device 30. This functional block diagram is formed by, for example, the CPU 31 and the GPU 42 executing a position estimation program stored in the system memory 40, the storage 41, or a non-transitory storage medium such as a DVD or a computer hard disk.

[0071] exist Figure 5 In Figure 2 Part of the hardware structure and the functional blocks are also shown in a mixed state. Figure 5 In FIG. 4 , the image capturing device 35 , the display control unit 45 , and the display unit 46 are exemplified as the hardware configuration.

[0072] In addition, the AR display device 30 includes, as functional blocks, a position estimation unit 50, a transmission control unit 51, a transmission unit 52, an input unit 53, a navigation function unit 54, a reception unit 55, a direction estimation unit 56, a virtual object processing unit 57, an image recognition unit 58, and a learned model storage unit 59. These functional blocks are composed of the CPU 31, the system memory 40, the storage 41, the GPU 42, the frame memory 43, and the like.

[0073] The input unit 53 receives various input operations from the user of the AR display device 30. The input operations include, for example, an operation of photographing the camera 35 of the AR display device 30, an operation of activating the navigation function, and an operation of inputting a destination when the navigation function is executed.

[0074] The image recognition unit 58 receives the captured image data obtained by the camera 35 and performs image recognition. Image recognition includes recognition of an object in the captured image and estimation of the separation distance between the object and the AR display device 30. When performing such image recognition, as described above, the captured image data includes, for example, color image data obtained by capturing a scene in the real world and separation distance data of each object in the color image data from the camera 35.

[0075] The image recognition unit 58 uses the image recognition learning model stored in the learned model storage unit 59 to recognize the captured image. The learned model storage unit 59 stores, for example, a neural network for image recognition that has been learned by an external server or the like. For example, outdoor image data including the comprehensive entertainment facility 10, that is, data on which the segmentation and labeling of each object in the image have been completed, is prepared as training data. Using the training data, a multi-layer neural network that is machine-learned through supervised learning is formed and stored in the learned model storage unit 59. The neural network may be, for example, a convolutional neural network (CNN).

[0076] The virtual object processing unit 57 obtains the posture and / or orientation of the virtual object for guidance displayed in the AR image based on the camera angle obtained by the orientation estimation unit 56. The real-world image that has been image-recognized and the 3D data of the virtual object for guidance with posture and orientation information set are sent to the display control unit 45. The display control unit 45 selects the passage portion that is part of the guidance path from the real-world image and overlaps the virtual object for guidance in front of the direction of travel toward the destination. The overlapping image is described later. Figure 8 The display unit 46 displays the information in this manner.

[0077] <Route guidance when the user sets the destination>

[0078] exist Figure 6 , a flowchart of the route guidance process performed by the guidance display system according to the present embodiment, particularly the initial setting, is illustrated. In this example, an example is shown in which the navigation function is activated by the user (user) of the AR display device 30. The navigation function is a function of the AR display device 30. Figure 6 In FIG. 1 , the processing mainly executed by the server 70 is indicated by (S), and the processing mainly executed by the AR display device 30 is indicated by (D).

[0079] In addition, the route guidance is in the comprehensive entertainment facility 10 (refer to Figure 1 ) when moving between theme parks within. That is to say, when the user moves on the connecting channel 20, the path guidance service described below is provided.

[0080] Reference Figure 3 , Figure 5 , Figure 6 When the navigation function activation operation is input to the input unit 53 of the AR display device 30, the navigation function unit 54 is activated. In addition, the transmission control unit 51 refers to the current position information estimated by the position estimation unit 50 to determine whether the current position of the AR display device 30 is outside the theme parks 12 to 18 (S10).

[0081] When the current position of the AR display device 30 is within the theme parks 12 to 18, Figure 6 The process ends. On the other hand, when the current position of the AR display device 30 is outside the theme parks 12 to 18, the transmission control unit 51 transmits information of the current position of the AR display device 30 and the theme park that becomes the destination (hereinafter appropriately referred to as the destination park) to the server 70 via the transmission unit 52 (S12). In addition, here, the destination park is input or selected by the user who uses the navigation function.

[0082] The current position information of the AR display device 30 and the information of the destination park received by the receiving unit 83 of the server 70 are sent to the route search unit 85. The route search unit 85 searches and creates a guidance route from the current position of the AR display device 30 to the destination park (S14). For example, the shortest distance route from the current position to the destination park is searched, and the route becomes the guidance route. The created guidance route information is sent to the sending unit 87.

[0083] In addition, the information of the destination park is sent to the 3D model extraction unit 86. The 3D model extraction unit 86 sets the character set for the destination park as the guidance virtual object (S16) by referring to the theme park-specific character storage unit 82. When there are multiple characters pre-set for the destination park, for example, any character is randomly selected.

[0084] The 3D model data of the corresponding character set as the guidance virtual object is sent to the transmission unit 87. The information of the guidance route and the 3D model data of the guidance virtual object are transmitted from the transmission unit 87 to the AR display device 30 (S18).

[0085] The display control unit 45 of the AR display device 30, which receives the information of the guidance path and the 3D model data of the virtual object for guidance, causes the image of the virtual object for guidance to be displayed in an overlapping manner at a position ahead of the current position of the AR display device 30 in the captured image obtained by the camera 35. The forward direction of travel refers to the direction from the AR display device 30 toward the destination park along the guidance path. For example, the display control unit 45 sets a location at a predetermined distance from the destination park in the image area identified as a passage in the captured image obtained by the camera 35 as the display position (overlapping position) of the virtual object for guidance (S20). Furthermore, the display control unit 45 causes the display unit 46 to display an overlapping image (AR image) obtained by overlapping the scenery of the real world with the virtual object for guidance (S22).

[0086] For example, in Figure 7 20 is shown in a perspective view. In this figure, a view is shown when standing at the fork of the path 25 leading to the zoo 18 and the path 27 leading to the amusement park 14. In this example, it is assumed that the zoo 18 is selected as the destination park. Figure 8 As shown, when the location is photographed by the camera 35 of the AR display device 30 , the image recognition unit 58 recognizes the photographed image and specifies the path 25 , the sign 26 , and the like.

[0087] Furthermore, an image of a deer as a guidance virtual object 100 is superimposed at a position on the determined route 25 that is away from the AR display device 30 and that is away from the destination park (zoo 18). Figure 8 In order to make the illustration clear, the guiding virtual object 100 is depicted with a planar outline (silhouette), but the form is not limited to this, and a 3D image of the guiding virtual object 100 may also be displayed.

[0088] When the AR display device 30 moves forward toward the destination park along the route 25, the guiding virtual object 100 also moves forward accordingly. At this time, an animation such as a deer of the guiding virtual object 100 moving forward may be displayed.

[0089] As described above, in the guidance display system according to the present embodiment, since a guidance virtual image corresponding to the destination park is selected, route guidance can be performed without impairing the world view of the destination park.

[0090] <Route guidance to alleviate congestion>

[0091] exist Figure 6 In the process, the user of the AR display device 30 activates the navigation function and sets the destination park, and the guidance display system involved in this embodiment can also be used to alleviate congestion.

[0092] exist Fig. 9 The flowchart of the route guidance process for alleviating congestion, especially the initial setting, is shown in FIG. Figure 6 The steps with the same number in the process are repeated and thus their description will be omitted as appropriate.

[0093] This process starts when the shooting function of the AR display device 30 is activated. The transmission control unit 51 (see Figure 5 ) Referring to the current position estimated by the position estimation unit 50, it is determined whether the current position of the AR display device 30 is outside the theme parks 12 to 18 (S10). When the AR display device 30 is inside the theme parks 12 to 18, this flow ends.

[0094] On the other hand, when the AR display device 30 is outside the theme parks 12 to 18, typically in the connection passage 20, the transmission control unit 51 refers to the working status of the navigation function unit 54 to determine whether the navigation function is being executed (S30). When the navigation function is being executed, since the destination park is set by the user, this process ends.

[0095] On the other hand, when the navigation function is in the stopped state, the transmission control unit 51 transmits the current position of the AR display device 30 to the server 70 (S32). In the server 70, the user calculation unit 84 calculates the number of users of the theme parks 12 to 18 at the destination park setting time point (that is, the current time) based on the information on the number of visitors entering and leaving the entrance devices (not shown) of the theme parks 12 to 18. Furthermore, the user calculation unit 84 obtains the area of ​​the theme parks 12 to 18 from the facility map storage unit 81 and calculates the user density of the theme parks 12 to 18 (S34).

[0096] After the user calculation unit 84 calculates the user density of the theme parks 12 to 18 at the time point of the destination park setting (that is, the current time), the user calculation unit 84 further calculates the theme park with the lowest user density among the theme parks 12 to 18 and sets it as the destination park (S36). The route search unit 85 creates a guidance route from the current position of the AR display device 30 to the set destination park. After step S36, the following steps are performed based on the set destination park. Figure 6 The same route guidance process (S16~S22).

[0097] For example, generally speaking, users of the comprehensive amusement facility 10 include users who have a clear purpose of visiting, such as those who want to see elephants in the zoo 18. On the other hand, there are a certain number of users who intend to stay in the comprehensive amusement facility 10 for a predetermined period of time and have not decided which theme park to visit among the theme parks 12 to 18. By guiding such latter users to relatively empty theme parks 12 to 18, the congestion rate of each theme park 12 to 18 can be smoothed, and as a result, congestion can be alleviated.

[0098] <Display of virtual objects for farewell>

[0099] In the above embodiment, the character set for the destination park is displayed on the display unit 46 in an overlapping manner. In addition, the display control unit 45 may display the character of the theme park closest to the AR display device 30 among the theme parks 12 to 18 as a farewell virtual object on the display unit 46 in an overlapping manner.

[0100] For example, refer to Figure 3 In addition to the destination park, the user calculation unit 84 also provides the 3D model extraction unit 86 with information on the theme parks 12 to 18 (hereinafter referred to as the nearest parks as appropriate) closest to the current position of the AR display device 30 .

[0101] The 3D model extraction unit 86 extracts the 3D model data of the character set for the destination park from the theme park-specific character storage unit 82 as described above, and also extracts the 3D model data of the character set for the nearest park from the storage unit 82. The 3D model data of the characters of the destination park and the nearest park are transmitted from the transmission unit 87 to the AR display device 30 together with the guidance route information.

[0102] The display control unit 45 of the AR display device (see Figure 5 ) The character of the destination park is displayed as a virtual object for guidance and is overlapped on the display unit 46, and the character of the nearest park is displayed as a virtual object for farewell and is overlapped on the display unit 46.

[0103] The virtual object for seeing off can be processed as follows: when the theme park in which the character of the virtual object is set is away from a predetermined distance from the theme park 12 to 18, it is no longer displayed on the display unit 46. By having such a performance function, for example, a performance can be performed in which the character of the theme park 12 to 18 that has just left the user sees off from the park.

[0104] <Control of objects to be warned>

[0105] AR technology is different from VR (virtual reality) technology. Since images of the real world are displayed, the so-called immersion is low, and the user may bump into obstacles such as fences by focusing on the virtual object 100. Therefore, the display control unit 45 can also cause the display unit 46 to highlight these obstacles.

[0106] Reference Figure 5 When the image recognition unit 58 recognizes the captured image obtained by the camera 35, the warning target object is recognized in the captured image. The teacher data of the warning target object is stored in advance in the learned model storage unit 59. The warning target object includes, for example, a fence, a pool, etc.

[0107] When the display control unit 45 causes the display unit 46 to display the captured image, Fig.10 As shown in FIG. 1 , the fence 102 as the warning target object is highlighted. Fig.10 The middle fence 102 is shaded. By performing such an emphasized display, the user can be alerted.

[0108] In addition, refer to Figure 3 , when the location information of the warning target object is stored in the facility map storage unit 81, as shown in FIG. Fig.11 As shown in FIG. 1 , the path search unit 85 may also create a guide path so as to avoid the warning target object. That is, the virtual object 100 moves forward on the display unit 46 in a manner to avoid the warning target object. Here, when approaching the warning target object, the vibrator 38 (see FIG. 1 ) may be used to generate a guide path so as to avoid the warning target object. Figure 2 ) is driven by vibration to remind the user to avoid contact with the warning object.

[0109] In addition, for example, when the camera 35 of the AR display device 30 is directed toward the warning target object, a performance can be performed in which the virtual object 100 runs out of the frame of the display unit 46 and becomes invisible, thereby avoiding contact with the warning target object while using the virtual object 100 to provide safe path guidance.

[0110] <Collision Avoidance Control>

[0111] like Fig.12 As shown in FIG. 1 , the guide route R1 from the zoo 18 to the aquarium 16 and the guide route R2 from the aquarium 16 to the zoo 18 are close to each other during the movement, and the users may come into contact with each other. Figure 3 ) When a plurality of AR display devices 30 approach to within a predetermined threshold distance, the initial guidance path may be corrected and recreated as a guidance path that avoids collision.

[0112] The path search unit 85 receives the position information of all the AR display devices 30 in the comprehensive entertainment facility 10. The path search unit 85 obtains the separation interval of the plurality of AR display devices 30, and for a pair of AR display devices 30A and 30B (see FIG. 1 ) that are close to each other within a predetermined threshold distance (e.g., 1 meter) among the AR display devices 30, Fig.12 ), remake the boot path.

[0113] For example, when the connecting passage 20 is notified to the user as a right-hand traffic road, Fig.12 As shown in FIG. 1 , the route search unit 85 creates avoidance routes R11 and R21 so as to deviate to the right from the original routes R1 and R2. By recreating the guide routes in this way, it is possible to prevent users from colliding with each other.

[0114] <Warning on taking things out of the facility>

[0115] In addition, when the AR display device 30 is a rental device limited to use in the comprehensive entertainment facility 10, the server 70 may also display a warning image on the display unit 46 of the AR display device 30 located outside the venue of the comprehensive entertainment facility 10 based on the location information of the AR display device 30. For example, the display control unit 45 causes the display unit 46 to display a message urging the user to return to the comprehensive entertainment facility 10.

[0116] <Other examples of AR display devices>

[0117] In the above-mentioned embodiment, a smartphone as an image see-through display is exemplified as the AR display device 30, but the AR display device 30 involved in this embodiment is not limited to this embodiment. Fig.13 Like the head mounted display (HMD) illustrated in FIG. 8 , the AR display device 30 is constituted by an optical see-through display.

[0118] In this case, the AR display device 30 is configured to include a half mirror 114 corresponding to the display unit 46 , a projector 116 corresponding to the display control unit 45 , and a sensor unit 112 corresponding to the position estimating unit 50 and the orientation azimuth estimating unit 56 .

[0119] The semi-transparent mirror 114 may be, for example, a lens of glasses and / or goggles. The semi-transparent mirror 114 projects light (image) from the real world to the user. On the other hand, the projector 116 disposed above the semi-transparent mirror 114 projects the image of the virtual object onto the semi-transparent mirror 114. Thus, an AR image in which the image of the virtual object for guidance is superimposed on the scenery of the real world can be displayed.

[0120] When the virtual object is displayed in a superimposed manner, the direction and orientation of the user's line of sight are estimated by the sensor unit 112. The projector 116 obtains the direction and orientation of the 3D model of the guidance virtual object based on the direction and orientation.

[0121] In addition, in the case of such an optical perspective display, there is a case where a camera 35 for photographing real-world scenery is not provided. In this case, it is necessary to set a triggering function for the display. Fig. 9 For example, Fig. 9 The starting point of the process is set to step S10, that is, Fig. 9 The process is transformed into a process that is started when the user leaves the theme park 12-18.

[0122] In addition, as another example of the AR display device 30 of the optical see-through display, Fig.14 1 shows a head-up display (HUD) as a fixed display device installed in a mobile body (mobility) such as a vehicle 120. For example, when moving between theme parks 12 to 18, the vehicle 120 travels on a connecting passage 20. The vehicle 120 may be a small vehicle for a maximum of five people to travel as a family.

[0123] In this case, the AR display device 30 is configured to include a windshield 124 corresponding to the display unit 46 , a projector 126 corresponding to the display control unit 45 , and a GPS receiver 122 corresponding to the position estimating unit 50 .

[0124] The windshield 124 projects light (image) from the real world to the user. On the other hand, the projector 126 disposed below the windshield 124 projects the image of the virtual object onto the windshield 124. Thus, an AR image in which the image of the guiding virtual object is superimposed on the scenery of the real world can be displayed.

Claims

1. A guidance display system for providing guidance to a facility configured based on a specific theme as a destination, the guidance display system comprising: A display device comprising a display unit, a display control unit, a shooting unit, and an image recognition unit, wherein the display control unit can cause the display unit to display an augmented reality image, wherein the augmented reality image is an image obtained by superimposing an image of a virtual object for guidance on a scenery in the real world, the shooting unit shoots the scenery in the real world, and the image recognition unit recognizes an object included in the shot image; and A server is wirelessly connected to the display device. The server has: a storage unit storing information of roles set for the facility; an extracting unit that extracts a character of the facility set as a destination as the guiding virtual object; a sending unit configured to send the extracted image data of the guiding virtual object to the display control unit; as well as a route search unit that searches for and creates a guide route from the current position of the display device to the set destination, When the display device moves toward the set destination along the guidance path, the guidance virtual object also moves forward accordingly. When the warning target object is recognized by the image recognition unit in the captured image, the guidance virtual object moves out of the frame of the display unit and is not displayed.

2. The guidance display system according to claim 1, The server has: a user calculation unit for calculating the user density of each of the plurality of facilities configured based on different themes, The route search unit sets the facility having the smallest user density at a destination setting time point among the plurality of facilities as a destination, and creates a guidance route to the destination.

3. The guidance display system according to claim 2, The extracting unit extracts, as a farewell virtual object, a character of the facility that is closest to the display device at a destination setting time point, The transmitting unit transmits the extracted image data of the farewell virtual object to the display control unit.

4. The guidance display system according to claim 3, The display control unit displays the warning target object in an emphasized manner on the display unit when the image recognition unit recognizes the warning target object in the captured image.

5. The guidance display system according to claim 4, The route search unit creates the guide route so as to be a route that avoids the warning target object.

6. The guidance display system according to claim 5, When the plurality of display devices approach within a predetermined threshold distance, the route search unit recreates each of the guide routes so as to avoid each of the display devices.

7. The guidance display system according to claim 3, The display device is a light-transmitting display having a semi-transparent mirror, wherein the semi-transparent mirror is a mirror through which a real-world landscape image is transmitted and on which the guiding virtual object is projected.

8. A server of a guidance display system, which is wirelessly connected to a display device capable of displaying an augmented reality image obtained by superimposing an image of a virtual object for guidance on a scenery in the real world, the display device comprising a display unit, a display control unit, a shooting unit, and an image recognition unit, the display control unit being capable of causing the display unit to display the augmented reality image, the shooting unit shooting the scenery in the real world, and the image recognition unit recognizing an object included in the shot image, the server of the guidance display system comprising: a storage unit storing information on roles set for a facility constructed based on a specific theme; an extracting unit that extracts a character of the facility set as a destination as the guiding virtual object; a sending unit configured to send the extracted image data of the guiding virtual object to the display device; as well as a route search unit that searches for and creates a guide route from the current position of the display device to the set destination, When the display device moves toward the set destination along the guidance path, the guidance virtual object also moves forward accordingly. When the warning target object is recognized by the image recognition unit in the captured image, the guidance virtual object moves out of the frame of the display unit and is not displayed.

Images