Information presentation system and information presentation method

Abstract

One aspect of the present invention relates to a system comprising a plurality of transmissive display devices, a plurality of moving media that support the plurality of transmissive display devices such that the respective disposition states thereof are variable, and an information presentation control device that is connected to the plurality of transmissive display devices and the plurality of moving media, wherein the information presentation control device assesses the representation form of a content to be presented that is designated to be presented from among a plurality of contents having different representation forms. The disposition states of the plurality of transmissive display devices are variably controlled by determining the formation of the plurality of transmissive display devices according to the assessed representation form and operating the plurality of moving media according to the determined formation. In addition, according to the assessed representation form, display information for being displayed on the plurality of transmissive display devices the disposition states of which are variably controlled is generated on the basis of the content to be presented, and the generated display information is outputted to the transmissive display device corresponding thereto.

Description

Information presentation system and information presentation method

[0001] One aspect of the present invention relates to an information presentation system and an information presentation method for presenting, for example, video content using a plurality of displays.

[0002] A known method for presenting video content is to split and display content with a wide angle of view, such as panoramic still images or moving images, across multiple displays arranged side by side on a two-dimensional plane.

[0003] Another presentation method that has been proposed is to arrange two transparent displays in a fixed, front-to-back position, and display two images with varying brightness ratios on these displays, thereby expressing the depth perceived at any point between the two images, as described in Patent Document 1, for example.

[0004] Japanese Patent Application Publication No. 2000-214413

[0005] However, the presentation method using multiple displays arranged in a two-dimensional direction can present video content with a wide field of view, such as panoramic video, but cannot display images with a stereoscopic feel, such as 3D video. On the other hand, a presentation method using two transparent displays arranged in a front-to-back direction can display a stereoscopic image, but cannot display video content with a wide field of view, such as panoramic video. Furthermore, since the display positions are fixed, the expressiveness in the depth direction is also inevitably fixed. In other words, none of the information presentation methods proposed so far can accommodate the presentation of diverse forms of video content.

[0006] The present invention has been made in light of the above circumstances, and aims to provide a technology that can accommodate the presentation of content in a variety of expression formats.

[0007] In order to solve the above problem, one aspect of an information presentation system or method according to the present invention is a system including a plurality of transmissive display devices, a plurality of moving media supporting the plurality of transmissive display devices in a manner that allows their respective positions to be variably adjusted, and an information presentation control device connected to the plurality of transmissive display devices and the plurality of moving media, wherein, when a content to be presented is designated from a plurality of content having different expression formats, the information presentation control device determines a formation of the plurality of transmissive display devices in accordance with the expression format of the content to be presented, and variably controls the positions of the plurality of transmissive display devices to achieve the determined formation, and simultaneously divides the content to be presented into a plurality of information elements in accordance with the expression format, and outputs the divided information elements to the corresponding transmissive display devices.

[0008] According to one aspect of the present invention, the placement positions of the multiple transparent display devices are variably controlled depending on the expression format of the content to be presented, and in this state the content to be presented is divided into multiple information elements and displayed on the transparent display devices.

[0009] Therefore, even if the content has a variety of expression formats, the formation of multiple transparent display devices can be automatically changed according to the expression format, and the information elements corresponding to the content to be presented can be displayed on the transparent display devices whose formation has changed.

[0010] That is, according to one aspect of the present invention, it is possible to provide a technology that can accommodate the presentation of content in a variety of expression formats.

[0011] FIG. 1 is a diagram showing an example of the configuration of an information presentation system according to an embodiment of the present invention. FIG. 2 is a block diagram showing an example of the hardware configuration of an information presentation control device used in the information presentation system shown in FIG. 1. FIG. 3 is a block diagram showing an example of the software configuration of the information presentation control device used in the information presentation system shown in FIG. 1. FIG. 4 is a flowchart showing an example of the processing procedure and processing content of information presentation control processing executed by a control unit of the information presentation control device shown in FIG. 3. FIG. 5 is a flowchart showing an example of the processing procedure and processing content of single-layer display control in the information presentation control processing shown in FIG. 4. FIG. 6 is a flowchart showing an example of the processing procedure and processing content of multi-layer display control in the information presentation control processing shown in FIG. 4. FIG. 7 is a flowchart showing an example of the processing procedure and processing content of display control in Volumetric Type 1 in the information presentation control processing shown in FIG. 4. FIG. 8 is a flowchart showing an example of the processing procedure and processing content of display control in Volumetric Type 2 in the information presentation control processing shown in FIG. 4. FIG. 9 is a flowchart showing an example of the processing procedure and processing content of display control in Volumetric Type 3 in the information presentation control processing shown in FIG. 4. FIG. 10 is a diagram showing an example of content information stored in a content information storage unit of the information presentation control device shown in FIG. 3. FIG. 11 is a diagram showing an example of scene information stored in a scene information storage unit of the information presentation control device shown in FIG. 3. FIG. 12 is a diagram showing an example of display device information stored in a display device information storage unit of the information presentation control device shown in FIG. 3. FIG. 13 is a diagram showing an example of setting information stored in a setting information storage unit of the information presentation control device shown in FIG. 3. FIG. 14 is a diagram for explaining an example of operation of single-layer display control. FIG. 15 is a diagram for explaining an example of operation of multi-layer display control. FIG. 16A is a diagram for explaining an example of operation of display control in Volumetric Type 1. FIG. 16B is a diagram for explaining an example of operation of display control in Volumetric Type 1. FIG. 16C is a diagram for explaining an example of operation of display control in Volumetric Type 1. FIG. 17A is a diagram for explaining an example of operation of display control in Volumetric Type 2.Fig. 17B is a diagram for explaining an example of the operation of display control in Volumetric Type 2. Fig. 17C is a diagram for explaining an example of the operation of display control in Volumetric Type 2. Fig. 18A is a diagram for explaining an example of the operation of display control in Volumetric Type 3. Fig. 18B is a diagram for explaining an example of the operation of display control in Volumetric Type 3. Fig. 18C is a diagram for explaining an example of the operation of display control in Volumetric Type 3.

[0012] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0013] [One Embodiment] (Configuration Example) (1) System FIG. 1 is a diagram showing an example of the configuration of an information presentation system according to one embodiment of the present invention.

[0014] In the information presentation system according to one embodiment, a plurality of display devices DS1 to DSn can be connected to an information presentation control device CS via a network NW.

[0015] The network NW may be, for example, a local area network (LAN) or a wireless LAN established in an office. Alternatively, the network NW may be a network that employs a low-power wireless data communication standard such as Bluetooth (registered trademark) or a public wireless network that employs the 4G or 5G standard.

[0016] (2) Display Devices DS1 to DSn Each of the display devices DS1 to DSn is a carriage-type moving device MS equipped with a robot arm RB, and a display device DP mounted on the carriage-type moving device MS. Specifically, the display device DP is attached to the tip of the robot arm RB.

[0017] The display device DP is made up of a transmissive display, and receives and displays display information transmitted from the information presentation control device CS.

[0018] The moving device MS is a cart equipped with an electric moving mechanism, and moves the display device DP to a specified position in accordance with a movement control signal sent from the information presentation control device CS. The robot arm RB includes, for example, an articulated arm and a servo mechanism for operating this arm, and changes the orientation of the display device DP in three dimensions in accordance with an orientation control signal sent from the information presentation control device CS. The position of the display device DP is expressed, for example, by three-dimensional coordinates. The orientation is also expressed, for example, by three elements: roll, tilt, and pan.

[0019] (3) Information Presentation Control Device CS The information presentation control device CS is, for example, a general-purpose personal computer. Note that the information presentation control device CS may also be a server computer installed on the web or in the cloud.

[0020] 2 and 3 are block diagrams showing an example of the hardware configuration and software configuration of the information presentation control device CS, respectively.

[0021] The information presentation control device CS has a control unit 1 that uses a hardware processor such as a central processing unit (CPU), and this control unit 1 is connected via a bus 6 to a storage unit having a program storage unit 2 and a data storage unit 3, an input / output interface (hereinafter, interface will be abbreviated as I / F) unit 4, and a communication I / F unit 5.

[0022] Input / output devices 7 such as a keyboard, mouse, display, and external memory are connected to the input / output I / F unit 4, and the input / output I / F unit 4 receives content information, scene information representing its configuration, setting information, and various instruction information from the input / output devices 7.

[0023] The communication I / F unit 5 transmits display information and various control signals to the display devices DP of the display devices DS1 to DSn, the moving device MS, and the robot arm RB via the network NW in accordance with a communication protocol defined by the network NW.

[0024] The program storage unit 2 is configured by combining, for example, a nonvolatile memory such as a solid-state drive (SSD) as a storage medium that can be written to and read from at any time, and a nonvolatile memory such as a read-only memory (ROM), and stores middleware such as an operating system (OS), as well as application programs required to execute various controls according to an embodiment. Hereinafter, the OS and each application program will be collectively referred to as the program.

[0025] The data storage unit 3 is, for example, a combination of a non-volatile memory such as an SSD that can be written to and read from at any time as a storage medium, and a volatile memory such as a RAM (Random Access Memory), and its storage area is provided with a setting information storage unit 31, a content information storage unit 32, a scene information storage unit 33, and a display device information storage unit 34 as storage units according to one embodiment of the present invention.

[0026] The setting information storage unit 31 stores setting information that indicates the default value of the display type of content, input by an administrator or a user from the input / output device 7. Fig. 13 is a diagram showing an example of setting information stored in the setting information storage unit 31, and in this example, Volumetric Type 1 is set as the default display type.

[0027] The content information storage unit 32 stores a plurality of pieces of content information for managing content. Each piece of content information consists of a content ID associated with a scene name, a display type, and an area of ​​interest. Fig. 10 is a diagram showing an example of content information stored in the content information storage unit 32.

[0028] The scene information storage unit 33 stores information representing objects to be displayed in a scene in association with the scene name. The information representing the object includes the object type (e.g., 3D or 2D), the display position coordinates of the object on the image frame, and a scale value representing the display size of the object. Fig. 11 is a diagram showing an example of scene information stored in the scene information storage unit 33.

[0029] The display device information storage unit 34 stores display device information for managing each display device DP mounted on the display devices DS1 to DSn. The display device information associates identification information (device ID) of a display device with three-dimensional coordinates representing its resolution and current position. Fig. 12 is a diagram showing an example of the display device information stored in the display device information storage unit 34.

[0030] The control unit 1 has the processing functions necessary to implement one embodiment, including a setting information acquisition processing unit 11, a content information acquisition processing unit 12, an expression format determination processing unit 13, a formation determination processing unit, a display information generation processing unit 15, a formation control processing unit 16, and a display information output processing unit 17.

[0031] Each of the processing units 11 to 17 is realized by causing a hardware processor of the control unit 1 to execute an application program stored in the program storage unit 2. Note that some or all of the processing units 11 to 17 may be realized using hardware such as an LSI (Large Scale Integration) or an ASIC (Application Specific Integrated Circuit).

[0032] Prior to controlling information presentation, the setting information acquisition processing unit 11 receives information representing the default value of the content display type input by the system administrator or user on the input / output device 7 via the input / output I / F unit 4, and stores the received information representing the default value of the display type in the setting information storage unit 31.

[0033] The content information acquisition processing unit 12 acquires content information and scene information representing the content details input by the system administrator or user to the input / output device 7 via the input / output I / F unit 4, and stores the acquired content information and scene information in the content information storage unit 32 and the scene information storage unit 33. The content information acquisition processing unit 12 can also acquire content information and scene information via the network NW from, for example, another terminal device, server computer, camera, etc. (not shown).

[0034] The expression format determination processing unit 13 determines the expression format, i.e., the display type, to be used when presenting content designated as a presentation target from the input / output device 7, by referring to the content information storage unit 32, the scene information storage unit 33, and the setting information storage unit 31. There are five display types to be determined: single layer, multi-layer, volumetric type 1, volumetric type 2, and volumetric type 3.

[0035] The formation determination processing unit determines the formation (positional relationship) of the multiple display devices DP used to present the content information to be presented, based on the display type determined by the expression format determination processing unit 13 and the display device information stored in the display device information storage unit 34. An example of the formation determination process will be described in the operation example.

[0036] The display information generation processing unit 15 generates display information to be displayed on each display device DP for which the formation has been determined, based on the display type determined by the expression format determination processing unit 13 and the information stored in the content information storage unit 32 and the scene information storage unit 33. An example of this display information generation processing will also be described below as an operation example.

[0037] The formation control processing unit 16 generates a position control signal and an attitude control signal for each display device DP in order to arrange the display devices DP in accordance with the formation determined by the formation determination processing unit 14. The formation control processing unit 16 then transmits the generated position control signal and attitude control signal from the communication I / F unit 5 to the moving device MS and the robot arm RB of each of the display devices DS1 to DSn to be controlled, respectively.

[0038] The display information output processing section 17 transmits each piece of display information generated by the display information generation processing section 15 from the communication I / F section 5 to the display devices DP of the display devices DS1 to DSn as display destinations.

[0039] (Example of Operation) Next, an example of operation of the information presentation system configured as above will be described.

[0040] (1) Pre-registration of Information Prior to controlling the presentation of information, a system administrator or a user inputs pre-created content information and scene information representing the content details using the input / output device 7. In response to this, the control unit 1 of the information presentation control device CS, under the control of the content information acquisition processing unit 12, retrieves the content information and scene information via the input / output I / F unit 4, and stores the retrieved content information and scene information in the content information storage unit 32 and the scene information storage unit 33, respectively.

[0041] The content information and scene information may be acquired from another terminal device or a server computer via the network NW and stored in the content information storage unit 32 and the scene information storage unit 33 .

[0042] In some of the content information, the display type may not be set. In case of such a case, the system administrator or the user can arbitrarily set a default value for the display type.

[0043] That is, the system administrator or user inputs a default value for the display type in the input / output device 7. Under the control of the setting information acquisition processing unit 11, the control unit 1 of the information presentation control device CS receives the default value for the display type input from the input / output device 7 via the input / output I / F unit 4 and stores this input information in the setting information storage unit 31. Fig. 13 shows an example of setting information stored in the setting information storage unit 31, and in this example, Volumetric type 1 is set as the default value for the display type.

[0044] (2) Presentation Control of Content Information When content information to be presented is designated, the information presentation control device CS controls the presentation of the content information as follows.

[0045] FIG. 4 is a flowchart showing an example of the procedure and content of the presentation control process executed by the control unit 1 of the information presentation control device CS.

[0046] (2-1) Determining the Expression Format That is, for example, suppose that the user inputs the ID of content information to be presented via the input / output device 7. When the control unit 1 of the information presentation control device CS detects the content ID in step S10, under the control of the expression format determination processing unit 13, in step S11, the control unit 1 reads corresponding content information from the content information storage unit 32 using the input content ID as a key, and further reads scene information corresponding to a scene name defined in the content information from the scene information storage unit 33.

[0047] First, in step S12, the expression format determination processing unit 13 determines whether the display type of the content information is "single layer" or something else based on the content information that has been read in. If the result of this determination is that the display type of the content information to be presented is designated as "single layer," the expression format determination processing unit 13 proceeds to step S15 and executes single layer display control.

[0048] Note that even when there is one object defined in the scene information of the content information and the type is "2D," the expression format determination processing unit 13 determines the display type to be "single layer."

[0049] On the other hand, suppose that the display type of the content information to be presented is determined to be other than the above in step S11. In this case, the expression format determination processing unit 13 determines in step S13 whether the display type is "multi-layer" or "volumetric." This determination is made based on whether the display type defined in the content information to be presented is specified as "multi-layer" or "volumetric." If the result of this determination is that the display type is specified as "multi-layer," the expression format determination processing unit 13 proceeds to step S16, where it executes multi-layer display control.

[0050] In addition, even if the scene information includes multiple 2D objects and the positions of these 2D objects are set to different positions in the depth direction, the expression format determination processing unit 13 determines the display type to be "multi-layer."

[0051] Furthermore, if the display type is determined to be not multi-layer, i.e., "Volumetric," in step S13, the representation format determination processing unit 13 subsequently determines the "Volumetric" type in step S14. This determination is made based on whether "Volumetric Type 1," "Volumetric Type 2," or "Volumetric Type 3" is specified as the display type of the content information.

[0052] If no display type is specified for the content information or if "null" is specified, the expression format determination processing unit 13 sets the default value of the display type stored in the setting information storage unit 31 as the display type of the content information to be presented. For example, if the setting information is set as shown in Fig. 13, the display type is set to "Volumetric Type 1".

[0053] (2-2) Single-Layer Display Control The control unit 1 of the information presentation control device CS executes single-layer display control as follows.

[0054] FIG. 5 is a flowchart showing an example of the processing procedure and processing content of single-layer display control executed by the control unit 1 of the information presentation control device CS.

[0055] That is, in step S151, under the control of the formation determination processing unit 14, the control unit 1 of the information presentation control device CS first performs rendering based on the camera position coordinates contained in the scene information read from the scene information storage unit 33, thereby calculating the size of the content to be presented.

[0056] Then, the formation determination processing unit 14 calculates the zoom ratio of the content in step S152 based on the size of the content to be presented and the resolution of each display device DP stored in the display device information storage unit 34, and determines the formation of the multiple display devices DP in step S153. That is, the formation determination processing unit 14 selects the multiple display devices DP to be used and determines the placement position and orientation of each selected display device DP.

[0057] Next, in step S154, under the control of the formation control processing unit 16, the control unit 1 of the information presentation control device CS generates, for each of the selected display devices DP, a position control signal for moving the position of the display device DP and an attitude control signal for controlling the attitude of the display device DP, in accordance with the determined placement position and attitude.

[0058] The formation control processing unit 16 then transmits the generated position control signals and attitude control signals to the moving devices MS and robot arms RB of the display devices DS1 to DSn to be controlled from the communication I / F unit 5. As a result, the placement positions of the display devices DP are moved and controlled by the moving devices MS, and the attitudes of the display devices DP are simultaneously controlled.

[0059] In addition, in step S155, the control unit 1 of the information presentation control device CS, under the control of the display information generation processing unit 15, performs rendering based on a size that reflects the content enlargement / reduction ratio calculated in step S152, and generates display information to be displayed on each of the selected display devices DP.

[0060] Then, in step S156, the control unit 1 of the information presentation control device CS transmits each piece of display information from the communication I / F unit 5 to each display device DP as a display destination under the control of the display information output processing unit 17. As a result, the display information is displayed on each display device DP whose formation has been controlled.

[0061] 14 shows an example of the operation of single-layer display control. In this example, the original image content V0 is enlarged according to its size and the resolution of the display device DP. The enlarged image content is then divided into two, left and right, to form display images V1 and V2. These display images V1 and V2 are output and displayed on two display devices DP, each of which is arranged in a direction parallel to the display surface, i.e., side-by-side. Therefore, the user can view the presented content image V0 as an image enlarged by the two display devices.

[0062] (2-3) Multi-Layer Display Control The control unit 1 of the information presentation control device CS executes multi-layer display control as follows.

[0063] FIG. 6 is a flowchart showing an example of the processing procedure and processing content of the multi-layer display control executed by the control unit 1 of the information presentation control device CS.

[0064] That is, in step S161, the control unit 1 of the information presentation control device CS, under the control of the formation determination processing unit 14, first reads scene information corresponding to the content information to be presented from the scene information storage unit 33. In step S162, the formation determination processing unit 14 identifies the positions of objects in the content based on the read scene information, and performs clustering based on the identified positions, using the number of display devices as the number of clusters, to determine the objects included in each of the multiple layers.

[0065] In addition, the formation determination processing unit 14 selects display devices DS1 to DSn to place display devices DP at positions corresponding to each of the multiple layers on which the object is determined, and determines the formation of each display device DP of the selected display devices DS1 to DSn.

[0066] Next, in step S164, the control unit 1 of the information presentation control device CS generates, for each of the selected display devices DS1 to DSn, a position control signal for moving the position of the display device DP and an attitude control signal for controlling the attitude of the display device DP in accordance with the determined placement position and attitude, under the control of the formation control processing unit 16. Then, the formation control processing unit 16 transmits the generated position control signals and attitude control signals from the communication I / F unit 5 to the movement device MS and robot arm RB of each of the display devices DS1 to DSn to be controlled, respectively.

[0067] As a result, the movement of each display device DP is controlled by the movement device MS so that the display device DP is placed at the position of each set layer, and at the same time, the attitude of the display device DP is controlled. Thus, the display devices DP are arranged side by side at regular intervals in the depth direction.

[0068] At the same time, the control unit 1 of the information presentation control device CS generates display information for each layer in step S165 under the control of the display information generation processing unit 15. This display information generation process is performed by rendering data representing content for each layer based on, for example, the position coordinates of the camera included in the scene information.

[0069] Then, in step S166, the control unit 1 of the information presentation control device CS transmits each piece of display information generated for each layer from the communication I / F unit 5 to each display device DP as a display destination, under the control of the display information output processing unit 17. As a result, the display information generated for each layer is displayed on each display device DP arranged at a position corresponding to each layer.

[0070] 15 shows an example of the operation of multi-layer display control. In this example, the original image content V0 includes objects in two layers, foreground and background. In this case, the formation of two display devices DP is determined and arranged so that they are aligned perpendicular to the display surface, i.e., aligned in the depth direction, to correspond to the positions of the two layers, foreground and background. Then, object images corresponding to the foreground and background, respectively, are displayed as display information on each of the arranged display devices DP. Therefore, two 2D images corresponding to the foreground and background can be displayed as a 3D image with a sense of depth.

[0071] (2-4) Display Control in Volumetric Type 1 The control unit 1 of the information presentation control device CS executes display control in Volumetric Type 1 as follows.

[0072] Volumetric Type 1 presents information about the entire area of ​​3D content, and sets multiple layers at regular intervals in the depth direction of the 3D content. Display devices are then placed at positions corresponding to the layers, and display information corresponding to each layer is displayed on these display devices.

[0073] FIG. 7 is a flowchart showing an example of the procedure and content of Volumetric Type 1 display control executed by the control unit 1 of the information presentation control device CS.

[0074] That is, in step S171, the control unit 1 of the information presentation control device CS first sets a layer corresponding to the rearmost position in the depth direction of the content under the control of the formation determination processing unit 14. The rearmost layer is set, for example, to a position where all objects within the field of view of the content are displayed, that is, a position where the edge size of the field of view is maximum.

[0075] Next, in step S172, the control unit 1 of the information presentation control device CS, under the control of the display information generation processing unit 15, generates display information representing the object included in the position of the backmost layer by rendering the content data based on the camera position coordinates included in the scene information. Note that objects included in other layers are displayed using edges.

[0076] Next, in step S173, the control unit 1 of the information presentation control device CS sets the foreground layer in the depth direction of the content information under the control of the formation determination processing unit 14. The foreground layer is set at a position that includes the object located closest to the user in the depth direction of the content. Then, in step S174, under the control of the display information generation processing unit 15, the control unit 1 of the information presentation control device CS generates display information representing the object included in the foreground layer by rendering the content data based on the camera position coordinates included in the scene information.

[0077] Next, in step S175, the control unit 1 of the information presentation control device CS calculates the distance between the backmost layer and the foremost layer under the control of the formation determination processing unit 14, and sets at least one layer at an intermediate position between the backmost layer and the foremost layer by evenly allocating the layers based on the calculated distance. Then, in step S176, the control unit 1 of the information presentation control device CS generates display information corresponding to the layer set at the intermediate position by rendering content data.

[0078] Next, in step S177, the control unit 1 of the information presentation control device CS, under the control of the formation control processing unit 16, generates a position control signal and an attitude control signal for each layer in order to position the display device DP at a position corresponding to each of the set layers: the backmost layer, the foremost layer, and any intermediate positions.

[0079] Then, the formation control processing unit 16 transmits each of the position control signals and each of the attitude control signals generated for each of the layers to the moving devices MS and robot arms RB of the display devices DS1 to DSn selected corresponding to each of the layers from the communication I / F unit 5. As a result, the position of each of the display devices DP of the display devices DS1 to DSn is moved and controlled by the moving devices MS, and the attitude of each of the display devices DP is controlled by the robot arms RB, so that the display devices DP are positioned at the rearmost, foremost, and intermediate positions of each of the layers.

[0080] Finally, in step S178, the control unit 1 of the information presentation control device CS transmits each piece of display information generated corresponding to each layer to the corresponding display device DP from the communication I / F unit 5. As a result, the display information generated for each layer is displayed on each display device DP arranged at a position corresponding to each layer.

[0081] 16A to 16C are diagrams used to explain an example of the operation of display control in Volumetric Type 1. This example shows a case where image content including, for example, a tomographic image of a human US obtained by CT and a front image captured by a camera CM is presented as the presentation target, as shown in Fig. 16A.

[0082] In the display control of the Volumetric Type 1 information presentation control device CS, three layers P1, P2, and P3 are set in the depth direction based on the tomographic image of the person US, and three display devices DP are arranged at positions corresponding to these layers P1, P2, and P3. Then, each of the tomographic images at a position corresponding to each of the layers P1, P2, and P3 is displayed on each of the three display devices DP.

[0083] Fig. 16B shows an example of the tomographic images displayed on each of the three display devices DP. When viewed from the front, the three displayed tomographic images are visually recognized as a 3D composite tomographic image in which the images are superimposed on one another, as shown in Fig. 16C.

[0084] (2-5) Display Control in Volumetric Type 2 The control unit 1 of the information presentation control device CS executes display control in Volumetric Type 2 as follows.

[0085] Volumetric Type 2 extracts information about a region of interest from 3D content, sets multiple layers at regular intervals in the depth direction of this region of interest, and then places display devices at positions corresponding to each of the layers, and displays display information corresponding to each layer on these display devices.

[0086] FIG. 8 is a flowchart showing an example of the procedure and content of Volumetric Type 2 display control executed by the control unit 1 of the information presentation control device CS.

[0087] That is, the control unit 1 of the information presentation control device CS first selects an area of ​​interest in the content in step S181. The area of ​​interest is selected based on an object or coordinate values ​​defined as a point of interest in the content information.

[0088] Next, under the control of the formation determination processing unit 14, the control unit 1 of the information presentation control device CS first determines the layer corresponding to the rearmost position in the depth direction in the attention area of ​​the content in step S182. Then, under the control of the display information generation processing unit 15, the control unit 1 of the information presentation control device CS then generates display information representing the object included in the rearmost layer position of the attention area by rendering the content data of the attention area based on the camera position coordinates included in the scene information in step S183. Note that objects included in other layers are displayed as edges.

[0089] Next, in step S184, the control unit 1 of the information presentation control device CS determines the layer corresponding to the foreground in the depth direction of the region of interest under the control of the formation determination processing unit 14. The foreground layer is set to a position that includes an object included in the foreground position of the region of interest. Then, in step S185, under the control of the display information generation processing unit 15, the control unit 1 of the information presentation control device CS generates display information representing the object included in the foreground layer of the region of interest by rendering the content data of the region of interest, based on the camera position coordinates included in the scene information.

[0090] Next, in step S186, the control unit 1 of the information presentation control device CS calculates the distance between the backmost layer and the foremost layer under the control of the formation determination processing unit 14, and sets at least one layer at an intermediate position between the backmost layer and the foremost layer by evenly allocating the layers based on the calculated distance. Then, in step S187, the control unit 1 of the information presentation control device CS generates display information corresponding to the layer set at the intermediate position by rendering content data.

[0091] Next, in step S188, the control unit 1 of the information presentation control device CS, under the control of the formation control processing unit 16, generates a position control signal and an attitude control signal for each layer in order to place the display device DP at a position corresponding to each of the set layers: the backmost layer, the foremost layer, and any intermediate positions.

[0092] Then, the formation control processing unit 16 transmits each of the position control signals and each of the attitude control signals generated for each layer to the moving device MS and the robot arm RB of the display devices DS1 to DSn selected corresponding to each layer from the communication I / F unit 5. As a result, the position of each of the display devices DP of the display devices DS1 to DSn is moved and controlled by the moving device MS, and the attitude of each of the display devices DP is controlled by the robot arm RB, so that the display devices DP are positioned at the rearmost, foremost, and intermediate positions of each layer.

[0093] Finally, in step S189, the control unit 1 of the information presentation control device CS transmits each piece of display information generated corresponding to each layer to the corresponding display device DP from the communication I / F unit 5. As a result, the display information generated for each layer is displayed on each display device DP arranged at a position corresponding to each layer.

[0094] 17A to 17C are diagrams used to explain an example of the operation of display control in Volumetric Type 2. This example shows a case where, as shown in Fig. 17A, 3D content consisting of a tomographic image of a human US captured by CT is presented.

[0095] In the display control of the information presentation control device CS in Volumetric Type 2, a region of interest, for example, a region corresponding to a lung region, is selected from the tomographic image of the human US. Then, three layers are set in the depth direction for the tomographic image of the selected region of interest, as in the case of Volumetric Type 1, and three display devices DP are arranged at positions corresponding to these layers. Then, each tomographic image at a position corresponding to each of the layers is displayed on the three display devices DP.

[0096] 17B shows an example of the tomographic images VE1, VE2, and VE3 of the region of interest displayed on the three display devices DP. When viewed from the front, the three displayed tomographic images VE1, VE2, and VE3 are visually recognized as a 3D composite tomographic image VE0 superimposed on one another, as shown in FIG. 17C.

[0097] (2-6) Display Control in Volumetric Type 3 The control unit 1 of the information presentation control device CS executes display control in Volumetric Type 3 as follows.

[0098] In Volumetric Type 3, when multiple layers are set in the depth direction of 3D content, the layers are not spaced evenly, but are set unevenly at positions corresponding to objects in the region of interest of the 3D content. Display devices are then arranged at positions corresponding to the multiple layers with uneven spacing, and display information corresponding to each layer is displayed on these display devices.

[0099] FIG. 9 is a flowchart showing an example of the processing procedure and processing content of Volumetric Type 3 display control executed by the control unit 1 of the information presentation control device CS.

[0100] That is, the control unit 1 of the information presentation control device CS first sets a layer corresponding to the rearmost position in the depth direction of the 3D content in step S191. Then, under the control of the display information generation processing unit 15, the control unit 1 of the information presentation control device CS then generates display information representing objects included in the rearmost layer of the 3D content by rendering the content data based on the camera position coordinates included in the scene information in step S192. Note that objects included in other layers are displayed as edges.

[0101] Next, in step S193, the control unit 1 of the information presentation control device CS sets a layer corresponding to the foreground in the depth direction of the 3D content under the control of the formation determination processing unit 14. The foreground layer is set at a position including an object in a region of interest in the 3D content. Then, in step S194, under the control of the display information generation processing unit 15, the control unit 1 of the information presentation control device CS generates display information representing the object of interest included in the position of the foreground layer by rendering the content data, based on the camera position coordinates included in the scene information.

[0102] Next, in step S195, the control unit 1 of the information presentation control device CS calculates the distance between the backmost layer and the foremost layer under the control of the formation determination processing unit 14, and sets at least one layer at an intermediate position between the backmost layer and the foremost layer by evenly allocating the layers based on the calculated distance. Then, in step S196, the control unit 1 of the information presentation control device CS generates display information corresponding to the layer set at the intermediate position by rendering content data.

[0103] Next, in step S197, the control unit 1 of the information presentation control device CS, under the control of the formation control processing unit 16, generates position control signals and attitude control signals for each layer in order to position the display device DP at positions corresponding to the set backmost, frontmost and intermediate layers.

[0104] The formation control processing unit 16 then transmits the position control signals and attitude control signals generated for each layer to the moving devices MS and robot arms RB of the display devices DS1 to DSn selected corresponding to each layer from the communication I / F unit 5. As a result, the position of each display device DP of the display devices DS1 to DSn is controlled by the moving devices MS and the attitude of each display device DP is controlled by the robot arm RB so that the display devices DP are positioned at the backmost, foremost, and intermediate positions of each layer.

[0105] Finally, in step S198, the control unit 1 of the information presentation control device CS transmits each piece of display information generated corresponding to each layer to the corresponding display device DP from the communication I / F unit 5. As a result, the display information generated for each layer is displayed on each display device DP arranged at a position corresponding to each layer.

[0106] 18A to 18C are diagrams used to explain an example of the operation of display control in Volumetric Type 3. This example shows a case where, as shown in Fig. 18A, in 3D content consisting of tomographic images of a human US captured by CT, for example, an entire image and a partial image of a region G of interest are displayed on different layers.

[0107] In the display control of the information presentation control device CS in Volumetric Type 3, a layer for displaying the entire image is set at the rearmost position in the depth direction for the tomographic image of the person US, and a layer for including the eye region, which is the region of interest, as an object is set at the frontmost position. Two display devices DP are then arranged at positions corresponding to the layers, and a display image showing the entire image and a partial display image showing the eye region of interest are displayed on these display devices DP, respectively.

[0108] 18B shows an example of a display image VG1 showing the overall image and a partial display image VG2 showing the eye region of interest, which are displayed on the two display devices DP. When viewed from the front, the two displayed images VG1 and VG2 are visually recognized as a 3D composite image VG0 in which the images are superimposed on each other, as shown in FIG. 18C.

[0109] (Effects) As described above, in one embodiment, based on the display type defined in the content information or the object type defined in the scene information, it is determined whether the representation format of the content to be presented is single layer, multi-layer, Volumetric Type 1, Volumetric Type 2, or Volumetric Type 3. Then, depending on the determination result, the formation of the transmissive display devices DP is determined, and the positions and attitudes of the transmissive display devices DP are controlled by the moving devices MS and the robot arm RB, so that display information representing the corresponding objects is displayed on these display devices DP.

[0110] Therefore, even if the content has a variety of expression formats, the formation of multiple display devices DP can be automatically and variably controlled according to the expression format of the content to be presented, and display information including corresponding objects can be displayed on each display device DP whose formation has changed.

[0111] [Other Embodiments] (1) As another example of multi-layer display control, the following display control is possible. That is, for example, when content with movement, such as animation, is to be presented, a moving image of a ball being thrown by a character is assigned to the foreground layer, and an image including a catcher is assigned to the background layer. Then, a display device corresponding to the background layer is fixedly positioned while displaying an image of the catcher, while a display device corresponding to the foreground layer is moved to follow the movement of the ball while displaying an image of the ball. In this way, it is possible to present moving images, such as animation, in a way that gives a more realistic feeling.

[0112] (2) In the embodiment, a case has been described in which a carriage-type moving device MS equipped with a robot arm RB is used as a moving medium for moving the display device DP. However, the present invention is not limited to this, and a small drone or a lift-type moving medium suspended from the ceiling may also be used.

[0113] (3) In addition, various modifications can be made to the method for controlling the position and attitude of the display device, the type and functional configuration of the information presentation control device CS, the processing procedure and processing content, the type and configuration of the presented information, the expression format, etc., without departing from the spirit of this invention.

[0114] Although the embodiments of the present invention have been described in detail above, the above description is merely an example of the present invention in every respect. It goes without saying that various improvements and modifications can be made without departing from the scope of the present invention. In other words, when implementing the present invention, specific configurations according to the embodiments may be appropriately adopted.

[0115] In short, this invention is not limited to the above-described embodiments, and in the implementation stage, the components can be modified and embodied without departing from the spirit of the invention. Furthermore, various inventions can be formed by appropriately combining multiple components disclosed in the above-described embodiments. For example, some components may be omitted from all the components shown in the embodiments. Furthermore, components from different embodiments may be appropriately combined.

[0116] DS1 to DSn...display device CS...information presentation control device DP...display device MS...mobile device RB...robot arm NW...network 1...control unit 2...program storage unit 3...data storage unit 4...input / output I / F unit 5...communication I / F unit 6...bus 11...setting information acquisition processing unit 12...content information acquisition processing unit 13...expression format determination processing unit 14...formation determination processing unit 15...display information generation processing unit 16...formation control processing unit 17...display information output processing unit 31...setting information storage unit 32...content information storage unit 33...scene information storage unit 34...display device information storage unit

Claims

1. An information presentation system comprising: a plurality of transparent display devices; a plurality of moving media supporting the plurality of transparent display devices in a manner that allows the arrangement of each of the plurality of transparent display devices to be variably variably adjusted; and an information presentation control device connected to the plurality of transparent display devices and the plurality of moving media, wherein the information presentation control device comprises: a memory unit that stores a plurality of contents having different expression formats; a first processing unit that determines the expression format of a presentation target content designated as a presentation target from the plurality of contents; a second processing unit that determines a formation of the plurality of transparent display devices in accordance with the determined expression format and operates the plurality of moving media in accordance with the determined formation, thereby variably controlling the arrangement of the plurality of transparent display devices; and a third processing unit that generates display information based on the presentation target content to be displayed on the plurality of transparent display devices whose arrangement has been variably controlled in accordance with the determined expression format, and outputs the generated display information to the corresponding transparent display device.

2. The information presentation system of claim 1, wherein the first processing unit determines whether the representation format of the content to be presented is a single-layer format in which a two-dimensional object is to be presented, a multi-layer format in which a plurality of two-dimensional objects are presented in multiple hierarchies, or a volumetric format in which a three-dimensional object is presented across multiple hierarchies; and the second processing unit, when the representation format is the single-layer format, determines the formation so that a plurality of the transparent display devices are arranged in a line parallel to their display surfaces, and when the representation format is the multi-layer format or the volumetric format, determines the formation so that a plurality of the transparent display devices are arranged in a line in the depth direction corresponding to a plurality of the hierarchies.

3. The first processing unit determines whether the Volumetric format is a first type, a second type, or a third type; the second processing unit, when it is determined that the Volumetric format is the first type or the second type, arranges the plurality of the transmissive display devices at equal intervals corresponding to each of the plurality of hierarchical layers, and when it is determined that the Volumetric format is the third type, arranges the plurality of the transmissive display devices at uneven intervals corresponding to each of the plurality of hierarchical layers; the third processing unit, when the Volumetric format is the first type, generates a plurality of pieces of display information corresponding to each of the plurality of hierarchical layers with an entire region of the presentation target content as a presentation target, when the Volumetric format is the second type, generates a plurality of pieces of display information corresponding to each of the plurality of hierarchical layers with an attention region of the presentation target content as a presentation target, and when the Volumetric format is the third type, generates the display information corresponding to the entire region of the presentation target content corresponding to a first layer of the plurality of hierarchical layers, and generates the display information corresponding to the attention region of the presentation target content corresponding to a second layer of the plurality of hierarchical layers. The information presentation system according to claim 2 .

4. An information presentation method executed by an information presentation control device connected to a plurality of transparent display devices and a plurality of moving media supporting the plurality of transparent display devices in a manner that allows the arrangement of each of the plurality of transparent display devices to be variably adjusted, the information presentation method comprising: a step of acquiring a plurality of contents having different expression formats and storing them in a memory unit; a step of determining the expression format of a presentation target content designated as a presentation target from the plurality of contents; a step of determining a formation of the plurality of transparent display devices in accordance with the determined expression format, and variably controlling the arrangement of the plurality of transparent display devices by operating the plurality of moving media in accordance with the determined formation; and a step of generating display information based on the presentation target content to be displayed on the plurality of transparent display devices whose arrangement has been variably controlled in accordance with the determined expression format, and outputting the generated display information to the corresponding transparent display device.

Images