Video display method, video information creation device, and video information display system

Abstract

This video display method includes: a user position acquisition step in which a processor acquires the position of a user; a specific target position acquisition step in which the processor acquires position information of a specific target for which guidance is to be provided as a target of interest to the user; an area calculation step in which the processor calculates an area where a virtual line connecting the position of the user and the position of the specific target intersects a display; an identification object generation step in which the processor generates an identification object for identifying the user; and a display step in which the processor adds the identification object to guidance information indicating the position of the specific target and displays the same in the area where the virtual line intersects the display.

Description

Video display method, video information creation device, and video information display system

[0001] The present technology relates to a video display method, a video display device, and a video information display system, and in particular to a video information creation device, a video information display system, and an information terminal device that provide guidance for finding a target object using AR (Augmented Reality).

[0002] Patent Document 1 discloses the following description: "The information processing device includes a display method setting unit that sets a display method for visual information to be displayed superimposed on the field of view of multiple users, and a display control unit that controls the display of the visual information according to the set display method, and the display methods include individual display that displays the visual information individually for each user, and integrated display that integrates and displays the visual information for multiple users. This technology can be applied, for example, to a vehicle that displays visual information using AR (abstract excerpt)."

[0003] Japanese Patent Application Laid-Open No. 2021-165766

[0004] The invention described in Patent Document 1 is a technology for an in-vehicle HUD (head-up display) that uses a stereo camera to measure the distance to an object and focuses an image (visual information) at the same position as the object, thereby making it difficult for the visual information to shift from the position of the object even if the driver's viewpoint shifts sideways. Therefore, since a vehicle is driven by only one person, Patent Document 1 does not take into consideration the possibility of multiple users viewing from various positions.

[0005] Therefore, even if Patent Document 1 is applied to a situation where it is desired to simultaneously provide multiple users with guidance information that allows them to see a sightseeing spot, such as an observation deck, because multiple users are trying to find the object from different viewpoints, there remains the problem that appropriate guidance cannot be provided from the perspective of each user, leading to the user's gaze being directed in the wrong direction, or that if guidance is provided to multiple users simultaneously, it becomes unclear who the guidance is intended for.

[0006] The present invention has been made to solve the above-mentioned problems, and has an object to simultaneously guide a plurality of users in the direction of an object as seen from each of their respective perspectives.

[0007] In order to solve the above problems, the present invention has the configurations described in the claims.

[0008] According to the present invention, it is possible to simultaneously guide a plurality of users in the direction of an object as seen from each of the users. Note that problems, configurations, and effects other than those described above will become apparent from the following description of the embodiments.

[0009] 1 is a hardware configuration diagram of a video information creation device of a first embodiment; FIG. 2 is a diagram showing a functional block configuration of the video information creation device of the first embodiment; FIG. 3 is a diagram showing a first example for explaining visual information of the video information creation device of the first embodiment; FIG. 4 is a diagram showing a main processing flow of the first example of the video information creation device of the first embodiment; FIG. 5 is a diagram showing a second example for explaining visual information of the video information creation device of the first embodiment; FIG. 6 is a diagram showing a main processing flow of the second example of the video information creation device of the first embodiment; FIG. 7 is a diagram showing a first example of object creation of the video information creation device of the first embodiment; FIG. 8 is a diagram showing a second example of object creation of the video information creation device of the first embodiment; FIG. 9 is a diagram showing a third example of object creation of the video information creation device of the first embodiment; FIG. 10 is a diagram showing a third example for explaining visual information of the video information creation device of the first embodiment; FIG. 11 is a diagram showing a main processing flow of the third example of the video information creation device of the first embodiment; FIG. 12 is a diagram showing a fourth A example for explaining visual information of the video information creation device of the first embodiment; FIG. 13 is a diagram showing a fourth B example for explaining visual information of the video information creation device of the first embodiment; FIG. 14 is a diagram showing a fourth C example for explaining visual information of the video information creation device of the first embodiment; FIG. 15 is a diagram showing main processing flows of the fourth A, fourth B, and fourth C examples of the video information creation device of the first embodiment. 1 is a diagram showing a fifth example for explaining visual information of the video information creation device of the first embodiment. FIG. 2 is a diagram showing a main processing flow of the fifth example of the video information creation device of the first embodiment. FIG. 3 is a diagram showing a sixth example for explaining visual information of the video information creation device of the first embodiment. FIG. 4 is a diagram showing a main processing flow of the sixth example of the video information creation device of the first embodiment. FIG. 5 is a diagram showing a functional block configuration of a video information display system of the second embodiment. FIG. 6 is a diagram showing a first example for explaining visual information of the video information display system of the second embodiment. FIG. 7 is a diagram showing a main processing flow of the first example of the video information display system of the second embodiment. FIG. 8 is a diagram showing a main processing flow of the first example of the video information display system of the second embodiment. FIG. 9 is a diagram showing a second example for explaining visual information of the video information display system of the second embodiment. FIG. 10 is a diagram showing a main processing flow of the second example of the video information display system of the second embodiment.1 is a diagram showing a main processing flow of a second example of the video information display system of the second embodiment. FIG. 2 is a diagram showing a functional block configuration of an information terminal device of a third embodiment. FIG. 3 is a diagram showing a first example for explaining visual information of an information terminal device of the third embodiment. FIG. 4 is a diagram showing a first example for explaining visual information of an information terminal device of the third embodiment. FIG. 5 is a diagram showing a main processing flow of the first example of the information terminal device of the third embodiment. FIG. 6 is a diagram showing a main transmission / reception processing flow of the first example of the information terminal device of the third embodiment. FIG. 7 is a diagram showing a second example for explaining visual information of an information terminal device of the third embodiment. FIG. 8 is a diagram showing another example of the second example for explaining visual information of the information terminal device of the third embodiment. FIG. 9 is a diagram showing a main processing flow of the second example of the information terminal device of the third embodiment. FIG. 10 is a diagram showing a transmission / reception processing flow with a server of the second example of the information terminal device of the third embodiment. FIG. 11 is a diagram showing a third example for explaining visual information of the information terminal device of the third embodiment. FIG. 12 is a diagram showing another example of the third example for explaining visual information of the information terminal device of the third embodiment. FIG. 13 is a diagram showing a fourth example for explaining visual information of the information terminal device of the third embodiment.

[0010] By displaying virtual reality objects and guiding multiple users in the same space to specific objects that can be viewed simultaneously, the present invention is expected to improve technology for creating more effective use of virtual reality displays in labor-intensive industries, and is therefore expected to contribute to Goal 8.2 of the United Nations' Sustainable Development Goals (SDGs) (increasing economic productivity through diversification, technological improvement and innovation, particularly in industries that add value to goods and services and labor-intensive industries).

[0011] An embodiment of the present invention and various examples will be described below using drawings and the like. The following description shows specific examples of the contents of the present invention, and the present invention is not limited to these descriptions. Various changes and modifications can be made by those skilled in the art within the scope of the technical ideas disclosed in this specification. Furthermore, in all drawings used to explain the present invention, parts having the same function are designated by the same reference numerals, and repeated explanations may be omitted. Below, matters common to all embodiments will be described, followed by descriptions of the features of each embodiment. Note that the effects described here are not necessarily limited and may be any of the effects described in this disclosure.

[0012] The video information creation device 100 in this embodiment simultaneously displays guidance for multiple users in the direction of a specific object that can be seen in large, medium, or small observation rooms, building windows, school windows, vehicle windows such as trains and tourist buses, HUDs, and aquariums.

[0013] 1A and 1B are diagrams showing the hardware configuration and functional block configuration of a video information creation device according to a first embodiment of the present invention.

[0014] The video information creation device 100 in this embodiment is, for example, a transmissive display, a head-up display, a smartphone, a tablet terminal, a personal computer, or the like.

[0015] 1A, the video information creation device 100 includes a main processor 101, a memory 110, an input / output interface 120, a display 134, a sensor group 150, a communication interface (I / F) 160, and an audio processing unit 170. These components are connected to each other via a bus 102.

[0016] The memory 110 includes a RAM 111, a ROM 112, a flash memory 113, and the like.

[0017] The input / output interface 120 includes various button switches 121, a touch panel 122, and the like.

[0018] The display 134 is configured as a transmissive display that allows the user to see through to the back (including a transmissive organic EL display, a transmissive inorganic EL display, a transmissive LCD display, and also glass that can reflect and display projected information).

[0019] The sensor group 150 includes a satellite positioning sensor 151, a geomagnetic sensor 152, a distance sensor 153, an in-camera 154, an out-camera 155, other sensors 156, and the like.

[0020] The communication interface 160 includes a wireless communication interface 161, a user information communication interface 162, a short-range wireless communication interface 163, and the like.

[0021] The audio processing unit 170 includes a speaker 171 as an audio output device and a microphone 172 as an audio input device.

[0022] The memory 110 stores programs and the like for controlling the entire video information creation device 100 by the main processor 101. Various data and the like to be processed by each component are temporarily stored via the bus 102.

[0023] The input / output interface 120 operates as follows: Various button switches 121 are switches that turn on or off the operation of this function, for example.

[0024] The touch panel 122 is configured with means for inputting information by touching buttons and the like displayed on the display 134 .

[0025] The sensor group 150 operates as follows. The satellite positioning sensor 151 is a sensor for measuring position using a satellite positioning system such as the Global Positioning System (GPS), and receives signals from multiple GPS satellites to obtain highly accurate three-dimensional current location information. It is also possible to use a Regional Network Satellite System (RNSS) as an SBAS (augmenting satellite constellation) used to reinforce and correct collected data in order to improve the positioning accuracy of GPS.

[0026] The geomagnetic sensor 152 detects the earth's magnetic force and measures two-dimensional and three-dimensional directions.

[0027] The distance sensor 153 measures the distance by emitting pulses of ultrasonic waves or laser light, hitting an object, and measuring the time it takes for the light to return.

[0028] The in-camera 154 is an image capturing device that captures an image of the user looking at the display 134 .

[0029] The outer camera 155 is an image capturing device that captures an image of what is in front of the display 134 as seen by the user.

[0030] The other sensors 156 are detectors for obtaining necessary information such as the surrounding environment (brightness, temperature, noise, etc.).

[0031] The operation of the communication interface 160 is as follows: The wireless communication interface 161 is an interface having a function of transmitting and receiving wireless communication used in mobile communication such as 4G / 5G.

[0032] The wireless communication interface 161 can obtain necessary information from a server on an external network.

[0033] The user information communication interface 162 is an interface having a function of transmitting and receiving data via medium-range wireless communication, such as Wi-Fi (registered trademark).

[0034] The short-range wireless communication interface 163 is an interface having a function of transmitting and receiving short-range wireless communication such as Bluetooth (registered trademark) or NFC (Near Field Communication).

[0035] As shown in FIG. 1B, the video information creation device 100 includes a video information processing unit 180 and a visual information generating unit 190 as main functional blocks.

[0036] The video information processing unit 180 includes an object superimposing unit 181 , a position information creating unit 182 , and a display method setting unit 183 .

[0037] The visual information generation unit 190 includes an object generation unit 191 , a display method change unit 192 , a user recognition unit 193 , and a gaze position detection unit 194 .

[0038] The operation of the video information processing unit 180 is as follows.

[0039] The position information creating unit 182 performs calculations to determine where on the display 134 the object for each user created by the object creating unit 191 or the display method changing unit 192 should be placed.

[0040] The display method setting section 183 sets the display method of the video information.

[0041] The object superimposing unit 181 superimposes and displays the object for each user created by the object generating unit 191 or the display method changing unit 192 at the arrangement position determined by the position information creating unit 182 .

[0042] The operation of the visual information generator 190 is as follows.

[0043] The user recognition unit 193 detects and recognizes the user using the in-camera 154 or the like.

[0044] The object generation unit 191 creates an object that enables identification of each user recognized by the user recognition unit 193 .

[0045] The gaze position detection unit 194 detects which part of the display 134 is visible for each user.

[0046] The display method change unit 192 appropriately changes the display method when the identification objects for each user cannot be displayed properly due to overlapping or the like.

[0047] The components included in the video information processing unit 180 and the visual information generating unit 190 are configured by the main processor 101 executing the video display program according to this embodiment stored in the memory 110, and the functions of each component are realized by the cooperation of hardware and programs.

[0048] (First Example of First Embodiment) The first example is characterized in that the specific object, which is the object that guides the user in the viewing direction, is a stationary object. Figure 2 is a diagram showing a first example for explaining visual information of the video information creation device of the first embodiment. Figure 3 is a diagram showing the main processing flow of the first example of the video information creation device of the first embodiment.

[0049] 2, the video information creation device 100 includes a transparent display 201 as the display 134 for viewing the scenery from inside the observation room, four sensor cameras 202a, 202b, 202c, and 202d as the in-cameras 154 arranged inside the four corners of the transparent display 201, and four sensor cameras 203a, 203b, 203c, and 203d as the out-cameras 155 arranged outside the four corners of the transparent display 201. Note that the number and positions of the cameras arranged as the in-cameras 154 and the out-cameras 155 are merely examples, and the number and positions of the cameras may be different from those shown in FIG. 2 as long as the desired video can be captured. This also applies to other drawings.

[0050] At least one object 204a that can be viewed from the observation room, such as Mt. Fuji or a sunset, is displayed. Of these objects 204a that can be viewed, an object selected to guide the user in the viewing direction is a specific object. Specific objects are selected one by one at a predetermined time interval from among the at least one object 204a. In FIG. 2, it is assumed that specific object 205a (Mt. Fuji) has been selected. The specific object selection process may be performed by the object generation unit 191, or the user may orally utter the name of the specific object about which the user wishes to be guided, and the microphone 172 may collect the sound, and the object generation unit 191 may perform voice analysis to select the specific object.

[0051] The object superimposing unit 181 displays guidance information in which an identification object 207a for A is added to an area on the transparent display 201 that is in the line of sight of user A 206a when looking at the specific object 205a (Mount Fuji). Note that the "area on the transparent display 201 that is in the line of sight of user A 206a when looking at the specific object 205a (Mount Fuji)" here refers to the line of sight when the user views the specific object 205a from their current location, not the line of sight when the user is not looking at the specific object. Therefore, more accurately, the object superimposing unit 181 determines the area where a virtual line connecting the user's position and the position of the specific object intersects with the display, and uses this area as the area to display the identification object 207a for A. The same applies to the following description. The guidance information is a graphic that includes part or all of the specific object, and by adding an identification object, it is possible to identify which user the guidance information is intended for.

[0052] Furthermore, the object superimposing unit 181 displays guidance information with an identification object 209a for B added in an area on the transparent display 201 that is in the line of sight of user B 208a when he or she sees the specific object 205a (Mount Fuji). At this time, the object superimposing unit 181 displays text information 210 such as "Mount Fuji is visible" in the upper left corner of the transparent display 201. The object 204a and the text information 210 are stored in advance in the memory 110 according to the installation location of the video information creation device 100.

[0053] Next, the processing flow of a first example of the first embodiment will be described with reference to FIG.

[0054] As shown in FIG. 3, when the video information creation device 100 is started (S801), the user recognition unit 193 recognizes a user (user A 206a or user B 208a) looking at the outdoor scenery through the transparent display 201 based on the image data obtained from four sensor cameras 202a, 202b, 202c, and 202d arranged inside the four corners of the transparent display 201 (S802).

[0055] In step 802, the position information creation unit 182 determines the three-dimensional position information of the specific object 205a (Mount Fuji) and the transmissive display 201 that it has previously determined, and the newly obtained three-dimensional position information of user A 206a and user B 208a. More precisely, the position information creation unit 182 detects the position information of the eyes of each user as the position information of user A 206a and user B 208a.

[0056] Next, the object generating unit 191 generates a different identification object for A 207a and an identification object for B 209a so that the detected user A 206a and user B 208a can be individually identified (S803).

[0057] Next, the position information creation unit 182 determines whether or not the user A 206a and the user B 208a are in a position where the specific object 205a (Mount Fuji) can be seen through the transparent display 201 (S804). If it is determined in step 804 that the user A 206a and the user B 208a are not in a position where the specific object 205a (Mount Fuji) can be seen (S804: No), the identification object 207a for A and the identification object 209a for B are not output, and the process proceeds to step 808 where other guidance information display processing is performed, and the process ends in step 809.

[0058] On the other hand, if the position information creation unit 182 determines that user A 206a and user B 208a are in positions where the specific object 205a (Mount Fuji) is visible (S804: Yes), then in step 805, the position for displaying the identification object 207a for user A and the identification object 209a for user B on the transparent display 201 connecting the specific object 205a (Mount Fuji) with user A 206a and user B 208a is calculated (S805). Note that if the distance between the specific object and the transparent display is sufficiently farther than the distance between the transparent display and the user and the distance between the users, the directions in which the users are viewing the specific object are considered to be approximately the same. For this reason, the object display position for each user may be calculated by calculating a three-dimensional direction vector of the specific object relative to the transparent display in advance, and then translating the direction vector according to the user's position to simply calculate the position of the identification object, thereby reducing the amount of computational processing. This is also true for other embodiments.

[0059] The position information creation unit 182 then determines whether the multiple identified objects for each user are adjacent to or overlapping on the transparent display 201 (S806). For the multiple identified objects for which it is determined that the multiple identified objects for each user are adjacent to or overlapping (S806: Yes), the display method change unit 192 converts the multiple identified objects into an integrated object (described later) and outputs the converted object (S807). The object superimposition unit 181 displays the integrated object on the transparent display 201 (S808), providing guidance on the position from which the specific target object 205a (Mount Fuji) can be seen. The process then ends in step 809.

[0060] On the other hand, if the position information creation unit 182 determines that the multiple identified objects for each user are not adjacent to or overlapping with each other (S806: No), the object generation unit 191 outputs the identified object 207a for A and the identified object 209a for B, and the object superimposition unit 181 displays guidance information to which the identified objects (the identified object 207a for A and the identified object 209a for B) have been added on the transparent display 201 (S808), thereby providing guidance to a position from which the specific object 205a (Mount Fuji) can be seen. The process then ends in step 809.

[0061] The process of FIG. 3 is repeated in a cycle of detecting or displaying one image on the sensor cameras 202a, 202b, 202c, and 202d or the transmissive display 201 (for example, 60, 120, or 240 images per second).

[0062] Although not described in the processing steps of FIG. 3, the main processor 101 may output the text information 210 from the speaker 171 as voice guidance information.

[0063] Although the example of seeing the real world through the transparent display 201 has been described, it is also possible to display images of the real world captured using the sensor cameras 203a-d. This has the effect of enabling the display of images captured in the past when a specific object cannot be seen due to weather, time of day, etc.

[0064] (Second Example of First Embodiment) The second example is characterized in that the specific object is a moving body. In the following, a sunset will be taken as an example of the specific object. The position of the sunset relative to the user changes from moment to moment. FIG. 4 is a diagram showing a second example for explaining the visual information of the video information creation device of the first embodiment. FIG. 5 is a diagram showing the main processing flow of the second example of the video information creation device of the first embodiment. In the second example, the specific object 205e (sunset) moves from moment to moment, as compared to the first example. In FIG. 4, the same symbols are used for components that are the same as or have the same functions as those in FIG. 2, and their description will be omitted.

[0065] The video information creation device 100 includes a transparent display 201 and sensor cameras 202a-d and 203a-d. A sunset is displayed as an example of an object 204e that can be seen from the observation room, and guidance information for a specific object 205e (sunset) is displayed.

[0066] The object superimposing unit 181 displays an identification object 207e for A in an area on the transparent display 201 that is in the line of sight of user A 206e when he sees the specific object 205e (sunset). Similarly, the object superimposing unit 181 displays an identification object 209e for B in an area on the transparent display 201 that is in the line of sight of user B 208e when he sees the specific object 205e (sunset). At this time, text information 210e such as "You can see the sunset" is displayed in the upper left corner of the transparent display 201.

[0067] Next, the processing flow of a second example of the first embodiment will be described with reference to Fig. 5. In Fig. 5, steps that perform the same processing or the same functions as those in Fig. 3 are given the same reference numerals, and their description will be omitted.

[0068] As shown in Figure 5, when the video information creation device 100 is started (started) (S801), the location information creation unit 182 detects a specific object 205e (sunset) based on the image data obtained from four sensor cameras 203a, 203b, 203c, and 203d arranged outside the four corners of the transparent display 201 (S812).

[0069] If the location information creation unit 182 cannot detect the specific object 205e (sunset) (S813: No), the object generation unit 191 does not output the identified object, but proceeds to step 808 to perform other guidance information display processing, and ends at step 809.

[0070] On the other hand, if the position information creation unit 182 determines that the specific object 205e (sunset) has been detected (S813: Yes), the position information creation unit 182 calculates the position of the specific object 205e (sunset) from the output of the sensor cameras 203a, 203b, 203c, and 203d (S814).

[0071] If the specific object 205e (sunset) is a celestial body such as the sun, the position information creation unit 182 calculates the position information of the video information creation device 100 in the world coordinate system using the satellite positioning sensor 151, etc., and communicates with a server on an external network via the wireless communication interface 161, and if the time at the location of the video information creation device 100 is known, the position information of the celestial body can be obtained.

[0072] The sensor cameras 203a to 203d are also provided with the function of the distance sensor 153. If the specific object is a closer object such as an airplane, a bird, or a fish in an aquarium, the position information creation unit 182 may successively calculate the movement of the specific object by triangulation using the outputs of at least two of the sensor cameras 203a to 203d. Thereafter, steps 802 to 809 are performed in the same manner as in FIG. 3.

[0073] The process of FIG. 5 is repeated in a cycle of detecting or displaying one image on the sensor cameras 202a to 202d, the sensor cameras 203a to 203d, or the transmissive display 201 (for example, 60, 120, or 240 images per second).

[0074] In this second example, an example was explained in which the specific object 205e (sunset) moves sequentially, but similarly, if the video information creation device 100 is attached to the window of a train, a tourist bus, etc., the position of the video information creation device 100 also moves sequentially, so in step 814, the position of the video information creation device 100 is calculated using a satellite positioning sensor 151, etc., and the relative position is calculated sequentially.

[0075] (Example of Identification Object Generation) Fig. 6 is a diagram showing a first example of the object generation unit 191 of the video information creation device of the first embodiment and the generation of an identification object in step 803. Fig. 7 is a diagram showing a second example of the object generation of the video information creation device of the first embodiment. Fig. 8 is a diagram showing a third example of the object generation of the video information creation device of the first embodiment.

[0076] FIG. 6 shows a user 221 and an identification object 222 generated by the object generation unit 191 in step 803 .

[0077] The object generation unit 191 captures the external characteristics of the user 221 with the in-camera 154 as identification information for the user 221, and generates and displays an identification object 222 that highlights and identifies the face or characteristic parts of the whole body of the user 221 (face shape, hairstyle, eyebrows, eyes, nose, mouth, glasses, hat, body shape, clothing). In this embodiment, the object generation unit 191 particularly extracts facial features, and creates an identification object 222 that represents a round body part and guides the user to see specific targets 205a and 205e within this circle.

[0078] 7 shows an example of a merged identification object. In this example, when an identification object 223 and an identification object 224 for two users are adjacent or overlapping, parts of the respective identification objects are merged to form a merged object 225.

[0079] When multiple identification objects for each user are adjacent to or overlap each other on the transparent display 201, the display method change unit 192 creates an integrated object 225 that uses a common round body part in step 807. This improves visibility compared to displaying two identification objects overlapping each other.

[0080] FIG. 8 shows another example of an integrated object, an integrated object 230 that integrates identification objects 226, 227, 228, and 229 of four users.

[0081] (Third Example of First Embodiment) The third example is an improved technology to make it easier to distinguish between identified objects of users who are close to the display 134 and those who are far from it. Fig. 9 is a diagram showing a third example for explaining visual information of the video information creation device of the first embodiment. Fig. 10 is a diagram showing a main processing flow of the third example of the video information creation device of the first embodiment. In Fig. 9, the same symbols are used for components that are the same as or have the same functions as those in Figs. 2 and 4, and descriptions thereof will be omitted.

[0082] The video information creation device 100 includes a transparent display 201 and sensor cameras 202a-d and 203a-d. Mount Fuji is selected as an object 204a that can be seen from the observation room, and guidance information for a specific object 205a (Mount Fuji) is displayed.

[0083] The object superimposing unit 181 displays an identification object 207f for A in an area on the transparent display 201 that is in the line of sight of user A 206f when looking at the specific object 205a (Mount Fuji). Similarly, the object superimposing unit 181 displays an identification object 209e for B in an area on the transparent display 201 that is in the line of sight of user B 208f when looking at the specific object 205a (Mount Fuji). At this time, text information 210a such as "Mount Fuji is visible" is displayed in the upper left corner of the transparent display 201.

[0084] In a third example, the display mode of an identified object is different between an identified object of a user closer to the video information creation device 100 and an identified object of a user farther from the video information creation device 100. As an example of changing the display mode, at least one of the size, resolution, brightness, and saturation of part or all of the identified object may be changed. This improves the ease with which each user can find their own identified object.

[0085] Next, the processing flow of a third example of the first embodiment will be described with reference to Fig. 10. In Fig. 10, steps that perform the same processing or the same functions as those in Figs. 3 and 5 are given the same reference numerals, and their description will be omitted.

[0086] 10 is executed by the display method change unit 192. In step 831, the display method change unit 192 calculates the distance from the transparent display 201 to each of the detected users (user A 206f and user B 208f). The calculation method is triangulation using outputs from at least two or more of the four sensor cameras 202a to 202d arranged inside the four corners of the transparent display 201. The sensor cameras 202a to 202d may be provided with the function of the distance sensor 153 to calculate the distance.

[0087] In step 832, the display method change unit 192 changes at least one of the size, resolution, brightness, and saturation of part or all of the identified object created for each user (user A 206f and user B 208f) in step 803 according to the distance calculated in step 831. In this embodiment, the identified object 209f for B, which is closer, is displayed with the size of the round circle of the entire body reduced compared to the identified object 209f for B, without changing the size of the facial features.

[0088] According to this embodiment, guidance information can be displayed to a user B 208f who approaches the transparent display 201, narrowing the range of positions where the specific object 205a can be seen more accurately. Also, the user B 208f can more easily distinguish between his / her own identification object 209f for B and the other user A's identification object 207f based on their positional relationship with the other user A 206f, making it easier to find his / her own object on the transparent display 201. In order to make this distinction easier, in addition to changing the size of a part of the object in this embodiment, the same effect can be obtained by changing the resolution, brightness, saturation, etc. according to the distance.

[0089] (Examples 4A, 4B, and 4C of the First Embodiment) Examples 4A, 4B, and 4C are examples in which, when the identified objects of a user who is close and a user who is far away overlap on the display 134, the identified object of the closer user is displayed preferentially. Also shown are examples of a method performed by the visual information generation unit 190 and the video information processing unit 180 to display the identified object of the farther user so as not to obstruct the identified object of the closer user. FIGS. 11A, 11B, and 11C are diagrams showing Examples 4A, 4B, and 4C for explaining visual information of the video information creation device of the first embodiment. FIG. 12 is a diagram showing the main processing flow of Examples 4A, 4B, and 4C of the video information creation device of the first embodiment. In FIGS. 11A, 11B, and 11C, components that are the same or have the same functions as those in FIGS. 2, 4, and 9 are designated by the same reference numerals, and their description will be omitted.

[0090] The video information creation device 100 includes a transparent display 201 and sensor cameras 202a-d and 203a-d. Mount Fuji is selected as an object 204a that can be seen from the observation room, and guidance information for a specific object 205a (Mount Fuji) is displayed.

[0091] 11A is an example in which, when the identification objects of a user who is closer to the display 134 and a user who is farther away overlap, the identification object of the closer user is preferentially displayed. When the display of the identification object 207g (not shown) for A and the identification object 209g for B are adjacent to or overlapping in an area on the transparent display 201 in the line of sight of user A 206g and user B 208g when they are looking at the specific object 205a (Mount Fuji), the object superimposing unit 181 preferentially displays the identification object 209g for B of user B 208g, which is closer to the display 134, and does not display part or all of the identification object 207g (not shown) for A of user A 206g, which is farther away from the display 134, thereby improving visibility on the transparent display 201.

[0092] The processing flow of the fourth A example of the video information creation device will be described with reference to Fig. 12. In Fig. 12, steps that perform the same processes or functions as those in Figs. 3, 5, and 10 are given the same reference numerals, and their description will be omitted.

[0093] In step 841, for each user (user A 206g and user B 208g), when the display of the identification object 207g (not shown) for A and the identification object 209g for B overlap in step 806, the display method change unit 192 preferentially displays the identification object 209g for B of the closer user B 208g, and does not output part or all of the identification object 207g (not shown) for A of the farther user A 206g.

[0094] If an example of the third example of the first embodiment (Figures 9 and 10) is used in conjunction with this, the identification object 207g for A of the distant user A 206g may be larger in size than the identification object 209g for B of the close user B 208g. In this case, the identification object 207g for A is displayed protruding behind the identification object 209g for B, which is displayed preferentially, and the guidance information is also displayed for the distant user A 206g at the same time in step 808.

[0095] 11B is one example of displaying an identification object 207g1 for A of a distant user A 206g. In the fourth example B, the identification object of a distant user is displayed outside of an area that preferentially displays a closer user.

[0096] 12, the user recognition unit 193 detects and recognizes each user (user A 206g and user B 208g), and the position information creation unit 182 calculates and determines the location of each user. Then, in step 841, the display method change unit 192, based on the user identification results and the calculation results of the user locations, prevents the display of the identification object 207g1 for A that overlaps with the area of ​​the transparent display 201 that is obscured by the shadow of the closer user B 208g as seen from the farther user A 206g (the dashed line area in FIG. 11B).

[0097] This allows the display of objects of nearby users to be more easily seen. Note that objects of distant users may be enlarged so that they are easier to see from a distance.

[0098] 11C is one of different display examples of the identification object 207g2 for A of a distant user A 206g. The fourth example C is outside the area that preferentially displays a nearby user, and the identification object of the distant user is more actively enlarged and displayed so that it can be easily seen from a distance.

[0099] In the fourth example C, in step 832 of FIG. 12, the display method change unit 192 changes the size of the identification object for A of the farther user A to be larger than the size of the identification object for B of the closer user B.

[0100] This allows the identification object 207g2 for A of user A 206g to be displayed larger so as not to obstruct user B 208g's view of the identification object 209g for B (and the specific object 205a visible beyond it) (as shown in Figure 11C).

[0101] According to Examples 4A, 4B, and 4C, the display of the identification object of a user farther from the display does not obscure the identification object and specific object of a user closer to the display, making it easier for the closer user to see the specific object. In addition, the farther user can display a large identification object that is easy to see from a distance.

[0102] (Fifth Example of First Embodiment) In the fifth example, the gaze position detection unit 194 and the display method change unit 192 identify users who are looking at the display 134 and users who are not, and display on the display 134 identified objects only for users who are looking. FIG. 13 is a diagram showing a fifth example for explaining visual information of the video information creation device of the first embodiment. FIG. 14 is a diagram showing a main processing flow of the fifth example of the video information creation device of the first embodiment. In FIG. 13, components that are the same as or have the same functions as those in FIGS. 2, 4, 9, 11A, 11B, and 11C are assigned the same reference numerals, and descriptions thereof will be omitted.

[0103] The video information creation device 100 includes a transparent display 201 and sensor cameras 202a-d and 203a-d. Mount Fuji is selected as an object 204a that can be seen from the observation room, and guidance information for a specific object 205a (Mount Fuji) is displayed.

[0104] When an identification object 207h (not shown) for A and an identification object 209h for B are displayed in an area on the transparent display 201 that is in the line of sight of user A 206h and user B 208h when they are looking at a specific object 205a (Mount Fuji), if user A 206h is not looking in the direction of the transparent display 201, the identification object 207h (not shown) for A is not displayed, thereby improving visibility on the transparent display 201.

[0105] The processing flow of the fifth example of the video information creation device will be described with reference to Fig. 14. In Fig. 14, steps that perform the same processing or the same functions as those in Figs. 3, 5, 10, and 12 are designated by reference numerals, and their description will be omitted.

[0106] In step 851, the gaze position detection unit 194 detects the eyes of each user from the user's face images obtained by the four sensor cameras 202a to 202d arranged inside the four corners of the transparent display 201, detects the gaze of each user, and generates gaze information.

[0107] More precisely, the gaze position detection unit 194 detects the gaze based on the position of the iris using the inner corner of the eye or the like as a reference point (for example, if the iris of the left eye is far from the inner corner of the eye, the user is looking to the left, and if the inner corner of the left eye and the iris are close, the user is looking to the right). Also, the field of view is set to, for example, approximately 60 degrees with the center of the gaze as the reference.

[0108] In step 852, if the gaze position detection unit 194 determines that a specific object 205a (Mount Fuji) is within the user's field of view via the transparent display 201 based on the user's gaze information (S852: Yes), the process proceeds to step 805 and the required identification object 209h for B is displayed.

[0109] On the other hand, if the gaze position detection unit 194 determines that the specific object a (Mount Fuji) is not within the user's field of view (S852: No), the process proceeds to step 808 and no guidance information is displayed using an identified object.

[0110] According to the fifth example, by not displaying the identification objects of users who are not facing the display 134, the number of identification objects displayed on the display 134 can be reduced, thereby preventing the viewing of the display 134 from being hindered by users who are looking at the display 134.

[0111] (Sixth Example of First Embodiment) The sixth example is an example of a method of displaying guidance information when a user who is not looking in the direction of the transparent display 201 of the fifth example starts to turn toward the transparent display 201. Fig. 15 is a diagram showing a sixth example for explaining visual information of the video information creation device of the first embodiment. Fig. 16 is a diagram showing a main processing flow of the sixth example of the video information creation device of the first embodiment. In Fig. 15, components that are the same as or have the same functions as those in Figs. 2, 4, 9, 11A, 11B, 11C, and 13 are assigned the same reference numerals, and descriptions thereof will be omitted.

[0112] The video information creation device 100 includes a transparent display 201 and sensor cameras 202a-d and 203a-d. The sunset and Mt. Fuji, which are the highlights of the object 204a seen from the observation room, are displayed, and guidance information for a specific object 205a (Mt. Fuji) is also displayed.

[0113] When user A206i's line of sight enters the transparent display 201 from the lower right of the transparent display 201, as shown in the identification object 207t1 (207t2, 207t3) for A, a part of the identification object is displayed with information (an arrow mark in this embodiment) indicating the direction in which the specific object 205a (Mount Fuji) can be seen. This makes it possible to guide the user's line of sight toward the specific object 205a when the user is looking in a direction different from the position where the specific object 205a is located.

[0114] The processing flow of the sixth example of the video information creation device will be described with reference to Fig. 16. In Fig. 16, steps that perform the same processing or the same functions as those in Figs. 3, 5, 10, 12, and 14 are given the same reference numerals, and their description will be omitted.

[0115] In steps S861, S862, and S863, the eye gaze position detection unit 194 is processed by the detection sensor 157, and step S864 is processed by the display method change unit 192 in FIG.

[0116] If the gaze position detection unit 194 determines that a specific object 205a (Mount Fuji) is present within the field of view via the transparent display 201 based on the user's gaze information (S861: Yes), it proceeds to step 808, where the object superimposition unit 181 displays the required identification object 209i for B (although not shown, steps 805 to S807 in Figure 16 may also be provided).

[0117] On the other hand, if the gaze position detection unit 194 determines that a specific object a (Mount Fuji) is not within the user's field of view (S861: No), the gaze position detection unit 194 determines whether or not part of the transparent display 201 is within the user's line of sight (S862).

[0118] If the gaze position detection unit 194 determines that the user is in a position where the entire transparent display 201 is not visible within the user's line of sight (S862: No), the identification object 207i for A is not displayed, and the process proceeds to step 808, where the object superimposition unit 181 displays other guidance information.

[0119] On the other hand, if the gaze position detection unit 194 determines that the user is in a position where part of the transparent display 201 is visible within the user's line of sight (S862: Yes), it calculates a position close to the position where the specific object a (Mount Fuji) is visible within the user's field of view within the transparent display 201 (S863), and the object superimposition unit 181 displays guidance information by adding information indicating the direction in which the specific object 205a (Mount Fuji) is visible to part of the identified object, as shown in the identified object 207t1 for A (S864).

[0120] When user A206i looks at the information on the direction of the identification object 207t1 for A in this guidance information and moves his / her gaze in the upper left direction, in the next processing step of Figure 16, which is repeated, the user's gaze information moves in the upper left direction and the identification objects 207t2 and 207t3 for A are displayed moving, and finally the guidance is displayed up to the identification object 207i for A.

[0121] Here, in order to easily attract the user's attention because they are not displayed at the center of the user's line of sight, the displayed identification objects 207t1 to 207t3 for A and the identification object 207i for A are highlighted using video effects such as blinking, zooming out, and fading in for a predetermined period of time. This processing is performed by the display method changing unit 192, for example.

[0122] Although the first to sixth examples of the video information creation device of the first embodiment have been described separately to explain the features of each, they can be used in combination as long as there is no technical contradiction.

[0123] (Second embodiment) This second embodiment relates to a video information display system that combines the video information creation device 300 (100) of the first embodiment with an information terminal device 400, and is an embodiment in which guidance information generated and displayed by the video information creation device is transmitted to the information terminal device carried by the user. Figure 17 is a diagram showing the functional block configuration of the video information display system of the second embodiment.

[0124] The video information creation device 300 of the second embodiment is, for example, a see-through display, a head-up display, a smartphone, a tablet terminal, a personal computer, etc., as described with reference to Figures 1 to 16. The information terminal device 400 is, for example, a portable smartphone, a tablet terminal, a personal computer, a head-up display, etc.

[0125] The video information creation device 300 in this embodiment includes a main processor 301, a memory 310, an input / output interface 320, a sensor group 350, a communication interface 360, an audio processing unit 370, etc. These components are connected to one another via a bus 302. The details of each unit are as described in FIGS. 1 to 16 of the first embodiment.

[0126] In addition, in the video information creation device 300, a video information processing unit 380 and a visual information generation unit 390 are configured by the main processor 301 executing the video display processing program according to this embodiment. For convenience of explanation in Fig. 17, the video information processing unit 380 and the visual information generation unit 390, which are functional blocks configured by cooperation between hardware and software, are also illustrated in Fig. 17 together with the hardware elements that configure the video information creation device 300, but the video information processing unit 380 and the visual information generation unit 390 are the same elements as the video information processing unit 180 and the visual information generation unit 190 described in Fig. 1B.

[0127] The information terminal device 400 includes a main processor 401, a memory 410, an input / output interface 420, a display 434, a video information processing unit 480, a sensor group 450, a communication interface 460, and an audio processing unit 470. These components are connected to one another via a bus 402.

[0128] The memory 410 includes a RAM 411, a ROM 412, a flash memory 413, and the like.

[0129] The input / output interface 420 includes various button switches 421, a touch panel 422, and the like.

[0130] The video information processing unit 480 includes an object superimposing unit 481 , a position information creating unit 482 , and a display method setting unit 483 .

[0131] The sensor group 450 includes a satellite positioning sensor 451, a geomagnetic sensor 452, a distance sensor 453, an in-camera 454, an out-camera 455, other sensors 456, an acceleration sensor 457, a gyro sensor 458, a biometric information acquisition sensor 459, and the like.

[0132] The communication interface 460 includes a wireless communication interface 461 , a user information communication interface 462 , and a short-range wireless communication interface 463 .

[0133] The audio processing unit 470 includes a speaker 471, a microphone 472, and the like.

[0134] The memory 410 stores programs and the like for controlling the entire information terminal device 400 by the main processor 401. Various data and the like to be processed by each component are temporarily stored via the bus 402.

[0135] The various button switches 421 are switches for switching the operation of this function on or off, for example. The touch panel 422 is configured with means for inputting by touching buttons and the like displayed on the display 434.

[0136] The display 434 receives and displays video information between the communication interface 460 and the communication interface 360 ​​directly or via a communication line 501 .

[0137] The satellite positioning sensor 451 is a sensor for measuring a position using a satellite positioning system such as GPS, and receives signals from multiple GPS satellites to obtain highly accurate three-dimensional current location information. Note that RNSS (Regional Network Satellite System) may also be used as an SBAS (augmenting satellite constellation) used to reinforce and correct collected data in order to improve the positioning accuracy of GPS.

[0138] The geomagnetic sensor 452 detects the earth's magnetic force and measures two-dimensional and three-dimensional directions.

[0139] The distance sensor 453 measures the distance by emitting pulses of ultrasonic waves or laser light, hitting the target, and measuring the time it takes for the light to return.

[0140] The in-camera 454 is an image capturing device that captures an image of the user looking at the display 434 .

[0141] The outer camera 455 is an image capturing device that captures an image of what is in front of the display 434 as seen by the user.

[0142] The other sensor 456 is a detector for obtaining necessary information such as the surrounding environment (brightness, temperature, noise, etc.).

[0143] The acceleration sensor 457 detects changes in the speed of the movement of the information terminal device 400 .

[0144] The gyro sensor 458 detects the rotation speed of the information terminal device 400 .

[0145] The biometric information acquisition sensor 459 acquires the user's fingerprint, palm print, vein pattern, face pattern, voice pattern, iris pattern, etc., and authenticates the owner of the information terminal device.

[0146] The wireless communication interface 461 is an interface having a function of transmitting and receiving wireless communication used for mobile communication such as 4G / 5G, etc. The wireless communication interface 461 can acquire necessary information via an external network between the communication interface 360 ​​of the video information creation device 300 and a server on the external network.

[0147] The user information communication interface 462 is an interface having a function of transmitting and receiving medium-range wireless communication such as Wi-Fi (registered trademark), and can communicate directly with the communication interface 360 ​​of the video information creation device 300 .

[0148] The short-range wireless communication interface 463 has a short-range wireless communication transmission / reception function such as Bluetooth (registered trademark) or NFC, and can directly communicate small amounts of information with the communication interface 360 ​​of the video information creation device 300.

[0149] The speaker 471 outputs, as audio information, guidance information created by the video information processing unit 480. The microphone 472 inputs information input via the input / output interface 420 as audio.

[0150] (First Example of Second Embodiment) In the first example, a technique will be described in which part or all of the guidance information displayed on a display installed on an observation deck or the like is transferred as transmission guidance information to an information terminal device owned by a user.

[0151] Fig. 18 is a diagram showing a first example for explaining visual information of the video information display system of the second embodiment. Fig. 19, Fig. 20, and Fig. 21 are diagrams showing the main processing flow of the first example of the video information display system of the second embodiment.

[0152] 18, the video information creation device 300 displays a see-through display 201 as the display 334 for viewing the scenery from the observation room, and at least one object 204a that is a highlight from the observation room, such as Mt. Fuji or a sunset. Of these highlight objects 204a, an object selected to guide the user in the viewing direction is a specific object. In FIG. 18, a specific object 205a (Mt. Fuji) is selected and displayed highlighted.

[0153] User A 206a owns an information terminal device 400, which has the form of a smartphone 492 shown in Fig. 18. The display 434 is configured with an organic EL display or a liquid crystal display 473. Here, the liquid crystal display 473 on the smartphone 492 displays transmission guide information 493 (indicated by a dashed line) including a part or all of the display screen of the transparent display 201.

[0154] Next, a processing flow of a first example of the video information display system of the second embodiment will be described with reference to Figures 19, 20, and 21. The video information creation device 300 in this second embodiment is the same as or has the same functions as the video information creation device 100 of the first embodiment. The information terminal device 400 allows a user to freely view the transparent display 201 using a smartphone 492 or the like in a situation where it is difficult to view the transparent display 201 directly.

[0155] Fig. 19 shows the processing steps of the video information creation device 300. In Fig. 19, steps that perform the same processing or the same functions as those in Figs. 3, 5, 10, 12, 14, and 16 are given the same reference numerals, and their explanation will be omitted.

[0156] In step 871, the transparent display 201 displays the transparent real world or an image of the real world captured by the outer camera 155, and simultaneously displays guidance information for user A 206a with an identification object 207a for A superimposed on the transparent display 201 on a line connecting the user's position and the specific object. Then, transmission guidance information 493 is generated, which includes part or all of the guidance information including at least the identification object 207a for A from the display contents of the transparent display 201, and is transmitted to the information terminal device 400 from the communication interface 360 ​​directly or via the communication line 501.

[0157] The process of FIG. 19 is repeated in a cycle of detecting or displaying one image on the sensor cameras 202a to 202d or the transmissive display 201 (for example, 60, 120, or 240 images per second).

[0158] The information terminal device 400 is a smartphone 492 with a built-in liquid crystal display 473. Figures 20 and 21 show processing steps of the information terminal device 400.

[0159] When the information terminal device 400 is started (initiated) in step 901 of Figure 20 (S901), the imaging unit of the outer camera 455 is started (S902), and a two-dimensional code indicating connection information with the video information creation device 300 is photographed and read by capturing an image of the two-dimensional code displayed on the periphery of the video information creation device 300 or on part of the transparent display 201 (S903).

[0160] The information terminal device 400 determines whether or not connection information has been obtained from the read two-dimensional code, and if connection information has not been obtained (S904: No), the process proceeds to step 906 and ends.

[0161] On the other hand, if the information terminal device 400 has obtained the connection information (S904: Yes), it connects the communication interface 460 of the information terminal device 400 with the communication interface 360 ​​of the image information production device 300, and proceeds to step 906 and ends.

[0162] When the connection between the communication interface 460 and the communication interface 360 ​​of the image information production device 300 is completed, the process of FIG. 21 is started (commences).

[0163] The information terminal device 400 receives, via the communication interface 360, the video of the real world transmitted from the video information creation device 300 in step 871 and the transmission guide information 493 to which the identification object 207a for A of user A 206a has been added (S907).

[0164] The display method setting unit 483 detects the display selection mode that has been set in advance by the input / output interface 420 (S908), and determines whether or not display according to the display selection mode is possible.

[0165] If the display method setting unit 483 determines that it is not possible to display in accordance with the display selection mode (S909: No), the process proceeds to step 911, where the identification object 207a for A is not displayed, and other guidance information (for example, video captured of the real world) is displayed (S911).

[0166] On the other hand, if the display method setting unit 483 determines that display according to the display selection mode is possible (S909: Yes), the object superimposing unit 481 superimposes (S910) the transmission guide information 493 including the guide information to which the identification object 207a for A of the user A 206a is added onto the video of the real world received from the video information creation device 300, and displays (S911).Then, the processing ends.

[0167] The display selection in the display selection mode includes not only whether or not to display guidance information, but also whether or not to display only a specific user A, etc.

[0168] The process of FIG. 21 is repeated in a cycle of displaying one image on the liquid crystal display 473 (for example, 60, 120, 240 frames per second).

[0169] According to this embodiment, guidance information with an identification object for each user added is displayed on the transparent display, but if the user wants to see it properly at hand, the display content of the transparent display including the user's identification object can be displayed on the mobile information terminal at hand as transmission guidance information. This allows the user to visually recognize the guidance information and specific objects for themselves on the information terminal device 400 at hand, improving usability.

[0170] Second Example of Second Embodiment In addition to the first example, a second example will explain a technique that makes it easier to find one's own identified object from among a plurality of identified objects displayed on the video information creation device 300 .

[0171] FIG. 22 is a diagram showing a second example for explaining visual information of the video information display system of the second embodiment.

[0172] 22 , the information terminal device 400 may have a user profile image registered in advance by the user himself / herself or an avatar 475 as his / her own alter ego. In the second example, the avatar 475 held by the information terminal device 400 is transmitted to the video information creation device 300, and an identification object 476 using the avatar 475 is generated and displayed, making it easier for the user to find his / her own familiar identification object 476 among the identification objects of other users.

[0173] Next, the processing flow of a second example of the video information display system of the second embodiment will be described with reference to Figures 23 and 24. Figures 23 and 24 are diagrams showing the main processing flow of the second example of the video information display system of the second embodiment. The video information creation device 300 in this example is the same as or has the same functions as the video information creation device 300 of the first example. Figure 23 shows the processing steps of the information terminal device 400. In Figure 23, steps that perform the same processes or the same functions as those in Figures 20 and 21 are given the same reference numerals, and their explanations will be omitted.

[0174] In step 921 , the information terminal device 400 transmits avatar information including an image of the avatar 475 pre-registered in the smartphone 492 from the communication interface 460 via the communication interface 360 ​​of the image information creation device 300 .

[0175] Fig. 24 shows the processing steps of the video information creation device 300. In Fig. 24, steps that perform the same processing or the same functions as those in Figs. 3, 5, 10, 12, 14, 16, and 19 are given the same reference numerals, and their explanation will be omitted.

[0176] In step 875 , the image information creation device 300 receives the avatar 475 that has been registered in advance in the smartphone 492 .

[0177] If the video information creation device 300 receives the avatar 475 (S876: Yes), the visual information generation unit 390 (display method change unit 392 not shown) changes the identification object created in step 803 to the avatar 475 and displays it as a guide in the processing from step 805 onwards.

[0178] On the other hand, if the video information creation device 300 does not receive the avatar 475 (S876: No), the visual information generation unit 390 (display method change unit 392 not shown) displays the guidance in the processing from step 805 onwards without changing the identification object created in step 803.

[0179] The process of FIG. 24 is repeated in a cycle of detecting or displaying one image on the sensor cameras 202a, 202b, 202c, and 202d or the transmissive display 201 (for example, 60, 120, or 240 images per second).

[0180] According to this embodiment, the avatar used by the user can be used as an identification object added to the guidance information, so that the user can easily find the guidance information relevant to him or her.

[0181] (First Example of Third Embodiment) The third embodiment is an extension of the second embodiment, and is an embodiment in which, when the user is in a position where the user cannot directly see the video information creation device 300, the user is guided to a position where the video information creation device 300 can be seen using the information terminal device 400. The information terminal device 400 of the third embodiment is, for example, a portable smartphone, tablet terminal, personal computer, head-up display, etc., as described in FIGS.

[0182] Fig. 25 is a diagram showing the functional block configuration of an information terminal device of the third embodiment. Figs. 26 and 27 are diagrams showing a first example for explaining visual information of an information terminal device of the third embodiment. Figs. 28 and 29 are diagrams showing the main processing flow of the first example of an information terminal device of the third embodiment. Figs. 30 and 31 are diagrams showing the main transmission / reception processing flow of the first example of an information terminal device of the third embodiment.

[0183] The information terminal device 400 in this embodiment includes a main processor 401, a memory 410, an input / output interface 420, a video information processing unit 480, a sensor group 450, a communication interface 460, and an audio processing unit 470. These components are connected to one another via a bus 402. The details of each unit are as described in FIGS. 17 to 24 of the second embodiment.

[0184] The communication interface 460 is connected to the video information creation device 300 or a related server 502 directly or via a communication line 501 to send and receive information.

[0185] 26 and 27, the same components as those in FIGS. 18 and 22 or those having the same functions are denoted by the same reference numerals, and the description thereof will be omitted.

[0186] 26, the image information creation device 300 is attached to a wall 511 inside the observation room. The specific object 205a (Mount Fuji) is not visible from the position of user A 206a because it is blocked by the wall 511 inside the observation room. Therefore, the image information creation device 300 transmits transmission guidance information to the information terminal device 400, and informs the user through the transmissive display of the image information creation device 300 that the specific object 205a is visible.

[0187] In FIG. 26, the smartphone 492 and the liquid crystal display 473 display images captured by the outer camera 455 and Mount Fuji, a sunset, and the like as objects 512 that are to be seen from the observation room.

[0188] User A 206 a can see from the image information 514 on the liquid crystal display 473 of the smartphone 492 that the specific object 205 a (Mount Fuji) is visible from the image information creation device 300 .

[0189] The user is then guided to move in that direction (to the left in Fig. 26). Fig. 27 shows the case where the user moves in accordance with the guidance in Fig. 26 and arrives at a position where the specific object 205a (Mount Fuji) can be seen through the transparent display of the image information creation device 300.

[0190] 26, user A is present within the angle of view of in-camera 454, and when an image of a virtual line of sight connecting user A 206a and specific object 205a (Mount Fuji) photographed by out-camera 455 is included in the image captured by out-camera 455 of information terminal device 400, guidance information 513 indicating the direction of specific object 205a (Mount Fuji) and image information 514 of specific object 205a (Mount Fuji) as seen from image information creation device 300 are displayed on the image on liquid crystal display 473. Note that the virtual line of sight is not the line of sight of the user because there is wall 511, but is a virtual line of sight of the user calculated by image information creation device 300, and if wall 511 were transparent, it would be the line when the user views specific object 205a.

[0191] Since user A 206a cannot be seen from the video information creation device 300, the guidance information 513 uses the avatar 475 registered on the smartphone 492 as described in the second example of the video information display system of the second embodiment of Figure 22.

[0192] The processing flow of a first example of the information terminal device of the third embodiment will be described with reference to Figures 28 and 29. In this embodiment, even if the specific object 205a (Mt. Fuji) is not visible to the user 206a, guidance information 513 is provided on the information terminal device 400 owned by the user 206a, guiding the user to the position of the image information creation device 300 in an observation room from which the specific object 205a (Mt. Fuji) can be seen. Additionally, image information 514 of the specific object 205a (Mt. Fuji) as seen from the image information creation device 300 is provided. In Figure 28, steps that perform the same processes or functions as those in Figures 20 and 23 are designated by the same reference numerals, and their description will be omitted.

[0193] Step 931 ends by displaying the image captured by the outer camera 455 and the surrounding object of interest 512 on the liquid crystal display 473.

[0194] In FIG. 29, steps that perform the same processing or the same functions as those in FIGS. 20 and 23 are given the same reference numerals, and their description will be omitted.

[0195] The user 206a selects from the objects 512 exemplified in step 931 of FIG. 28, or freely sets the name of the specific object 205 using the touch panel 422 on the liquid crystal display 473 (S932).

[0196] If the specific object 205 has not been set (S933: No), the process proceeds to step 911, where the object superimposing unit 481 ends the process without superimposing the guidance information 513.

[0197] On the other hand, if the object superimposition unit 481 determines that a specific object 205 has been set (S933: Yes), the object superimposition unit 481 acquires the location information of the information terminal device 400 using the satellite positioning sensor 451 or the like (S934).

[0198] Then, the information terminal device 400 transmits the image captured by the outer camera 455, the position information of the specific object 205, and the terminal position information to the image information creation device 300 (S935).

[0199] As described in the first embodiment, the video information creation device 300 creates guidance information 513 if there is a line connecting the user 206a (using the location information of the terminal owned) and the specific object 205a within the received captured video.

[0200] The information terminal device 400 receives the video of the real world captured by the outer camera 155 of the video information creation device 300 and the guidance information 513 (S936), and performs display processing of the guidance information (S911) and ends the process.

[0201] The processes in FIGS. 28 and 29 are repeated in a cycle of capturing or displaying one image on the outer camera 455 or the display 434 (for example, 60, 120, or 240 images per second).

[0202] Next, an example of a more specific main process of the transmission and reception operation between the information terminal device 400 and the video information production device 300 in steps 935 and 936 will be described with reference to Figures 30 and 31. Steps 935 and 936 in Figures 30 and 31 are the same processes as those in Figures 29 and 30.

[0203] FIG. 30 explains the processing in the information terminal device 400.

[0204] The information terminal device 400 transmits the captured image, the specific object, and the terminal position information (S935), which are received by the image information creation device 300 (S951). Based on the terminal position information received by the image information creation device 300, the received captured image is compared with 3D map information from a VPS (Visual Positioning System) that has previously generated an indoor map using detailed captured images of the building and a distance sensor, etc., and is capable of measuring the image by correlating it with a world coordinate system, thereby determining a more accurate terminal position (S952).

[0205] The image information creation device 300 calculates the line connecting this more accurate terminal position information and the specific object 205a (Mount Fuji) and the position information on the received captured image, and calculates the specific object display position information (S953).

[0206] If the received captured image includes an image of the display 334 (134) of the image information creation device 300, the image information creation device 300 replaces that image with an image of the real world captured by the outer camera 355 (155) of the image information creation device 300 and transmits the image and specific object display position information (S954).

[0207] The information terminal device 400 receives this information (S936), and executes the guidance information display process to display it (S911).

[0208] In this way, highly accurate guidance information can be provided using highly accurate VPS technology that uses images captured by satellite positioning sensors.

[0209] The process of calculating the position in the position information generating unit 482 in the information terminal device 400 will be described with reference to FIG.

[0210] The information terminal device 400 transmits a request for indoor map information (S935a), which is received by the image information creation device 300 (S961). The image information creation device 300 creates and transmits indoor map information (S962), which is received by the information terminal device 400 (S936a).

[0211] The information terminal device 400 transmits a specific object information request (S935b), and the image information creation device 300 receives the specific object information request (S963). The image information creation device 300 transmits the specific object information (S964), and the information terminal device 400 receives it (S936b).

[0212] The information terminal device 400 transmits a video information request to the video information creation device 300 (S935c), which is received by the video information creation device 300 (S965). The video information creation device 300 transmits video information captured by the outer camera 155 of the video information creation device 300 (S966), which is received by the information terminal device 400 (S936c). The position information creation unit 482 of the information terminal device 400 calculates the position of the guidance information, and the object superimposition unit 481 superimposes the guidance information 513. As described above, the guidance information to be displayed in step 911 is created by the information terminal device 400.

[0213] (Second Example of Third Embodiment) The second example provides a method of guiding to the position of the image information creation device 300 in an observation room where the specific object 205a (Mount Fuji) can be seen from a more distant position than in the first example.

[0214] Figures 32 and 33 are diagrams showing a second example and another example for explaining visual information of the information terminal device of the third embodiment. Figure 34 is a diagram showing the main processing flow of the second example of the information terminal device of the third embodiment. Figure 35 is a diagram showing the processing flow for sending and receiving with the server 502 of the second example of the information terminal device of the third embodiment. In Figures 32 and 33, parts that are the same as or have the same functions as those in Figures 26 and 27 are assigned the same reference numerals, and their explanations will be omitted.

[0215] The smartphone 492 displays on the liquid crystal display 473 the image captured by the outer camera 455, the positions of windows (image information creation devices 300) of multiple observation rooms or the like from which the specific object 205a (Mt. Fuji) closest to the current location can be seen, and image information 488, 489 of the specific object 205a (Mt. Fuji) visible from those positions. Furthermore, the smartphone 492 displays on the liquid crystal display 473 map information 498 to that point and guidance information (directions 484, 485 to the building and indoor guidance 486, 487 within the building).

[0216] Next, the processing flow of a second example of the third embodiment will be described with reference to Fig. 34. In Fig. 34, steps that perform the same processing or the same functions as those in Figs. 20, 21, 23, 28, and 29 are given the same reference numerals, and their description will be omitted.

[0217] In step 971, the information terminal device 400 (smartphone 492) determines whether there are multiple locations from which the specific object 205a (Mount Fuji) received in step 936 can be seen, and if there are not multiple locations (S971: No), displays the guidance information (S911).

[0218] On the other hand, if there are multiple locations (S971: Yes), the multiple locations are displayed as shown in Figure 32, and the user can select one of them (S972). Video information 488, 489 of specific object 205a (Mount Fuji) as seen from multiple locations is displayed as information for this selection. The displayed video information 488, 489 may be current video of the location (video information creation device 300) received via server 502, or it may be a prediction of the time it will take to reach the location from the current position and a display of video from a past image that will be seen at that time. This is particularly effective when the specific object 205e (sunset) is constantly moving.

[0219] The information terminal device 400 transmits information about the specified location to the server 502 (S973), receives guidance information (Figure 33) to the specified location from the server 502 (S974), and guides the user to a location where the specific object 205a (Mount Fuji) can be seen (S911).

[0220] Here, a detailed processing flow of transmission and reception between the information terminal device 400 and the server 502 (or the video information creation device 300) will be described with reference to Fig. 35. Steps S935(d, e), S936(d, e), and S951(d, e) to S954(d, e) in Fig. 35 are the same processes as those in Figs. 29, 30, and 31.

[0221] When the information terminal device 400 transmits the captured image, the specific object, and the terminal location information (S935d), the server 502 receives it (S951d), and based on the received terminal location information, it compares the received captured image with the VPS three-dimensional map information that has been created in advance by extracting characteristic parts from satellite images, images taken while driving around on roads and sidewalks, and information taken inside buildings, to determine a more accurate terminal location (S952).

[0222] The server 502 calculates the display position information of the specific object (S953), and if there are multiple video information creation devices 300 from which the specific object can be seen in the received captured video, the server 502 obtains video images of the real world as seen from each video information creation device 300 and transmits this information (S954d), which is received by the information terminal device 400 (S936d).

[0223] 33, the information terminal device 400 displays a plurality of locations, and the user specifies a location that he or she wishes to view from the plurality of locations (S981). The information terminal device 400 transmits information about the specified location (S982), and the server 502 receives the information about the specified location (S983).

[0224] The server 502 again transmits specific object position guidance information for only the designated location and the image of the image information creation device (S954e), which are received by the information terminal device 400 (S936e) and executes display processing of the guidance information (S911).

[0225] The process of FIG. 34 is repeated in a cycle of capturing or displaying one image on the outer camera 455 or the display 434 (for example, 60, 120, or 240 images per second).

[0226] According to this embodiment, the user can be guided to the position of the image information creation device 300 in the observation room where the specific object 205a (Mount Fuji) can be seen from a greater distance compared to the first example of the third embodiment.

[0227] (Third example of the third embodiment) In the third example, when a specific object that is difficult to find, such as a small fish in an aquarium tank, is displayed on the smartphone 492, this guidance information is also provided to a user C who does not own a smartphone 492.

[0228] 36 and 37 are diagrams showing a third example and another example for explaining visual information of the information terminal device of the third embodiment. The liquid crystal display of the smartphone 492 in Fig. 36 displays a message 210j indicating that a specific object, such as a small fish in an aquarium tank, can be seen.

[0229] In addition, the LCD display of the smartphone 492 also displays an identification object 523 for A that provides guidance information for user A206a in the situation of the first example shown in Figure 27, and an identification object 491 for C that provides guidance information if there is a user C nearby who does not own a smartphone.

[0230] Here, user A selects identification object 491 for C on smartphone 492 and lends smartphone 492 to user C, so that similar guidance information (FIG. 37) can be provided to user C who does not own a smartphone (or does not use the present invention). This display mode change is performed by the processing shown in steps 908 to S910 in FIG. 21.

[0231] (Fourth Example of Third Embodiment) FIG. 38 is a diagram illustrating a fourth example of visual information of the information terminal device of the third embodiment. In the fourth example, eyeglass-type smart glasses 495 are shown as one of the present embodiments. Smart glasses are convenient for hands-free use and are suitable for the purpose of providing guidance information of the present invention. All of the components of the information terminal device 400 shown in FIG. 25 may be built in, or at least the outer camera 496, the display 497, and part of the communication interface 460 may be built in, and the other components may be realized via wireless communication with a separately owned smartphone 492.

[0232] Although the embodiments of the present invention have been described above, it goes without saying that the configurations for realizing the technology of the present invention are not limited to the above-described embodiments, and various modifications are possible. For example, the above-described embodiments have been described in detail to clearly explain the present invention, and are not necessarily limited to those including all of the described configurations. Furthermore, it is possible to replace part of the configuration of one embodiment with the configuration of another embodiment, and it is also possible to add the configuration of another embodiment to the configuration of one embodiment. All of these fall within the scope of the present invention. Furthermore, numerical values, messages, etc. appearing in the text and figures are merely examples, and the effects of the present invention will not be impaired even if different ones are used.

[0233] The programs described in each processing example may be independent programs, or multiple programs may constitute a single application program. The order in which each process is performed may also be changed.

[0234] Some or all of the functions of the present embodiment described above may be implemented in hardware, for example, by designing them as integrated circuits. Alternatively, they may be implemented in software by a microprocessor unit, CPU, or the like interpreting and executing an operating program that implements each function. Furthermore, the scope of software implementation is not limited, and hardware and software may be used together. Alternatively, some or all of the functions may be implemented in a server. The server may be any type of server, such as a local server, cloud server, edge server, or internet service, as long as it can execute functions in cooperation with other components via communications. Information such as programs, tables, and files that implement each function may be stored in a memory, a recording device such as a hard disk or solid-state drive (SSD), or a recording medium such as an IC card, SD card, or DVD, or may be stored in a device on a communications network.

[0235] Furthermore, the control lines and information lines shown in the diagram are those considered necessary for explanation, and do not necessarily represent all of the control lines and information lines on the product. In reality, it can be assumed that almost all components are interconnected.

[0236] The above embodiments include the following inventions: (Supplementary Note 1) An image display method, comprising: a user position acquisition step of acquiring the position of a user, a specific object position acquisition step of acquiring position information of a specific object to be guided to the user as an object that the user should pay attention to, an area calculation step of calculating an area where a virtual line connecting the position of the user and the position of the specific object intersects with a display, an identified object generation step of generating an identified object that identifies the user, and a display step of adding the identified object to guidance information indicating the position of the specific object and displaying the information in the area where the virtual line intersects with the display. (Supplementary Note 2) A video information creation device comprising: a processor; an in-camera; an out-camera; and a display, wherein the processor acquires a user's position based on an image captured by the in-camera; acquires position information of a specific object to be guided to the user as an object that the user should pay attention to based on an image captured by the out-camera; calculates an area where a virtual line connecting the user's position and the specific object intersects with the display; generates an identification object that identifies the user; and adds the identification object to guidance information indicating the position of the specific object, and displays it in the area where the virtual line intersects with the display.(Supplementary Note 3) A video information display system in which a video information creation device and an information terminal device are communicatively connected, the video information creation device comprising: a first processor, an in-camera, an out-camera, a first display, and a first communication device, the first processor acquires a user's position based on an image captured by the in-camera, acquires position information of a specific object to be guided to the user as an object to which the user should pay attention based on an image captured by the out-camera, calculates an area where a virtual line connecting the user's position and the specific object intersects with the first display, generates an identification object that identifies the user, adds the identification object to guidance information indicating the position of the specific object and displays it in the area where the virtual line intersects with the first display, transmits a display screen on the first display showing the identification object and the guidance information from the first communication device to the information terminal device as transmission guidance information, and the information terminal device comprises: a second processor, a second display, and a second communication device, the second processor displays the transmission guidance information received by the second communication device on the second display.

[0237] 100: Video information creation device 101: Main processor 102: Bus 110: Memory 111: RAM 112: ROM 113: Flash memory 120: Input / output interface 121: Button switch 122: Touch panel 134: Display 150: Sensor group 151: Satellite positioning sensor 152: Geomagnetic sensor 153: Distance sensor 154: In-camera 155: Out-camera 156: Other sensors 157: Detection sensor 160: Communication interface 161: Wireless communication interface 162: User information communication interface 163: Short-range wireless communication interface 170: Audio processing unit 171: Speaker 172: Microphone 180: Video information processing unit 181: Object superimposition unit 182: Position information creation unit 183: Display method setting unit 190: Visual information generation unit 191: Object generation unit 192: Display method change unit 193: User recognition unit 194: Gaze position detection unit 201: Transmissive display 202a: Sensor camera 202b: Sensor camera 202c: Sensor camera 202d: Sensor camera 203a: Sensor camera 203b: Sensor camera 203c: Sensor camera 203d: Sensor camera 204a: Object 204e: Object 205: Specific object 205a: Specific object 205e: Specific object 206a: User 206e: User 206f: User 206g: User 206h: User 206i: User 207a: Identified object 207e: Identification object 207f: Identification object 207g: Identification object 207g1: Identification object 207g2: Identification object 207h: Identification object 207i: Identification object 207t1: Identification object 207t2: Identification object 207t3: Identification object 208a: User 208e: User 208f: User 208g: User208h: User 209a: Identification object 209e: Identification object 209f: Identification object 209g: Identification object 209h: Identification object 209i: Identification object 210: Character information 210a: Character information 210e: Character information 210j: Message 221: User 222: Identification object 223: Identification object 224: Identification object 225: Integrated object 226: Identification object 227: Identification object 228: Identification object 229: Identification object 230: Integrated object 300: Video information creation device 301: Main processor 302: Bus 310: Memory 320: Input / output interface 334: Display 350: Sensor group 355: Out camera 360: Communication interface 370: Audio processing unit 380: Video information processing unit 390: Visual information generation unit 392: Display method change unit 400: Information terminal device 401: Main processor 402: Bus 410: Memory 411: RAM 412: ROM 413: Flash memory 420: Input / output interface 421: Button switch 422: Touch panel 434: Display 450: Sensor group 451: Satellite positioning sensor 452: Geomagnetic sensor 453: Distance sensor 454: In-camera 455: Out-camera 456: Other sensors 457: Acceleration sensor 458: Gyro sensor 459: Biometric information acquisition sensor 460: Communication interface 461: Wireless communication interface 462: User information communication interface 463: Short-range wireless communication interface 470: Audio processing unit 471: Speaker 472: Microphone 473: Liquid crystal display 475: Avatar 476: Identification object 480: Video information processing unit 481: Object superimposition unit 482: Position information creation unit483: Display method setting unit 484: Route guidance 485: Route guidance 486: Indoor guidance 487: Indoor guidance 488: Video information 489: Video information 491: Identified object 492: Smartphone 493: Transmission guidance information 495: Smart glasses 496: Outer camera 497: Display 498: Map information 501: Communication line 502: Server 511: Wall 512: Object 513: Guidance information 514: Video information 523: Identified object

Claims

1. A method for displaying video, comprising: a user position acquisition step for acquiring the position of a user; a specific object position acquisition step for acquiring position information of a specific object to be guided to the user as an object that the user should pay attention to; an area calculation step for calculating an area where a virtual line connecting the position of the user and the position of the specific object intersects with a display; an identification object generation step for generating an identification object that identifies the user; and a display step for adding the identification object to guidance information indicating the position of the specific object and displaying it in the area where the virtual line intersects with the display.

2. An image display method according to claim 1, wherein in the identification object generating step, the processor generates the identification object indicating facial features of the user based on an image captured of the user.

3. A video display method according to claim 2, wherein in the display step, the processor displays the guidance information in which the identification object is added to a figure that encompasses all or a part of the specific object.

4. A method for displaying an image as described in claim 3, wherein the display is a transparent display, and the processor, in the specific object position acquisition step, acquires position information of the specific object based on an image captured of an actual outdoor scene seen through the transparent display, in the area calculation step, calculates an area where the virtual line connecting the position of the user indoors and the position of the specific object intersects with the transparent display, and in the display step, displays the guidance information superimposed on the actual outdoor scene viewed by the user through the transparent display.

5. A video display method as described in claim 1, wherein, when a first user and a second user are present, the processor, in the area calculation step, calculates a first area where a virtual line connecting the position of the first user and the specific object intersects with the display, and a second area where a virtual line connecting the position of the second user and the specific object intersects with the display; in the identification object generation step, generates a first identification object that identifies the first user, and a second identification object that identifies the second user; and, in the display step, displays the guidance information with the first identification object added in the first area, and displays the guidance information with the second identification object added in the second area.

6. A video display method as claimed in claim 1, wherein, when there are a first user and a second user, the processor, in the area calculation step, calculates a first area where a virtual line connecting the position of the first user and the specific object intersects with the display, and a second area where a virtual line connecting the position of the second user and the specific object intersects with the display; in the identification object generation step, generates a first identification object that identifies the first user and a second identification object that identifies the second user, and if the first area and the second area are not separated by a predetermined distance, generates an integrated object in which the first identification object and the second identification object are added to one piece of guidance information; in the display step, displays the integrated object in an area including a part or all of the first area or the second area; and in the identification object generation step, if the first area and the second area are separated by a predetermined distance, generates a first identification object that identifies the first user and a second identification object that identifies the second user, the display step displays the guidance information with the first identification object added in the first area, and displays the guidance information with the second identification object added in the second area.

7. A method for displaying an image as described in claim 5, wherein, in the display step, the processor displays the first identified object relatively small in the first area and displays the second identified object relatively large in the second area when the position of the first user is closer to the display than the position of the second user.

8. A video display method as described in claim 5, wherein, when the position of the first user is between the position of the second user and the specific object with respect to the display, in the display step, the processor hides the second identified object and the guidance information to which the second identified object is added within an area of ​​the second area where the line of sight of the first user is blocked as seen from the second user, and displays only the second identified object and the guidance information to which the second identified object is added within an area of ​​the second area where the line of sight of the first user is not blocked as seen from the second user.

9. A method for displaying an image as described in claim 1, wherein the processor further executes a gaze determination step for determining whether the user's gaze is directed toward the display, and in the display step, if the user's gaze is directed toward the display, the identification object and guidance information to which the identification object has been added are displayed in the display area, and if the user's gaze is not directed toward the display, the identification object and the guidance information to which the identification object has been added are not displayed in the display area.

10. A method for displaying an image as described in claim 1, wherein the processor further executes a gaze determination step for determining whether the user's gaze is directed toward the display, and in the display step, if the user's gaze deviates from the virtual line, the identification object and the guidance information with the identification object added are displayed in an area where the user's gaze and the display intersect, and the identification object and the guidance information with the identification object added are moved and displayed toward an area where the virtual line intersects with the display.

11. A video information creation device comprising a processor, an in-camera, an out-camera, and a display, wherein the processor acquires a user's position based on an image captured by the in-camera, acquires position information of a specific object to be guided to the user as an object that the user should pay attention to based on an image captured by the out-camera, calculates an area where a virtual line connecting the position of the user and the position of the specific object intersects with the display, generates an identification object that identifies the user, adds the identification object to guidance information indicating the position of the specific object, and displays it in the area where the virtual line intersects with the display.

12. A video information display system in which a video information creation device and an information terminal device are connected via communication, the video information creation device comprising a first processor, an in-camera, an out-camera, a first display, and a first communication device, the first processor acquires a user's position based on an image captured by the in-camera, acquires position information of a specific object to be guided to the user as an object to which the user should pay attention based on an image captured by the out-camera, calculates an area where a virtual line connecting the position of the user and the position of the specific object intersects with the first display, generates an identification object that identifies the user, adds the identification object to guidance information indicating the position of the specific object and displays it in the area where the virtual line intersects with the first display, and transmits a display screen on the first display in which the identification object and the guidance information are displayed from the first communication device to the information terminal device as transmission guidance information, the information terminal device comprising a second processor, a second display, and a second communication device, the second processor displays the transmission guidance information received by the second communication device on the second display.

13. A video information display system as described in claim 12, wherein avatar information representing an avatar of the user is transmitted from the information terminal device to the video information creation device, and the first processor uses an avatar image represented by the avatar information as the identification object.

Images