Image display device, information processor, and method for controlling image display

Abstract

To analyze a user state before a display unit and execute processing for switching a display content according to an analysis result, such as processing for switching a still image to a moving image.SOLUTION: A data processing unit for executing control of switching a display content of a display unit executes processing for analyzing an image taken by an imaging unit, determines whether a user detected from the taken image is satisfying a stipulated content switch condition, more specifically whether the user is imaging a display content, switches the display content of the display unit when the user is determined to be satisfying the content switch condition, and executes processing for switching a still image to a moving image.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] The present disclosure relates to an image display device, an information processing device, and an image display control method. More specifically, the present disclosure relates to an image display device, an information processing device, and an image display control method that execute control of display data, such as changing the display data of a display device, such as a signage device installed in a public area or the like, according to the actions of a user in front of the display device.

Background Art

[0002] Recently, display devices for information display have been widely used in public areas such as shopping centers, stations, airports, and busy streets. As an example of a display device, there is a signage device. A signage device, also called an electronic billboard or an electronic bulletin board, is a device that displays various information according to the installation location.

[0003] The signage device can output various contents, such as advertising information and guide information of various products and stores, further advertising information of movies, etc., and advertising images of performances of singers, actors, idols, etc.

[0004] For example, when displaying advertising information on a signage device, a high advertising effect can be obtained by displaying an advertisement according to the age and gender of people who gather in large numbers at the location where the signage device is placed. For example, Patent Document 1 (International Publication WO2018 / 105217) discloses a system that provides an advertisement according to the attributes of an advertisement-providing target.

[0005] However, the system described in this Patent Document 1 only changes the provided content according to the attributes of the advertisement-providing target user, such as gender and age group, and does not have an interactive configuration such as switching and displaying content according to user actions. Note that as an interactive content display control configuration, there is only a degree to which, for example, a user touches a screen to switch the display of content.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0007] The present disclosure has been made, for example, in view of the above problems, and an object thereof is to provide an image display device that switches display content of a display device such as a signage device according to the operation of a user near the display device or the like, an information processing device, and an image display control method.

[0008] One embodiment of the present disclosure executes switching control of display content when it is detected that a user near the display device, for example, has performed an operation of turning a smartphone toward the display device.

Means for Solving the Problems

[0009] The first aspect of the present disclosure is a display unit, an imaging unit that captures an image including at least the vicinity of the display unit, and a data processing unit that executes switching control of the display content of the display unit, wherein the data processing unit executes analysis processing of the captured image of the imaging unit, determines whether a user detected from the captured image is in a state corresponding to a specified content switching condition, and when it is determined that the detected user is in a state corresponding to the specified content switching condition, the image display device switches the display content of the display unit.

[0010] Furthermore, the second aspect of the present disclosure is a communication unit that communicates with the image display device, It has a data processing unit that executes switching control of the display content of the display device. The communication unit receives a captured image including at least the vicinity of the image display device from the image display device, The data processing unit executes analysis processing of the captured image, and determines whether or not the user detected from the captured image is in a state corresponding to a specified content switching condition. When it is determined that the detected user is in a state corresponding to the specified content switching condition, it is an information processing device that transmits, via the communication unit, at least one of a switching control command for the display content or the switching display content to the image display device.

[0011] Furthermore, a third aspect of the present disclosure is an image display control method executed in an image display device, The image display device has a display unit, an imaging unit that captures an image including at least the vicinity of the display unit, and a data processing unit that executes switching control of the display content of the display unit. The data processing unit executes analysis processing of the captured image of the imaging unit, and determines whether or not the user detected from the captured image is in a state corresponding to a specified content switching condition. It is an image display control method for switching the display content of the display unit when it is determined that the detected user is in a state corresponding to the specified content switching condition.

[0012] Still other objects, features, and advantages of the present disclosure will become apparent from more detailed descriptions based on the embodiments of the present disclosure and the accompanying drawings described later. Note that in this specification, a system is a logical collective configuration of a plurality of devices, and the devices of each configuration are not limited to those within the same housing.

[0013] According to the configuration of an embodiment of the present disclosure, a configuration is realized that analyzes the user state in front of the display unit and executes a switching process of display content according to the analysis result, for example, a switching process from a still image to a moving image. Specifically, for example, a data processing unit that executes switching control of the display content of the display unit executes analysis processing of the captured image of the imaging unit, and determines whether the user detected from the captured image meets a specified content switching condition, for example, whether the user is capturing the display content. When it is determined that the user is in a state that meets the content switching condition, the display content of the display unit is switched. For example, a process of switching a still image to a moving image is executed. With this configuration, a configuration is realized that analyzes the user state in front of the display unit and executes a switching process of display content according to the analysis result, for example, a switching process from a still image to a moving image. Note that the effects described in this specification are merely examples and are not limited, and there may be additional effects.

Brief Description of the Drawings

[0014]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Figure 26

Figure 27

Figure 28

Figure 29

Figure 30

Figure 31

Figure 32

DETAILED DESCRIPTION OF THE INVENTION

[0015] Hereinafter, with reference to the drawings, details of the image display device, information processing device, and image display control method of the present disclosure will be described. The description will be made according to the following items. 1. Outline of the processing executed by the image display device of the present disclosure 2. Configuration example of the image display device of the present disclosure 3. Sequence of the processing executed by the image display device of the present disclosure 4. Example of the switching condition of the content output by the image display device of the present disclosure 5. Example of the implementation in which an external device such as a server connected to the image display device via a network executes content switching control 6. Configuration example of the hardware such as the image display device 7. Summary of the configuration of the present disclosure

[0016] [1. Outline of the processing executed by the image display device of the present disclosure] First, with reference to FIG. 1 and below, the outline of the processing executed by the image display device of the present disclosure will be described.

[0017] FIG. 1 shows an image display device 10 and a user 30 who is a pedestrian walking in front of it. The image display device 10 is, for example, a signage device. As described above, a signage device is also called an electronic billboard, an electronic bulletin board, etc., and is a device that displays various information according to the installation location. The signage device can output various contents, for example, advertising information and guide information of various products and stores, further promotional information such as movies, and promotional images of performers such as singers, actors, and idols.

[0018] An image of a performer 20 is output to the display unit 11 of the image display device (signage device) 10 shown in FIG. 1. Note that a performer is, for example, a singer, an actor, a famous person, etc. The performer image 20a output to the display unit 11 of the image display device (signage device) 10 shown in FIG. 1 is a normal image (for example, a still image).

[0019] As shown in FIG. 1, the image display device 10 has an imaging unit (camera) 12. The imaging unit (camera) 12 continuously captures an image in front of the image display device 10, and the captured image is input to a data processing unit inside the image display device 10 and analyzed in the data processing unit.

[0020] The data processing unit inside the image display device 10 inputs the captured image of the imaging unit (camera) 12 and analyzes the actions (such as the actions) of the user 30 in front of the image display device 10. The data processing unit inside the image display device 10 executes the switching process of the content displayed on the display unit 11 according to the analysis result.

[0021] For example, when it is detected that the user 30 in front of the image display device 10 is trying to capture the displayed content on the display unit 11 by pointing a smartphone at the image display device 10, or is in the process of capturing, the data processing unit inside the image display device 10 executes the switching process of the displayed content.

[0022] Referring to FIG. 2, a specific example of the switching process of the displayed content will be described. FIG. 2 shows a state where the user 30 in front of the image display device 10 is pointing a smartphone at the image display device 10 and capturing the displayed content on the display unit 11.

[0023] The data processing unit of the image display device 10 analyzes the actions (actions) of the user 30 by using the captured image of the imaging unit (camera) 12. As a result of this analysis process, it is determined that the user 30 is performing an action of capturing the displayed content on the display unit 11 with a smartphone. Based on this action determination, the data processing unit of the image display device 10 executes the switching process of the displayed content on the display unit 11.

[0024] Specifically, the display of the performance image (normal image (still image)) 20a that was being displayed before the user action was confirmed is stopped, and the output of the performance image (special image (video)) 20b is started. That is, the image of the performance 20 is switched from a still image to a video, and a moving image is output. In addition, in conjunction with the output of the video, it is also possible to configure to output music, voices, such as the call or singing voice of the performance 20. The user 30 can record the video data and audio data output to the display unit 11 of the image display device 10 using the smartphone 40.

[0025] As described above, the image display device 10 of the present disclosure analyzes the user state such as the movement of the user such as a passerby in front of the display unit 11 of the image display device 10, and when it is determined that the user state satisfies the content switching condition defined in advance, the content switching process for the content displayed on the display unit 11 is executed.

[0026] In this embodiment, the "user state that satisfies the content switching condition defined in advance" is defined as the "state of executing display unit shooting by the user", but this is an example of the content switching condition.

[0027] The "user state that satisfies the content switching condition defined in advance" is not limited to the "state of executing display unit shooting process by the user", and it is possible to apply various user states such as other user actions, the user's belongings, and clothes. Specific examples will be described later.

[0028] In this embodiment, the display contents before and after the switching are a combination of still image content and video content, but this is an example, and the combination of the display contents before and after the switching is not limited to the combination of still images and videos, and various combinations such as the combination of still image a and still image b, and the combination of video a and video b are possible.

[0029] Note that the image display device 10 shown in FIG. 2 is configured to output an image of one performer. However, for example, as shown in FIG. 3, a configuration using an image display device 10 that outputs images of a plurality of different performers may be used.

[0030] The image display device 10 shown in FIG. 3 is configured to display three different performers, namely performer a, performer b, and performer c, side by side on the display unit 11. Note that the display unit 11 of the image display device 10 is configured as a single display unit, which is divided into three regions, and individual contents are output in units of regions.

[0031] Normally, a still image of performer a is displayed in the left region, a still image of performer b is displayed in the central region, and a still image of performer c is displayed in the right region. A user 30 who is a passerby is walking in front of the display unit 11 of this image display device 10.

[0032] FIG. 4 shows that the user 30 takes out the smartphone 40 while looking at the image displayed on the display unit 11, and takes a picture of the image of performer a displayed in the left region of the display unit 11 of the image display device 10. The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12 and determines that the user 30 is performing an operation of taking a picture of the image of performer a displayed in the left region of the display unit 11 with the smartphone. Based on this operation determination, the data processing unit of the image display device 10 executes a switching process for the image of performer a displayed in the left region of the display unit 11.

[0033] Specifically, the display of the still image of performer a that was being displayed before the user operation was confirmed is stopped, and the image of performer a is switched to a moving image (special image) to start the display of the moving image. In addition, in conjunction with the output of the moving image, it is also possible to configure to output music, voices, such as the call or song of performer 20, etc.

[0034] The user 30 can use the smartphone 40 to record the moving image data and voice data of performer a displayed in the left region of the display unit 11 of the image display device 10.

[0035] Note that at this time, the image of performer b in the central region and the image of performer c in the right region of the display unit 11 that are not the shooting targets by the smartphone 40 of the user 30 continue to be output as still images (normal images).

[0036] Next, Figure 5 shows a state where the user 30 has moved from the left side (p1) to the right side (p2) of the image display device 10 and started taking a picture of the image of performer c displayed in the right area of the display unit 11 of the smartphone 40.

[0037] The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12 and determines that the user 30 is performing an operation of taking a picture of the image of performer c displayed in the right area of the display unit 11 with the smartphone. Based on this operation determination, the data processing unit of the image display device 10 executes a process of switching the image of performer c displayed in the right area of the display unit 11 from a still image to a moving image.

[0038] Note that the image of performer a in the left area of the display unit 11 that is not the shooting target by the smartphone 40 of the user 30 is switched to a still image (normal image) at the end of the playback of the video content. Also, the image of performer b in the central area continues to be output as a still image (normal image).

[0039] The data processing unit of the image display device 10 determines the position of the user 30 and the shooting direction of the smartphone 40 by analyzing the captured image of the imaging unit (camera) 12, and executes a process of selecting only the image of the area in the shooting direction of the smartphone 40, switching it to a moving image, and playing it back. Details of the analysis process of the position of the user 30 and the shooting direction of the smartphone 40 executed by the data processing unit of the image display device 10 will be described later.

[0040] Figure 6 is a diagram showing an example of the transition of the display image of the display unit 11 when the user 30 moves from left to right while taking a picture with the smartphone 40. Time elapses in the order from the upper left (step S01) (time t = t1) of Figure 6 to the upper right center (step S02) (time t = t2) and the lower left (step S03) (time t = t3).

[0041] In the upper left (step S01) (time t = t1), the user 30 is taking a picture by turning the smartphone towards the image of performer a from the left end of the image display device 10. At this time, the data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the user 30's smartphone is the display area of performer a on the left side of the display unit 11, and switches only the image of performer a to a video. The images in the display areas of performers b and c remain still images.

[0042] Next, in the upper right center (step S02) (time t = t2), the user 30 is in a state of taking a picture by turning the smartphone towards the image of performer b in the center of the image display device 10. At this time, the data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the user 30's smartphone is the display area of performer b in the central area of the display unit 11, and switches the image of performer b to a video.

[0043] The image of performer a is switched to a still image (normal image) as the video content being played ends. The image in the display area of performer c remains a still image.

[0044] Next, in the lower left (step S03) (time t = t3), the user 30 is in a state of taking a picture by turning the smartphone towards the image of performer c on the right side of the image display device 10. At this time, the data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the user 30's smartphone is the display area of performer c in the right area of the display unit 11, and switches the image of performer c to a video.

[0045] The images of performers a and b are switched to still images (normal images) as the video content being played ends.

[0046] FIG. 7 is a diagram showing an example of processing when a plurality of users in front of the image display device 10 capture images of performers in different regions displayed on the display unit 11.

[0047] (Step S11) in FIG. 7 shows a state where users x and 30x are taking pictures by pointing their smartphones at the image of performer c on the right side of the image display device 10. The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the smartphone of users x and 30x is the display area of performer c in the right area of the display unit 11, and switches the image of performer c to a video. The images of performers a and b remain as still images (normal images).

[0048] (Step S12) shows a state where, while users x and 30x are shooting performer c in the right area of the display unit, another user, users y and 30y, starts shooting by pointing their smartphones at the image of performer a on the left side of the image display device 10.

[0049] The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, and determines that the shooting direction (shooting target) of the smartphone of users x and 30x is the display area of performer c in the right area of the display unit 11, and further, the shooting direction (shooting target) of the smartphone of users y and 30y is the display area of performer a in the left area of the display unit 11.

[0050] Based on this determination, the data processing unit of the image display device 10 continues to play the video of performer c in the right area of the display unit 11, and at the same time, switches the image of performer a in the left area of the display unit 11 from a still image to a video and starts playing the video.

[0051] FIG. 8 shows the state of (Step S13) after (Step S12) shown in FIG. 7. As shown in the upper left of FIG. 8 (step S12), it is in the same state as (step S12) in the lower right of FIG. 7. The image of performer c in the right region of display unit 11, which is the shooting target of user x, 30x, and the image of performer a in the left region of display unit 11, which is the shooting target of user y, 30y, are switched to a video and video playback is being executed.

[0052] As shown in the lower right of FIG. 8 (step S13), it shows the state where the shooting by user x, 30x has ended and user x, 30x is leaving the image display device 10. The other user y, 30y is continuing to shoot the image of performer a in the left region of display unit 11.

[0053] In this state, the image of performer c in the right region of display unit 11 is switched to a still image (normal image) as the playback video content ends. On the other hand, the image of performer a in the left region of display unit 11 continues the video playback process and will be switched to a still image when the playback content ends.

[0054] FIG. 9 is a diagram for explaining an example of a process of selecting and playing a video to be played according to the position of the user performing the image shooting, the orientation and direction of the smartphone by which the user performs the image shooting.

[0055] First, as shown in the upper left of FIG. 9 (step S21), it shows the state where user x, 30x is shooting the image of performer b in the center of image display device 10 with the smartphone facing from the left side. The data processing unit of image display device 10 analyzes the captured image of imaging unit (camera) 12, determines that the shooting direction (shooting target) of user x, 30x's smartphone is the display area of performer b in the central region of display unit 11, and switches the image of performer b to a video (video p (video content p)).

[0056] In the example of the process shown in FIG. 9, the data processing unit of image display device 10 also analyzes the position of user x, 30x based on the captured image of imaging unit (camera) 12. The data processing unit of the image display device 10 analyzes that the position of user x, 30x is on the left side of the image of performer b, and user x, 30x is shooting the image of performer b from the left side. According to this analysis result, when the data processing unit of the image display device 10 switches the image of performer b to a video, it selects and plays the optimal video according to the user's position and shooting direction, that is, "video p (video content p)". That is, it selects and plays the optimal video content for the user on the left side of the image of performer b.

[0057] In the storage unit (content storage unit) inside the image display device 10, multiple types of video content are stored for each performer. The data processing unit of the image display device 10 selects and plays the optimal video according to the position and shooting direction of the user who shoots the image of the performer.

[0058] Next, (step S22) on the right of FIG. 9 shows a state where the playback process of "video p (video content p)" has ended while user x, 30x is shooting the central performer b of the image display device 10 from the left. With the end of the playback of this video p (video content p), the image of performer b is switched to a still image (normal image).

[0059] Furthermore, afterwards, (step S23) shows a state where user x, 30x starts walking in front of the image display device 10 and moving to the right, and shooting the image of performer b from the right.

[0060] The data processing unit of the image display device 10 analyzes based on the captured image of the imaging unit (camera) 12 that user x, 30x is shooting performer b from the right. Based on this analysis result, the data processing unit of the image display device 10 switches the still image of performer b to "video q (video content q)" and starts video playback. This "video q (video content q)" is different from the "video p (video content p)" played in (step S21).

[0061] That is, it is content for users on the right side of the image of performer b or users taking pictures of the image from the right side, and it is video content optimized for users x, 30x.

[0062] FIG. 10 is a diagram for explaining an example of processing when a plurality of users take pictures of the same performer's image. First, (step S31) in the upper left of FIG. 10 shows a state where user x, 30x is taking a picture of the image of performer b in the center of the image display device 10 with the smartphone facing it from the left side. The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the smartphone of user x, 30x is the display area of performer b in the central area of the display unit 11, and switches the image of performer b to a video (video p (video content p)).

[0063] The data processing unit of the image display device 10 also analyzes the position of user x, 30x based on the captured image of the imaging unit (camera) 12. The data processing unit of the image display device 10 analyzes that the position of user x, 30x is on the left side of the image of performer b and user x, 30x is taking a picture of the image of performer b from the left side. According to this analysis result, when switching the image of performer b to a video, the data processing unit of the image display device 10 selects and plays the optimal video according to the position and shooting direction of the user, that is, "video p (video content p)". That is, it selects and plays the video content optimal for the user on the left side of the image of performer b.

[0064] Next, (step S32) in the lower right of FIG. 10 shows a state where while user x, 30x is taking a picture of performer b in the center of the image display device 10 from the left, another user z, 30z starts the process of taking a picture of the same performer b from the right.

[0065] The data processing unit of the image display device 10 analyzes based on the captured image of the imaging unit (camera) 12 that user x, 30x and user z, 30z are respectively taking pictures of performer b from the left and right. Based on this analysis result, the data processing unit of the image display device 10 executes a playback video switching process to switch the "video p (video content p)" being played to "video r (video content r)". That is, it switches to and plays "video r (video content r)", which is the optimal video content that can satisfy both the user on the left side and the user on the right side of the image of performer b.

[0066] Note that during the video content switching process, in order to eliminate the discontinuity between the video before switching and the video after switching and perform a smooth switch, a process of playing an interpolation image composed of intermediate images of the content before and after switching is performed.

[0067] That is, an interpolation image for smoothly transitioning between the images of the video content before and after switching is played between the images of the video content before and after switching.

[0068] This interpolation image may be generated by the data processing unit of the image display device 10 when performing the content switching process, or the interpolation image data stored in the storage unit (content storage unit) in advance may be used. Note that a specific example of the generation process of the interpolation image during this content switching will be described later.

[0069] FIG. 11 is also a diagram for explaining an example of a process for switching playback video content according to the user position, the direction of the smartphone, and the like. (Step S41) shows a state where the user x, 30x is taking a picture of the image of performer b at the center of the image display device 10 with the smartphone facing from the left side.

[0070] The data processing unit of the image display device 10 analyzes the captured image of the imaging unit (camera) 12, determines that the shooting direction (shooting target) of the user x, 30x's smartphone is the display area of performer b in the central area of the display unit 11, and selects and plays "video p (video content p)", which is the optimal video for the user x, 30x on the left side of the image of performer b in the central area.

[0071] The following (step S42) shows a state where the "video p" content of performer b has been played until the end, the video playback has ended, and the image of performer b has been switched to a still image.

[0072] Furthermore, afterwards, (step S43) shows a state where another user z, 30z has started the process of taking a picture of performer b from the right. At this point, user x, 30x has the smartphone pointed downwards and not towards the image.

[0073] The data processing unit of the image display device 10 analyzes, based on the captured image of the imaging unit (camera) 12, that user z, 30z is taking a picture of performer b from the right. Based on this analysis result, the data processing unit of the image display device 10 executes a playback video switching process of switching the currently playing "video p (video content p)" to "video q (video content q)". That is, it switches to and plays "video q (video content q)", which is the optimal video content for user z, 30z who is taking a picture of the image of performer b from the right side.

[0074] Furthermore, the subsequent (step S44) is a state where user x, 30x who noticed the new video playback has started the process of taking a picture of performer b from the left.

[0075] The data processing unit of the image display device 10 analyzes, based on the captured image of the imaging unit (camera) 12, that user x, 30x and user z, 30z are respectively taking pictures of performer b from the left and the right. Based on this analysis result, the data processing unit of the image display device 10 executes a playback video switching process of switching the currently playing "video q (video content q)" to "video r (video content r)". That is, it switches to and plays the optimal video content that can satisfy both the user on the left side and the user on the right side of the image of performer b.

[0076] In this way, the data processing unit of the image display device 10 analyzes the position of the user who performs the image capturing and the orientation of the smartphone, and selects and plays the optimal video according to the analysis result.

[0077] Furthermore, the data processing unit of the image display device 10 of the present disclosure also executes a process of analyzing the user status (user state) such as the user's clothing and possessions based on the captured image of the imaging unit (camera) 12, and selecting and playing the content (video content) to be played based on the analysis result.

[0078] Referring to FIG. 12, one processing example will be described. As shown in the upper left of FIG. 12 (step S51), it shows a state where the user y, 30y is walking while taking an image with a smartphone from the left side of the image display device 10 and approaching.

[0079] At this point, the data processing unit of the image display device 10 detects based on the captured image of the imaging unit (camera) 12 that the user y, 30y is capturing the performer a in the left area of the display unit, and performs a process of switching the image of the performer a from a still image to a moving image.

[0080] Furthermore, the data processing unit of the image display device 10 detects based on the captured image of the imaging unit (camera) 12 that the user y, 30y is holding the registered goods 50 which is the registered back in advance.

[0081] Note that user status information such as specific goods, possessions, wear, and clothing is registered in the storage unit of the image display device 10. When the user in the image captured by the imaging unit (camera) 12 corresponds to these registered statuses, a special image (special image corresponding to the user status) corresponding to the status is output to the display unit 11.

[0082] The example shown in FIG. 12 is a processing example when the data processing unit of the image display device 10 detects that "user y, 30y is photographing performer a in the left area of the display unit" and also detects that the user has a registered item 50 which is a pre-registered background.

[0083] In this case, the data processing unit of the image display device 10 first switches the image of performer a from a still image to a moving image as shown in the upper left of FIG. 12 (step S51). Furthermore, at the timing when the moving image playback ends, the data processing unit of the image display device 10 selects and plays and displays a special "moving image s content (special image corresponding to user status)" as shown in the lower right of FIG. 12 (step S52).

[0084] However, if the registered item 50 is in the blind spot of the camera's field of view and the data processing unit of the image display device 10 can only detect that user y, 30y is photographing performer a in the left area of the display unit and cannot detect that the user has the registered item 50, the data processing unit of the image display device 10 only performs the process of switching the image of performer a from a still image to a moving image as shown in the upper left of FIG. 12 (step S51). After that, if the registered item 50 is detected due to the movement of user y, 30y or the like, after the playback of the moving image ends, or immediately after the detection of the registered item 50, a special "moving image s content (special image corresponding to user status)" may be played.

[0085] Similar to FIG. 12, FIG. 13 is a diagram for explaining an example of a process of analyzing the user status (user state) such as the user's clothing and possessions and selecting and playing content (moving image content) to be played based on the analysis result.

[0086] (Step S61) in FIG. 13 shows a state where user y, 30y is photographing performer a in the left area of the display unit. Based on the captured image of the imaging unit (camera) 12, the data processing unit of the image display device 10 detects that the user y, 30y is capturing the performer a in the left area of the display unit, and performs a process of switching the image of the performer a from a still image to a moving image.

[0087] Furthermore, based on the captured image of the imaging unit (camera) 12, the data processing unit of the image display device 10 detects that the user y, 30y is holding the registered goods 50 which is a pre-registered background.

[0088] Originally, after the playback of the moving image of the performer a ends, that is, it is desired to transition to the playback of special moving image s content immediately. However, as shown in (step S62) of FIG. 13, at the end of the playback of the moving image of the performer a, another user x, 30x is performing a process of capturing the performer c from the right side of the image, and the user z, 30z is also performing a process of capturing the performer b from the left side of the image.

[0089] Thus, due to the shooting actions of the users x, 30x and the user z, 30z, the playback of the moving images of the performers b and c has already started. Therefore, the data processing unit of the image display device 10 waits until the playback of the moving images of the performers b and c ends.

[0090] (Step S63) of FIG. 13 shows a state where the playback of the moving images of the performers b and c has ended. After the playback of the moving images of the performers b and c ends, the data processing unit of the image display device 10 selects and plays and displays a special "moving image s content (special image corresponding to user status)" as shown in (step S64) at the lower left of FIG. 13.

[0091] Note that the processing example described here is a processing example in which, after waiting until all video playback on the display unit has ended, a special "Video s Content (Special Image Corresponding to User Status)" is played. However, even if the video playback of Performers b and c has started due to the shooting actions of Users x, 30x and User z, 30z, the actions of User y, 30y who has the registered goods are prioritized, and the video playback of Performers b and c is terminated halfway before the end of the video playback of Performers b and c, and a special "Video s Content (Special Image Corresponding to User Status)" may be played.

[0092] As described above, the image display device 10 of the present disclosure also executes processing of analyzing the user status (user state) such as the user's clothing and possessions, and selecting and playing the content (video content) to be played based on the analysis result.

[0093] [2. Configuration Example of Image Display Device of the Present Disclosure] Next, a configuration example of the image display device of the present disclosure will be described.

[0094] FIG. 14 is a diagram showing a configuration example of the image display device 100 of the present disclosure. The image display device 100 shown in FIG. 14 corresponds to the image display device 10 shown in FIG. 1 and others. Note that the configuration of the image display device 100 shown in FIG. 14 is a diagram showing only the main configuration of the image display device 100. The image display device 100 has, in addition to the configuration shown in FIG. 14, for example, an audio output unit, an input unit operable by the user, a storage unit storing a control program, and the like.

[0095] As shown in FIG. 14, the image display device 100 mainly includes an imaging unit 101, a data processing unit 102, a display unit 103, a user information storage unit 104, a content storage unit 105, and a communication unit 106.

[0096] The imaging unit 101 is a camera and corresponds to the imaging unit (camera) 11 attached to the image display device 10 described with reference to FIG. 1 and others. The imaging unit 101 captures an image in front of the image display device. For example, it captures a user who is a passerby. The captured image captures the user's actions, such as the situation of image capture processing using a smartphone. The image data captured by the imaging unit 101 is input to the data processing unit 102.

[0097] The data processing unit 102 executes analysis processing of the image data captured by the imaging unit 101, switching control processing of the content to be displayed on the display unit 103 based on the analysis result, and the like.

[0098] The display unit 103 corresponds to the display unit 11 of the image display device 10 described with reference to FIG. 1 and other figures, and displays various information such as still images and video images of performers. For example, it is configured by a liquid crystal display device, an organic EL display device, or the like.

[0099] The user information storage unit 104 is a storage unit that stores information such as the user detected from the captured image of the imaging unit 101, that is, the state of the user such as a passerby in front of the display unit 103, and the state of the smartphone owned by these users. Information analyzed by the data processing unit 102 based on the image data captured by the imaging unit 101 is recorded. The user analysis processing by the data processing unit 102 and the details of the stored data in the user information storage unit 104 will be described later.

[0100] The content storage unit 105 is a storage unit that stores various contents to be displayed on the display unit 103. For example, a plurality of contents such as still images and videos of various performers are stored.

[0101] The data processing unit 102 performs processing of selecting the content to be output from the contents stored in the content storage unit 105 and displaying and playing it on the display unit 103. For example, when the user in front of the display unit 103 starts capturing an image of a certain performer as described above, the processing is to switch from the still image content of that performer to the video content and output it.

[0102] Specific examples of the content stored in the content storage unit 105 and details of the content selection and output processing by the data processing unit will be described later.

[0103] The communication unit 106 is a communication unit that executes communication with an external device such as an external server. For example, it is used in cases such as receiving new content from an external server, receiving and inputting an update program for control, etc.

[0104] Next, the detailed configuration of the data processing unit 102 will be described with reference to FIG. 15. As shown in FIG. 15, the data processing unit 102 includes a photographed image analysis unit 121, an output image control unit 122, a content acquisition unit 123, and an interpolated image generation unit 124.

[0105] The photographed image analysis unit 121 executes analysis processing of the image data photographed by the imaging unit 101. The photographed image analysis unit 121 inputs, from the imaging unit 101, an image that has photographed the state of a passerby or the like in front of the display unit 103 of the image display device 100, and executes analysis processing of the input image. Specifically, it detects a user such as a passerby from the photographed image, analyzes various statuses (states) of the user and the status (state) of the smartphone owned by the user, and stores the analysis information in the user information storage unit 104. Details of the information stored in the user information storage unit 104 will be described later.

[0106] The output image control unit 122 executes control processing of the image to be displayed on the display unit 103 based on the analysis information of the photographed image analysis unit 121. Specifically, it executes the output image control processing as described above with reference to FIGS. 1 to 13. For example, when it is detected that the user is photographing a performance image with a smartphone, it executes processing such as switching the performance image from a still image (normal image) to a moving image (special image).

[0107] Alternatively, it executes processing for switching the output image according to the user's position, processing for switching the output image according to the user's belongings (goods), clothing, and the like. Regarding the detailed sequence of the output image control process by the output image control unit 122, it will be described with reference to the flowchart in the subsequent section.

[0108] In the determination process of whether the user is taking a performance image with a smartphone, which is executed by the output image control unit 122, for example, multiple determination processes such as "determination of the angle of the smartphone" and "whether a part of the user's face area has been replaced by the smartphone" are executed to perform a highly accurate determination. Also, for example, it may be configured to store "the shape of the user's hand when holding the smartphone" and the like as learning data in the storage unit, and increase the determination accuracy by performing a determination process using the learning data in the storage unit.

[0109] The content acquisition unit 123 executes a process of acquiring the content stored in the content storage unit 105, for example, still images and moving images of each performer. It is the output image control unit 122 that executes the process of selecting the output image for the display unit 103. The content acquisition unit 123 executes a process of acquiring the content determined by the output image control unit 122 from the content storage unit 105 and providing it to the output image control unit 122.

[0110] A large number of still images and moving images of each performer are stored in the content storage unit 105. Examples of the stored data in the content storage unit 105 will be described in the subsequent section. Note that the content acquisition unit 123 may also acquire new content via the communication unit 106.

[0111] The interpolation image generation unit 124 executes a process of generating an interpolation image for smoothly executing the video switching process. For example, as described above with reference to FIG. 10 etc., when switching the playback video from video p to video r, if the switching of the playback video is executed instantaneously, a video without continuity will be played. The interpolation image generation unit 124 executes interpolation image generation processing to solve this problem.

[0112] That is, the interpolation image generation unit 124 generates an interpolation image composed of intermediate images between the images before and after the switching in order to eliminate the discontinuity of the reproduced video and perform smooth switching, and to make the video content before and after the switching transition smoothly.

[0113] With reference to FIGS. 16 and 17, a specific example of the interpolation image generation processing executed by the interpolation image generation unit 124 will be described.

[0114] FIG. 16(1) is a diagram showing a state in which the user 30 is shooting the performer b from the left side (user position = p1) of the image of the performer b. In this state, the output image control unit 122 of the data processing unit 102 of the image display device 100 selects and reproduces the video content optimal for the user 30 who is shooting the image of the performer b from the left side. Assume that this selected video content is the video content bM1 stored in the content storage unit 105 as shown on the right side of FIG. 16.

[0115] Next, as shown in FIG. 16(2), assume that the user 30 moves from the left side (user position = p1) to the right side (user position = p2) of the image of the performer b and continues shooting the image of the performer b. Before the user 30 moves, the reproduced "video content bM1" is the optimal image for the user on the left side (user position = p1) of the image of the performer b, and is not optimal for the user who has moved to the right side (user position = p2).

[0116] Therefore, the output image control unit 122 performs a process of switching the currently reproduced "video content bM1" to the video content optimal for the user who has moved to the right side (user position = p2), that is, the "video content bM2" stored in the content storage unit 105 shown on the right side of FIG. 16.

[0117] However, if such different video contents are instantaneously switched, image discontinuity will occur at the boundary of the video contents before and after the switching. The interpolation image generation unit 124 generates an interpolation image composed of intermediate images of the contents before and after the switching in order to eliminate such image discontinuity and perform smooth switching.

[0118] FIG. 17 shows an example of an interpolation image sequence (interpolation image series) generated by the interpolation image generation unit 124. FIG. 17 shows an example of an interpolation image sequence (interpolation image series) generated by the interpolation image generation unit 124 during the switching process between two video contents, namely video content bM1 and video content bM2, stored in the content storage unit 105.

[0119] The leftmost image at the bottom of FIG. 17 is the final image frame (frame identifier = bM1 - fxxx) played immediately before the switching of the video content bM1, which is the video being played before the switching. The rightmost image is the first image frame (frame identifier = bM2 - fyyy) played immediately after the switching of the video content bM2, which is the video after the switching.

[0120] These two image frames before and after the switching, namely, the image frame of video content bM1 (frame identifier = bM1 - fxxx) the image frame of video content bM2 (frame identifier = bM2 - fyyy) These two images have different movements. If they suddenly switch, it will result in an obviously unnatural movement. That is, image discontinuity occurs at the boundary of the video contents before and after the switching.

[0121] The interpolation image generation unit 124 generates an interpolation image to eliminate such image discontinuity and make the video contents before and after the switching smoothly transition, that is, an interpolation image composed of intermediate images of the images before and after the switching. Specifically, an interpolation image sequence (interpolation image series) as shown in FIG. 17 is generated.

[0122] The interpolation image sequence (interpolation image series) generated by such an interpolation image generation unit 124 is input to the output image control unit 122. The output image control unit 122 plays back the interpolation image sequence (interpolation image series) generated by the interpolation image generation unit 124 after the final image frame (frame identifier = bM1-fxxx) immediately before the switching of the video content bM1, and then executes playback control to play back the first image frame (frame identifier = bM2-fyyy) immediately after the switching of the video content bM2.

[0123] By such playback control, the video content bM1 and the images before and after the switching of the video content bM2 are played back as if they were one video content with smooth movement, and the user, who is the observer, can view the video without discomfort.

[0124] Next, with reference to FIG. 18, an example of the stored data in the user information storage unit 104 will be described. As described above, the data analyzed by the imaging image analysis unit 121 of the data processing unit 102 of the image display device 100 shown in FIG. 15 based on the captured image of the imaging unit 101 is stored in the user information storage unit 104.

[0125] As shown in FIG. 18, the following data is stored in the user information storage unit 104. (a) Attributes of the user (person) included in the input image (b) Status of the user (person) included in the input image (c) Status of the smartphone held by the user (person) included in the input image (d) Performer (favorite performer) presumed to be the preference of the user (person) included in the input image (e) Video (video) viewing history information of the user (person) included in the input image Each of the information (a) to (e) is recorded in the user information storage unit 104 in units of detected users.

[0126] "The ""(a) attributes of the user (person) included in the input image"" refer to the user's gender, age, etc. However, this attribute information is information estimated from the image and is not necessarily 100% correct information."

[0127] "The ""(b) status of the user (person) included in the input image"" refers to information including information on goods such as the bag held by the user, the color of the clothes the user is wearing, the position of the user's line of sight, etc." "The ""(c) status of the smartphone held by the user (person) included in the input image"" refers to the position information of the smartphone and the orientation information of the smartphone (camera), etc."

[0128] The shooting image analysis unit 121 analyzes based on the shooting image of the imaging unit 101 and stores it in the user information storage unit 104. The position (x, y) of the user's line of sight, the position (x, y) of the smartphone, and specific examples of the direction of the smartphone will be described with reference to FIG. 19."

[0129] FIG. 19 shows a user 30 standing in front of the display unit 103 of the image display device 100 and taking a picture of the image of performer a displayed at the left end of the display unit 103 with the smartphone 40. Note that FIG. 19 shows the state of the user corresponding to the recording information of the user (U001) shown in FIG. 18."

[0130] The imaging unit (camera) 101 of the image display device 100 captures an image of the user 30. The captured image is input to the shooting image analysis unit 121 of the data processing unit 102." The shooting image analysis unit 121 analyzes the position (x, y) of the user's line of sight 30 and further analyzes the position (x, y) and direction of the smartphone 40."

[0131] As shown in FIG. 19, the display surface of the display unit 103 of the image display device 100 is set as the xy plane. The x-axis is the horizontal axis of the display unit 103 of the image display device 100 set on the user's walking surface. The y-axis is the vertical axis of the display unit 103 of the image display device 100 set at the left end of the display unit 103 of the image display device 100." The origin (0, 0) of the xy plane is the intersection of the y-axis set at the left end of the display unit 103 of the image display device 100 and the x-axis set on the user's walking surface. The z-axis is an axis extending in a direction away from the plane of the display unit 103 (xy plane) of the image display device 100.

[0132] In the example shown in FIG. 19, the user 30 stands at a position on the z-axis, that is, at (x, y) = (0, 0), and is taking a picture of the image of performer a with the smartphone 40. In such a state, the captured image analysis unit 121 of the data processing unit 102 of the image display device 100 analyzes the position (x, y) of the user 30's line of sight and the position (x, y) and direction of the smartphone 40 as follows.

[0133] The captured image analysis unit 121 calculates the intersection position of the vector indicating the line-of-sight direction of the user 30 and the display unit 103 as the line-of-sight position (x, y) of the user 30, and stores it in the user information storage unit 104. In the example shown in FIG. 19, the line-of-sight position (x, y) of the user 30 is (55, 174), and this (x, y) = (55, 174) is recorded in the user information storage unit 104 as the line-of-sight position (x, y) of the user state (user status). This user's line-of-sight position information corresponds to the line-of-sight position information (x, y) = (55, 174) of the user (U001) in the data shown in FIG. 18.

[0134] Furthermore, the captured image analysis unit 121 analyzes the position (x, y) and direction of the smartphone 40. In the state shown in FIG. 19, the position (x, y) of the smartphone 40 is (x, y) = (0, 169). The captured image analysis unit 121 records the position (x, y) = (0, 169) of the smartphone 40 in the user information storage unit 104. This smartphone position information corresponds to the smartphone position information (x, y) = (0, 169) of the user (U001) in the stored data of the user information storage unit 104 shown in FIG. 18. Note that although the stored data in the user information storage unit 104 shown in FIG. 18 does not include z-direction data, it may be configured to analyze and store z-direction data as well.

[0135] Furthermore, the photographed image analysis unit 121 analyzes the direction of the smartphone 40. The direction of the smartphone 40 is the rotation angle around the vertical axis (the vertical axis parallel to the Y axis), and in the example shown in FIG. 19, it is the rotation angle of "18° to the right".

[0136] The photographed image analysis unit 121 records the direction of the smartphone 40 = (18° to the right) in the user information storage unit 104. This smartphone position information corresponds to the smartphone direction information = (18° to the right) of the user (U001) of the data shown in FIG. 18.

[0137] The output image analysis unit 122 of the data processing unit 102 shown in FIG. 15 uses information such as the position of the user, the position and orientation of the smartphone stored in the user information storage unit 104 to determine the performer and shooting direction that the user is shooting, and determines the content to be output to the display unit 103 based on this determination result, and acquires and reproduces it from the content storage unit 105.

[0138] Returning to FIG. 18, the description of the stored data in the user information storage unit 104 is continued. For "the performer (favorite performer) estimated to be the preference of the user (person) included in the input image", for each performer displayed on the display unit 1103, the probability value (0 to 100%) that the performer is the user's preference, that is, the favorite performer, is recorded.

[0139] The probability value (0 to 100%) that each performer is the user's favorite performer is calculated by the photographed image analysis unit 121 through the analysis process of the shadow image. Specifically, for example, the line-of-sight occupancy rate of the user for each performer displayed on the display unit 103 is recorded as the favorite performer probability value (0 to 100%).

[0140] The captured image analysis unit 121 performs user gaze tracking, for example, by analyzing the captured image, analyzes the position, timing, number of seconds, etc. being stared at, and estimates the performer (favorite performer) being focused on. Further, for example, it may be configured to strengthen the speculation of "favorite" by weighting the performance of the portion stared at immediately before or after the user turns the smartphone.

[0141] The " (e) video (image) viewing history information of the user (person) included in the input image" is information obtained by counting the number of times the user has viewed the video playback of the image display device 100 in the past.

[0142] Although not shown in FIG. 18, the captured image analysis unit 121 of the data processing unit 102 of the image display device 100 executes user identification processing based on the captured image of the imaging unit 101, and stores user identification information including user characteristic information such as the face image of each user in the user information storage unit 104.

[0143] The user unit data shown in FIG. 18 is separately linked to the stored user identification information, and the captured image analysis unit 121 analyzes and records the observation execution date and time of each user's performance video (special image) using this link data. The captured image analysis unit 121 records the observation execution date and time of the performance video (special image) for each performance unit for each user in this " (e) video (image) viewing history information of the user (person) included in the input image" area.

[0144] For example, in the example shown in FIG. 18, the user (U001) has the following performance video viewing history. Video observation of performance c at 12:20:05 on December 4, 2023, Video observation of performance b at 12:21:15 on December 4, 2023, Video observation of performance c at 12:21:55 on December 4, 2023,

[0145] As described above, in the user information storage unit 104 shown in FIG. 18, (a) Attributes of the user (person) included in the input image (b) Status of the user (person) included in the input image (c) Status of the smartphone held by the user (person) included in the input image (d) Presumed performer (favorite performer) of the user (person) included in the input image (e) Video (image) viewing history information of the user (person) included in the input image Each of the information (a) to (e) is recorded as user unit information.

[0146] Each of the information (a) to (e) is analyzed by the captured image analysis unit 121 of the data processing unit 102 of the image display device 100 shown in FIG. 15 based on the captured image of the imaging unit 101 and stored in the user information storage unit 104.

[0147] Next, with reference to FIG. 20, an example of the stored data in the content storage unit 105 will be described. As shown in FIG. 20, the content storage unit 105 stores a large number of still image contents and video contents in units of performers to be output to the display unit 103 of the image display device 100.

[0148] For example, for performer a, the following image contents are stored. (1) Still image content a1 (2) Video content for display area L (left) = video contents aL1, aL2, aL3, ··· (3) Video content for display area M (center) = video contents aM1, aM2, aM3, ··· (4) Video content for display area R (right) = video contents aR1, aR2, aR3, ···

[0149] “(1) Still image content a1” is a still image of performer a and is image data corresponding to a normal image.

[0150] The "Video Content for Display Area L (Left)" is video content (special image) to be displayed in the left display area in the setting of a three-partitioned display area in partner units, as shown in, for example, FIGS. 3 and 4. The "Video Content for Display Area M (Center)" is video content (special image) to be displayed in the central display area of the three-partitioned display area. The "Video Content for Display Area R (Right)" is video content (special image) to be displayed in the right display area of the three-partitioned display area.

[0151] Note that a plurality of different video contents are recorded in each of the video contents for these display areas (2) to (4), and the output image control unit 122 selects the optimal content according to the situation of the user in front of the display unit 103 and outputs it to each display area.

[0152] For example, the image content to be output when the user is shooting from the right side of the performer's image, as described with reference to FIGS. 9 and 10 above, the image content to be output when the user is shooting from the left side, the image content to be output when the user is shooting from the front, the image content to be output when multiple users are shooting, and further, as described with reference to FIG. 12, the image content to be output according to the user's belongings and clothing, etc. A large number of different contents to be output according to the situation of the user in front of the display unit 103 are stored as the video contents for each of the display areas (2) to (4).

[0153] The output image control unit 122 selects the optimal content from these numerous contents according to the situation of the user in front of the display unit 103 and outputs it.

[0154] [3. Sequence of Processes Executed by the Image Display Device of the Present Disclosure] Next, the sequence of processes executed by the image display device 100 of the present disclosure will be described.

[0155] Refer to the flowchart below Figure 21 to describe the sequence of processes executed by the image display device 100 of the present disclosure. Note that the flowchart shown below Figure 21 is executed by the data processing unit 102 of the image display device 100. The data processing unit 102 has a CPU or the like with a program execution function, and executes processing according to the flowchart according to the program stored in the storage unit of the image display device 100. Hereinafter, the processing of each step of the flowchart shown in Figure 21 will be sequentially described.

[0156] (Step S101) First, in step S101, the data processing unit 102 of the image display device 100 inputs the captured image of the imaging unit (camera) 101 of the image display device 101, executes analysis of the input image, and determines whether there is a user such as a passerby near the display unit 103, that is, determines the presence or absence of a user.

[0157] Note that this processing is the processing executed by the captured image analysis unit 121 of the data processing unit 102 of the image display device 100 shown in Figure 15.

[0158] (Step S102) Step S102 is a branch step based on the determination result of the presence or absence of a user in step S101. If it is determined that there is a user such as a passerby near the display unit 103, the process proceeds to step S103. On the other hand, if it is determined that there is no user, the process proceeds to step S121.

[0159] (Step S103) In step S102, if it is determined that there is a user such as a passerby near the display unit 103, the data processing unit 102 executes the following processing in step S103.

[0160] The user information of the user attributes (such as gender, age, etc.), user status (clothing, belongings, line of sight direction, etc.), and smartphone status (position, camera direction, etc.) of the detected user from the captured image of the imaging unit (camera) 101 is analyzed and stored in the user information storage unit 104.

[0161] This process is also a process executed by the captured image analysis unit 121 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0162] For the user detected from the captured image, the captured image analysis unit 121 analyzes the user's attributes (such as gender, age, etc.), the user's status (clothing, belongings, line of sight direction, etc.), the smartphone status (position, camera direction, etc.) of the detected user's smartphone, analyzes these and stores them in the user information storage unit 104.

[0163] As described above with reference to FIG. 18, the following data is stored in the user information storage unit 104. (a) Attributes of the user (person) included in the input image (b) Status of the user (person) included in the input image (c) Status of the smartphone held by the user (person) included in the input image (d) Performer (favorite performer) presumed to be the preference of the user (person) included in the input image (e) Video (video) viewing history information of the user (person) included in the input image Each of these pieces of information (a) to (e) is recorded in the user information storage unit 104 in units of users detected from the captured image.

[0164] Although only the analysis and storage processing of the information (a) to (c) are described in the flow shown in FIG. 21, the captured image analysis unit 121 of the data processing unit 102 also analyzes each of the information (d) and (e) and records them in the user information storage unit 104 as described above with reference to FIG. 18.

[0165] (Step S104) Next, the data processing unit 102 of the image display device 100 determines, in step S104, whether the user state detected from the captured image is a user state that satisfies the predefined display content switching condition.

[0166] This process is the process executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0167] Note that the "user state that satisfies the predefined display content switching condition" is, for example, "the execution state of the display unit capturing process by the user". However, as described above, the "user state that satisfies the predefined content switching condition" is not limited to the "execution state of the display unit capturing process by the user", and other user operations and user states may be applied. Specific examples will be described later.

[0168] In step S104, if it is determined that the user state detected from the captured image is a user state that satisfies the predefined display content switching condition, the process proceeds to step S111. On the other hand, if it is determined that the user state detected from the captured image is not a user state that satisfies the predefined display content switching condition, the process proceeds to step S121.

[0169] (Step S111) First, the processes below step S111 will be described with reference to the flowchart shown in FIG. 22.

[0170] The processes below step S111 are the processes executed when it is determined in step S104 that the user state detected from the captured image is a user state that satisfies the predefined display content switching condition.

[0171] In this case, the data processing unit 102 of the image display device 100 determines, in step S111, whether the image currently output to the display unit is a normal image (such as a still image).

[0172] This process is the process executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0173] Note that the image to be processed below step S111 is the output image of the image area that the user is targeting for shooting. For example, if three performers a to c are displayed side by side on the display unit and the user is shooting the image of performer a, only the image of performer a is the image to be processed below step S111.

[0174] In step S111, if it is determined that the image output to the display unit is a normal image (such as a still image), the process proceeds to step S112. If the image output to the display unit is not a normal image (such as a still image), the process proceeds to step S113.

[0175] (Step S112) In step S111, if it is determined that the image output to the display unit is a normal image (such as a still image), the data processing unit 102 of the image display device 100 executes an output content switching process of switching the normal image (such as a still image) output to the display unit to a special image (such as a moving image) in step S112.

[0176] That is, it executes a switching process from a still image to a moving image. Note that, as described above, in conjunction with this moving image switching process, it may be configured to output audio or the like.

[0177] This process is the process executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0178] In the process of this step S112, when the output image control unit 122 switches a normal image (such as a still image) to a special image (such as a moving image), it selects and outputs the moving image content determined to be optimal according to the user position or the like from among a number of moving image contents stored in the content storage unit 105.

[0179] The detailed sequence of this step S112 will be described with reference to FIG. 23. FIG. 23 shows a flow chart indicating the detailed sequence of step S112. As shown in FIG. 23, step S112 is composed of the process of step S112a and the process of step S112b. These two processes will be described. Note that these two processes are processes executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0180] (Step S112a) In step S112a, the output image control unit 122 analyzes the user position, smartphone position, camera direction, etc. of the user based on the captured image of the imaging unit 101. Note that this data may be input by the output image control unit 122 from the captured image analysis unit 121, or the data stored in the user information storage unit 104 may be used.

[0181] (Step S112b) Next, in step S112b, the output image control unit 122 selects and outputs a special image (such as a moving image) optimal for the user position, smartphone position, camera direction, etc. of the user analyzed in step S112a.

[0182] For example, when the user is taking a picture from the left side of the performer image, the moving image content optimal for the user on the left side is selected and output. Alternatively, when the user is taking a picture from the right side of the performer image, the moving image content optimal for the user on the right side is selected and output, and so on.

[0183] Furthermore, when the user position, the camera position, or the camera direction changes in step S112b, the output image control unit 122 also executes content switching processing in accordance with these changes.

[0184] For example, when the user's position changes as shown in FIGS. 9 and 10 described above, content switching is executed in accordance with the change in the user position. Note that when switching the content, an interpolation image is output between the switching contents as necessary. That is, as described above with reference to FIGS. 16 and 17, an interpolation image for smoothly transitioning the video content before and after switching is generated and output so that unnatural movement does not occur at the switching portion between different video contents.

[0185] Note that as described above with reference to FIG. 10, when a plurality of users are performing shooting, optimal video content for these plurality of users is selected and output. Also, as described above with reference to FIG. 12, when it is determined that items such as a bag owned by the user or clothing satisfy the specified content switching conditions, the output is switched to special content according to the conditions.

[0186] For example, data associating information on goods that are the user's belongings, those goods, or the color of clothes with the performer displayed on the display unit may be stored in the storage unit, and content switching control may be performed using this association data. For example, when the user is wearing an item of a specific color, processing such as switching the image of the member corresponding to that color to the video content is performed.

[0187] Returning to FIG. 22, the description of the flowchart shown in FIG. 22 will be continued. (Step S113) In step S111, if it is determined that the image output to the display unit is not a normal image (such as a still image), that is, a special image (such as a video), the process proceeds to step S113.

[0188] In this case, the data processing unit 102 of the image display device 100 continues to output the special image (such as a video) output to the display unit as it is in step S113.

[0189] This process is the process executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0190] Note that also in the process of step S113, similar to the process of step S112 described above, in the process of continuously outputting the special image (such as a video), a process of selecting and outputting the optimal video content according to the user position and the like is performed.

[0191] The detailed sequence of this step S113 will be described with reference to FIG. 24. FIG. 24 shows a flow indicating the detailed sequence of step S113. As shown in FIG. 24, step S113 is composed of the process of step S113a and the process of step S113b. These two processes will be described. Note that these two processes are the processes executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0192] (Step S113a) In step S113a, the output image control unit 122 analyzes the user position, smartphone position, camera direction, etc. of the user based on the captured image of the imaging unit 101. Note that this data may be input by the output image control unit 122 from the captured image analysis unit 121, or the data stored in the user information storage unit 104 may be used.

[0193] (Step S113b) Next, in step S113b, the output image control unit 122 selects and outputs a special image (such as a video) that is optimal for the user position, smartphone position, camera direction, etc. of the user analyzed in step S113a.

[0194] For example, when the user is taking a picture from the left side of the performance image, video content optimal for the user on the left side is selected and output. Alternatively, when the user is taking a picture from the right side of the performance image, video content optimal for the user on the right side is selected and output, and so on.

[0195] Furthermore, in step S113b, when the user position, camera position, or camera direction changes, the output image control unit 122 executes switching processing of the output content in accordance with these changes. Also, when switching content, an interpolation image is output between the switching content as necessary. These processes are the same as the processes in step S112 described above.

[0196] Note that after the completion of the processes in step S112 or step S113 of the flowchart shown in FIG. 22, the process returns to step S101, and the processes below step S101 are repeated.

[0197] Next, the processes to be executed when it is determined as No in step S102 of the flowchart shown in FIG. 21 or when it is determined as No in step S104 will be described with reference to the flowchart shown in FIG. 25.

[0198] The case where it is determined as No in step S102 of the flowchart shown in FIG. 21 means that it is determined that there is no user such as a passerby in the vicinity of the display unit 103. Also, the case where it is determined as No in step S104 means that it is determined that the user state detected from the captured image of the imaging unit 101 is not a user state that satisfies the previously defined display content switching conditions. In these cases, the processes according to the flowchart shown in FIG. 25 are executed. The processing of each step of the flow shown in FIG. 25 will be described below.

[0199] (Step S121) In any of the above cases, that is, when it is determined in step S102 that there is no user such as a passerby in the vicinity of the No= display unit 103, when the user state detected from the captured image of the No= imaging unit 101 in step S104 does not satisfy the predefined display content switching condition, in any of these cases, the data processing unit 102 of the image display device 100 determines in step S121 whether the image output to the display unit is a normal image (such as a still image).

[0200] This processing is the processing executed by the output image control unit 122 of the data processing unit 102 of the image display device 100 shown in FIG. 15.

[0201] If it is determined that the image output to the display unit 103 is a normal image (such as a still image), the process proceeds to step S122. On the other hand, if it is determined that the image output to the display unit 103 is a special image (such as a video), the process proceeds to step S123.

[0202] (Step S122) If it is determined in step S121 that the image output to the display unit 103 is a normal image (such as a still image), the data processing unit 102 of the image display device 100 continuously outputs the normal image (such as a still image) output to the display unit as it is as a normal image (still image) in step S122.

[0203] (Step S123) On the other hand, if it is determined in step S121 that the image output to the display unit 103 is a special image (such as a video), the data processing unit 102 of the image display device 100 executes a process of switching the special image (such as a video) output to the display unit to a normal image (still image) in step S123.

[0204] After the completion of the process in step S122 or step S123, return to step S101 and repeat the processes below step S101.

[0205] Note that when the process is performed according to the flow shown in FIGS. 21 to 25, for example, when a certain user points the smartphone at the display unit 103, the playback of video content, which is a special image, starts. However, if the user continues to point the smartphone at the display unit 103 even after the end of the playback of the video content, processes such as "video playback ⇒ end ⇒ video playback ⇒ end ⇒ video playback ⇒ ···" will be repeated.

[0206] In order to avoid such repetition of processes, when the user continues to point the smartphone at the display unit 103 even after the end of the playback of the video content, it may be configured to perform an exception process of not performing repeated playback of the video content. However, even when this exception process is performed, if the user once stops pointing the smartphone at the display unit 103 and then points the smartphone at the display unit 103 again, the playback of the video content is started.

[0207] [4. Examples of content switching conditions for the image display device according to the present disclosure] Next, examples of content switching conditions for the image display device according to the present disclosure will be described.

[0208] As described above, the image display device according to the present disclosure analyzes the user state such as the operation of the user such as a passerby in front of the display unit, and when it is determined that the user state satisfies the content switching conditions defined in advance, it executes the content switching process for the content displayed on the display unit.

[0209] Note that in the above-described embodiment, an example in which "the state of the user executing shooting of the display unit by the user" is given as an example of "the user state satisfying the content switching conditions defined in advance" has been described. However, as described above, this is an example of the content switching conditions.

[0210] The "user state that satisfies the pre-defined content switching condition" is not limited to the "state of executing the display unit photographing process by the user", and it is possible to configure to apply various other user operations and user states. Hereinafter, examples of user operations and user states applicable to the content switching condition will be described.

[0211] With reference to FIG. 26 and below, user states and the like applicable as content switching conditions will be described. FIG. 26(A) is an example in which content switching is performed on the condition that it is detected that the user x, 30x in front of the image display device 100 is directing his / her line of sight towards the display unit of the image display device 100 for a pre-defined time (for example, 5 seconds) or more.

[0212] Thus, even when not performing image shooting with a smartphone, if a user who is looking at the display image (such as a performer's image) of the display unit for a certain period of time is detected, the output image control unit 122 of the data processing unit 102 of the image display device 100 executes content switching processing such as switching the display content from a still image to a moving image.

[0213] FIG. 26(B) is an example in which content switching is performed on the condition that it is detected that the user y, 30y in front of the image display device 100 is pushing a baby stroller.

[0214] If a user pushing a baby stroller or a user carrying a carry bag is detected, the output image control unit 122 of the data processing unit 102 of the image display device 100 executes content switching processing such as switching the display content from a still image to a moving image.

[0215] FIG. 27(C) is an example in which content switching is performed on the condition that a parent and child are detected in front of the image display device 100. FIG. 27(D) is an example in which content switching is performed on the condition that two adults, a man and a woman, are detected in front of the image display device 100. In addition, when a family member is detected, or when two or more groups are detected, etc., these may be used as content switching conditions.

[0216] FIG. 28(E) shows an example of content switching on the condition that a person (user) standing backward in front of the image display device 100 is detected. This is effective as a process for making the user turn around and view the display image on the display unit of the image display device.

[0217] In addition to these, for example, when the user has a specific good, is walking with a pet, is wearing specific clothes, is wearing clothes of a specific color, etc., various user states can be set as content switching conditions.

[0218] [5. Example of an external device such as a server connected to the image display device via a network executing content switching control] Next, an example in which an external device such as a server connected to the image display device via a network executes content switching control will be described.

[0219] In the above-described embodiment, the image display device has been described as a stand-alone type device that executes processing alone. That is, as described with reference to FIGS. 14 and 15, the data processing unit 102 of the image display device 100 analyzes the captured image of the imaging unit 101 and performs switching control of the content output to the display unit 103 according to the analysis result.

[0220] The processing of the present disclosure can be applied not only to such a stand-alone type device but also to a network-connected configuration as shown in FIG. 29, for example. The configuration shown in FIG. 29 is a configuration in which a large number of image display devices (signage devices) 100 are network-connected to a server 70.

[0221] A large number of network-connected image display devices (signage devices) 100 transmit the captured images of the imaging units (cameras) mounted on their respective devices to the server 70. The server 70 receives the captured images from each of the plurality of image display devices (signage devices) 100, analyzes the received images, determines whether a user state that satisfies the content switching condition is detected, and sends a content switching instruction or the content to be switched to each of the image display devices 100 according to the determination result.

[0222] When configured such that the server 70 transmits the display content selected based on the analysis result to the image display device (signage device) 100, the image display device (signage device) 100 performs the process of displaying the content received from the server 70. In this configuration, the image display device (signage device) 100 only needs to have an imaging unit, a display unit, and a communication unit.

[0223] Also, each of the image display devices (signage devices) 100 may be configured to have a content storage unit. In such a configuration, the server 70 transmits a content switching control command including the designation information of the content to be displayed in each image display device based on the analysis result to the image display device (signage device) 100.

[0224] The image display device (signage device) 100 acquires and displays the content specified by the server 70 from the content storage unit. In this configuration, the image display device (signage device) 100 only needs to have an imaging unit, a display unit, a communication unit, and further a content storage unit.

[0225] In any case, since the image display device (signage device) 100 does not need to perform processes with high loads such as the analysis process of the captured image, cost reduction can be achieved in the case of a system having a large number of image display devices (signage devices) 100.

[0226] Fig. 30 shows a configuration example of the server 70 and the image display device (signage device) 100 in the case of a network-type image display system as shown in Fig. 29.

[0227] The configuration example shown in Fig. 29 is a system configuration example in the case where the server 70 receives captured images from each of a plurality of image display devices (signage devices) 100, analyzes the received images, determines whether a user state satisfying the content switching condition is detected, and individually transmits the content to be switched to each of the image display devices 100 according to the determination result. The server 70 holds the content.

[0228] In this system configuration, as shown in Fig. 30, the image display device (signage device) 100 includes an imaging unit 221, a communication unit 222, and a display unit 223. The image display device (signage device) 100 captures an image captured by the imaging unit 221, that is, an image showing the situation of a passerby (user) in front of the image display device (signage device) 100, and transmits it to the server 70 via the communication unit 222.

[0229] On the other hand, the server 70 includes a data processing unit 201, a communication unit 202, a user information storage unit 203, and a content storage unit 204. Fig. 31 shows a detailed configuration example of the data processing unit 201 of the server 70. As shown in Fig. 31, the data processing unit 201 of the server 70 includes a captured image analysis unit 251, an output image control unit 252, a content acquisition unit 253, and an interpolation image generation unit 254.

[0230] The captured image analysis unit 251 executes analysis processing of the image data captured by the imaging unit 221 of each image display device (signage device) 100 received by the communication unit 202. The captured image analysis unit 251 performs analysis processing on an image that captures the state of passers-by or the like in front of the display unit 223 of each image display device 100. Specifically, it detects users such as passers-by from the captured image, analyzes various statuses (states) of the users and the status (state) of the smartphones owned by the users, and stores the analysis information in the user information storage unit 203.

[0231] The information stored in the user information storage unit 203 is the same as the information described with reference to FIG. 18 above. However, the stored information in the user information storage unit 203 is stored in a setting classified as information per each image display device (signage device) 100.

[0232] The output image control unit 252 executes a process of determining an image to be displayed on the display unit 223 of each image display device (signage device) 100 based on the analysis information of the captured image analysis unit 251.

[0233] For example, when it is detected that a user entering in front of a certain image display device (signage device) 100 is taking a picture of a performer's image with a smartphone, a process of switching the performer's image displayed on that image display device (signage device) 100 from a still image (normal image) to a moving image (special image) is executed. Specifically, the content to be transmitted to that image display device (signage device) 100 is acquired from the content storage unit 204 via the content acquisition unit 253, and the acquired content is transmitted to that image display device (signage device) 100 via the communication unit 202.

[0234] Note that in the data processing unit 201 of the server 70, the image analysis process, the content selection process, the content transmission process, and each of these processes are executed individually and in parallel for each image display device (signage device) 100.

[0235] The content acquisition unit 253 executes a process of acquiring the content stored in the content storage unit 204, for example, still images and moving images of each performer. Note that it is the output image control unit 252 that selects the images to be displayed on the display units 223 of each of the image display devices (signage devices) 100. The content acquisition unit 253 acquires the display content for each of the image display devices (signage devices) 100 determined by the output image control unit 252 from the content storage unit 204 and provides it to the output image control unit 252. The output image control unit 252 transmits these contents to each of the image display devices (signage devices) 100.

[0236] Note that the stored data in the content storage unit 105 is the same as the data described with reference to FIG. 20 earlier, and a large number of still images and moving images of each performer are stored.

[0237] The interpolation image generation unit 254 executes an interpolation image generation process for realizing a smooth transition when switching video contents, which was described with reference to FIGS. 16 and 17 earlier. Note that this process is also executed individually for each of the image display devices (signage devices) 100.

[0238] In the network type system described with reference to FIGS. 29 to 31, the server 70 receives the captured images from each of the plurality of image display devices (signage devices) 100, analyzes the received images, and executes content switching control for each of the image display devices (signage devices) 100.

[0239] As a result, each of the image display devices (signage devices) 100 does not need to perform a highly load process such as analysis processing of the captured images. Therefore, in the case of a system having a large number of image display devices (signage devices) 100, cost reduction can be achieved.

[0240] [6. Hardware Configuration Examples of Image Display Devices, etc.] Next, hardware configuration examples of the image display device (signage device) 100 and the server 70 will be described.

[0241] FIG. 32 is a diagram showing an example of the hardware configuration of the image display device (signage device) 100 and the server 70 of the present disclosure. Hereinafter, the hardware configuration shown in FIG. 32 will be described.

[0242] The CPU (Central Processing Unit) 301 functions as a control unit and a data processing unit that executes various processes according to programs stored in the ROM (Read Only Memory) 302 or the storage unit 308. For example, it executes the processes according to the sequence described in the above-described embodiments. Programs and data that the CPU 301 executes are stored in the RAM (Random Access Memory) 303. These CPU 301, ROM 302, and RAM 303 are interconnected by a bus 304.

[0243] The CPU 301 is connected to the input / output interface 305 via the bus 304. Connected to the input / output interface 305 are an input unit 306 composed of various switches, a keyboard, a mouse, a microphone, a sensor, etc., and an output unit 307 composed of a display, a speaker, etc. The CPU 301 executes various processes in response to commands input from the input unit 306 and outputs the processing results to, for example, the output unit 307.

[0244] The storage unit 308 connected to the input / output interface 305 is composed of, for example, a hard disk, etc., and stores programs and various data that the CPU 301 executes. The communication unit 309 functions as a transmission / reception unit for data communication via networks such as Wi-Fi communication, Bluetooth (registered trademark) (BT) communication, and other Internet and local area networks, and communicates with external devices.

[0245] The drive 310 connected to the input / output interface 305 drives a removable medium 311 such as a magnetic disk, an optical disk, a magneto-optical disk, or a semiconductor memory such as a memory card, and executes data recording or reading.

[0246] [Summary of the Configuration of the Present Disclosure] As described above, the embodiments of the present disclosure have been described in detail with reference to specific examples. However, it is obvious that those skilled in the art can modify or substitute the examples without departing from the gist of the present disclosure. That is, the present invention has been disclosed in the form of examples and should not be construed in a limited manner. To determine the gist of the present disclosure, the claims section should be referred to.

[0247] Note that the technology disclosed in this specification can be configured as follows. (1) A display unit, An imaging unit that captures an image including at least the vicinity of the display unit, A data processing unit that executes switching control of the display content of the display unit, The data processing unit, Executes analysis processing of the captured image of the imaging unit, determines whether the detected user is in a state that meets the specified content switching condition, An image display device that switches the display content of the display unit when it is determined that the detected user is in a state that meets the specified content switching condition.

[0248] (2) When the data processing unit, Determines that the detected user is in a state that meets the specified content switching condition, The image display device according to (1), which switches the display content of the display unit from still image content to moving image content.

[0249] (3) The user state that meets the specified content switching condition is The state of the captured execution of the display content of the display unit by the detected user, When the data processing unit, Determines that the detected user is capturing the display content of the display unit, the image display device according to (1) or (2), which switches the display content of the display unit.

[0250] (4) The data processing unit analyzes the position of the user who is photographing the display content of the display unit, and switches the display content of the display unit to optimal content according to the user position. The image display device according to (3).

[0251] (5) The data processing unit switches the display content of the display unit to optimal video content according to the user position. The image display device according to (4).

[0252] (6) The data processing unit when there are a plurality of users who are photographing the display content of the display unit, switches the display content of the display unit to optimal content according to the positions of the plurality of users who are photographing the display content. The image display device according to (4) or (5).

[0253] (7) The display unit is configured to display individual image contents in a plurality of display areas, and the data processing unit identifies the display area being photographed by the detected user, and switches the display content being displayed in the identified display area. The image display device according to any one of (3) to (6).

[0254] (8) The data processing unit identifies the display area being photographed by the detected user, and switches the display content being displayed in the identified display area from still image content to video content. The image display device according to (7).

[0255] (9) The display unit is configured to display images of individual performers in a plurality of display areas, and the data processing unit identifies the display area being photographed by the detected user, The image display device according to (7) or (8), which switches the performance image being displayed in a specific display area from still image content to moving image content.

[0256] (10) The data processing unit analyzes at least one of the position of the user who is photographing the display content of the display unit, the position of the image capturing device, or the angle, to identify the display area being photographed by the user, The image display device according to (9), which switches the performance image being displayed in the identified display area from still image content to moving image content.

[0257] (11) The data processing unit analyzes at least one of the position of the detected user who is photographing the display content of the display unit, the position of the image capturing device, or the angle, and executes a process of estimating the favorite performer of the detected user, the push performer (9) or (10) described above.

[0258] (12) The user state corresponding to the specified content switching condition is a state in which the belongings or clothing of the detected user match the pre-specified belongings or clothing, The data processing unit When it is determined that the belongings or clothing of the detected user match the pre-specified belongings or clothing, the image display device according to any one of (1) to (11) that switches the display content of the display unit.

[0259] (13) The user state corresponding to the specified content switching condition is a state in which the detected user is a plurality of users, and the combination of the plurality of users matches the pre-specified user combination, The data processing unit When the detected user is a plurality of users and the combination of the plurality of users matches a user combination defined in advance, the image display device according to any one of (1) to (12) that switches the display content of the display unit.

[0260] (14) The image display device according to (13), wherein the predefined user combination is a parent-child or a pair of a man and a woman.

[0261] (15) The data processing unit When switching the video content being displayed on the display unit to different video content, an interpolation image for making the video content before and after the switching smoothly transition is displayed between the video content before and after the switching. The image display device according to any one of (1) to (14).

[0262] (16) A communication unit that executes communication with the image display device, having a data processing unit that executes switching control of the display content of the display device, The communication unit receives a captured image including at least the vicinity of the image display device from the image display device, The data processing unit executes analysis processing of the captured image, determines whether the user detected from the captured image is in a state that meets a specified content switching condition, When it is determined that the detected user is in a state that meets the specified content switching condition, An information processing device that transmits at least one of a switching control command for the display content or the switched display content to the image display device via the communication unit.

[0263] (17) An image display control method executed in an image display device, The image display device has a display unit, an imaging unit that captures an image including at least the vicinity of the display unit, and a data processing unit that executes switching control of the display content of the display unit, The data processing unit Execute the analysis process of the captured image of the imaging unit, and determine whether the user detected from the captured image is in a state that meets the specified content switching condition. An image display control method for switching the display content of the display unit when it is determined that the detected user is in a state that meets the specified content switching condition.

[0264] In addition, the series of processes described in the specification can be executed by hardware, software, or a combined configuration of both. When executing the process by software, install the program recording the process sequence into the memory in a computer incorporated into dedicated hardware and execute it, or it is possible to install the program into a general-purpose computer capable of executing various processes and execute it. For example, the program can be recorded in advance on a recording medium. In addition to installing from the recording medium into the computer, the program can be received via a network such as a LAN (Local Area Network) or the Internet and installed into a recording medium such as an internal hard disk.

[0265] Note that the various processes described in the specification are not only executed in time series according to the description, but may also be executed in parallel or individually according to the processing ability of the device executing the process or as necessary. Also, in this specification, a system is a logical set configuration of a plurality of devices, and the devices of each configuration are not limited to those in the same housing.

Industrial Applicability

[0266] As described above, according to the configuration of an embodiment of the present disclosure, a configuration is realized that analyzes the user state in front of the display unit and executes a switching process of the display content, for example, a switching process from a still image to a moving image, according to the analysis result. Specifically, for example, a data processing unit that executes switching control of display content on the display unit performs analysis processing of a captured image by the imaging unit, and determines whether a user detected from the captured image meets a specified content switching condition, for example, whether the user is capturing the display content. When it is determined that the user is in a state that meets the content switching condition, the display content on the display unit is switched. For example, a process of switching a still image to a moving image is executed. With this configuration, a configuration is realized that analyzes the user state in front of the display unit and executes switching processing of display content, for example, switching processing from a still image to a moving image, according to the analysis result.

Description of Signs

[0267] 10 Image display device (signage device) 11 Display unit 12 Imaging unit (camera) 20 Performer 30 User (pedestrian) 40 Smartphone 70 Server 100 Image display device 101 Imaging unit 102 Data processing unit 103 Display unit 104 User information storage unit 105 Content storage unit 106 Communication unit 121 Captured image analysis unit 122 Output image control unit 123 Content acquisition unit 124 Interpolation image generation unit 201 Data processing unit 202 Communication unit 203 Content storage unit 204 User information storage unit 221 Imaging unit 222 Communication unit 223 Display unit 251 Captured image analysis unit 252 Output image control unit 253 Content acquisition unit 254 Interpolation Image Generation Unit 301 CPU 302 ROM 303 RAM 304 Bus 305 Input / Output Interface 306 Input Unit 307 Output Unit 308 Storage Unit 309 Communication Unit 310 Drive 311 Removable Media

Claims

1. A display unit, an imaging unit that captures an image including at least the vicinity of the display unit, and a data processing unit that executes switching control of the display content of the display unit, wherein the data processing unit executes analysis processing of the captured image of the imaging unit, determines whether a detected user is in a state corresponding to a specified content switching condition, and when it is determined that the detected user is in a state corresponding to the specified content switching condition, switches the display content of the display unit. An image display device.

2. The data processing unit when it is determined that the detected user is in a state corresponding to the specified content switching condition, switches the display content of the display unit from still image content to moving image content. The image display device according to claim 1.

3. The user state corresponding to the specified content switching condition is a state in which the display content of the display unit is being captured by the detected user, and the data processing unit when it is determined that the detected user is capturing the display content of the display unit, switches the display content of the display unit. The image display device according to claim 1.

4. The data processing unit analyzes the position of a user who is capturing the display content of the display unit, and switches the display content of the display unit to optimal content according to the user position. The image display device according to claim 3.

5. The data processing unit switches the display content of the display unit to optimal moving image content according to the user position. The image display device according to claim 4.

6. The data processing unit when there are a plurality of users who are capturing the display content of the display unit, switches the display content of the display unit to optimal content according to the positions of the plurality of users who are capturing the display content. The image display device according to claim 4.

7. The display unit is configured to display individual image content in a plurality of display areas, and the data processing unit identifies the display area being captured by the detected user, and switches the display content being displayed in the identified display area. The image display device according to claim 3.

8. The data processing unit identifies the display area being captured by the detected user, and switches the display content being displayed in the identified display area from still image content to moving image content. The image display device according to claim 7.

9. The display unit configured to display images of individual performers in a plurality of display areas, wherein the data processing unit, identifies the display area being photographed by the detected user, and switches the image of the performer being displayed in the identified display area from still image content to video content. The image display device according to claim 7. **Claim 10** wherein the data processing unit, analyzes at least one of the position of the user photographing the display content of the display unit, the position of the image capturing device, or the angle, to identify the display area being photographed by the user, and switches the image of the performer being displayed in the identified display area from still image content to video content. The image display device according to claim 9. **Claim 11** wherein the data processing unit, analyzes at least one of the position of the detected user photographing the display content of the display unit, the position of the image capturing device, or the angle, and executes a process of estimating the favorite performer of the detected user, the push performer. The image display device according to claim 9. **Claim 12** wherein the user state corresponding to the specified content switching condition is, a state in which the belongings or clothing of the detected user match the pre-specified belongings or clothing, wherein the data processing unit, when it is determined that the belongings or clothing of the detected user match the pre-specified belongings or clothing, switches the display content of the display unit. The image display device according to claim 1. **Claim 13** wherein the user state corresponding to the specified content switching condition is, a state in which the detected user is a plurality of users and the combination of the plurality of users matches the pre-specified user combination, wherein the data processing unit, when the detected user is a plurality of users and the combination of the plurality of users matches the pre-specified user combination, switches the display content of the display unit. The image display device according to claim 1. **Claim 14** The pre-specified user combination according to claim 13 is a parent-child or a pair of a man and a woman. The image display device according to claim 13. **Claim 15** wherein the data processing unit, when switching the video content being displayed on the display unit to different video content, displays an interpolation image for making the video content before and after the switching smoothly transition between the video content before and after the switching. The image display device according to claim 1. **Claim 16** a communication unit that communicates with the image display device, It has a data processing unit that executes switching control of the display content of the display device. The communication unit receives a captured image including at least the vicinity of the image display device from the image display device. The data processing unit executes analysis processing of the captured image and determines whether the user detected from the captured image is in a state that meets a specified content switching condition. When it is determined that the detected user is in a state that meets the specified content switching condition, an information processing device that transmits either a switching control command for the display content or at least one of the switched display contents to the image display device via the communication unit.

17. An image display control method executed in an image display device, wherein the image display device has a display unit, an imaging unit that captures an image including at least the vicinity of the display unit, and a data processing unit that executes switching control of the display content of the display unit. The data processing unit executes analysis processing of the captured image of the imaging unit and determines whether the user detected from the captured image is in a state that meets a specified content switching condition. An image display control method for switching the display content of the display unit when it is determined that the detected user is in a state that meets the specified content switching condition.

Images