Image display device, information processing device, and image display control method

Abstract

An image processing system, method and computer program product control a change of electronic signage based on a detected action of at least one person within viewing range of the electronic signage.　Circuitry is configured to identify an indication of the action of the at least one person within viewing range of electronic signage displayed on an electronic signage device, and control a change of content of the electronic signage based on the indication.

Description

IMAGE DISPLAY DEVICE, INFORMATION PROCESSING DEVICE, AND IMAGE DISPLAY CONTROL METHOD

[0001] The present disclosure relates to an image display device, an information processing device, and an image display control method.　More specifically, the present disclosure relates to an image display device, an information processing device, and an image display control method for executing control of display data on a display device such as a signage device placed in a public area by changing the display data depending on an operation of a user in front of the display device.

[0002] In recent years, display devices for information display have been frequently used in public areas such as shopping malls, stations, airports, and downtown. Examples of the display devices include a signage device.　The signage device, which is also called a digital signage and an electronic billboard, is a device that displays various kinds of information depending on where the device is located.

[0003] The signage device can output various contents, such as advertisement information on various products and stores, guidance information, advertisement information on movies, and advertisement images of performers such as a singer, an actor, and an idol.

[0004] For example, in a case where advertisement information is displayed on a signage device, high advertising effectiveness can be obtained by displaying advertisements depending on the age and sex of persons that gather in a place where the signage device is placed. For example, PTL 1 (WO 2018 / 105217) discloses a system that provides advertisements depending on attributes of advertisement providing targets.

[0005] However, the system described in PTL 1 merely changes provided contents depending on attributes of users of advertisement providing targets, for example, the sex and age groups, and does not have an interactive configuration in which contents are switched and displayed depending on, for example, user actions. Note that known interactive contents display control configurations are, for example, display switching of contents by touching a screen by a user.

[0006] WO 2018 / 105217Summary

[0007] The present disclosure has been made in view of the above-mentioned problems, for example, and it is an object thereof to provide an image display device, an information processing device, and an image display control method for switching display contents on a display device such as a signage device depending on an operation of a user near the display device.

[0008] In one example of the present disclosure, for example, when it is detected that a user near a display device has performed an operation of directing a smartphone toward the display device, control of switching display contents is executed.

[0009] A first aspect of the present disclosure resides in an image display device, including: a display unit; an imaging unit configured to take an image including at least a neighborhood of the display unit; and a data processing unit configured to execute control of switching display contents on the display unit, in which the data processing unit is configured to: execute processing for analyzing the image taken by the imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and switch the display contents on the display unit when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition.

[0010] Furthermore, a second aspect of the present disclosure resides in an information processing device, including: a communication unit configured to execute communication with an image display device; and a data processing unit configured to execute control of switching display contents on the display device, in which: the communication unit is configured to receive, from the image display device, a taken image including at least a neighborhood of the image display device; and the data processing unit is configured to: execute processing for analyzing the taken image to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition, transmit at least one of a switching control command for the display contents and switching display contents to the image display device through the communication unit.

[0011] Furthermore, a third aspect of the present disclosure resides in an image display control method to be executed in an image display device, the image display device including: a display unit; an imaging unit configured to take an image including at least a neighborhood of the display unit; and a data processing unit configured to execute control of switching display contents on the display unit, the image display control method including: executing, by the data processing unit, processing for analyzing the image taken by the imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and switching, by the data processing unit, the display contents on the display unit when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition.

[0012] Other objects, features, and advantages of the present disclosure become apparent from more detailed descriptions based on examples of the present disclosure and the accompanying drawings described later.　It should be noted that a system as used herein is a logical set configuration of a plurality of devices, and devices in each configuration are not necessarily required to be included in the same casing.

[0013] The configuration in one example of the present disclosure implements a configuration in which processing for analyzing the state of a user in front of a display unit (electronic signage) and switching display contents depending on an analysis result, for example, processing for switching the display contents from a still image to a moving image is executed. Specifically, for example, a data processing unit configured to execute switching (change) control of display contents on a display unit executes processing for analyzing an image taken by an imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition, for example, whether the user is photographing display contents, and when it is determined that the user is in the state corresponding to the contents switching condition, switches the display contents on the display unit.　For example, the data processing unit executes processing for switching a still image to a moving image. This configuration implements a configuration in which processing for analyzing the state of a user in front of a display unit and switching display contents depending on an analysis result (e.g., a signal or message that serves as an indication of at least one person in a viewing range of the electronic signage) , for example, processing for switching the display contents from a still image to a moving image is executed. Note that the effects as disclosed herein are merely an example, and not limited, and additional effects may be provided.

[0014] Fig. 1 is a diagram for describing the outline of processing executed by an image display device in the present disclosure.Fig. 2 is a diagram for describing the outline of processing executed by the image display device in the present disclosure.Fig. 3 is a diagram for describing the outline of processing executed by the image display device in the present disclosure.Fig. 4 is a diagram for describing the outline of processing executed by the image display device in the present disclosure.Fig. 5 is a diagram for describing the outline of processing executed by the image display device in the present disclosure.Fig. 6 is a diagram for describing a transition example of display images in a case where a user moves from left to right of the image display device while photographing images with a smartphone.Fig. 7 is a diagram for describing a processing example in a case where a plurality of users in front of the image display device photograph images of different performers in different regions displayed on the display unit.Fig. 8 is a diagram for describing a processing example in a case where a plurality of users in front of the image display device photograph images of different performers in different regions displayed on the display unit.Fig. 9 is a diagram for describing an example of processing for selecting a moving image to be reproduced on the basis of the position of a user who photographs an image and the orientation and direction of a smartphone that photographs the image, and reproducing the selected moving image.Fig. 10 is a diagram for describing a processing example in a case where a plurality of users photograph an image of the same performer.Fig. 11 is a diagram for describing an example of processing for switching reproduced moving image contents depending on the position of a user and the direction of a smartphone.Fig. 12 is a diagram for describing an example of processing for analyzing a user status (user state) such as clothes and belongings of a user, selecting contents (moving image contents) to be reproduced on the basis of an analysis result, and reproducing the selected contents.Fig. 13 is a diagram for describing an example of processing for analyzing a user status (user state) such as clothes and belongings of a user, selecting contents (moving image contents) to be reproduced on the basis of an analysis result, and reproducing the selected contents.Fig. 14 is a diagram for describing a configuration example of the image display device in the present disclosure.Fig. 15 is a diagram for describing a detailed configuration of a data processing unit in the image display device in the present disclosure.Fig. 16 is a diagram for describing a specific example of interpolated image generation processing executed by an interpolated image generation unit.Fig. 17 is a diagram for describing a specific example of interpolated image generation processing executed by the interpolated image generation unit.Fig. 18 is a diagram for describing an example of data stored in a user information storage unit.Fig. 19 is a diagram for describing specific examples of a position (x,y) of the line of sight of a user, a position (x,y) of a smartphone, and a direction of the smartphone that are stored in the user information storage unit.Fig. 20 is a diagram for describing an example of data stored in a contents storage unit.Fig. 21 is a diagram illustrating a flowchart for describing a sequence of processing executed by the image display device in the present disclosure.Fig. 22 is a diagram illustrating a flowchart for describing a sequence of processing executed by the image display device in the present disclosure.Fig. 23 is a diagram illustrating a flowchart for describing a sequence of processing executed by the image display device in the present disclosure.Fig. 24 is a diagram illustrating a flowchart for describing a sequence of processing executed by the image display device in the present disclosure.Fig. 25 is a diagram illustrating a flowchart for describing a sequence of processing executed by the image display device in the present disclosure.Fig. 26 is a diagram for describing user states that can be applied as contents switching conditions.Fig. 27 is a diagram for describing user states that can be applied as contents switching conditions.Fig. 28 is a diagram for describing a user state that can be applied as a contents switching condition.Fig. 29 is a diagram for describing a configuration example in which a large number of image display devices (signage devices) are connected to a server through a network.Fig. 30 is a diagram for describing a configuration example of a server and an image display device (signage device) in the case of a network image display system.Fig. 31 is a diagram for describing a configuration example of a data processing unit in a server in the case of a network image display system.Fig. 32 is a diagram for describing a hardware configuration example of an image display device and a server.

[0015] Details of an image display device, an information processing device, and an image display control method in the present disclosure are described below with reference to the drawings.　Note that the description is given in the order of the following matters. 1. Outline of processing executed by image display device in the present disclosure 2. Configuration example of image display device in the present disclosure 3. Sequence of processing to be executed by image display device in the present disclosure 4. Example of switching conditions for contents to be output by image display device in the present disclosure 5. Examples in which external device such as server connected to image display device through network executes contents switching control 6. Hardware configuration examples of image display device, etc. 7. Summary of configuration in the present disclosure

[0016] (1. Outline of processing executed by image display device in the present disclosure) First, the outline of processing executed by an image display device in the present disclosure is described with reference to Fig. 1 and subsequent drawings.

[0017] Fig. 1 illustrates an image display device 10 and a user 30 who is a passerby walking in front of the image display device 10 (withing viewing range, e.g., 100 feet or less and / or within a field of view of the imaging unit 101). The image display device 10 is, for example, a signage device.　As described above, the signage device, which is also called a digital signage and an electronic billboard, is a device that displays various kinds of information depending on where the device is located. In the signage device, various contents such as advertisement information on various products and stores, guidance information, advertisement information on movies, and advertisement images of performers such as a singer, an actor, and an idol can be output.　While the term "performer" is used for convenience, other entities such as celebrities, animals, etc. may be displayed as well.

[0018] An image of a performer 20 is output to a display unit 11 in the image display device (signage device) 10 illustrated in Fig. 1.　Note that the performer is, for example, a singer, an actor, or a famous person.　A performer image 20a output to the display unit 11 in the image display device (signage device) 10 illustrated in Fig. 1 is a normal image (for example, a still image).

[0019] As illustrated in Fig. 1, the image display device 10 has an imaging unit (camera) 12.　The imaging unit (camera) 12 continuously takes images in front of (within the field of view of the camera 12) the image display device 10, and the taken images are input to a data processing unit inside the image display device 10 and analyzed in the data processing unit.　While the term "unit" such as "data processing unit" is used herein, it should be understood that "unit" or in this case, "data processing unit" may be implemented as circuitry.　The circuitry may be hardwired circuitry, such as an application specific integrated circuit, and / or programmable circuitry such as one or more processors (e.g., CPUs and / or GPUs) and at least one or more memories that hold executable code.　The one or more processors are configured to implement the functions of the data processing unit by the execution of the stored executable code.　The one or more processors may be disposed locally within the image display device 10, or may be distributed.　A distributed processor architecture may include one or more remote processors (e.g., cloud computing resources) that communicate with one or more local processors via wired and / or wireless communications.

[0020] The data processing unit inside the image display device 10 inputs an image taken by the imaging unit (camera) 12, and analyzes operations (actions) of the user 30 in front of the image display device 10.　While the term "user" is used herein, it should be understood that "user" is intended to cover a single person or a plurality of people, such as a group of teenagers walking together, or parents pushing a baby stroller.　The actions of the plurality of people may include, for example, walking by, posing in front of the display, etc.　One or more actions of the plurality of people includes actions by an individual in the group (e.g., someone pointing at the displayed image) or the group itself (e.g., walking, dancing, posing in front of the displayed image). The data processing unit inside the image display device 10 executes processing for switching contents displayed on the display unit 11 in accordance with an analysis result.　The term "inside" refers to at least a portion of the processing circuitry that is located inside the display unit 11.

[0021] For example, when it is detected that the user 30 in front of the image display device 10 is about to photograph display contents on the display unit 11 or is photographing the display contents with his / her smartphone directed to the image display device 10, the data processing unit inside the image display device 10 executes processing for switching the display contents.

[0022] Referring to Fig. 2, a specific example of the processing for switching display contents is described. Fig. 2 illustrates a state in which the user 30 in front of the image display device 10 photographs display contents on the display unit 11 with his / her smartphone directed to the image display device 10.

[0023] The data processing unit in the image display device 10 uses an image taken by the imaging unit (camera) 12 to analyze an operation (action) of the user 30.　As a result of this analysis processing, it is determined that the user 30 is performing an operation of photographing the display contents on the display unit 11 with the smartphone. The data processing unit in the image display device 10 executes the processing for switching the display contents on the display unit 11 on the basis of this operation determination.

[0024] Specifically, the data processing unit stops displaying the performer image (normal image (still image)) 20a, which was displayed before the checking of user operation, and starts outputting a performer image (special image (moving image)) 20b. In other words, the data processing unit switches the image of the performer 20 from a still image to a moving image, and outputs a moving image.　Note that the data processing unit may be configured to output music or voice, for example, voice such as a call or singing voice of the performer 20 together with the output of the moving image. The user 30 can use a smartphone 40 to record moving image data and voice data output to the display unit 11 in the image display device 10.

[0025] In this manner, the image display device 10 in the present disclosure analyzes a user state such as operation of a user such as a passerby in front of the display unit 11 in the image display device 10, and when it is determined that the user state satisfies a contents switching condition prescribed in advance, executes processing for switching contents to be displayed on the display unit 11.

[0026] Note that the "user state that satisfies a contents switching condition prescribed in advance" in the present example is a "state in which photographing of the display unit is executed by a user", which is an example of a contents switching condition.

[0027] The "user state that satisfies a contents switching condition prescribed in advance" is not limited to the "state in which photographing of the display unit is executed by a user", and various user states such as other user operations and belongings and clothes of a user can be applied.　Specific examples thereof are described later.

[0028] Furthermore, in the present example, the display contents before and after switching are a combination of still image contents and moving image contents, but this is an example.　The combination of display contents before and after switching is not limited to a combination of a still image and a moving image, and various combinations such as a combination of a still image a and a still image b and a combination of a moving image a and a moving image b can be used.

[0029] Note that the image display device 10 illustrated in Fig. 2 is configured to output an image of a single performer, but, for example, as illustrated in Fig. 3, an image display device 10 configured to output images of a plurality of different performers may be used.

[0030] The image display device 10 illustrated in Fig. 3 is configured to display three different performers, that is, a performer a, a performer b, and a performer c, side by side on the display unit 11. Note that the display unit 11 in the image display device 10 is configured as a single display unit.　The single display unit is divided into three regions, and individual contents are output in units of regions.

[0031] In normal cases, a still image of the performer a is displayed in a left region, a still image of the performer b is displayed in a center region, and a still image of the performer c is displayed in a right region. The user 30, who is a passerby, is walking in front of the display unit 11 in the image display device 10.

[0032] In Fig. 4, the user 30 looks at an image displayed on the display unit 11, and takes out the smartphone 40 to photograph an image of the performer a displayed in the left region of the display unit 11 in the image display device 10.　The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12, and determines that the user 30 is performing an operation of photographing the image of the performer a displayed in the left region of the display unit 11 with the smartphone. The data processing unit in the image display device 10 executes, on the basis of this operation determination, processing for switching the image of the performer a displayed in the left region of the display unit 11.

[0033] Specifically, the data processing unit stops displaying a still image of the performer a, which was displayed before the checking of user operation, and switches the image of the performer a to a moving image (special image) to start displaying the moving.　Note that the data processing unit may be configured to output music or voice, for example, voice such as a call or a song of the performer 20 together with the output of the moving image.

[0034] The user 30 can use the smartphone 40 to record moving image data and voice data of the performer a displayed in the left region of the display unit 11 in the image display device 10.

[0035] Note that, at this time, the output of the image of the performer b in the center region of the display unit 11 and the image of the performer c in the right region, which are not photographing targets by the smartphone 40 of the user 30, is continued as still images (normal images).

[0036] The next Fig. 5 illustrates a state in which the user 30 has moved from the left side (p1) to the right side (p2) of the image display device 10 and starts photographing an image of the performer c displayed in the right region of the display unit 11 in the image display device 10 with the smartphone 40.

[0037] The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12, and determines that the user 30 executes an operation of photographing the image of the performer c displayed in the right region of the display unit 11 with the smartphone. The data processing unit in the image display device 10 executes, on the basis of this operation determination, processing for switching the image of the performer c displayed in the right region of the display unit 11 from a still image to a moving image.

[0038] Note that the image of the performer a in the left region of the display unit 11, which is not a photographing target by the smartphone 40 of the user 30, is switched to a still image (normal image) at a time when the reproduction of the moving image contents is finished.　Furthermore, the output of the image of the performer b in the center region is continued as a still image (normal image).

[0039] The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine the position of the user 30 and a photographing direction of the smartphone 40, and executes processing for selecting only an image in a region corresponding to the photographing direction of the smartphone 40, switching the image to a moving image, and reproducing the moving image. Note that details of the processing for analyzing the position of the user 30 and the photographing direction of the smartphone 40, which is executed by the data processing unit in the image display device 10, are described later.

[0040] Fig. 6 is a diagram illustrating a transition example of images displayed on the display unit 11 in a case where the user 30 has moved from left to right of the image display device 10 while photographing the images by the smartphone 40.　Time elapses in the order of (Step S01) (time t=t1) at the upper left of Fig. 6, (Step S02) (time t=t2) at the right center, and (Step S03) (time t=t3) at the lower left.

[0041] In (Step S01) (time t=t1) at the upper left, the user 30 is photographing from the left end of the image display device 10 with the smartphone directed to the image of the performer a. At this time, the data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user 30 is a display region of the performer a on the left side of the display unit 11, and switches only the image of the performer a to a moving image. The images in the display regions of the performers b and c remain as still images.

[0042] Next, (Step S02) (time t=t2) at the right center is a state in which the user 30 is photographing with the smartphone directed to the image of the performer b at the center of the image display device 10. At this time, the data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that the photographing direction (photographing target) of the smartphone of the user 30 is a display region of the performer b in the center region of the display unit 11, and switches the image of the performer b to a moving image.

[0043] Along with the finish of the reproduced moving image contents, the image of the performer a is switched to a still image (normal image).　The image in the display region of the performer c remains as a still image.

[0044] Next, (Step S03) (time t=t3) at the lower left is a state in which the user 30 is photographing with the smartphone directed to the image of the performer c on the right side of the image display device 10. At this time, the data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that the photographing direction (photographing target) of the smartphone of the user 30 is a display region of the performer c in the right region of the display unit 11, and switches the image of the performer c to a moving image.

[0045] Along with the finish of the reproduced moving image contents, the images of the performers a and b are switched to still images (normal images).

[0046] Fig. 7 is a diagram illustrating a processing example in a case where a plurality of users in front of the image display device 10 photograph images of different performers in different regions displayed on the display unit 11.

[0047] (Step S11) in Fig. 7 indicates a state in which a user x 30x is photographing with his / her smartphone directed to an image of the performer c on the right side of the image display device 10. The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user x 30x is a display region of the performer c in the right region of the display unit 11, and switches the image of the performer c to a moving image. The images of the performers a and b remain as still images (normal images).

[0048] (Step S12) indicates a state in which, while the user x 30x is photographing the performer c in the right region of the display unit, a user y 30y as another user starts photographing with his / her smartphone directed to the image of the performer a on the left side of the image display device 10.

[0049] The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user x 30x is a display region of the performer c in the right region of the display unit 11 and that a photographing direction (photographing target) of the smartphone of the user y 30y is a display region of the performer a in the left region of the display unit 11.

[0050] On the basis of this determination, the data processing unit in the image display device 10 switches the image of the performer a in the left region of the display unit 11 from a still image to a moving image and starts reproducing the moving image while continuing the reproduction of the moving image of the performer c in the right region of the display unit 11.

[0051] Fig. 8 illustrates a state of (Step S13), which is a state after (Step S12) illustrated in Fig. 7. (Step S12) illustrated at the upper left of Fig. 8 is the same state as (Step S12) at the lower right of Fig. 7, and is a state in which the image of the performer c in the right region of the display unit 11, which is a photographing target of the user x 30x, and the image of the performer a in the left region of the display unit 11, which is a photographing target of the user y 30y, have been switched to moving images and the reproduction of the moving images is executed.

[0052] (Step S13) illustrated at the lower right of Fig. 8 indicates a state in which the photographing by the user x 30x has been finished and the user x 30x is going away from the image display device 10. The other user y 30y continues photographing the image of the performer a in the left region of the display unit 11.

[0053] In this state, the image of the performer c in the right region of the display unit 11 is switched to a still image (normal image) along with the finish of the reproduced moving image contents.　On the other hand, the image of the performer a in the left region of the display unit 11 is continuously subjected to moving image reproduction processing, and is switched to a still image at a time when reproduced contents are finished.

[0054] Fig. 9 is a diagram for describing a processing example in which a moving image to be reproduced is selected and reproduced depending on the position of a user who photographs an image and the orientation and direction of a smartphone that is used by the user to photograph an image.

[0055] First, (Step S21) at the upper left of Fig. 9 indicates a state in which a user x 30x is photographing with his / her smartphone directed to an image of the performer b at the center of the image display device 10. The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user x 30x is a display region of the performer b in the center region of the display unit 11, and switches the image of the performer b to a moving image (moving image p (moving image contents p)).

[0056] In the processing example illustrated in Fig. 9, the data processing unit in the image display device 10 also analyzes the position of the user x 30x on the basis of the image taken by the imaging unit (camera) 12. The data processing unit in the image display device 10 analyzes that the position of the user x 30x is located on the left side of the image of the performer b and the user x 30x is photographing the image of the performer b from the left side.　The data processing unit in the image display device 10 switches the image of the performer b to a moving image in accordance with the analysis result by selecting and reproducing a moving image optimal for the position of a user or a photographing direction, that is, the "moving image p (moving image contents p)".　In other words, the data processing unit selects and reproduces moving image contents optimal for the user located on the left side of the image of the performer b.

[0057] In a storage unit (contents storage unit) inside the image display device 10, a plurality of kinds of moving image contents for each performer are stored, and the data processing unit in the image display device 10 selects and reproduces a moving image optimal for the position of a user who photographs an image of a performer and a photographing direction.

[0058] Next, (Step S22) on the right of Fig. 9 indicates a state in which the processing for reproducing the "moving image p (moving image contents p)" has been finished while the user x 30x is photographing the performer b at the center of the image display device 10 from left.　Along with the finish of the reproduction of the moving image p (moving image contents p), the image of the performer b is switched to a still image (normal image).

[0059] Furthermore, after that, (Step S23) indicates a state in which the user x 30x has walked in front of the image display device 10 and moved in the right direction and has started processing for photographing an image of the performer b from right.

[0060] The data processing unit in the image display device 10 analyzes, on the basis of an image taken by the imaging unit (camera) 12, that the user x 30x is photographing the performer b from right. On the basis of this analysis result, the data processing unit in the image display device 10 switches the still image of the performer b to a "moving image q (moving image contents q)", and starts reproducing the moving image.　The "moving image q (moving image contents q)" is moving image contents different from the "moving image p (moving image contents p)" reproduced in (Step S21).

[0061] Specifically, the moving image q (moving image contents q) are contents for a user located on the right side of the image of the performer b and a user who is photographing the image from the right side, and are moving image contents optimal for the user x 30x.

[0062] Fig. 10 is a diagram for describing a processing example in a case where a plurality of users photograph images of the same performer. First, (Step S31) at the upper left of Fig. 10 indicates a state in which the user x 30x is photographing with his / her smartphone directed to an image of the performer b at the center of the image display device 10 from the left side. The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user x 30x is a display region of the performer b in the center region of the display unit 11, and switches the image of the performer b to a moving image (moving image p (moving image contents p)).

[0063] The data processing unit in the image display device 10 also analyzes the position of the user x 30x on the basis of the image taken by the imaging unit (camera) 12. The data processing unit in the image display device 10 analyzes that the position of the user x 30x is on the left side of the image of the performer b and the user x 30x is photographing the image of the performer b from the left side.　The data processing unit in the image display device 10 switches the image of the performer b to a moving image in accordance with the analysis result by selecting and reproducing a moving image optimal for the position of a user or a photographing direction, that is, the "moving image p (moving image contents p)".　That is, the data processing unit in the image display device 10 selects and reproduces moving image contents optimal for the user located on the left side of the image of the performer b.

[0064] Next, (Step S32) at the lower right of Fig. 10 indicates a state in which another user z 30z starts processing for photographing the same performer b from right while the user x 30x is photographing the performer b at the center of the image display device 10 from left.

[0065] The data processing unit in the image display device 10 analyzes, on the basis of an image taken by the imaging unit (camera) 12, that the user x 30x and the user z 30z are photographing the performer b from left and right, respectively. On the basis of this analysis result, the data processing unit in the image display device 10 executes reproduced moving image switching processing for switching a "moving image p (moving image contents p)" that is being reproduced to a "moving image r (moving image contents r)".　Specifically, the data processing unit switches and reproduces a "moving image r (moving image contents r)" that is optimal moving image contents that can be satisfied by both the user located on the left side and the user located on the right side of the image of the performer b.

[0066] Note that, in the processing for switching moving image contents, in order to solve the discontinuity of a moving image before switching and a moving image after switching to achieve smooth switching, processing for reproducing an interpolated image configured by an intermediate image of the contents before and after the switching is performed.

[0067] That is, an interpolated image for implementing a moving image of moving image contents before and after switching is reproduced between images of the moving image contents before and after the switching.

[0068] This interpolated image may be generated when the data processing unit in the image display device 10 performs the processing for switching the contents, or interpolation image data stored in the storage unit (contents storage unit) in advance may be used. Note that specific examples of the processing for generating an interpolated image at the time of the contents switching are described later.

[0069] Fig. 11 is also a diagram for describing an example of processing for switching reproduced moving image contents depending on the position of a user and the direction of a smartphone. (Step S41) indicates a state in which the user x 30x is photographing with his / her smartphone directed to an image of the performer b at the center of the image display device 10 from the left side.

[0070] The data processing unit in the image display device 10 analyzes an image taken by the imaging unit (camera) 12 to determine that a photographing direction (photographing target) of the smartphone of the user x 30x is a display region of the performer b in the center region of the display unit 11, and selects and reproduces, as a moving image of the performer b, a "moving image p (moving image contents p)" that is a moving image optimal for the user x 30x located on the left side of the image of the performer b in the center region.

[0071] The next (Step S42) indicates a state in which the "moving image p" contents of the performer b have been reproduced by the end, and the reproduction of the moving image is finished such that the image of the performer b has been switched to a still image.

[0072] Furthermore, after that, (Step S43) indicates a state in which another user z 30z has started processing for photographing the image of the performer b from right.　At this time, the user x 30x takes down the smartphone without directing the smartphone to the image.

[0073] The data processing unit in the image display device 10 analyzes, on the basis of an image taken by the imaging unit (camera) 12, that the user z 30z is photographing the performer b from right. On the basis of this analysis result, the data processing unit in the image display device 10 executes reproduced moving image switching processing for switching the "moving image p (moving image contents p)" that is being reproduced to a "moving image q (moving image contents q)".　Specifically, the data processing unit switches the moving image that is being reproduced to a "moving image q (moving image contents q)" that is moving image contents optimal for the user z 30z who is photographing the image from the right side of the image of the performer b, and reproduces the selected moving image.

[0074] Furthermore, the subsequent (Step S44) is a state in which the user x 30x who has been aware of the reproduction of the new moving image starts processing for photographing the performer b from left.

[0075] The data processing unit in the image display device 10 analyzes, on the basis of an image taken by the imaging unit (camera) 12, that the user x 30x and the user z 30z are photographing the performer b from left and right, respectively. On the basis of this analysis result, the data processing unit in the image display device 10 executes reproduced moving image switching processing for switching the "moving image q (moving image contents q)" that is being reproduced to a "moving image r (moving image contents r)".　Specifically, the data processing unit switches and reproduces optimal moving image contents that can be satisfied by both the user located on the left side and the user located on the right side of the image of the performer b.

[0076] In this manner, the data processing unit in the image display device 10 analyzes the position of a user who photographs an image and the orientation of a smartphone, and selects and reproduces a moving image optimal for an analysis result.

[0077] Furthermore, the data processing unit in the image display device 10 in the present disclosure also executes processing for analyzing, on the basis of an image taken by the imaging unit (camera) 12, a user status (user state) such as clothes and belongings of a user, and selecting contents (moving image contents) to be reproduced on the basis of an analysis result, and reproducing the selected contents.

[0078] Referring to Fig. 12, a processing example is described. (Step S51) illustrated on the upper left in Fig. 12 indicates a state in which the user y 30y is walking and approaching from the left side of the image display device 10 while photographing an image with a smartphone.

[0079] At this time, the data processing unit in the image display device 10 performs processing for detecting, on the basis of an image taken by the imaging unit (camera) 12, that the user y 30y is photographing the performer a in a left region of the display unit, and switching the image of the performer a from a still image to a moving image.

[0080] Furthermore, the data processing unit in the image display device 10 detects, on the basis of the image taken by the imaging unit (camera) 12, that the user y 30y is carrying a registered item 50 that is a bag registered in advance.

[0081] Note that user status information such as particular items, belongings, wear, and clothes is registered in the storage unit in the image display device 10, and when a user within an image taken by the imaging unit (camera) 12 corresponds to such a registered status, a special image (user status-related special image) corresponding to the status is output to the display unit 11.

[0082] The example illustrated in Fig. 12 is a processing example in which, on the basis of an image taken by the imaging unit (camera) 12, the data processing unit in the image display device 10 detects that "the user y 30y is photographing the performer a in the left region of the display unit and detects that the user y 30y is carrying a registered item 50 that is a bag registered in advance".

[0083] In this case, the data processing unit in the image display device 10 first switches an image of the performer a from a still image to a moving image as indicated by (Step S51) at the upper left of Fig. 12. Furthermore, at a timing at which the reproduction of the moving image is finished, the data processing unit in the image display device 10 selects, reproduces, and displays special "moving image s contents (user status-related special image)" as indicated by (Step S52) at the lower right of Fig. 12.

[0084] However, when the registered item 50 is in a blind area of the field of view of the camera and it cannot be detected that the user y 30y carries the registered item 50 but it can be only detected that the user y 30y is photographing the performer a in the left region of the display unit, the data processing unit in the image display device 10 performs processing for only switching the image of the performer a from a still image to a moving image as indicated by (Step S51) at the upper left of Fig. 12. After that, in the case where the registered item 50 is detected because the user y 30y has moved, special "moving image s contents (user status-related special image)" may be reproduced immediately after the finish of the reproduction of the moving image or after the detection of the registered item 50.

[0085] Similarly to Fig. 12, Fig. 13 is also a diagram for describing an example of processing for analyzing a user status (user state) such as clothes or belongings of a user, selecting contents (moving image contents) to be reproduced on the basis of an analysis result, and reproducing the selected contents.

[0086] (Step S61) in Fig. 13 is a state in which a user y 30y is photographing the performer a in a left region of the display unit. The data processing unit in the image display device 10 performs processing for detecting, on the basis of an image taken by the imaging unit (camera) 12, that the user y 30y is photographing the performer a in the left region of the display unit, and switching an image of the performer a from a still image to a moving image.

[0087] Furthermore, the data processing unit in the image display device 10 detects, on the basis of the image taken by the imaging unit (camera) 12, that the user y 30y is carrying a registered item 50 that is a bag registered in advance.

[0088] Originally, the reproduction needs to be transitioned to special moving image s contents immediately after the finish of reproduction of the moving image of the performer a, but as indicated by (Step S62) in Fig. 13, at the time when the reproduction of the moving image of the performer a is finished, another user x 30x is performing processing for photographing the performer c from the right side of the image, and the user z 30z is also performing processing for photographing the performer b from the left side of the image.

[0089] As described above, the reproduction of the moving images of the performers b and c has started due to the photographing actions of the user x 30x and the user z 30z, and hence the data processing unit in the image display device 10 once stands by until the reproduction of the moving images of the performers b and c is finished.

[0090] (Step S63) in Fig. 13 is a state in which the reproduction of the moving images of the performers b and c has been finished.　After the reproduction of the moving images of the performers b and c is finished, the data processing unit in the image display device 10 selects, reproduces, and displays special "moving image s contents (user status-related special image)" as indicated by (Step S64) at the lower left of Fig. 13.

[0091] Note that the processing example described here is a processing example in which after the data processing unit stands by until the reproduction of all moving images on the display unit is finished, the data processing unit outputs a sequence of reproducing special "moving image s contents (user status-related special image)", but even when the reproduction of moving images of the performers b and c has started due to photographing actions of the user x 30x and the user z 30z, the action of the user y 30y carrying a registered item is prioritized and the reproduction of moving images of the performers b and c is finished in the middle before the finish of the reproduction of the moving images of the performers b and c, and special "moving image s contents (user status-related special image)" is reproduced.　Such a sequence may be employed.

[0092] In this manner, the image display device 10 in the present disclosure also executes processing for analyzing a user status (user state) such as clothes and belongings of a user, selecting contents (moving image contents) to be reproduced on the basis of an analysis result, and reproducing the selected contents.

[0093] (2. Configuration example of image display device in the present disclosure) Next, a configuration example of an image display device in the present disclosure is described.

[0094] Fig. 14 is a diagram illustrating a configuration example of an image display device 100 in the present disclosure.　The image display device 100 illustrated in Fig. 14 corresponds to the image display device 10 illustrated in Fig. 1 and other drawings. Note that the configuration of the image display device 100 illustrated in Fig. 14 illustrates only main components of the image display device 100.　The image display device 100 has, in addition to the configuration illustrated in Fig. 14, for example, a voice output unit, an input unit that can be operated by a user, and a storage unit having a control program stored therein.

[0095] As illustrated in Fig. 14, the image display device 100 has, its main components, an imaging unit 101, a data processing unit 102, a display unit 103 (e.g., display), a user information storage unit 104 (a memory), a contents storage unit 105 (memory), and a communication unit 106 (receiver, transmitter, transceiver).

[0096] The imaging unit 101 is a camera, and corresponds to the imaging unit (camera) 11 mounted to the image display device 10 described above with reference to Fig. 1 and other drawings.　The imaging unit 101 takes an image in front of the image display device.　For example, the imaging unit 101 photographs a user as a passerby. In a taken image, an operation of a user, such as a situation of image photographing processing by a smartphone is photographed.　Data on images taken by the imaging unit 101 is input to the data processing unit 102.

[0097] The data processing unit 102 executes processing for analyzing data on images taken by the imaging unit 101 and processing for controlling switching of contents to be displayed on the display unit 103 on the basis of an analysis result.

[0098] The display unit 103 corresponds to the display unit 11 in the image display device 10 described above with reference to Fig. 1 and other drawings, and displays various kinds of information such as still images and moving images of a performer.　For example, the display unit 103 is configured by a liquid crystal display device or an organic EL display device.

[0099] The user information storage unit 104 is a storage unit that stores therein information on states of users detected from an image taken by the imaging unit 101, that is, users such as a passerby in front of the display unit 103, and states of smartphones owned by the users.　Information analyzed by the data processing unit 102 on the basis of data on images taken by the imaging unit 101 is recorded. Details of user analysis processing by the data processing unit 102 and data stored in the user information storage unit 104 are described later.

[0100] The contents storage unit 105 is a storage unit that stores therein various kinds of contents to be displayed on the display unit 103.　For example, a plurality of contents such as various still images and moving images of performers.

[0101] The data processing unit 102 performs processing for selecting contents to be output from among contents stored in the contents storage unit 105, and displaying and reproducing the contents on the display unit 103. For example, when a user in front of the display unit 103 starts photographing an image of a performer as described above, the data processing unit 102 performs processing for outputting the image of the performer by switching from still image contents to moving image contents.

[0102] Specific examples of contents stored in the contents storage unit 105 and details of contents selection and output processing by the data processing unit are described later.

[0103] The communication unit 106 is a communication unit configured to execute communication with an external device, such as an external server. For example, the communication unit 106 is used for new contents reception processing, control update program reception, and input processing from an external server.

[0104] Next, a detailed configuration of the data processing unit 102 is described with reference to Fig. 15. As illustrated in Fig. 15, the data processing unit 102 has a taken image analysis unit 121, an output image control unit 122, a contents acquisition unit 123, and interpolated image generation unit 124.

[0105] The taken image analysis unit 121 executes processing for analyzing data on an image taken by the imaging unit 101. The taken image analysis unit 121 inputs, from the imaging unit 101, an image that photographs a situation of a passerby ahead of the display unit 103 of the image display device 100, and executes processing for analyzing the input image.　Specifically, the taken image analysis unit 121 detects a user such as a passerby from the taken image, analyzes various statuses (states) of the user and statuses (states) of a smartphone owned by the user, and stores the analyzed information in the user information storage unit 104. Details of the information stored in the user information storage unit 104 are described later.

[0106] The output image control unit 122 executes processing for controlling an image to be displayed on the display unit 103 on the basis of information analyzed by the taken image analysis unit 121. Specifically, the output image control unit 122 executes output image control processing as described above with reference to Fig. 1 to Fig. 13.　For example, when it is detected that a user is photographing an image of a performer with a smartphone, the output image control unit 122 executes processing for switching the image of the performer from a still image (normal image) to a moving image (special image).

[0107] Alternatively, the output image control unit 122 executes processing for switching an image to be output depending on the position of a user and processing for switching an output image depending on belongings (item) and clothes of a user. Note that a detailed sequence of the output image control processing by the output image control unit 122 is described later with reference to flowcharts.

[0108] In the processing executed by the output image control unit 122 for determining whether a user is photographing an image of a performer with a smartphone, for example, a plurality of kinds of determination processing, such as determination of "angle of smartphone" and determination as to "whether a part of a face region of a user has been replaced with a smartphone" are executed to perform highly accurate determination. Furthermore, for example, the "shape of a hand of a user when carrying a smartphone" may be stored in the storage unit as learning data, and determination processing using the learning data in the storage unit may be performed to increase the accuracy of determination.

[0109] The contents acquisition unit 123 executes processing for acquiring contents stored in the contents storage unit 105, for example, still images and moving images of each performer. Note that the processing for selecting an image to be output to the display unit 103 is executed by the output image control unit 122, and the contents acquisition unit 123 executes processing for acquiring the contents determined by the output image control unit 122 from the contents storage unit 105 and providing the contents to the output image control unit 122.

[0110] In the contents storage unit 105, a large number of still images and moving images of each performer are stored.　Examples of data stored in the contents storage unit 105 are described later. Note that the contents acquisition unit 123 may acquire new contents through the communication unit 106.

[0111] The interpolated image generation unit 124 executes interpolated image generation processing that enables the switching processing for moving images to be smoothly executed.　For example, as described above with reference to Fig. 10 and the like, when a reproduced moving image is switched from the moving image p to the moving image r, if the switching of the reproduced moving image is instantaneously executed, discontinuous moving images are reproduced. The interpolated image generation unit 124 executes interpolated image generation processing for solving this problem.

[0112] Specifically, in order to solve the discontinuity of reproduced moving images to achieve smooth switching such that moving image contents before and after switching become a moving image that smoothly transitions, the interpolated image generation unit 124 generates an interpolated image configured by an intermediate image of the images before and after the switching.

[0113] Referring to Fig. 16 and Fig. 17, a specific example of interpolated image generation processing executed by the interpolated image generation unit 124 is described.

[0114] Fig. 16(1) is a diagram illustrating a state in which the user 30 is photographing the performer b from the left side (user position=p1) of the image of the performer b. In this state, the output image control unit 122 in the data processing unit 102 in the image display device 100 selects and reproduces moving image contents optimal for the user 30 who is photographing the image of the performer b from the left side. It is assumed that the selected moving image contents are moving image contents bM1 stored in the contents storage unit 105 as illustrated on the right side of Fig. 16.

[0115] Next, as illustrated in Fig. 16(2), the user 30 has moved from the left side (user position=p1) of the image of the performer b to the right side (user position=p2) and continues photographing the image of the performer b. The "moving image contents bM1", which were reproduced before the user 30 moved, are an image optimal for a user located on the left side (user position=p1) of the image of the performer b, and are not optimal for a user who has moved to the right side (user position=p2).

[0116] Thus, the output image control unit 122 preforms processing for switching the "moving image contents bM1" that are being reproduced to moving image contents optimal for a user who has moved to the right side (user position=p2), that is, "moving image contents bM2" stored in the contents storage unit 105 illustrated on the right side of Fig. 16.

[0117] However, if such different moving image contents are instantaneously switched, the discontinuity of images occurs at a boundary between moving image contents before and after the switching. In order to solve the discontinuity of images to achieve smooth switching, the interpolated image generation unit 124 generates an interpolated image configured by an intermediate image of contents before and after the switching.

[0118] Fig. 17 illustrates an example of an interpolated image sequence generated by the interpolated image generation unit 124. Fig. 17 illustrates an example of an interpolated image sequence generated by the interpolated image generation unit 124 in processing of switching two moving image contents stored in the contents storage unit 105, that is, moving image contents bM1 and moving image contents bM2.

[0119] An image at the lower left end of Fig. 17 is the final image frame (frame identifier=bM1-fxxx) that is reproduced immediately before the switching of the moving image contents bM1 as a reproduced moving image before the switching. An image at the right end is the first image frame (frame identifier=bM2-fyyy) that is reproduced immediately after the switching of the moving image contents bM2 as a moving image after the switching.

[0120] These two image frames before and after switching, that is, the image frame (frame identifier=bM1-fxxx) of the moving image contents bM1 and the image frame (frame identifier=bM2-fyyy) of the moving image contents bM2 are images with different motions.　If the images are suddenly switched, an apparently unnatural motion occurs.　In other words, the discontinuity of images occurs at a boundary between moving image contents before and after the switching.

[0121] The interpolated image generation unit 124 generates an interpolated image for solving the discontinuity of images to obtain a moving image with smooth transition of moving image contents before and after the switching, that is, an interpolated image configured by an intermediate image of the images before and after the switching. Specifically, the interpolated image generation unit 124 generates an interpolated image sequence as illustrated in Fig. 17.

[0122] Such an interpolated image sequence generated by the interpolated image generation unit 124 is input to the output image control unit 122. The output image control unit 122 executes reproduction control in which after the final image frame (frame identifier=bM1-fxxx) immediately before the switching of the moving image contents bM1, an interpolated image sequence generated by the interpolated image generation unit 124 is reproduced, and then the first image frame (frame identifier=bM2-fyyy) immediately after the switching of the moving image contents bM2 is reproduced.

[0123] With such reproduction control, images before and after the switching of the moving image contents bM1 and the moving image contents bM2 are reproduced like single moving image contents with smooth motions, and hence a user as an observer can observe moving images with a comfortable feeling.

[0124] Next, an example of data stored in the user information storage unit 104 is described with reference to Fig. 18. As described above, in the user information storage unit 104, data analyzed by the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15 on the basis of an image taken by the imaging unit 101 is stored.

[0125] As illustrated in Fig. 18, the following data is stored in the user information storage unit 104. (a) Attributes of a user (person) included in an input image (b) State (status) of the user (person) included in the input image (c) State (status) of a smartphone held by the user (person) included in the input image (d) Performer (favorite performer) estimated to be liked by the user (person) included in the input image (e) Moving image (video) observation history information on the user (person) included in the input image These pieces of information (a) to (e) are recorded in the user information storage unit 104 in units of detected users.

[0126] The "(a) attributes of a user (person) included in an input image" are the sex, age, and the like of the user.　Note that the attribute information is information estimated from the image, and is thus not always 100% correct information.

[0127] The "(b) state (status) of the user (person) included in the input image" is information including information on an item such as a bag held by a user, color of clothes worn by the user, and the position of the line of sight of the user. The "(c) state (status) of a smartphone held by the user (person) included in the input image" is information on the position of the smartphone and information on the orientation of the smartphone (camera).

[0128] Specific examples of a line-of-sight position (x,y) of a user, a smartphone position (x,y), and a smartphone direction that are analyzed by the taken image analysis unit 121 on the basis of an image taken by the imaging unit 101 and stored in the user information storage unit 104 are described with reference to Fig. 19.

[0129] Fig. 19 illustrates a user 30 who stands in front of the display unit 103 in the image display device 100 and is photographing an image of a performer a displayed on the left end of the display unit 103 with a smartphone 40.　Note that Fig. 19 illustrates the state of the user corresponding to recorded information on a user (U001) illustrated in Fig. 18.

[0130] The imaging unit (camera) 101 in the image display device 100 takes an image of the user 30.　The taken image is input to the taken image analysis unit 121 in the data processing unit 102. The taken image analysis unit 121 analyzes the position (x,y) of the line of sight of the user 30, and further analyzes the position (x,y) and orientation of the smartphone 40.

[0131] As illustrated in Fig. 19, a display surface of the display unit 103 in the image display device 100 is set as an xy plane.　The x axis is an axis in a lateral direction (horizontal direction) of the display unit 103 of the image display device 100 that is set on a walking surface of a user.　The y axis is an axis in a vertical direction (perpendicular direction) of the display unit 103 in the image display device 100 that is set at the left end of the display unit 103 in the image display device 100. The origin (0,0) of the xy plane is an intersection of the y axis set at the left end of the display unit 103 in the image display device 100 and the x axis set at a walking surface of a user. Note that the z axis is an axis extending in a direction separating from the plane (xy plane) of the display unit 103 in the image display device 100.

[0132] In the example illustrated in Fig. 19, the user 30 photographs an image of the performer a with the smartphone 40 while standing on the z axis, that is, at the position (x,y)=(0,0). In such a state, the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 analyzes the position (x,y) of the line of sight of the user 30 and the position (x,y) and orientation of the smartphone 40 as follows.

[0133] The taken image analysis unit 121 calculates an intersection position between a vector indicating the direction of the line of sight of the user 30 and the display unit 103 as a line of sight position (x,y) of the user 30, and stores the line of sight position (x,y) in the user information storage unit 104. In the example illustrated in Fig. 19, the position of the line of sight of the user 30 is (x,y)=(55,174), and (x,y)=(55,174) is recorded in the user information storage unit 104 as the position (x,y) of the line of sight of the user state (user status). This user line-of-sight position information corresponds to line-of-sight position information (x,y)=(55,174) on the user (U001) in the data illustrated in Fig. 18.

[0134] Furthermore, the taken image analysis unit 121 analyzes the position (x,y) and orientation of the smartphone 40. In the state illustrated in Fig. 19, the position (x,y) of the smartphone 40 is (x,y)=(0,169). The taken image analysis unit 121 records the position (x,y)=(0,169) of the smartphone 40 in the user information storage unit 104. This smartphone position information corresponds to smartphone position information (x,y)=(0,169) on the user (U001) in the data stored in the user information storage unit 104 illustrated in Fig. 18. Note that the data stored in the user information storage unit 104 illustrated in Fig. 18 does not include data in the z direction, but data in the z direction may be analyzed and stored.

[0135] Furthermore, the taken image analysis unit 121 analyzes the direction of the smartphone 40.　The direction of the smartphone 40 is a rotation angle about a vertical axis (vertical axis parallel to Y axis), and is a rotation angle of "18° to right" in the example illustrated in Fig. 19.

[0136] The taken image analysis unit 121 records the direction of the smartphone 40=(right 18°) in the user information storage unit 104. This smartphone position information corresponds to smartphone direction information=(right 18°) on the user (U001) in the data illustrated in Fig. 18.

[0137] The output image analysis unit 122 in the data processing unit 102 illustrated in Fig. 15 uses information on the position of a user and the position and orientation of a smartphone stored in the user information storage unit 104 to determine a performer photographed by the user and a photographing direction, determines contents to be output to the display unit 103 on the basis of this determination result, acquires the contents from the contents storage unit 105, and reproduces the contents.

[0138] Referring back to Fig. 18, the description of data stored in the user information storage unit 104 is continued. In "(d) performer (favorite performer) estimated to be liked by a user (person) included in the input image", in regard to each performer displayed on the display unit 1103, the value (0 to 100%) of probability that the performer is favorite to a user, that is, a favorite performer, is recorded.

[0139] The value (0 to 100%) of probability that each performer is a favorite performer for a user is calculated by the taken image analysis unit 121 by taken image analysis processing.　Specifically, for example, the rate of occupancy by which the line of sight of a user with respect to each performer displayed on the display unit 103 is recorded as a favorite performer probability value (0 to 100%).

[0140] For example, the taken image analysis unit 121 tracks the line of sight of a user through taken image analysis processing, analyzes a gazed position, timing, and the number of seconds, and estimates a performer of interest (favorite performer).　Furthermore, for example, the estimation of "favorite" may be enhanced by weighting a performer at a part where the user gazes immediately before or immediately after the user directs his / her smartphone.

[0141] The "(e) moving image (video) observation history information on a user (person) included in the input image" is information obtained by counting the number of times by which the user viewed the reproduction of moving images on the image display device 100 in the past.

[0142] Although not illustrated in Fig. 18, the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 executes user identification processing based on an image taken by the imaging unit 101, and stores user identification information including user feature information such as a face image of each user in the user information storage unit 104.

[0143] The data in units of users illustrated in Fig. 18 is linked (associated) with user identification information separately stored, and the taken image analysis unit 121 uses the linked data to analyze and record observation execution date and time of a performer moving image (special image) for each user. For each user, the taken image analysis unit 121 records observation execution date and time of a performer moving image (special image) in units of performers in the region of the "(e) moving image (video) observation history information on the user (person) included in the input image".

[0144] For example, in the example illustrated in Fig. 18, the user (U001) has the following observation history of the performer moving image. Observation of moving image of performer c at 12:20:05 on 4 December 2023 Observation of moving image of performer b at 12:21:15 on 4 December 2023 Observation of moving image of performer c at 12:21:55 on 4 December 2023

[0145] As described above, in the user information storage unit 104 illustrated in Fig. 18, (a) attributes of user (person) included in input image (b) states (statuses) of user (person) included in input image (c) states (statuses) of smartphone held by user (person) included in input image (d) performer (favorite performer) estimated to be liked by user (person) included in input image (e) moving image (video) observation history information on user (person) included in input image These pieces (a) to (e) of information are recorded as information in units of users.

[0146] These pieces (a) to (e) of information are analyzed by the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15 on the basis of an image taken by the imaging unit 101, and stored in the user information storage unit 104.

[0147] Next, an example of data stored in the contents storage unit 105 is described with reference to Fig. 20. As illustrated in Fig. 20, in the contents storage unit 105, a large number of still image contents and moving image contents in units of performers that are to be output to the display unit 103 in the image display device 100 are stored.

[0148] For example, in regard to the performer a, the following image contents are stored. (1) Still image contents a1 (2) Moving image contents for display region L (left)=moving image contents aL1, aL2, aL3, ... (3) Moving image contents for display region M (mid)=moving image contents aM1, aM2, aM3, ... (4) Moving image contents for display region R (right)=moving image contents aR1, aR2, aR3, ...

[0149] The "(1) still image contents a1" is a still image of the performer a, and is image data corresponding to a normal image.

[0150] The "(2) moving image contents for display region L (left)" are moving image contents (special image) to be displayed in a left display region in the setting of display regions divided into three regions in units of partners as illustrated in Fig. 3 and Fig. 4, for example. The "(3) moving image contents for display region M (mid)" are moving image contents (special image) to be displayed in a center display region in the display regions divided into three regions. The "(4) moving image contents for display region R (right)" are moving image contents (special image) to be displayed in a right display region in the display regions divided into three regions.

[0151] Note that, in each of the moving image contents (2) to (4) for the display regions, a plurality of different moving image contents are recorded, and the output image control unit 122 selects contents optimal for situations of a user in front of the display unit 103, and outputs the contents to each display region.

[0152] For example, image contents that are output when a user is photographing an image of a performer from the right side, image contents that are output when a user is photographing from the left side, image contents that are output when a user is photographing from the front, and image contents that are output when a plurality of users are photographing as described above with reference to Fig. 9 and Fig. 10 and image contents that are output depending on belongings and clothes of a user as described above with reference to Fig. 12, that is, a large number of different contents to be output depending on the situations of a user in front of the display unit 103 are stored as the moving image contents (2) to (4) for the display regions.

[0153] The output image control unit 122 selects and outputs, from among the large number of contents, contents optimal for the situations of a user in front of the display unit 103.

[0154] (3. Sequence of processing to be executed by image display device in the present disclosure) Next, a sequence of processing to be executed by the image display device 100 in the present disclosure is described.

[0155] Referring to the flowcharts in Fig. 21 and subsequent drawings, a sequence of processing to be executed by the image display device 100 in the present disclosure is described. Note that the flowcharts illustrated in Fig. 21 and subsequent drawings are executed by the data processing unit 102 in the image display device 100.　Note that the data processing unit 102 has a CPU having a program execution function, and executes processing according to the flows in accordance with programs stored in a storage unit in the image display device 100. Pieces of processing in steps in the flowchart illustrated in Fig. 21 are sequentially described below.

[0156] (Step S101) First, in Step S101, the data processing unit 102 in the image display device 100 inputs an image taken by the imaging unit (camera) 101 in the image display device 101, and executes analysis of the input image to determine whether there is a user such as a passerby in the neighborhood of the display unit 103, that is, determine the presence / absence of a user.

[0157] Note that this processing is processing executed by the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0158] (Step S102) Step S102 is a branch step based on a determination result of the presence / absence of a user in Step S101, and when it is determined that there is a user such as a passerby in the neighborhood of the display unit 103, the flow proceeds to Step S103.　On the other hand, when it is determined that there is no user, the flow proceeds to Step S121.

[0159] (Step S103) When it is determined in Step S102 that there is a user such as a passerby in the neighborhood of the display unit 103, the data processing unit 102 executes the following processing in Step S103.

[0160] The data processing unit 102 analyzes user information on user attributes (such as sex and age) of a user detected from an image taken by the imaging unit (camera) 101, user states (such as wearing items, belongings, and line of sight), and smartphone states (such as position and camera direction) of a smartphone of the detected user, and stores the information in the user information storage unit 104.

[0161] This processing is also processing executed by the taken image analysis unit 121 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0162] The taken image analysis unit 121 analyzes, in regard to a user detected from the taken image, attributes (such as sex and age) of the user, states (such as wearing items, belongings, and line of sight) of the user, and smartphone states (such as position and camera direction) of a smartphone of the detected user, and stores the information in the user information storage unit 104.

[0163] Note that, as described above with reference to Fig. 18, the following data is stored in the user information storage unit 104. (a) Attributes of user (person) included in input image (b) State (status) of user (person) included in input image (c) State (status) of smartphone held by user (person) included in input image (d) Performer (favorite performer) estimated to be liked by user (person) included in input image (e) Moving image (video) observation history information on user (person) included in input image These pieces (a) to (e) of information are recorded in the user information storage unit 104 in units of a user detected from a taken image.

[0164] In the flow illustrated in Fig. 21, only the analysis and storage processing for the above-mentioned pieces (a) to (c) of information is described, but as described above with reference to Fig. 18, the taken image analysis unit 121 in the data processing unit 102 also analyzes the above-mentioned pieces (d) and (e) of information and records the information in the user information storage unit 104.

[0165] (Step S104) Next, in Step S104, the data processing unit 102 in the image display device 100 determines whether a user state detected from the taken image is a user state that satisfies a display contents switching condition prescribed in advance.

[0166] This processing is processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0167] Note that the "user state that satisfies a display contents switching condition prescribed in advance" is, for example, a "state in which processing of photographing of the display unit is executed by a user". However, as described above, the "user state that satisfies a contents switching condition prescribed in advance" is not limited to a "state in which processing of photographing of the display unit is executed by a user", and other user operations and user states may be applied.　Specific examples thereof are described later.

[0168] When it is determined in Step S104 that the user state detected from the taken image is a user state that satisfies a display contents switching condition prescribed in advance, the flow proceeds to Step S111. On the other hand, when it is determined that the user state detected from the taken image is not a user state that satisfies a display contents switching condition prescribed in advance, the flow proceeds to Step S121.

[0169] (Step S111) First, processing in Step S111 and subsequent steps is described with reference to a flowchart illustrated in Fig. 22.

[0170] Processing in Step S111 and subsequent steps is processing executed when it is determined in Step S104 that the user state detected from the taken image is a user state that satisfies a display contents switching condition prescribed in advance.

[0171] In this case, in Step S111, the data processing unit 102 in the image display device 100 determines whether the image that is currently output to the display unit is a normal image (such as still image).

[0172] This processing is processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0173] Note that an image to be processed in Step S111 and subsequent steps is an output image in an image region as a photographing target of a user. For example, when three performers a to c are displayed side by side on the display unit and a user is photographing an image of the performer a, only the image of the performer a is a processing target image in Step S111 and subsequent steps.

[0174] When it is determined in Step S111 that the image output to the display unit is a normal image (such as still image), the flow proceeds to Step S112, and when the image output to the display unit is not a normal image (such as still image), the flow proceeds to Step S113.

[0175] (Step S112) When it is determined in Step S111 that the image output to the display unit is a normal image (such as still image), in Step S112, the data processing unit 102 in the image display device 100 executes output contents switching processing for switching the normal image (such as still image) output to the display unit to a special image (such as moving image).

[0176] In other words, the data processing unit 102 executes processing for switching the image from a still image to a moving image. Note that voice may be output together with the moving image switching processing as described above.

[0177] This processing is processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0178] Note that, in the processing in Step S112, when switching a normal image (such as still image) to a special image (such as moving image), the output image control unit 122 performs processing for selecting and outputting moving image contents that have been determined as optimal for the user position from among a large number of moving image contents stored in the contents storage unit 105.

[0179] A detailed sequence of Step S112 is described with reference to Fig. 23. Fig. 23 illustrates a flow indicating the detailed sequence of Step S112. As illustrated in Fig. 23, Step S112 is configured by processing in Step S112a and processing in Step S112b. The two pieces of processing are described. Note that the two pieces of processing are processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0180] (Step S112a) In Step S112a, the output image control unit 122 analyzes a user position of a user, a smartphone position, and a camera direction on the basis of an image taken by the imaging unit 101. Note that these data may be input by the output image control unit 122 from the taken image analysis unit 121, or data stored in the user information storage unit 104 may be used.

[0181] (Step S112b) Next, in Step S112b, the output image control unit 122 selects and outputs a special image (such as moving image)) optimal for the user position of the user, the smartphone position, or the camera direction analyzed in Step S112a.

[0182] For example, when a user is photographing a performer image from the left side of the image, moving image contents optimal for the user located on the left side are selected and output. Alternatively, when a user is photographing a performer image from the right side of the image, moving image contents optimal for the user located on the right side are selected and output.

[0183] Furthermore, when the user position, the camera position, or the camera direction changes, in Step S112b, the output image control unit 122 executes processing for switching the output contents in accordance with the change.

[0184] For example, when the position of a user changes as illustrated in Fig. 9 and Fig. 10 described above, the switching of contents is executed in accordance with the change in user position. Note that, in the switching of the contents, an interpolated image is output between the switching contents as necessary. Specifically, as described above with reference to Fig. 16 and Fig. 17, an interpolated image is generated and output in order not to generate an unnatural motion at a switching part of different moving image contents, so that the moving image contents before and after the switching become moving images with smooth transition.

[0185] Note that, as described above with reference to Fig. 10, when a plurality of users are photographing, moving image contents optimal for the plurality of users are selected and output. Furthermore, as described above with reference to Fig. 12, when it is determined that belongings such as a bag owned by a user or clothes satisfy a prescribed contents switching condition, the contents are switched to special contents corresponding to the condition and output.

[0186] For example, data that associates information on items as belongings of users and colors of the items and clothes with performers displayed on the display unit may be stored in the storage unit, and the associated data may be used to perform contents switching control. For example, when a user wears an item of a particular color, processing for switching an image of a member corresponding to the color to moving image contents is performed.

[0187] Returning to Fig. 22, the description of the flowchart illustrated in Fig. 22 is continued. (Step S113) When it is determined in Step S111 that the image output to the display unit is not a normal image (such as still image), that is, the image is a special image (such as moving image), the flow proceeds to Step S113.

[0188] In this case, in Step S113, the data processing unit 102 in the image display device 100 continuously outputs the special image (such as moving image) output to the display unit without any change.

[0189] This processing is processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0190] Note that, also in the processing in Step S113, similarly to the processing in Step S112 described above, processing for selecting and outputting moving image contents optimal for a user position is performed in the continuous output processing of a special image (such as moving image).

[0191] A detailed sequence of Step S113 is described with reference to Fig. 24. Fig. 24 illustrates a flow indicating the detailed sequence of Step S113. As illustrated in Fig. 24, Step S113 is configured by processing in Step S113a and processing in Step S113b. The two pieces of processing are described. Note that the two pieces of processing are processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0192] (Step S113a) In Step S113a, the output image control unit 122 analyzes a user position of a user, a smartphone position, and a camera direction on the basis of an image taken by the imaging unit 101. Note that these data may be input by the output image control unit 122 from the taken image analysis unit 121, or data stored in the user information storage unit 104 may be used.

[0193] (Step S113b) Next, in Step S113b, the output image control unit 122 selects and outputs a special image (such as moving image)) optimal for the user position of a user, the smartphone position, and the camera direction analyzed in Step S113a.

[0194] For example, when a user is photographing a performer image from the left side of the image, the output image control unit 122 selects and outputs moving image contents optimal for the user located on the left side. Alternatively, when a user is photographing a performer image from the right side of the image, the output image control unit 122 selects and outputs moving image contents optimal for the user located on the right side.

[0195] Furthermore, when the user position, the camera position, or the camera direction changes, in Step S113b, the output image control unit 122 executes processing for switching the output contents in accordance with the change. In the switching of the contents, the output image control unit 122 outputs an interpolated image between the switching contents as necessary. These pieces of processing are the same as the processing in Step S112 described above.

[0196] Note that, after the completion of the processing in Step S112 or Step S113 in the flow illustrated in Fig. 22, the flow returns to Step S101, and the processing in Step S101 and subsequent steps is repeated.

[0197] Next, with reference to the flowchart illustrated in Fig. 25, processing to be executed when No is determined in Step S102 or when No is determined in Step S104 in the flow illustrated in Fig. 21 is described.

[0198] The case where No is determined in Step S102 in the flow illustrated in Fig. 21 is the case where it is determined that there is no user such as a passerby in the neighborhood of the display unit 103. Furthermore, the case where No is determined in Step S104 is the case where a user state detected from the image taken by the imaging unit 101 is not a user state that satisfies a display contents switching condition prescribed in advance. In these cases, the processing according to the flow illustrated in Fig. 25 is executed. Processing in each step of the flow illustrated in Fig. 25 is described below.

[0199] (Step S121) In any of the above-mentioned cases, that is, when No is determined in Step S102 that there is no user such as a passerby in the neighborhood of the display unit 103, or when No is determined in Step S104 that the user state detected from the image taken by the imaging unit 101 is not a user state that satisfies the display contents switching condition prescribed in advance, the data processing unit 102 in the image display device 100 determines in Step S121 whether the image output to the display unit is a normal image (such as still image).

[0200] This processing is processing executed by the output image control unit 122 in the data processing unit 102 in the image display device 100 illustrated in Fig. 15.

[0201] When it is determined that the image output to the display unit 103 is a normal image (such as still image), the flow proceeds to Step S122. On the other hand, when the image output to the display unit 103 is a special image (such as moving image), the flow proceeds to Step S123.

[0202] (Step S122) When it is determined in Step S121 that the image output to the display unit 103 is a normal image (such as still image), in Step S122, the data processing unit 102 in the image display device 100 continuously outputs the normal image (such as still image) output to the display unit as a normal image (still image) without ant change.

[0203] (Step S123) On the other hand, when it is determined in Step S121 that the image output to the display unit 103 is a special image (such as moving image), in Step S123, the data processing unit 102 in the image display device 100 executes processing for switching the special image (such as moving image) output to the display unit to a normal image (still image).

[0204] After the completion of the processing in Step S122 or Step S123, the flow returns to Step S101, and the processing in Step S101 and subsequent steps is repeated.

[0205] Note that, in the case where the processing is performed in accordance with the flows illustrated in Fig. 21 to Fig. 25, for example, when a user directs a smartphone to the display unit 103, the reproduction of moving image contents as a special image is started.　However, if the state in which the user directs the smartphone to the display unit 103 is continued even after the finish of the reproduction of moving image contents, the processing of "moving image reproduction → finish → reproduction of moving image → finish → reproduction of moving image → ..." is repeated.

[0206] In order to prevent such repetition of the processing, if a user continues directing the smartphone to the display unit 103 even after the reproduction of moving image contents is finished, exceptional processing for preventing repetitive reproduction of moving image contents may be performed. However, also in the case where the exceptional processing is performed, if the user once stops directing the smartphone to the display unit 103 and then again directs the smartphone to the display unit 103, the reproduction of moving image contents is started.

[0207] (4. Example of switching conditions for contents to be output by image display device in the present disclosure) Next, an example of switching conditions for contents to be output by the image display device in the present disclosure is described.

[0208] As described above, the image display device in the present disclosure analyzes a user state such as operation of a user such as a passerby in front of the display unit, and when it is determined that the user state is in a state that satisfies a contents switching condition prescribed in advance, executes processing for switching contents displayed on the display unit.

[0209] Note that, in the above-mentioned example, a "state in which processing for photographing the display unit is executed by a user" is described as an example of the "user state that satisfies a contents switching condition prescribed in advance", but as described above, this is an example of the contents switching condition.

[0210] The "user state that satisfies a contents switching condition prescribed in advance" is not limited to the "state in which processing for photographing the display unit is executed by a user", and other various user operations and user states can be applied. Examples of user operations user states that can be applied to the contents switching condition are described.

[0211] Referring to Fig. 26 and subsequent drawings, user states that can be applied to the contents switching condition are described. Fig. 26(A) is an example in which contents are switched under a condition that it is detected that a user x 30x in front of the image display device 100 is directing the line of sight toward the display unit in the image display device 100 for a time (for example, 5 seconds) prescribed in advance or more.

[0212] In this manner, even when a user is not photographing an image with a smartphone, if the user who views the displayed image (such as image of performer) on the display unit for a given time or more has been detected, the output image control unit 122 in the data processing unit 102 in the image display device 100 executes contents switching processing for switching the display contents from a still image to a moving image.

[0213] Fig. 26(B) is an example in which contents are switched under a condition that it is detected that a user y 30y in front of the image display device 100 is pushing a stroller.

[0214] When a user pushing a stroller or a user carrying a suitcase is detected, the output image control unit 122 in the data processing unit 102 in the image display device 100 executes contents switching processing for switching the display contents from a still image to a moving image.

[0215] Fig. 27(C) is an example in which contents are switched under a condition that a parent and a child are detected in front of the image display device 100. Fig. 27(D) is an example in which contents are switched under a condition that a pair of two female and male persons are detected in front of the image display device 100. In addition, the case where a family is detected and the case where a group of two or more persons is detected may be used as a contents switching condition.

[0216] Fig. 28(E) is an example in which contents are switched under a condition that a person (user) standing back in front of the image display device 100 is detected. This is effective as processing for prompting the user to turn his / her head to view an image displayed on the display unit in the image display device.

[0217] Note that, in addition to the above, various user states such as the case where a user has a particular item, the case where a user is walking with a pet, the case where a user wears particular clothes, and the case where a user wears clothes of particular color can be set as a contents switching condition.

[0218] (5. Example in which external device such as server connected to image display device through network executes contents switching control) Next, an example in which an external device such as a server connected to an image display device through a network executes contents switching control is described.

[0219] In the above-mentioned examples, the image display device is configured to execute processing along as a stand-alone device. Specifically, as described above with reference to Fig. 14 and Fig. 15, the data processing unit 102 in the image display device 100 analyzes an image taken by the imaging unit 101 and performs control of switching contents to be output to the display unit 103 in accordance with an analysis result.

[0220] The processing in the present disclosure can be implemented by not only such a stand-alone device but also, for example, a device connected to a network as illustrated in Fig. 29. The configuration illustrated in Fig. 29 is a configuration in which a large number of image display devices (signage devices) 100 are connected to a server 70 and a network.

[0221] A large number of image display devices (signage devices) 100 connected to the network transmit images taken by imaging units (camera) mounted to the devices to the server 70. The server 70 receives the taken images from the plurality of image display devices (signage devices) 100, analyzes the received images to determine whether a user state that satisfies a contents switching condition is detected, and transmits a contents switching command or contents to be switched to the image display devices 100 individually in accordance with a determination result.

[0222] When the server 70 is configured to transmit display contents selected on the basis of the analysis result to the image display device (signage device) 100, the image display device (signage device) 100 performs processing for displaying the contents received from the server 70.　In this configuration, the image display device (signage device) 100 only needs to have an imaging unit, a display unit, and a communication unit.

[0223] Furthermore, each image display device (signage device) 100 may have a contents storage unit.　In this configuration, the server 70 transmits a contents switching control command including information of designating contents to be displayed on each image display device on the basis of an analysis result to the image display device (signage device) 100.

[0224] The image display device (signage device) 100 acquires, from the contents storage unit, contents designated from the server 70 and displays the acquired contents. In this configuration, the image display device (signage device) 100 only needs to have an imaging unit, a display unit, a communication unit, and a contents storage unit.

[0225] In any case, the image display device (signage device) 100 is not required to perform high load processing such as processing for analyzing a taken image, and hence the cost can be reduced in the case of a system having a large number of image display devices (signage devices) 100.

[0226] Fig. 30 illustrates a configuration example of the server 70 and the image display device (signage device) 100 in the case where the server 70 and the image display device (signage device) 100 are configured as a network image display system as illustrated in Fig. 29.

[0227] The configuration example illustrated in Fig. 29 is a system configuration example in which the server 70 receives a taken image from each of a plurality of image display devices (signage devices) 100, analyses the received image, determines whether a user state that satisfies a contents switching condition is detected, and transmits contents to be switched in accordance with a determination result individually to each of the image display devices 100.　The server 70 holds contents.

[0228] In such a system configuration, as illustrated in Fig. 30, the image display device (signage device) 100 has an imaging unit 221, a communication unit 222, and a display unit 223. The image display device (signage device) 100 photographs an image taken by the imaging unit 221, that is, an image indicating a situation of a passerby (user) in front of the image display device (signage device) 100, and transmits the image to the server 70 through the communication unit 222.

[0229] On the other hand, the server 70 has a data processing unit 201, a communication unit 202, a user information storage unit 203, and a contents storage unit 204. Fig. 31 illustrates a detailed configuration example of the data processing unit 201 in the server 70. As illustrated in Fig. 31, the data processing unit 201 in the server 70 has a taken image analysis unit 251, an output image control unit 252, a contents acquisition unit 253, and an interpolated image generation unit 254.

[0230] The taken image analysis unit 251 executes processing for analyzing data on an image taken by the imaging unit 221 in each image display device (signage device) 100, which has been received by the communication unit 202. The taken image analysis unit 251 executes processing for analyzing an image where a situation of a passerby in front of the display unit 223 in each image display device 100 is included.　Specifically, the taken image analysis unit 251 detects a user such as a passerby from the taken image, analyzes various statuses (states) of the user and statuses (states) of a smartphone owned by the user, and stores the analyzed information in the user information storage unit 203.

[0231] Information stored in the user information storage unit 203 is the same as the information described above with reference to Fig. 18. However, the information is stored in the user information storage unit 203 while being set so as to be divided in units of the image display device (signage device) 100.

[0232] The output image control unit 252 executes, on the basis of information analyzed by the taken image analysis unit 251, processing for determining an image to be displayed on the display unit 223 in each image display device (signage device) 100.

[0233] For example, when it is detected that a user in front of an image display device (signage device) 100 is photographing an image of a performer with a smartphone, the output image control unit 252 executes processing for switching the image of the performer displayed on the image display device (signage device) 100 from a still image (normal image) to a moving image (special image). Specifically, the output image control unit 252 acquires contents to be transmitted to the image display device (signage device) 100 from the contents storage unit 204 through the contents acquisition unit 253, and transmits the acquired contents to the image display device (signage device) 100 through the communication unit 202.

[0234] Note that the data processing unit 201 in the server 70 executes image analysis processing, contents selection processing, contents transmission processing independently and concurrently for each image display devices (signage device) 100.

[0235] The contents acquisition unit 253 executes processing of acquiring contents stored in the contents storage unit 204, for example, still images and moving images of each performer. Note that an image to be displayed on the display unit 223 in each image display device (signage device) 100 is selected by the output image control unit 252, and the contents acquisition unit 253 acquires display contents for each image display device (signage device) 100 determined by the output image control unit 252 from the contents storage unit 204, and provides the acquired contents to the output image control unit 252. The output image control unit 252 transmits these contents to each image display device (signage device) 100.

[0236] Note that data stored in the contents storage unit 105 is the same as the data described above with reference to Fig. 20, and a large number of still images and moving images for each performer are stored.

[0237] The interpolated image generation unit 254 executes processing for generating an interpolated image in order to implement smooth transition in the switching of moving image contents described above with reference to Fig. 16 and Fig. 17. Note that this processing is also executed individually for each image display device (signage device) 100.

[0238] In the network systems described above with reference to Fig. 29 to Fig. 31, the server 70 receives a taken image from each of the plurality of image display devices (signage devices) 100, and analyzes the received image to execute the control of switching the contents of each image display device (signage device) 100.

[0239] As a result, each image display device (signage device) 100 is not required to perform high load processing, such as processing for analyzing a taken image, and hence the cost can be reduced in the case of a system having a large number of image display devices (signage device) 100.

[0240] (6. Hardware configuration examples of image display device, etc.) Next, hardware configuration examples of the image display device (signage device) 100 and the server 70 are described.

[0241] Fig. 32 is a diagram illustrating an example of hardware configurations of the image display device (signage device) 100 and the server 70 in the present disclosure. The hardware configuration illustrated in Fig. 32 is described below.

[0242] A CPU (Central Processing Unit) 301 functions as a control unit and a data processing unit configured to execute various kinds of processing in accordance with programs stored in a ROM (Read Only Memory) 302 or a storage unit 308.　For example, the CPU 301 executes processing in accordance with the sequence described above in the example.　In a RAM (Random Access Memory) 303, programs to be executed by the CPU 301 and data are stored.　The CPU 301, the ROM 302, and the RAM 303 are connected to one another by a bus 304.

[0243] The CPU 301 is connected to an input / output interface 305 through the bus 304.　To the input / output interface 305, an input unit 306 configured by various kinds of switches, a keyboard, a mouse, a microphone, and a sensor and an output unit 307 configured by a display and a speaker are connected.　The CPU 301 executes various kinds of processing in accordance with instructions input from the input unit 306, and outputs processing results to the output unit 307, for example.

[0244] The storage unit 308 connected to the input / output interface 305 is configured by, for example, a hard disk, and stores therein programs to be executed by the CPU 301 and various kinds of data.　A communication unit 309 functions as a transmission / reception unit for Wi-Fi communication, Bluetooth (registered trademark) (BT) communication, and other data communications through networks such as the Internet and a local area network, and communicates with external devices.

[0245] A drive 310 connected to the input / output interface 305 drives a removable media 311 such as a magnetic disk, an optical disc, a magneto-optical disk, or a semiconductor memory such as a memory card, and executes recording and reading of data.

[0246] (7. Summary of configuration in the present disclosure) The examples of the present disclosure have been described above in detail with reference to particular examples.　However, it is apparent that the examples can be modified and substituted by a person skilled in the art within the range not departing from the gist of the present disclosure.　That is, the present invention is disclosed in an illustrative manner, and should not be interpreted in a limited manner.　To determine the gist of the present disclosure, the claims should be taken into consideration.

[0247] Furthermore, a series of processing described herein can be executed by hardware, software, or a combination thereof.　When the processing is executed by software, a program having a processing sequence recorded therein is installed in a memory in a computer incorporated in dedicated hardware and executed, or a program is installed in a general-purpose computer that can execute various kinds of processing and executed.　For example, the program can be recorded in a recording medium in advance.　In addition to installing the program into a computer from a recording medium, the program may be received through a network such as a LAN (Local Area Network) or the Internet, and installed in a recording medium such as incorporated hardware.

[0248] Note that various kinds of processing described herein are not only executed in a time series in accordance with the description but also executed concurrently or individually depending on the capacity of processing of a device that executes processing or as necessary.　Furthermore, the system as used herein is a logical set configuration of a plurality of devices, and devices in each configuration are not necessarily required to be included in the same casing.

[0249] As described above, the configuration in one example of the present disclosure implements a configuration in which a state of a user in front of a display unit is analyzed, and processing for switching display contents depending on an analysis result, for example, processing for switching the display contents from a still image to a moving image is executed. Specifically, for example, a data processing unit configured to execute switching control of display contents on a display unit executes processing for analyzing an image taken by an imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition, for example, whether the user is photographing display contents, and when it is determined that the user is in the state corresponding to the contents switching condition, switches the display contents on the display unit.　For example, the data processing unit executes processing for switching a still image to a moving image. This configuration implements a configuration in which processing for analyzing the state of a user in front of a display unit and switching display contents depending on an analysis result, for example, processing for switching the display contents from a still image to a moving image is executed.

[0250] According to present disclosure various aspects are presented such as, (Aspect 1), which is an image processing system comprising: 　　circuitry configured to 　　　　identify an indication of an action of at least one person within viewing range of electronic signage displayed on an electronic signage device; and control a change of content of the electronic signage based on the indication. (Aspect 2) 　　The image processing system of aspect 1, wherein the circuitry is further configured to receive image data of an image within a field of view of a camera positioned to have the field of view that captures an area in front of the electronic signage device within the viewing range. (Aspect 3) The image processing device of aspect 2, wherein the circuitry is further configured to analyze the at least one person and determine the action of the at least one person. (Aspect 4) The image processing system of aspect 3, wherein the circuitry is further configured to change the content from a still image to a moving image based on the indication. (Aspect 5) The image processing system of aspect 1, wherein the indication is an indication of the at least one person photographing the content of the electronic signage. (Aspect 6) The image processing system of aspect 1, wherein the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, and the circuitry, based on the indication, is configured to change the first image of the first entity from a still image to a moving image, but not change the second image of the second entity from being a still image. (Aspect 7) The image processing system of aspect 6, wherein in response to the circuitry having identified that the action includes the at least one person having moved closer to the second image of the second entity than the first image of the first entity, the circuitry is configured to change the second image of the second entity from a still image to a moving image. (Aspect 8) The image processing system of aspect 7, wherein the content of the electronic signage includes a third image of a third entity, and in response to the circuitry having identified that the action includes the at least one person having moved closer to the third image of the third entity than the second image of the second entity, the circuitry changes the third image of the third entity. (Aspect 9) The image processing system of aspect 1, wherein the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, the at least one person includes a first person position closer to the first image than the second image, and a second person position closer to the second image than the first image, and the circuitry, is configured to change the first image of the first entity from a still image to a moving image based on the position of the first person, and change the second image of the second entity from another still image to another moving image based on the position of the second person. (Aspect 10) The image processing system of aspect 5, the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, the at least one person includes a first person photographing the first image, and a second person photographing the first image, the circuitry, is configured to present the first image as a moving image that has optimal moving image contents so as be photographed by the first person at a first position, and the second person even though the second person is at a different position. (Aspect 11) The image processing system of aspect 5, the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, the at least one person includes a first person photographing the electronic signage with a first image capturing device, and a second person photographing the electronic signage with a second image capturing device, the circuitry, is configured to control a change of the content displayed on the electronic signage device based on the indication, the indication being an indication of an orientation of at least one of the first image capturing device and the second image capturing device. (Aspect 12) The image processing system of aspect 11, wherein the circuitry is further configured to estimate a favorite displayed entity of the at least one person based on at least one of a detected line of sight of the at least one person, and a length of gaze time of the at least one person. (Aspect 13) The image processing system of aspect 11, wherein the circuitry, is configured to present the content displayed of the electronic signage device as a moving image that has optimal moving image contents for the first image capturing device and the second image capturing device. (Aspect 14) The image processing system of aspect 1, wherein the indication of the action of at least one person includes a status of the at least one person, the status including at least one of clothing worn by the at least one person or an item brought by the at least one person, and the circuitry is configured to change the content of the electronic signage to content associated with at least one of the clothing or the item. (Aspect 15) The image processing system of aspect 1, wherein the circuitry is configured generate interpolated image content as the electronic signage to obtain a smooth transition of moving image content before and after content to be displayed is switched from first moving image contents to second moving image contents. (Aspect 16) The image processing system of aspect 1, wherein the circuitry in connected to a network that includes a plurality of electronic signage devices that display related content to different people at different locations. (Aspect 17) The image processing system of aspect 1, further comprising: the display that displays the electronic signage. (Aspect 18) The image processing system of aspect 2, further comprising: the camera that is positioned with the field of view positioned to capture the area in front of the electronic signage device within the viewing range. (Aspect 19) An image processing method comprising: identifying with circuitry an indication of an action of at least one person within viewing range of electronic signage; and controlling with the circuitry a change of content of the electronic signage based on the indication. (Aspect 20) A non-transitory computer readable storage device having stored therein computer readable code that when executed by a programmable processor, causes the programmable processor to perform operations comprising: identifying an indication of an action of at least one person within viewing range of electronic signage; and controlling with the circuitry a change of content of the electronic signage based on the indication. (Aspect 21) An image display device, comprising: a display unit; an imaging unit configured to take an image including at least a neighborhood of the display unit; and a data processing unit configured to execute control of switching display contents on the display unit, wherein the data processing unit is configured to: execute processing for analyzing the image taken by the imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and switch the display contents on the display unit when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition. (Aspect 22) The image display device according to claim 1, wherein the data processing unit is configured to switch, when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition, the display contents on the display unit from still image contents to moving image contents. (Aspect 23) The image display device according to claim 1, wherein: the user state corresponding to the prescribed contents switching condition is a state in which photographing of the display contents on the display unit is executed by the detected user; and the data processing unit is configured to switch the display contents on the display unit when it is determined that the detected user is photographing the display contents on the display unit. (Aspect 24) The image display device according to claim 3, wherein the data processing unit is configured to: analyze a position of a user who is photographing the display contents on the display unit; and switch the display contents on the display unit to contents optimal for the position of the user. (Aspect 25) The image display device according to claim 4, wherein the data processing unit is configured to switch the display contents on the display unit to moving image contents optimal for the position of the user. (Aspect 26) The image display device according to claim 4, wherein the data processing unit is configured to switch, when a plurality of users are photographing the display contents on the display unit, the display contents on the display unit to contents optimal for positions of the plurality of users who are photographing the display contents. (Aspect 27) The image display device according to claim 3, wherein: the display unit is configured to display individual image contents in a plurality of display regions; and the data processing unit is configured to: specify a display region where the detected user is photographing; and switch display contents that are being displayed in the specified display region. (Aspect 28) The image display device according to claim 7, wherein the data processing unit is configured to: specify a display region where the detected user is photographing; and switch display contents that are being displayed in the specified display region from still image contents to moving image contents. (Aspect 29) The image display device according to claim 7, wherein: the display unit is configured to display images of individual performers in a plurality of display regions; and the data processing unit is configured to: specify a display region where the detected user is photographing; and switch an image of a performer that is being displayed in the specified display region from still image contents to moving image contents. (Aspect 30) The image display device according to claim 9, wherein the data processing unit is configured to: analyze at least one of a position of a user who is photographing display contents of the display unit, a position of an image photographing device, and an angle of the image photographing device, and specify a display region where the user is photographing; and switch an image of a performer that is being displayed in the specified display region from still image contents to moving image contents. (Aspect 31) The image display device according to claim 9, wherein: the data processing unit is configured to execute processing for: analyzing at least one of a position of the detected user who is photographing display contents on the display unit, a position of an image photographing device, and an angle of the image photographing device; and estimating a favorite performer who is a performer liked by the detected user. (Aspect 32) The image display device according to claim 1, wherein: the user state corresponding to the prescribed contents switching condition is a state in which belongings or clothes of the detected user match belongings or clothes prescribed in advance; and the data processing unit is configured to switch the display contents on the display unit when it is determined that the belongings or the clothes of the detected user match the belongings or the clothes prescribed in advance. (Aspect 33) The image display device according to claim 1, wherein: a user state corresponding to the prescribed contents switching condition is a state in which there are a plurality of the detected users and a combination of the plurality of the users matches a user combination prescribed in advance; and the data processing unit is configured to switch the display contents on the display unit when there are a plurality of the detected users and a combination of the plurality of the users matches the user combination prescribed in advance. (Aspect 34) The image display device according to claim 13, wherein the user combination prescribed in advance is a parent and a child or a pair of male and female persons. (Aspect 35) The image display device according to claim 1, wherein the data processing unit is configured to, when switching moving image contents that are being displayed on the display unit to different moving image contents, display an interpolated image, which is for making moving image contents before and after switching be a moving image with smooth transition, between the moving image contents before and after the switching. (Aspect 36) An information processing device, comprising: a communication unit configured to execute communication with an image display device; and a data processing unit configured to execute control of switching display contents on the display device, wherein: the communication unit is configured to receive, from the image display device, a taken image including at least a neighborhood of the image display device; and the data processing unit is configured to: execute processing for analyzing the taken image to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition, transmit at least one of a switching control command for the display contents and switching display contents to the image display device through the communication unit. (Aspect 37) An image display control method to be executed in an image display device, the image display device comprising: a display unit; an imaging unit configured to take an image including at least a neighborhood of the display unit; and a data processing unit configured to execute control of switching display contents of the display unit, the image display control method comprising: executing, by the data processing unit, processing for analyzing the image taken by the imaging unit to determine whether a user detected from the taken image is in a state corresponding to a prescribed contents switching condition; and switching, by the data processing unit, the display contents on the display unit when it is determined that the detected user is in the state corresponding to the prescribed contents switching condition.

[0251] 10 Image display device (signage device) 11 Display unit 12 Imaging unit (camera) 20 Performer 30 User (passerby) 40 Smartphone 70 Server 100 Image display device 101 Imaging unit 102 Data processing unit 103 Display unit 104 User information storage unit 105 Contents storage unit 106 Communication unit 121 Taken image analysis unit 122 Output image control unit 123 Contents acquisition unit 124 Interpolated image generation unit 201 Data processing unit 202 Communication unit 203 Contents storage unit 204 User information storage unit 221 Imaging unit 222 Communication unit 223 Display unit 251 Taken image analysis unit 252 Output image control unit 253 Contents acquisition unit 254 Interpolated image generation unit 301 CPU 302 ROM 303 RAM 304 Bus 305 Input / output interface 306 Input unit 307 Output unit 308 Storage unit 309 Communication unit 310 Drive 311 Removable media

Claims

1. An image processing system comprising: 　　circuitry configured to 　　　　identify an indication of an action of at least one person within viewing range of electronic signage displayed on an electronic signage device; and 　　　 control a change of content of the electronic signage based on the indication.

2. The image processing system of claim 1, wherein the circuitry is further configured to receive image data of an image within a field of view of a camera positioned to have the field of view that captures an area in front of the electronic signage device within the viewing range.

3. The image processing device of claim 2, wherein the circuitry is further configured to analyze the at least one person and determine the action of the at least one person.

4. The image processing system of claim 3, wherein the circuitry is further configured to change the content from a still image to a moving image based on the indication.

5. The image processing system of claim 1, wherein the indication is an indication of the at least one person photographing the content of the electronic signage.

6. The image processing system of claim 1, wherein 　　　 the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, and 　　　 the circuitry, based on the indication, is configured to change the first image of the first entity from a still image to a moving image, but not change the second image of the second entity from being a still image.

7. The image processing system of claim 6, wherein in response to the circuitry having identified that the action includes the at least one person having moved closer to the second image of the second entity than the first image of the first entity, the circuitry is configured to change the second image of the second entity from a still image to a moving image.

8. The image processing system of claim 7, wherein the content of the electronic signage includes a third image of a third entity, and 　　　 in response to the circuitry having identified that the action includes the at least one person having moved closer to the third image of the third entity than the second image of the second entity, the circuitry changes the third image of the third entity.

9. The image processing system of claim 1, wherein 　　　 the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, 　　　 the at least one person includes a first person position closer to the first image than the second image, and a second person position closer to the second image than the first image, and 　　　 the circuitry, is configured to change the first image of the first entity from a still image to a moving image based on the position of the first person, and change the second image of the second entity from another still image to another moving image based on the position of the second person.

10. The image processing system of claim 5, 　　　 the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, 　　　 the at least one person includes a first person photographing the first image, and a second person photographing the first image, 　　　 the circuitry, is configured to present the first image as a moving image that has optimal moving image contents so as be photographed by the first person at a first position, and the second person even though the second person is at a different position.

11. The image processing system of claim 5, 　　　 the content of the electronic signage includes a first image of a first entity, and a second image of a second entity, 　　　 the at least one person includes a first person photographing the electronic signage with a first image capturing device, and a second person photographing the electronic signage with a second image capturing device, 　　　 the circuitry, is configured to control a change of the content displayed on the electronic signage device based on the indication, the indication being an indication of an orientation of at least one of the first image capturing device and the second image capturing device.

12. The image processing system of claim 11, wherein the circuitry is further configured to estimate a favorite displayed entity of the at least one person based on at least one of a detected line of sight of the at least one person, and a length of gaze time of the at least one person.

13. The image processing system of claim 11, wherein 　　　 the circuitry, is configured to present the content displayed of the electronic signage device as a moving image that has optimal moving image contents for the first image capturing device and the second image capturing device.

14. The image processing system of claim 1, wherein 　　　 the indication of the action of at least one person includes a status of the at least one person, the status including at least one of clothing worn by the at least one person or an item brought by the at least one person, and 　　　 the circuitry is configured to change the content of the electronic signage to content associated with at least one of the clothing or the item.

15. The image processing system of claim 1, wherein the circuitry is configured generate interpolated image content as the electronic signage to obtain a smooth transition of moving image content before and after content to be displayed is switched from first moving image contents to second moving image contents.

16. The image processing system of claim 1, wherein 　　　 the circuitry in connected to a network that includes a plurality of electronic signage devices that display related content to different people at different locations.

17. The image processing system of claim 1, further comprising: 　　　 the display that displays the electronic signage.

18. The image processing system of claim 2, further comprising: 　　　 the camera that is positioned with the field of view positioned to capture the area in front of the electronic signage device within the viewing range.

19. An image processing method comprising: 　　　 identifying with circuitry an indication of an action of at least one person within viewing range of electronic signage; and 　　　 controlling with the circuitry a change of content of the electronic signage based on the indication.

20. A non-transitory computer readable storage device having stored therein computer readable code that when executed by a programmable processor, causes the programmable processor to perform operations comprising: 　　　 identifying an indication of an action of at least one person within viewing range of electronic signage; and 　　　 controlling with the circuitry a change of content of the electronic signage based on the indication.

Images