Information processing device, information processing method, and program

Abstract

An information processing device comprising a control unit for displaying a virtual object on three-dimensional coordinates associated with a real space and causing a prescribed operation to be executed by the virtual object in accordance with a prescribed sound that is reproduced from a real sound source positioned in the real space, the control unit performing a delay amount setting process for progressively increasing the amount of delay of an operation start timing at which the prescribed operation is executed by the virtual object relative to a reproduction timing at which the real sound source reproduced the prescribed sound, in correspondence to an increase in the distance between the position of the real sound source in the real space and the position on three-dimensional coordinates of the virtual object. FIG. 4

Description

[Technical field]

[0001] The present disclosure relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There have been proposals for controlling the playback timing of content (see, for example, Patent Documents 1 and 2 below). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2018-183483 A [Patent Document 2] JP 2015-073182 A Summary of the Invention [Problem to be solved by the invention]

[0004] Incidentally, in a live concert venue, the longer the distance to the stage, the later the sound emitted on the stage reaches the user. In recent years, a live performance format has been proposed in which sound is reproduced from speakers on a stage while a video of a virtual singer singing and dancing on the stage, rather than a real singer, is reproduced. Even in such a live performance format, realism is required regarding the delay of the sound reproduced from the speakers, but the technology described in the above-mentioned patent document has difficulty in appropriately reproducing the realism regarding the delay of the sound.

[0005] An object of the present disclosure is to provide an information processing device, an information processing method, and a program that can appropriately reproduce the reality regarding the delay of sound reproduced from a real sound source. [Means for solving the problem]

[0006] The present disclosure relates to, for example, a control unit that displays a virtual object on three-dimensional coordinates associated with a real space and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to set a delay amount corresponding to a timing at which a predetermined sound reaches a position of the user in a real space; The amount of delay is A delay in the timing at which a virtual object starts to execute a predetermined action relative to the timing at which a real sound source reproduces a predetermined sound. and The control unit may adjust the distance between the real sound source in the real space and the virtual object in the three-dimensional coordinate system to be larger. Slow Increase the extension amount R It is an information processing device.

[0007] The present disclosure relates to, for example, a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to set a delay amount corresponding to a timing at which a predetermined sound reaches a position of the user in a real space; The amount of delay is A delay in the timing at which a virtual object starts to execute a predetermined action relative to the timing at which a real sound source reproduces a predetermined sound. and The control unit may adjust the distance between the real sound source in the real space and the virtual object in the three-dimensional coordinate system to be larger. Slow Increase the extension amount R It is an information processing method.

[0008] The present disclosure relates to, for example, a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to set a delay amount corresponding to a timing at which a predetermined sound reaches a position of the user in a real space; The amount of delay is A delay in the timing at which a virtual object starts to execute a predetermined action relative to the timing at which a real sound source reproduces a predetermined sound. and The control unit may adjust the distance between the real sound source in the real space and the virtual object in the three-dimensional coordinate system to be larger. Slow Increase the extension amount R It is a program that causes a computer to execute the information processing method. [Brief description of the drawings]

[0009] [Figure 1] FIG. 1 is a diagram illustrating a playback system according to an embodiment. [Diagram 2] FIG. 2 is a diagram to which reference will be made in describing the issues to be considered in one embodiment. [Diagram 3] FIG. 3 is a diagram for explaining an overview of an embodiment. [Figure 4] 4A to 4D are diagrams for explaining an overview of one embodiment. [Diagram 5] FIG. 5 is a diagram for explaining a specific example of a playback system. [Figure 6] FIG. 6 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 7] FIG. 7 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 8] FIG. 8 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 9] FIG. 9 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 10] FIG. 10 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 11] FIG. 11 is a diagram for explaining a specific example of the process performed in the playback system according to an embodiment. [Figure 12] FIG. 12 is a table showing that the processing performed in the playback system according to one embodiment can achieve realism regarding sound delay. [Figure 13] FIG. 13 is a diagram that is referred to when explaining an example of an arithmetic expression for calculating the amount of delay in the operation start timing in one embodiment. [Figure 14] FIG. 14 is a diagram showing an example of the configuration of a system including a plurality of devices in one embodiment. [Figure 15] FIG. 15 is a diagram illustrating an example of the configuration of each of a server, a smart display, and a speaker according to an embodiment. [Figure 16] FIG. 16 is a flowchart that will be referred to when describing the flow of processing performed in one embodiment. [Figure 17] FIG. 17 is a flowchart that will be referred to when explaining the flow of processing performed in one embodiment. [Figure 18] FIG. 18 is a flowchart that will be referred to when explaining the flow of processing performed in one embodiment. [Figure 19] FIG. 19 is a diagram to be referred to when explaining the modified example. [Figure 20] FIG. 20 is a diagram to be referred to when explaining the modified example. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The description will be made in the following order. <One embodiment> <Modification> The embodiments and the like described below are preferred specific examples of the present disclosure, and the contents of the present disclosure are not limited to these embodiments and the like.

[0011] <One embodiment> [Processing performed in the embodiment] First, the processing performed in this embodiment will be described. In this embodiment, a relatively large-scale playback system such as that in a live music venue is assumed.

[0012] (Outline of the playback system) Fig. 1 is a diagram showing a schematic diagram of a playback system (playback system 1) according to this embodiment. The playback system 1 includes, for example, a stage 2, and speakers 3, which are an example of real sound sources, are arranged on or around the stage 2. Actual sounds (music, sound effects, announcement sounds, etc.) are played back from the speakers 3. Fig. 1 shows an example in which the speakers 3 are arranged on both the left and right sides of the stage 2, but the number and arrangement of the speakers 3 can be changed as appropriate.

[0013] Users, who are spectators, are present facing the stage 2. The users watch images in which virtual objects imitating singers and dancers are displayed while listening to sounds reproduced from the speaker 3. The virtual objects may be real people, not just virtual objects such as artificially created singers, and the images may be images in which such real people are displayed. The images are displayed on a smart display carried or worn by the user. The smart display is capable of drawing virtual information in real space, and specifically, examples of such smart displays include the display of a smartphone owned by the user, a glasses-type display (also called AR (Augmented Reality) glasses, etc.), and a HUD (Head Up Display). Each user can view the same video content by, for example, installing the same application on a device having a smart display and reproducing the content. In this way, the reproduction system 1 according to this embodiment is a system in which live reproduction is performed by fusing a real stage with virtual objects.

[0014] (Issues to consider) Next, a description will be given of problems to be considered in the above-mentioned playback system 1. When trying to reproduce the reality of the delay between the arrival of a played sound source from a real sound source while displaying a video on a smart display, the appropriate timing for a virtual object to perform a predetermined action (motion) differs depending on the distance to the stage 2 of the user and the position of the virtual object.

[0015] FIG. 2 is a diagram for explaining an example of reality corresponding to a user position and a virtual object position. In the table shown in FIG. 2, the position of a virtual object as a virtual object is defined in the vertical direction. Specifically, a pattern in which the virtual object is located on a stage and a pattern in which the virtual object is located in front of the user are defined. In addition, in the table shown in FIG. 2, the relationship between the user position and the stage position is defined in the horizontal direction. Specifically, a pattern in which the user position is close to the stage and a pattern in which the user position is far from the stage are defined.

[0016] In the case of a pattern where the virtual object is on the stage and the user is close to the stage (hereinafter referred to as pattern P1), the timing of the image and sound matches, which gives the reality. In the case of a pattern where the virtual object is in front of the user and the user is close to the stage (hereinafter referred to as pattern P2), the timing of the image and sound matches, which gives the reality. In light of patterns P1 and P2, the timing of the image and sound always matches, which gives the reality.

[0017] In the case where the virtual object is located on the stage and the user is far from the stage (hereinafter referred to as pattern P3), the sound arrives with a delay from the video, which is realistic. In the case where the virtual object is located in front of the user and the user is far from the stage (hereinafter referred to as pattern P4), the timing of the video and the sound matches, which is realistic. Considering patterns P3 and P4, the display timing differs depending on the position of the virtual object, which is realistic.

[0018] In addition, considering patterns P1 and P3, the virtual object display timing is the same regardless of the user's distance from the stage, which is realistic. In addition, considering patterns P2 and P4, the virtual object display timing is different depending on the user's distance from the stage, which is realistic. The technology described in the above-mentioned patent documents has a problem in that it is not possible to reproduce these realities.

[0019] (Overview of one embodiment) Next, an overview of this embodiment will be described with reference to Fig. 3 and Fig. 4. Fig. 3 is a diagram showing the above-mentioned four patterns (patterns P1, P2, P3, and P4) in a schematic manner. In Fig. 3, a user close to the stage 2 is a user UA, and a user far from the stage 2 is a user UB. Fig. 3 also shows a virtual object C displayed on the smart display according to each pattern.

[0020] 4A to 4D are diagrams for explaining an outline of the processing performed according to each pattern. Fig. 4A shows the processing performed in the case of pattern P1. In the case of pattern P1, processing is performed to bring the timing for causing the virtual object to execute a predetermined action (hereinafter, appropriately referred to as the action start timing) closer to the playback timing of the sound reproduced from the real sound source.

[0021] 4B shows the process performed in the case of pattern P2. In the case of pattern P2, a process is performed in which the timing at which the virtual object starts to execute a predetermined action is brought closer to the timing at which the sound is reproduced from the real sound source.

[0022] 4C shows the process performed in the case of pattern P3. In the case of pattern P2, the process is performed to bring the timing at which the virtual object starts to execute a predetermined action closer to the timing at which the sound is reproduced from the real sound source.

[0023] 4D shows the process performed in the case of pattern P4. In the case of pattern P4, a process is performed in which the timing at which the virtual object starts to execute a predetermined action is delayed from the timing at which sound is reproduced from the real sound source.

[0024] (Example of processing) The outline of the above-described process will be described with a specific example. FIG. 5 shows a specific example of the reproduction system 1. As described above, the reproduction system 1 has a stage 2 and a speaker 3. A position PO1 on the stage 2 indicates a position of the virtual object when the position is on the stage 2. The position of the virtual object here is a position of the virtual object on a three-dimensional coordinate system associated with the real space, and is a position obtained by performing a calculation described later. In this example, the positions of the stage 2 and the speaker 3 are the same position. Therefore, when the virtual object is at the position PO1 on the stage 2, the distance between the virtual object and the speaker 3 is set to 0.

[0025] The position PO2 in the reproduction system 1 indicates the position of the user UA who is close to the stage 2. The position of the user UA is equivalent to the position of a smart display carried or worn by the user UA, for example. The distance between the user UA and the stage 2 is, for example, 10 m. The user UA listens to the sound of the sound source reproduced from the speaker 3, and visually recognizes the virtual object through the eyeglass-type AR glasses 11A, which are an example of a smart display. Note that when the virtual object is displayed in front of the user UA's eyes, that is, when the AR glasses 11A are displayed in such a manner that the virtual object is displayed in front of the user UA's eyes, the position of the virtual object substantially coincides with the position PO2.

[0026] Moreover, the position PO3 in the reproduction system 1 indicates the position of the user UB, who is far from the stage 2. The position of the user UB is equivalent to, for example, the position of a smart display carried or worn by the user UB. The distance between the user UB and the stage 2 is, for example, 200 m. The user UB listens to the sound of the sound source reproduced from the speaker 3, and visually recognizes the virtual object through the eyeglass-type AR glasses 11B, which are an example of a smart display. Note that when the virtual object comes in front of the user UB's eyes, that is, when the AR glasses 11B are displayed in such a manner that the virtual object comes in front of the user UB's eyes, the position of the virtual object approximately coincides with the position PO3.

[0027] In the reproduction system 1 in the real space described above, the distance between the speaker 3 and the position of the virtual object is assumed to be D (m). In this case, after a sound corresponding to a predetermined action is reproduced from the speaker 3, D*1000 / 340(msec)...(Formula 1) The action start timing at which the virtual object performs the predetermined action is set to a timing delayed from the predetermined timing by the amount of the delay. (However, the speed of sound is assumed to be 340 m / sec.) Therefore, the greater the distance between the position of the speaker 3 in real space and the position of the virtual object in the three-dimensional coordinate system, the greater the delay amount set for the action start timing at which the virtual object executes a predetermined action relative to the playback timing at which the speaker 3 plays a predetermined sound. From the set action start timing, an image of the virtual object performing the predetermined action is rendered on the AR glasses.

[0028] The amount of delay from when a sound is reproduced from the speaker 3 until when the virtual object starts to perform a predetermined action is specifically set as follows. (1) For user UA Time it takes for the sound played from Speaker 3 to reach the user UA: Approximately 30 msec Delay when virtual object is on stage 2: 0msec (Because the distance between speaker 3 and the virtual object is 0 m, D = 0 in (Equation 1)) Delay in the start of an action when the virtual object is in front of the user UA: Approximately 30 msec (Because the distance between speaker 3 and the virtual object is 10 m, D in (Equation 1) is 10) (2) For user UB Time it takes for the sound played from speaker 3 to reach user UB: Approximately 600 msec Delay when virtual object is on stage 2: 0msec (Because the distance between speaker 3 and the virtual object is 0 m, D = 0 in (Equation 1)) Delay in starting the action when the virtual object is in front of the user UB: Approximately 600 msec (Because the distance between speaker 3 and the virtual object is 200 m, D in (Equation 1) is 200) Taking the above into consideration, a specific explanation will be given regarding the operation start timing, etc.

[0029] FIG. 6 is a timing chart in the case where the user UA is located near the speaker 3 and the virtual object C is located on the stage 2. As an example of the timing of the sound reproduced from the speaker 3 as a real sound source, timings T1, T2, and T3 are specified in chronological order. When sound delay and the like are not taken into consideration, the virtual object C performs a predetermined action at timing T1. When the timing of the sound that causes the predetermined action is timing t (t=T1 in this example), the sound reproduced from the speaker 3 reaches the user UA after (t+30) msec. Note that the action performed by the virtual object C is an example action including actions M1, M2, and M3. Also, in FIG. 6 and other figures, marks of the same musical note indicate the same sound.

[0030] To facilitate understanding of the process performed in this embodiment (hereinafter also referred to as this proposal), reference examples 1 and 2 will be described in addition to this proposal. Reference example 1 is an example in which the operation start timing is always aligned with the playback timing of a specific sound. Reference example 2 is an example in which the operation start timing is always aligned with the arrival timing of a specific sound.

[0031] 6, in reference example 1, at timing t when sound is reproduced from speaker 3, virtual object C starts action M1, and then performs actions M2 and M3. That is, the action start timing is set to timing t. In this case, although there is a lag of 30 msec between the timing when the sound arrives and the timing when the action starts, the lag of 30 msec is small enough that it does not give the user an uncomfortable auditory sensation, and therefore, according to reference example 1, reality regarding sound delay is realized.

[0032] In addition, in reference example 2, the virtual object C starts the action M1 at the timing (t+30 msec) when the sound reproduced from the speaker 3 reaches the user UA, and then performs the actions M2 and M3. That is, the action start timing is set to the timing (t+30 msec). In this case, the sound arrival timing and the action start timing coincide. Since the user UA is located close to the stage 2, according to reference example 2, reality regarding the delay in the sound is realized.

[0033] Furthermore, as described above, in this proposal, the delay amount of the operation start timing is set to 0. Therefore, similar to the first reference example, this proposal achieves reality regarding sound delay.

[0034] 7 is a timing chart in the case where the user is a user UB who is located far from the speaker 3, and the position of the virtual object C is on the stage 2. In this case, the sound reproduced from the speaker 3 reaches the user UB after (t+600 msec).

[0035] In reference example 1, at timing t when sound is reproduced from the speaker 3, the virtual object C starts an action M1, and then performs actions M2 and M3. That is, the action start timing is set to timing t. Also, the sound reproduced from the speaker 3 reaches the user UB after (t+600 msec). According to reference example 1, reality regarding sound delay is realized.

[0036] In reference example 2, at the timing (t+600 msec) when the sound reproduced from the speaker 3 arrives, the virtual object C starts the action M1, and then performs the actions M2 and M3. In reality, the sound recognized aurally should arrive with a delay relative to the image recognized visually, but in reference example 2, the timing when the sound reaches the user UB coincides with the timing when the action starts. Therefore, according to reference example 2, reality regarding the delay in the sound cannot be realized.

[0037] According to this proposal, since the virtual object is on stage 2, the delay amount is set to 0, that is, the action start timing is set to timing t. Also, the sound reproduced from speaker 3 reaches user UB after (t+600 msec). Therefore, according to this proposal, reality regarding sound delay has been achieved.

[0038] FIG. 8 is a timing chart in the case where the user UA is located near the speaker 3 and the position of the virtual object C is in front of the user. Since the virtual object C is an object displayed on the smart display, its position can be freely changed in the content, unlike an actual live performance. For example, the virtual object C may be displayed in such a manner that it flies from the stage 2 and comes right in front of the user UA. In this case, as described above, since the virtual object C is located in front of the user UA, the timing at which the action starts and the timing at which the sound is heard by the user UA match, which provides reality regarding the sound delay.

[0039] In reference example 1, at timing t when sound is reproduced from the speaker 3, the virtual object C starts an action M11, and then performs actions M12 and M13. That is, the action start timing is set to timing t. In this case, the sound arrival timing and the action start timing are approximately the same to an extent that no discomfort is caused to the auditory sense (with an error of 30 msec). Therefore, according to reference example 1, reality regarding sound delay is realized.

[0040] Also, in reference example 2, the virtual object C starts the action M11 at the timing (t+30 msec) when the sound played from the speaker 3 reaches the user UA, and then performs the actions M12 and M13. That is, the action start timing is set to the timing (t+30 msec). In this case, the sound arrival timing and the action start timing match. From the above, according to reference example 2, reality regarding the sound delay is realized.

[0041] In addition, in this proposal, as described above, the operation start timing is set to (t+30 msec). Therefore, similar to the second reference example, this proposal achieves reality regarding sound delay.

[0042] 9 is a timing chart in the case where the user is a user UB who is located far from the speaker 3, and the position of the virtual object C is in front of the user. In this case, the sound reproduced from the speaker 3 reaches the user UB after (t+600 msec). As described above, since the virtual object C is located in front of the user, the timing when the action starts and the timing when the sound is heard by the user UB match, which provides reality regarding the delay in the sound.

[0043] In reference example 1, at timing t when sound is reproduced from speaker 3, virtual object C starts action M11, and subsequently performs actions M12 and M13. That is, the action start timing is set to timing t. On the other hand, the sound reaches user UB after (t+600 msec), and user UB recognizes the sound. Therefore, according to reference example 1, reality regarding sound delay cannot be realized.

[0044] In addition, in Reference Example 2, the virtual object C starts the action M11 at the timing (t+600 msec) when the sound reproduced from the speaker 3 reaches the user UB, and then performs the actions M12 and M13. Since the timing when the sound is heard by the user UB and the timing when the actions start coincide with each other, Reference Example 2 realizes a reality regarding the delay in the sound.

[0045] In addition, in this proposal, as described above, the action start timing is set to (t+600 msec), so the timing at which the user UB hears the sound coincides with the action start timing. Therefore, as in Reference Example 2, this proposal achieves reality regarding sound delay.

[0046] 10 is a timing chart in the case where the user UA is located close to the speaker 3 and the position of the virtual object C is midway between the speaker 3 and the user UA (the distance between the position of the speaker 3 and the position of the virtual object C is 5 m). Since the position of the virtual object C can be regarded as being substantially the same as its position on the stage 2, the fact that the timing at which the sound is heard and the timing of the action are substantially the same provides a sense of reality regarding the sound delay.

[0047] In Reference Example 1, the timing t is the operation start timing. Therefore, reality regarding sound delay is realized. According to Reference Example 2, the timing (t+30 msec) is the operation start timing. As described above, since a lag of 30 msec is considered to have almost no effect on the auditory sense, Reference Example 2 also realizes reality regarding sound delay.

[0048] In this proposal, by substituting D=5 in (Equation 1), the set delay amount is 15 msec. The timing (t+15 msec) is the action start timing. Since the 15 msec delay is considered to have almost no effect on the auditory sense, this proposal also achieves reality regarding sound delay. When the action start timing arrives, an image of the virtual object C performing a series of actions M21, 22, and 23 is rendered on the AR glasses 11A.

[0049] 11 is a timing chart in the case where the user is a user UB located far from the speaker 3, and the position of the virtual object C is midway between the speaker 3 and the user UA (the distance between the position of the speaker 3 and the position of the virtual object C is 100 m, and the distance between the virtual object C and the user UB is 100 m). In this case, since the distance between the virtual object C and the user UB is 100 m, the sound reaches the user UB 300 msec after the virtual object C is played, which provides reality in terms of sound delay.

[0050] In reference example 1, the operation start timing is t. Then, 600 msec after t, the sound reproduced from the speaker 3 reaches the user UB. Therefore, in reference example 1, there is no reality regarding the delay in the sound.

[0051] In reference example 2, the action start timing is (t+600 msec). The action start timing coincides with the timing at which the sound reaches the user UB. Therefore, in reference example 2, there is no reality regarding the delay in the sound.

[0052] In this proposal, by substituting D=100 in (Equation 1), the delay amount is set to 300 msec. The timing (t+300 msec) is the action start timing. Then, the sound reaches the user UB 300 msec after the timing (t+300 msec). Therefore, in this proposal, there is reality regarding the sound delay. When the action start timing arrives, an image of the virtual object C performing a series of actions M21, 22, and 23 is rendered on the AR glasses 11B.

[0053] Fig. 12 is a table summarizing the results of the specific processing described above. In the table, a circle indicates that the reality regarding the sound delay can be realized. Also, in the table, a cross indicates that the reality regarding the sound delay cannot be realized. In comparison with Reference Examples 1 and 2, according to this proposal, the reality regarding the sound delay can be realized in both cases.

[0054] FIG. 13 is a diagram for explaining parameters required for setting the amount of delay in the operation start timing, and an example of a calculation using the parameters. The position (absolute coordinates) of the real sound source (speaker 3) in the three-dimensional coordinate system related to the real space: AC R 3D coordinate position (absolute coordinate) related to the user's real space: AC U Virtual object coordinates relative to the user's position: RC O Conversion coefficient between coordinate distance and actual distance (m): α In this case, as shown in Figure 13, Absolute coordinates of virtual object: AC U +RC O Distance between the real sound source (Speaker 3) and the virtual object: Distance(AC R ,A.C. U +RC O )*α(m) The delay in the start of the operation can be expressed as follows: Distance(ACR ,A.C. U +RC O )*α*1000 / 340(msec)...(Formula 2) It can be expressed as:

[0055] (System configuration example) 14 is a diagram showing an example of the configuration of a system (system 100) configured by multiple devices in one embodiment. The system 100 includes, for example, a server 30, a smart display (for example, the above-mentioned AR glasses 11A and 11B) 40, and a speaker (for example, the above-mentioned speaker 3) 50. Various data and commands can be exchanged between the devices through communication.

[0056] (Example of each device configuration) "Server Configuration Example" With reference to FIG. 15, configuration examples of the server 30 according to the present embodiment, the smart display 40 which is an example of an information processing device, and the speaker 50 will be described.

[0057] The server 30 includes, for example, a server control unit 301, a sound source position coordinates storage unit 302, a progress information storage unit 303, and a server communication unit 304. The server control unit 301 comprehensively controls each unit of the server 30. Processing performed by the server 30, which will be described later, is performed under the control of the server control unit 301 unless otherwise specified.

[0058] The sound source position coordinate holding unit 302 is a memory that holds the position (absolute position) of the speaker 50, which is an example of a real sound source, in three-dimensional coordinates associated with the real space. The progress information holding unit 303 is a memory that holds progress information of the content (e.g., playback time). Note that the sound source position coordinate holding unit 302 and the progress information holding unit 303 may be the same memory or different memories.

[0059] The server communication unit 304 is configured to communicate with the smart display 40 and the speaker 50, and includes a modulation / demodulation circuit and an antenna that comply with a communication standard. The server communication unit 304 may be a communication unit that complies with a network such as the Internet, a communication unit that complies with short-range wireless communication such as a LAN (Local Area Network), or a communication unit that complies with both.

[0060] "Example of smart display configuration" The smart display 40 has a terminal control unit 401, a terminal communication unit 402, a sound source position information storage unit 403, a progress information storage unit 404, a self-position information acquisition unit 405, a content drawing unit 406, and a display 407.

[0061] The terminal control unit 401 comprehensively controls the entire smart display 40. The terminal control unit 401 has, for example, a virtual object position information storage unit 401A, a display timing storage unit 401B, a virtual object absolute position calculation unit 401C, a distance calculation unit 401D, a start time calculation unit 401E, a start time comparison unit 401F, and an animation control unit 401G.

[0062] The virtual object position information storage unit 401A stores the position information of the virtual object. The display timing storage unit 401B stores the display timing set to the virtual object by default. The position information and display timing of the virtual object are acquired, for example, when an application including video content is installed on the smart display 40. The position information and display timing of the virtual object may be acquired by communication or the like. The position information and display timing of the virtual object are defined along a time series. The position information of the virtual object is information defined by a relative position with respect to the position of the smart display 40. The display timing of the virtual object is a timing at which a video of the virtual object performing a predetermined operation is displayed, and is defined in accordance with the passage of the playback time of the video content. The position information of the virtual object is supplied from the virtual object position information storage unit 401A to the virtual object absolute position calculation unit 401C. The display timing of the virtual object is supplied from the display timing storage unit 401B to the start time calculation unit 401E.

[0063] The virtual object absolute position calculation unit 401C calculates the absolute position of the virtual object based on the position information of the virtual object and the position information of the smart display 40 supplied from the self-position information acquisition unit 405. The absolute position of the virtual object is a predetermined position on a three-dimensional coordinate system associated with the real space. The virtual object is displayed (drawn) at a position on the display 407 corresponding to the absolute position calculated by the virtual object absolute position calculation unit 401C.

[0064] The distance calculation unit 401D calculates the distance between the speaker 50 and the virtual object based on sound source position information, which is the absolute position of the speaker 50, transmitted from the server 30 and the absolute position of the virtual object supplied from the virtual object absolute position calculation unit 401C.

[0065] The start time calculation unit 401E calculates the start time at which the virtual object executes a predetermined action, that is, the action start timing. More specifically, the start time calculation unit 401E performs a delay amount setting process to increase the delay amount of the action start timing at which the virtual object executes a predetermined action relative to the playback timing at which the speaker 50 plays a predetermined sound, the greater the distance between the position of the speaker 50 in the real space and the position of the virtual object in the three-dimensional coordinate system. The action start timing calculated by the start time calculation unit 401E is supplied to the start time comparison unit 401F.

[0066] The start time comparison unit 401F judges whether the timing specified by the operation start timing exceeds the time indicated by the progress information (hereinafter, appropriately referred to as the progress time) based on the progress information supplied from the progress information storage unit 404 and the operation start timing started by the start time calculation unit 401E. If the operation start timing exceeds the progress time, the start time comparison unit 401F notifies the animation control unit 401G of that effect.

[0067] When the action start timing exceeds the progress time, the animation control unit 401G generates an animation in which the virtual object performs a series of predetermined actions. The animation generated by the animation control unit 401G is supplied to the content drawing unit 406.

[0068] The terminal communication unit 402 communicates with an external device such as the server 30. Through communication performed by the terminal communication unit 402, the smart display 40 acquires sound source position information and progress information supplied from the server 30. The sound source position information acquired by the terminal communication unit 402 is retained (stored) in the sound source position information retention unit 403, and the progress information is retained in the progress information retention unit 404. The sound source position information retained in the sound source position information retention unit 403 is supplied to a distance calculation unit 401D. In addition, the progress information retained in the progress information retention unit 404 is supplied to a start time comparison unit 401F.

[0069] The self-location information acquisition unit 405 as a location information acquisition unit acquires self-location information, which is the current location of the smart display 40. The self-location information may be directly input by the user, or may be acquired by the user performing an operation of reading a QR code (registered trademark) attached to the user's location (such as a seat) indicating the location. The self-location information may be acquired via a location sensor such as a Global Navigation Satellite System (GNSS), or may be acquired by estimating the self-location using Simultaneous Localization and Mapping (SLAM) or a feature amount map captured in advance. The location of the user having the smart display 40 may change (is variable). In this case, the self-location information also changes according to the change in the user's location.

[0070] The content drawing unit 406 performs processing to draw the animation supplied from the animation control unit 401G on the display 407. Specifically, a virtual object singing on the stage 2 and a virtual object flying around the venue are drawn on the display 407. This allows the user to visually recognize the virtual objects displayed on the display 407.

[0071] The display 407 is, for example, an optical see-through display. The user recognizes the virtual objects displayed on the display 407 while visually recognizing real objects such as the stage 2 through the display 407.

[0072] "Speaker configuration example" The speaker 50 has a speaker communication unit 501, a sound source reproduction unit 502, and a speaker unit 503. The speaker communication unit 501 communicates with the server 30 and the smart display 40. The speaker communication unit 501 receives, for example, a sound reproduction command transmitted from the server 30 or the smart display 40.

[0073] The sound source reproducing unit 502 performs known audio signal processing for reproducing the sound source data. The sound source data may be stored in an appropriate memory of the speaker 50, or may be acquired by communication via the speaker communication unit 501. The sound source data processed by the sound source reproducing unit 502 is reproduced from the speaker unit 503.

[0074] When multiple virtual objects are included in the video content, the position information and display timing of these objects are specified for each virtual object. The process described below is also performed for each virtual object. There may be no display timing, and the virtual object may be preprogrammed to perform a predetermined action.

[0075] The progress information may be managed by the smart display 40, not by the server 30. The smart display 40 may acquire the current progress information by using marker recognition, camera recognition using a feature map, or sound fingerprinting, not from the server 30. Sound source position information of the speaker 50 may be directly supplied from the speaker 50 to the smart display 40.

[0076] "Processing flow" Next, the flow of the processing performed in this embodiment (mainly the processing performed in the smart display 40) will be described with reference to Figs. 16, 17, and 18. Fig. 16 is a flowchart showing the overall processing flow. Fig. 17 is a flowchart showing the flow of the virtual object display timing control processing in Fig. 16. Fig. 18 is a flowchart showing the flow of the operation start timing calculation processing in Fig. 17. Note that, in the following description, it is assumed that an application for executing the processing according to this embodiment is installed in the smart display 40.

[0077] 16, in step ST11, the user position, in other words, the position of the smart display 40 carried or worn by the user, is acquired by the self-position information acquisition unit 405. As described above, the user position may be input by a user operation or may be acquired by a sensor or the like. Then, the process proceeds to step ST12.

[0078] In step ST12, sound source position information of the speaker 50, which is an example of a real sound source, is acquired from the server 30, and the acquired sound source position information is stored in the sound source position information storage unit 403. Then, the process proceeds to step ST13.

[0079] In step ST13, content playback is started. Content playback is started, for example, by an instruction from the server 30 made in response to a content playback operation on the smart display 40. When a content playback command is supplied from the server 30 to the speaker 50, sound source data is played from the speaker unit 503. When content playback is started, the smart display 40 periodically acquires progress information from the server 30. Then, the process proceeds to step ST14.

[0080] In step ST14, control related to the playback of the content is performed in accordance with the playback of the content, and the process then proceeds to step ST15.

[0081] In step ST15, it is determined whether or not X seconds or more have passed since the previous progress information was obtained. This determination is made, for example, by the terminal control unit 401. The terminal control unit 401 determines, for example, whether or not X seconds have passed since the progress information stored in the progress information storage unit 404 was updated to the progress information transmitted from the server 30. X seconds can be set appropriately, for example, from several seconds to 5 seconds. If X seconds have not passed since the previous progress information was obtained, the process proceeds to step ST18. If X seconds have passed since the previous progress information was obtained, the process proceeds to step ST16.

[0082] In step ST16, the smart display 40 acquires the progress information from the server 430. Then, the process proceeds to step ST17.

[0083] In step ST17, the progress information acquired in the process in step ST16 is updated by being stored in the progress information storage unit 404. Then, the process proceeds to step ST18.

[0084] In this embodiment, the user position may change. When the user position changes, the process in step ST18 is performed, and the new user position is supplied from the self-position information acquisition unit 405 to the terminal control unit 401, thereby updating the user position. Then, a process based on the updated user position is performed. Then, the process proceeds to step ST19.

[0085] In step ST19, a virtual object display timing control process is performed. The details of the virtual object display timing control process will be described later. The virtual object display timing control process obtains the operation start timing of the virtual object. Then, the process proceeds to step ST20.

[0086] In step ST20, a virtual object that performs a predetermined action depending on the lapse of the action start timing is drawn on the display 407. Note that, if there is a virtual object that does not depend on the action start timing, the virtual object is always drawn on the display 407 during content playback, for example.

[0087] Next, the flow of the virtual object display timing control process in this embodiment will be described with reference to the flowchart in FIG.

[0088] For example, after the process in step ST18 shown in Fig. 16 is performed, the virtual object display timing control process is started in step ST31. Then, the process proceeds to step ST32.

[0089] In step ST32, a motion start timing calculation process is performed. The motion start timing calculation process obtains the motion start timing of the virtual object. The motion start timing calculation process will be described later in detail. Then, the process proceeds to step ST33.

[0090] In step ST33, the start time comparison unit 401F compares the operation start timing obtained in step ST32 with the progress time to determine whether the operation start timing has exceeded the progress time. If the operation start timing has not exceeded the progress time, the process proceeds to step ST35. If the operation start timing has exceeded the progress time, the process proceeds to step ST34.

[0091] In step ST34, animation of a predetermined movement of the target virtual object is started. This process is performed by the animation control unit 401G. Then, the process proceeds to step ST35.

[0092] In step ST35, the terminal control unit 401 judges whether or not all timing controls of the operations that have operation start timings after the progress time have been completed. If all timing controls of the operations have not been completed, the process returns to step ST32. If all timing controls of the operations have been completed, the process proceeds to step ST36.

[0093] In step ST36, the virtual object display timing control process is completed. Then, the process in step ST20 shown in Fig. 16 is performed, and an image of the virtual object performing a predetermined action, which is generated by the animation control unit 401G, is displayed on the display 407.

[0094] Next, the flow of the operation start timing calculation process in this embodiment will be described with reference to the flowchart in FIG.

[0095] In step ST41, the start time when the virtual object performs a predetermined action is set to t seconds (in milliseconds) after the start of playback. The value of t is defined by the installed application. Then, the process proceeds to step ST42.

[0096] In step ST42, the position of the virtual object is converted into an absolute position, which is a position on a three-dimensional coordinate system associated with a real space, based on the user position. That is, the relative position of the virtual object with respect to the smart display 40 defined by the application is converted into an absolute position based on the position of the smart display 40 supplied from the self-position information acquisition unit 405. Then, the process proceeds to step ST43.

[0097] In step ST43, the distance calculation section 401D calculates the distance D between the absolute position of the virtual object calculated in step ST42 and the position indicated by the sound source position information of the speaker 50. Then, the process proceeds to step ST44.

[0098] In step ST44, start time calculation unit 401E performs a delay amount setting process to calculate the delay amount based on the above-mentioned (Equation 2). As described above, in the delay amount setting process, the delay amount is set to be larger as distance D increases. Then, the process proceeds to step ST45.

[0099] In step ST45, the operation start timing is determined by adding the delay amount to a predetermined t, and then the process from step ST33 onwards shown in FIG.

[0100] The above-described processes such as the delay amount setting process are performed for each virtual object or for each series of actions of a virtual object.

[0101] According to the present embodiment described above, it is possible to appropriately set the action start timing for causing a virtual object to execute a predetermined action.

[0102] <Modification> Although one embodiment of the present disclosure has been specifically described above, the contents of the present disclosure are not limited to the above-described embodiment, and various modifications based on the technical concept of the present disclosure are possible. Modifications will be described below.

[0103] "Variation 1" Predetermined synchronization information may be set for the content to be played. The synchronization information is, for example, a flag indicating whether the content is synchronous or asynchronous. For example, in the case of "synchronous", a logical value of "1" is set as the synchronization information, and in the case of "asynchronous", a logical value of "0" is set as the synchronization information. Of course, the logical value may be the opposite. The synchronization information is information that can be set for each action of a virtual object in one piece of content or for each virtual object. For example, there are scenes such as a countdown where it is desired to show the same image to multiple users at the same timing. In such a case, "1" is set as the synchronization information.

[0104] The processes shown in patterns P11 and P12 in FIG. 19 are processes when "0" is set as the synchronization information, that is, "asynchronous". When "0" is set as the synchronization information, the operation start timing of the virtual object is set based on the delay amount setting process, as in the above-mentioned embodiment. The processes shown in pattern P13 in FIG. 19 are processes when "1" is set as the synchronization information, that is, "synchronous". When "1" is set as the synchronization information, no delay in display occurs regardless of the distance between the stage and the virtual object, in other words, a synchronization process is performed in which the operation start timing of the virtual object for each user is set to the same predetermined timing. Such a synchronization process is performed, for example, under the control of the terminal control unit 401. By performing the synchronization process, it is possible to draw an image of a predetermined scene at the same timing for multiple users. This timing is common to a predetermined smart display (for example, a smart display owned by a user UA) and another smart display (for example, a smart display owned by a user UB) that operates with the same application.

[0105] When the synchronization process is performed, information corresponding to the synchronization process may be drawn on the display 407. For example, countdown characters may be drawn on the display 407 as the information corresponding to the synchronization process.

[0106] "Variation 2" Predetermined sharing information may be set for the content to be played back. The sharing information is, for example, a flag indicating whether it is shared or not shared. For example, in the case of "shared", a logical value of "1" is set as the sharing information, and in the case of "not shared", a logical value of "0" is set as the sharing information. Of course, the logical values ​​may be opposite. The sharing information is information that can be set for each action of a virtual object or for each virtual object in one piece of content.

[0107] When "1" is set as the shared information, the predetermined motion of the virtual object and its position are shared among users having smart displays that operate with the same application. When "0" is set as the shared information, the predetermined motion of the virtual object is shared among users, but the position of the virtual object is different for each user. A specific example of the latter is, for example, a case where the flying motion of a virtual object is shared among users, but the position is, for example, in front of user UA and closer to stage 2 for user UB.

[0108] With reference to FIG. 20, a specific example in which "1" is set as the shared information will be described. As shown in FIG. 20, when a virtual object moves to a certain position PO8 (a position 150 m from the stage 2) between the user UA and the user UB, an image in which the back of the virtual object is visible is drawn on the smart display of the user UA, and an image in which the front of the virtual object is visible is drawn on the smart display of the user UB. That is, when "shared" is set as the predetermined shared information, the virtual object is drawn in a manner corresponding to the positional relationship between the position of the smart display and the position of the virtual object on the three-dimensional coordinates. Also, when non-shared is set as the predetermined shared information, the virtual object is displayed in a manner corresponding to each information processing device.

[0109] The delay amount in the case shown in FIG. 20 is set in the same manner as in the case of non-sharing, that is, in the first embodiment. Specifically, it is calculated as D=150. In the case of user UA, the sound of the real sound source arrives after 30 msec, and the virtual object starts to move after 440 msec from the generation of the sound of the real sound source. In the case of user UB, the sound of the real sound source arrives after 30 msec, and the virtual object starts to move after 440 msec from the generation of the sound of the real sound source.

[0110] The synchronization information and the sharing information are set when the content is produced. It is also possible to set both the synchronization information and the sharing information. The synchronization information and the sharing information may be changeable after the content is produced.

[0111] "Other Modifications" In the above-described embodiment, in the case where a virtual object approaches from the stage, a series of animations may be used to represent the virtual object's jumping out from the stage to the point of view. In this case, the timing for the virtual object to jump out from the stage is determined taking into consideration the time it takes to move, or the speed at which the virtual object jumps out from the stage to the point of view is adjusted appropriately.

[0112] In addition to the realistic sound coming from the real sound source, the user can also wear open-type earphones and listen to the sound source being played from the smart display at the same time. By using 3D sound to localize the sound source at the position of the virtual object, the user can experience the sound coming from a different position from the real sound source.

[0113] The real sound source may be a concert hall that is picked up by a microphone and reproduced. This embodiment can be applied not only to live performances, but also to parades and the like in which virtual objects are updated.

[0114] The configurations, methods, steps, shapes, materials, and values ​​given in the above-mentioned embodiments and modifications are merely examples, and different configurations, methods, steps, shapes, materials, and values ​​may be used as necessary, and may be replaced with known ones. In addition, the configurations, methods, steps, shapes, materials, and values ​​in the embodiments and modifications may be combined with each other to the extent that no technical contradiction occurs. In addition, the present disclosure may be realized in any form, such as a control method or a manufacturing device for electronic devices.

[0115] It should be noted that the contents of the present disclosure should not be construed as being limited to the effects exemplified in this specification.

[0116] The present disclosure may also have the following configurations. (1) a control unit that displays a virtual object on three-dimensional coordinates associated with a real space and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to increase a delay amount of an action start timing for causing the virtual object to execute the predetermined action relative to a playback timing at which the real sound source reproduces the predetermined sound, the greater the distance between a position of the real sound source in the real space and a position of the virtual object in the three-dimensional coordinate system. Information processing device. (2) The control unit acquires a position of the virtual object in the three-dimensional coordinate system based on a relative position of the virtual object defined with respect to a position of the information processing device. An information processing device as described in (1). (3) The control unit calculates a distance between a position of the real sound source and a position of the virtual object on the three-dimensional coordinate system, and sets the amount of delay based on a calculation result. An information processing device according to (2). (4) The position of the real sound source in the three-dimensional coordinate system is supplied from an external device. An information processing device according to (3). (5) The position of the information processing device is variable. An information processing device according to any one of (2) to (4). (6) A location information acquisition unit that acquires the location of the information processing device An information processing device according to any one of (2) to (5). (7) The control unit performs a synchronization process to set the operation start timing to a predetermined timing regardless of the delay amount. An information processing device according to any one of (1) to (6). (8) The control unit determines the operation start timing based on the synchronization process when synchronization is set as the predetermined synchronization information, and determines the operation start timing based on the delay amount setting process when asynchronous is set as the predetermined synchronization information. An information processing device according to (7). (9) The synchronization information is set for each virtual object. An information processing device according to (8). (10) The predetermined timing is a timing common to other information processing devices that operate with the same application. An information processing device according to (7) or (8). (11) When the synchronization process is performed, the control unit displays information corresponding to the synchronization process. An information processing device according to any one of (7) to (10). (12) When "shared" is set as predetermined shared information, the control unit displays the virtual object so as to correspond to a positional relationship between a position of the information processing device and a position of the virtual object on the three-dimensional coordinate system, and when "non-shared" is set as predetermined shared information, displays the virtual object so as to correspond to each information processing device. An information processing device according to any one of (1) to (12). (13) The shared information is set for each virtual object. An information processing device according to (12). (14) The control unit performs the delay amount setting process for each of a plurality of virtual objects to be displayed. An information processing device according to any one of (1) to (13). (15) The virtual object is displayed on a portable wearable display. An information processing device according to any one of (1) to (14). (16) The delay amount setting process is performed in each of a plurality of information processing devices that operate with the same application. An information processing device according to any one of (1) to (15). (17) a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to increase a delay amount of an action start timing for causing the virtual object to execute the predetermined action relative to a playback timing at which the real sound source reproduces the predetermined sound, the greater the distance between a position of the real sound source in the real space and a position of the virtual object in the three-dimensional coordinate system. Information processing methods. (18) a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; The control unit performs a delay amount setting process to increase a delay amount of an action start timing for causing the virtual object to execute the predetermined action relative to a playback timing at which the real sound source reproduces the predetermined sound, the greater the distance between a position of the real sound source in the real space and a position of the virtual object in the three-dimensional coordinate system. A program that causes a computer to execute an information processing method. [Explanation of symbols]

[0117] 1. Playback system 2. Stage 3,50···Speaker 30 Servers 40. Smart Display 401...Control section 401C: Virtual object absolute position calculation unit 401D...Distance calculation section 401E...Start time calculation section 405... Self-location information acquisition unit C. Virtual Object UA, UB...User

Claims

1. a control unit that displays a virtual object on three-dimensional coordinates associated with a real space and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; the control unit performs a delay amount setting process for setting a delay amount corresponding to a timing at which the predetermined sound reaches a position of the user in the real space; the delay amount is a delay amount of an action start timing for causing the virtual object to execute the predetermined action with respect to a playback timing at which the real sound source reproduces the predetermined sound, The control unit increases the amount of delay as the distance between the position of the real sound source in the real space and the position of the virtual object on the three-dimensional coordinate system increases. Information processing device.

2. The control unit acquires a position of the virtual object in the three-dimensional coordinate system based on a relative position of the virtual object defined with respect to a position of the information processing device. The information processing device according to claim 1 .

3. The control unit calculates a distance between a position of the real sound source and a position of the virtual object on the three-dimensional coordinate system, and sets the amount of delay based on a result of the calculation. The information processing device according to claim 2 .

4. The position of the real sound source in the three-dimensional coordinate system is supplied from an external device. The information processing device according to claim 3 .

5. The position of the information processing device is variable. The information processing device according to claim 2 .

6. A location information acquisition unit that acquires the location of the information processing device The information processing device according to claim 2 .

7. The control unit performs a synchronization process to set the operation start timing to a predetermined timing regardless of the delay amount. The information processing device according to claim 1 .

8. The control unit determines the operation start timing based on the synchronization process when synchronization is set as the predetermined synchronization information, and determines the operation start timing based on the delay amount setting process when asynchronous is set as the predetermined synchronization information. The information processing device according to claim 7.

9. The synchronization information is set for each virtual object. The information processing device according to claim 8.

10. The predetermined timing is a timing common to other information processing devices that operate with the same application. The information processing device according to claim 7.

11. When the synchronization process is performed, the control unit displays information corresponding to the synchronization process. The information processing device according to claim 7.

12. When "shared" is set as predetermined shared information, the control unit displays the virtual object so as to correspond to a positional relationship between a position of the information processing device and a position of the virtual object on the three-dimensional coordinate system, and when "non-shared" is set as predetermined shared information, displays the virtual object so as to correspond to each information processing device. The information processing device according to claim 1 .

13. The shared information is set for each virtual object. The information processing device according to claim 12.

14. The control unit performs the delay amount setting process for each of a plurality of virtual objects to be displayed. The information processing device according to claim 1 .

15. The virtual object is displayed on a portable wearable display. The information processing device according to claim 1 .

16. The delay amount setting process is performed in each of a plurality of information processing devices that operate with the same application. The information processing device according to claim 1 .

17. a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; the control unit performs a delay amount setting process for setting a delay amount corresponding to a timing at which the predetermined sound reaches a position of the user in the real space; the delay amount is a delay amount of an action start timing for causing the virtual object to execute the predetermined action with respect to a playback timing at which the real sound source reproduces the predetermined sound, The control unit increases the amount of delay as the distance between the position of the real sound source in the real space and the position of the virtual object on the three-dimensional coordinate system increases. Information processing methods.

18. a control unit that displays a virtual object on three-dimensional coordinates associated with a real space, and causes the virtual object to perform a predetermined action in response to a predetermined sound reproduced from a real sound source disposed in the real space; the control unit performs a delay amount setting process for setting a delay amount corresponding to a timing at which the predetermined sound reaches a position of the user in the real space; the delay amount is a delay amount of an action start timing for causing the virtual object to execute the predetermined action with respect to a playback timing at which the real sound source reproduces the predetermined sound, The control unit increases the amount of delay as the distance between the position of the real sound source in the real space and the position of the virtual object on the three-dimensional coordinate system increases. A program that causes a computer to execute an information processing method.

Images