Image transmission system and image transmission device

Abstract

According to the present invention, a first imaging device captures a first image of a performer in a theater from the position of a virtual first seat that is set in the theater. A second imaging device captures a second image including the performer from a position different from the position of the first imaging device. A visual line detection unit detects the direction of the visual line of the performer. A transmission control unit transmits the second image to a terminal device of a viewer associated with the virtual first seat, and transmits the first image to the terminal device on the basis of the direction of the visual line of the performer having been pointed toward the virtual first seat.

Description

Technical Field

[0001] The present invention relates to an image transmission system and an image transmission device.

Background Art

[0002] Conventionally, so-called online live shows that transmit images (generally moving images) showing performances such as music, drama, and dance to a user's terminal device via a network have become widespread. For example, Patent Document 1 below discloses a live video distribution system. The video data processing device switches and outputs a plurality of input camera videos based on the results of analysis performed by a music analysis data or video analysis unit. In addition, a specific person is detected from a plurality of camera videos, and the video is output by switching to a video in which the specific person is the main subject according to changes in the sound or video during the music. Further, the video data processing device switches and outputs a video according to a request transmitted from a user terminal, and transmits it to the user terminal via a video distribution device and a streaming server.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] Online live shows have high convenience in that viewers can watch the live performance even if they cannot come directly to the live venue. On the other hand, experiences such as individual interactions between the performer and the viewer, for example, the performer and the viewer making eye contact, cannot be obtained in an online live show. For example, even if the performer looks at the imaging device during an online live show, the image is viewable by all viewers watching the online live show. Therefore, the viewer does not perceive that the performer is looking at him / herself. Thus, watching an online live show has the problem that the relationship between the viewer and the performer becomes diluted compared to watching a live performance at the live venue, and the satisfaction of the viewer is low.

[0005] An object of the present invention is to improve the satisfaction of viewers when transmitting an image of a live performance.

Means for Solving the Problem

[0006] An image transmission system according to an aspect of the present invention includes a first imaging device that captures a first image of a first performer in the real space from a position of a virtual first seat set in the real space, a second imaging device that captures a second image including the performer from a position different from the position of the first imaging device, a transmission control unit that transmits the second image to a terminal device of a viewer associated with the virtual first seat, and a detection unit that detects the direction of the line of sight of the performer. The transmission control unit transmits the first image to the terminal device based on the fact that the direction of the line of sight of the performer is directed toward the virtual first seat.

[0007] In addition, an image transmission device according to an aspect of the present invention includes a first acquisition unit that acquires a first image of a performer in the real space, which is captured by a first imaging device from a position of a virtual first seat set in the real space; a second acquisition unit that acquires a second image including the performer, which is captured by a second imaging device from a position different from the first imaging device; a transmission control unit that transmits the second image to a terminal device of a viewer associated with the virtual first seat; and a detection unit that detects the direction of the performer's line of sight. The transmission control unit transmits the first image to the terminal device based on the fact that the performer's line of sight is directed toward the virtual first seat.

Advantages of the Invention

[0008] According to an aspect of the present invention, it is possible to improve the satisfaction of viewers when transmitting a live performance video.

Brief Description of the Drawings

[0009]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12A

Figure 12B

Figure 13

Figure 14A

Figure 14B

Figure 15

Mode for Carrying Out the Invention

[0010] A. First Embodiment Hereinafter, the configuration of the image transmission system 1 according to the first embodiment of the present invention will be described.

[0011] A-1. System Configuration FIG. 1 is a block diagram showing the configuration of an image transmission system 1 according to the first embodiment. The image transmission system 1 includes a first imaging device 10, a moving mechanism 50, a second imaging device 20, and an image transmission device 30A. The image transmission device 30A is connected to a plurality of terminal devices 40-1 to 40-n (n is an arbitrary integer of 1 or more) via a communication network N. Each of the terminal devices 40-1 to 40-n is held by different viewers V1 to Vn. In the present embodiment, seats C1 to C7 (see FIG. 2) described later are associated with viewers V1 to V7 in a one-to-one manner. The viewers V1 to V7 hold the terminal devices 40-1 to 40-7, respectively. Therefore, in FIG. 1, the terminal devices 40-1 to 40-7 are illustrated as an example of the terminal devices 40-1 to 40-n connected to the image transmission device 30A. That is, the present embodiment exemplifies the case where n = 7. Hereinafter, when representing any one of the terminal devices 40-1 to 40-7, it is denoted as the terminal device 40-i (i is an arbitrary integer from 1 to 7). Also, when representing any one of the viewers V1 to V7, it is denoted as the viewer Vi (i is an arbitrary integer from 1 to 7).

[0012] The image transmission system 1 is a system for transmitting images (a first image PC1 and a second image PC2 described later) captured by the first imaging device 10 and the second imaging device 20 to the terminal devices 40-1 to 40-7. More specifically, the first imaging device 10 and the second imaging device 20 capture an actor P (see FIG. 2) performing a performance such as singing. The viewer Vi uses the terminal device 40-i to view an image in which the actor P performing the performance appears. That is, the image transmission system 1 is used for online live transmission.

[0013] In the present embodiment, the case where the online live is a so-called live distribution that transmits the image of the performance in real time is considered. However, the online live is not limited to this, and it may be a method of transmitting the pre-recorded image of the performance afterwards.

[0014] The terminal device 40-i is an information processing device such as a smartphone, a tablet terminal, a personal computer, smart glasses, or VR glasses. The terminal device 40-i includes, for example, an image display device such as a display, a speaker, and a communication device connected to the communication network N.

[0015] The first imaging device 10 has an imaging optical system and an imaging element. The imaging optical system is an optical system including at least one imaging lens. For example, the imaging optical system may have various optical elements such as a prism, or may have a zoom lens or a focus lens, etc. The imaging element is, for example, a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary MOS) image sensor, etc.

[0016] Also, the first imaging device 10 includes an interface for outputting image data to the image transmission device 30A. The first imaging device 10 and the image transmission device 30A may be connected by wire or wirelessly. The second imaging device 20 also has the same configuration as the first imaging device 10. The first imaging device 10 and the second imaging device 20 are, in one example, video cameras.

[0017] The image captured by the first imaging device 10 is called the first image PC1, and the image data corresponding to the first image PC1 is called the first image data. Also, the image captured by the second imaging device 20 is called the second image PC2, and the image data corresponding to the second image PC2 is called the second image data. In this embodiment, the first imaging device 10 and the second imaging device 20 capture moving images. The first imaging device 10 captures images at a predetermined frame rate, generates the first image data corresponding to the first image PC1, and outputs the generated first image data to the image transmission device 30A. Similarly, the second imaging device 20 generates the second image data corresponding to the second image PC2 and outputs the generated second image data to the image transmission device 30A.

[0018] FIG. 2 is an explanatory diagram schematically showing the arrangement of the first imaging device 10 and the second imaging device 20. FIG. 2 is a view of the theater G from above. The space inside the theater G is an example of the real space. The first imaging device 10 and the second imaging device 20 image the performer P inside the theater G. In the present embodiment, the performer P performs a live performance such as singing on the stage S installed inside the theater G. The live performance is not limited to singing and may be, for example, a play or a dance. In the present embodiment, the space inside the theater G is relatively small, and the direction L of the line of sight of the performer P can be identified from each of the seats C1 to C7.

[0019] A plurality of virtual seats C1 to C7 are arranged inside the theater G. For example, if seat C1 is the first seat and seat C2 is the second seat, seat C2 is arranged at a physical position different from the physical position of seat C1 inside the theater G. Hereinafter, when representing any one of the seats C1 to C7, it is denoted as seat Ci (i is an arbitrary integer from 1 to 7). The seats C1 to C7 are obtained by, for example, dividing the floor of the theater G into a plurality of sections and attaching identification information (seat number) to each section. Note that the seats C1 to C7 may be chairs actually installed in the theater G. Each of the seats C1 to C7 is associated with a viewer V1 to V7, respectively. For example, viewer V1 is associated with seat C1. That is, viewer Vi is associated with seat Ci. The terminal device 40-i held by viewer Vi may be denoted as the terminal device 40-i associated with seat Ci.

[0020] When viewer Vi applies to watch an online live show (or after the application and before the start of the live show), seat Ci is assigned. For viewer Vi, which one of seats C1 to C7 is assigned may be determined by viewer Vi himself / herself or by lottery or the like. If a payment is required to watch the online live show, the fee may vary according to the positions of seats C1 to C7. For example, it may be set that the closer seat Ci is to the center of the stage S, the higher the fee. Also, a plurality of viewers Vi-1, Vi-2,... may be associated with one seat Ci. On the other hand, in this embodiment, not all of the viewers V1 to Vj (j is an arbitrary integer) who watch the online live show are assigned to the same seat Ci. This is because if all of the viewers V1 to Vj are assigned to the same seat Ci, all of the viewers V1 to Vj will watch the same image, which is substantially equivalent to not having assigned seat Ci at all.

[0021] The first imaging device 10 captures a first image PC1 of the performer P who performs in the theater G from the position of the seat Ci that is virtually arranged in the theater G. In the first embodiment, the first imaging device 10 is movable between seats C1 to C7. The first imaging device 10 captures an image of the performer P as the first image PC1 from any one of the positions of seats C1 to C7. For example, when the first imaging device 10 is at the position of seat C1, an image of the performer P from the position of seat C1 is captured as the first image PC1. Also, when the first imaging device 10 is at the position of seat C2, an image of the performer P from the position of seat C2 is captured as the first image PC1.

[0022] The first imaging device 10 is mounted on the moving mechanism 50. The moving mechanism 50 moves the first imaging device 10 between seats C1 to C7. The moving mechanism 50 moves the first imaging device 10 based on the control of the movement control unit 314 described later. The moving mechanism 50 includes a rail 52, a main body 54, and an arm 56. The rail 52 is installed along the arrangement of seats C1 to C7. The main body 54 houses a driving wheel that moves on the rail 52 and an actuator or the like that rotates the driving wheel. One end of the arm 56 is fixed to the main body 54, and the other end is fixed to the first imaging device 10. When the main body 54 moves on the rail 52, the first imaging device 10 moves left and right within the theater G. The arm 56 is connected to an actuator (not shown) and can expand and contract vertically. Therefore, the first imaging device 10 can also move vertically within the theater G.

[0023] Note that instead of using the moving mechanism 50 to move the first imaging device 10, the imaging operator may move the first imaging device 10.

[0024] FIGS. 3A and 3B are explanatory diagrams schematically showing the first image PC1. In the present embodiment, the imaging magnification of the first imaging device 10 is mainly set so that the face of the performer P occupies a large portion of the image. By viewing the first image PC1, the viewer Vi can grasp the expression of the performer P and the direction L of the line of sight (see FIG. 2), etc.

[0025] FIG. 3A shows the first image PC1-A captured from the position of seat C4 when the direction L of the line of sight of the performer P faces seat C4 as shown in FIG. 2. In the first image PC1-A, since the performer P is facing the first imaging device 10 (seat C4), the viewer Vi viewing the first image PC1-A appears to see the performer P facing himself / herself.

[0026] On the other hand, FIG. 3B shows the first image PC1-B captured from the position of seat C1 when the direction L of the line of sight of the performer P faces seat C4 as shown in FIG. 2. Since the performer P is not facing the first imaging device 10 in the first image PC1-B, the viewer Vi viewing the first image PC1-B does not appear to see the performer P facing himself / herself.

[0027] The second imaging device 20 captures a second image PC2 including the performer P from a position different from that of the first imaging device 10. In the present embodiment, the second imaging device 20 captures the entire stage S from a position farther from the stage S than the first imaging device 10. The position of the second imaging device 20 may be fixed, or it may be mounted on the moving mechanism 50 and be movable within the theater G in the same manner as the first imaging device 10. Also, a plurality of imaging devices corresponding to the second imaging device 20 may be installed within the theater G. For example, as shown in FIG. 2, in addition to the imaging device that captures the stage S from the front, an imaging device that captures the stage S from the right side and an imaging device that captures the stage S from the left side may be installed as the second imaging device 20.

[0028] FIG. 4 is an explanatory diagram schematically showing the second image PC2. In the second image PC2 shown in FIG. 4, a wide range of the stage S and the entire body of the performer P are shown. By viewing the second image PC2, the viewer Vi can grasp the movement of the body of the performer P and the performance on the stage S.

[0029] As shown in FIGS. 3A, 3B, and 4, the ratio of the performer P in the first image PC1 is larger than the ratio of the performer P in the second image PC2. In other words, the imaging magnification of the first image PC1 is larger than the imaging magnification of the second image PC2.

[0030] A-2. Image transmission device 30A FIG. 5 is a block diagram showing the configuration of the image transmission device 30A. The image transmission device 30A is, for example, a computer, and transmits the first image PC1 and the second image PC2 to the terminal devices 40-1 to 40-7. The image transmission device 30A includes an input device 301, a display device 302, a communication device 303, an image input interface (I / F) 304, a storage device 305, a processing device 306A, and a bus 307. The input device 301, the display device 302, the communication device 303, the image input interface 304, the storage device 305, and the processing device 306A are interconnected by a bus 307 for communicating information. The bus 307 may be configured using a single bus, or may be configured using different buses for each device.

[0031] The input device 301 is a device that receives operations from the administrator of the image transmission device 30A. For example, the input device 301 includes a pointing device such as a keyboard, a touch pad, a touch panel, or a mouse. When the input device 301 includes a touch panel, the input device 301 may also serve as the display device 302.

[0032] The display device 302 is a device that displays images and character information. The display device 302 displays various images under the control of the processing device 306A. For example, various display panels such as a liquid crystal display panel and an organic EL display panel are preferably used as the display device 302.

[0033] The communication device 303 communicates with the terminal devices 40-1 to 40-7 using wireless communication or wired communication. In the present embodiment, the communication device 303 includes an interface connectable to the communication network N, and communicates with the terminal devices 40-1 to 40-7 via the communication network N.

[0034] The image input interface 304 is a connection interface with the first imaging device 10 and the second imaging device 20. The first image data and the second image data are input into the image transmission device 30A via the image input interface 304.

[0035] The storage device 305 is a recording medium readable by the processing device 306A. The storage device 305 includes, for example, a non-volatile memory and a volatile memory. The non-volatile memory is, for example, a ROM (Read Only Memory), an EPROM (Erasable Programmable Read Only Memory), and an EEPROM (Electrically Erasable Programmable Read Only Memory). The volatile memory is, for example, a RAM (Random Access Memory). The storage device 305 stores the program PG1. The program PG1 is a program for operating the image transmission device 30A.

[0036] The processing device 306A includes one or more CPUs (Central Processing Unit). One or more CPUs are an example of one or more processors. Each of the processor and the CPU is an example of a computer.

[0037] The processing device 306A reads the program PG1 from the storage device 305. By executing the program PG1, the processing device 306A functions as an image acquisition unit 310, a gaze detection unit 312A, a movement control unit 314, and a transmission control unit 316A. At least one of the image acquisition unit 310, the gaze detection unit 312A, the movement control unit 314, and the transmission control unit 316A may be configured by a circuit such as a DSP (Digital Signal Processor), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field Programmable Gate Array).

[0038] The image acquisition unit 310 acquires the first image PC1 from the first imaging device 10. Also, the image acquisition unit 310 acquires the second image PC2 from the second imaging device 20. The image acquisition unit 310 is an example of the first acquisition unit and the second acquisition unit. The image acquisition unit 310 continuously acquires the first image PC1 and the second image PC2 during the live performance of the performer P.

[0039] The gaze detection unit 312A detects the direction L of the gaze of the performer P. The gaze detection unit 312A is an example of the detection unit. In the first embodiment, the gaze detection unit 312A detects the direction L of the gaze of the performer P based on the second image PC2. The gaze detection unit 312A performs image analysis on the second image PC2 to detect the position of the performer P on the stage S and the direction of the face of the performer P. The gaze detection unit 312A estimates that the gaze is directed, for example, in the direction in which the face of the performer P is facing. Alternatively, the gaze detection unit 312A may further detect the position of the black eyes of the performer P from the second image PC2 to more precisely detect the direction L of the gaze of the performer P.

[0040] The movement control unit 314 controls the movement mechanism 50. The movement control unit 314 drives the movement mechanism 50 to move the first imaging device 10 to the intersection of the direction L of the gaze of the performer P detected by the gaze detection unit 312A and the rail 52. That is, the movement control unit 314 moves the first imaging device 10 to the movement mechanism 50 between the seats C1 to C7 based on the direction L of the gaze of the performer P detected by the gaze detection unit 312A.

[0041] For example, as shown in FIG. 2, when the direction L of the gaze of the performer P is directed at the seat C4, the movement control unit 314 moves the first imaging device 10 to the position of the seat C4. Also, for example, when the direction L of the gaze of the performer P is directed at the seat C1, the movement control unit 314 moves the first imaging device 10 to the position of the seat C1. In the first embodiment, since the first imaging device 10 is moved in the direction L of the gaze of the performer P, the first image PC1 mainly consists of an image in which the performer P is looking in the direction as shown in FIG. 3A.

[0042] Note that, depending on the choreography of the singing, for example, there may be a case where the direction L of the line of sight of the performer P faces a range where the moving mechanism 50 cannot move, such as when the performer P turns his / her back to the seats C1 to C7 or looks up at the sky. In this case, the movement control unit 314 may move the first imaging device 10 to a predetermined reference position (for example, the central position of the rail 52) and wait, or may wait the first imaging device 10 at a position where it can no longer follow the direction L of the line of sight.

[0043] The transmission control unit 316A transmits the image of the performer P to the terminal devices 40-1 to 40-7. The image of the performer P is the first image PC1 or the second image PC2. The transmission control unit 316A switches the image to be transmitted to the terminal devices 40-1 to 40-7 between the first image PC1 and the second image PC2 based on the direction L of the line of sight of the performer P. The images transmitted to the terminal devices 40-1 to 40-7 are different from each other for each of the terminal devices 40-1 to 40-7.

[0044] More specifically, the transmission control unit 316A transmits the second image PC2 to the terminal devices 40-1 to 40-7 during normal times. Normal times mean, for example, when taking the terminal device 40-1 as an example, the direction L of the line of sight of the performer P is not facing the seat C1 associated with the terminal device 40-1. On the other hand, when the direction L of the line of sight of the performer P faces the seat Ci associated with the viewer Vi holding the terminal device 40-i, the transmission control unit 316A transmits the first image PC1 to the terminal device 40-i held by the viewer Vi. That is, the transmission control unit 316A transmits the first image PC1 to the terminal device 40-i based on the fact that the direction L of the line of sight of the performer P faces the seat Ci.

[0045] FIG. 6 is an explanatory diagram schematically showing the direction L of the line of sight of the speaker P. In FIG. 6, the period Tjk (j and k are integers, k = j + 1) indicates the period from time tj to time tk. For example, consider the case where the speaker P sequentially directs the line of sight to the seats C1 to C7. The period during which the line of sight is directed to seat C1 is the period T12, the period during which the line of sight is directed to seat C2 is the period T23, the period during which the line of sight is directed to seat C3 is the period T34, the period during which the line of sight is directed to seat C4 is the period T45, the period during which the line of sight is directed to seat C5 is the period T56, the period during which the line of sight is directed to seat C6 is the period T67, and the period during which the line of sight is directed to seat C7 is the period T78.

[0046] FIG. 7 is an explanatory diagram schematically showing the switching timing of the transmitted images corresponding to the direction L of the line of sight shown in FIG. 6. For example, to the terminal device 40-1 held by the viewer V1 corresponding to seat C1, the second image PC2 is transmitted until time t1. From time t1 when the speaker P directs the line of sight to seat C1 to time t2, the first image PC1 is transmitted. After time t2 when the line of sight of the speaker P deviates from seat C1, the second image PC2 is transmitted.

[0047] Also, for example, to the terminal device 40-2 held by the viewer V2 corresponding to seat C2, the second image PC2 is transmitted until time t2. From time t2 when the speaker P directs the line of sight to seat C2 to time t3, the first image PC1 is transmitted. After time t3 when the line of sight of the speaker P deviates from seat C2, the second image PC2 is transmitted. Similarly for the other terminal devices 40-3 to 40-7, the first image PC1 is transmitted while the speaker P directs the line of sight to seats C3 to C7.

[0048] In the above description, the transmission control unit 316A considered the case of switching the image to be transmitted to the terminal device 40-i between the first image PC1 and the second image PC2. That is, at a certain point in time, the image transmitted to the terminal device 40-i was either the first image PC1 or the second image PC2. Not limited to this, the transmission control unit 316A may, for example, always transmit the first image PC1 and the second image PC2 to the terminal device 40-i and transmit a control signal for switching the image to be displayed on the terminal device 40-i between the first image PC1 and the second image PC2.

[0049] Also, when the image displayed on the terminal device 40-i switches to the first image PC1, there is a possibility that the viewer Vi is not looking at the screen, or the viewer Vi does not notice that the image has switched and watches it casually. Therefore, the transmission control unit 316A may transmit an alert to the terminal device 40-i before the image transmitted to the terminal device 40-i switches from the second image PC2 to the first image PC1. More specifically, the transmission control unit 316A may transmit an alert to the terminal device 40-i substantially simultaneously with the time of transmitting the first image PC1 to the terminal device 40-i, or before transmitting the first image PC1 to the terminal device 40-i.

[0050] An example of the transmission timing of the alert is indicated by the symbol N in FIG. 7. Transmitting an alert may be, for example, transmitting a control signal to activate the vibration function of the terminal device 40-i, or transmitting a control signal to output an alert sound on the terminal device 40-i. Also, transmitting an alert may be transmitting a control signal to display a video or image indicating an alert on the terminal device 40-i. The control signal may be transmitted to the terminal device 40-i together with, for example, the first image PC1, and may be a signal that generates an alert immediately before the output of the first image PC1 or simultaneously with the output of the first image PC1. The viewer Vi who receives the alert can recognize that the image switches to the first image PC1 and can prevent missing the first image PC1.

[0051] In addition, in order to send an alert in advance, it is necessary to predict the direction L of the line of sight of the speaker P in advance. The line-of-sight detection unit 312A may continuously track the direction L of the line of sight of the speaker P, analyze the change in the direction L of the line of sight, and estimate the direction L of the line of sight of the speaker P after a predetermined time. In this case, the transmission control unit 316A sends an alert to the terminal device 40-i corresponding to the seat Ci towards which the line of sight of the speaker P turns after a predetermined time.

[0052] A-3. Operation of the processing device 306A FIG. 8 is a flowchart showing the operation of the processing device 306A. In the flowchart of FIG. 8, for convenience of explanation, the image transmission process for one terminal device 40-i is shown. The processing device 306A performs the following processes in parallel for each of the terminal devices 40-1 to 40-7.

[0053] The processing device 306A functions as an image acquisition unit 310 and acquires the first image PC1 from the first imaging device 10 and the second image PC2 from the second imaging device 20, respectively (step S100). The acquisition of the first image PC1 and the second image PC2 is continuously performed during the subsequent processing. The processing device 306A functions as a transmission control unit 316A and transmits the second image PC2 to the terminal device 40-i of the viewer Vi (step S102).

[0054] The processing device 306A functions as a line-of-sight detection unit 312A and detects the direction L of the line of sight of the speaker P based on the second image PC2 (step S104). The processing device 306A functions as a movement control unit 314 and moves the first imaging device 10 onto the extension line of the line of sight of the speaker P (step S106).

[0055] In addition, the processing device 306A functions as a transmission control unit 316A and determines whether there is a seat Ci on the extension line of the line of sight of the speaker P, that is, whether the line of sight of the speaker P is directed towards the seat Ci (step S108). If the line of sight of the speaker P is not directed towards the seat Ci (step S108: NO), the processing device 306A returns the process to step S104.

[0056] When the line of sight of the performer P is directed at the seat Ci (step S108: YES), the processing device 306A functions as a transmission control unit 316A and transmits the first image PC1 to the terminal device 40-i of the viewer Vi corresponding to the seat Ci (step S110). At this time, the processing device 306A may transmit an alert to the terminal device 40-i.

[0057] The processing device 306A waits until the line of sight of the performer P moves away from the seat Ci (step S112: NO). During this period, the first image PC1 is transmitted to the terminal device 40-i. When the line of sight of the performer P moves away from the seat Ci (step S112: YES), the processing device 306A functions as a transmission control unit 316A and transmits the second image PC2 to the terminal device 40-i of the viewer Vi corresponding to the seat Ci (step S114). Then, the processing device 306A returns the process to step S100.

[0058] As described above, according to the image transmission system 1 according to the first embodiment, when the line of sight of the performer P is directed at the virtually set seat Ci, the first image PC1 obtained by imaging the performer P from the seat Ci is transmitted to the viewer Vi. The viewer Vi can experience, even in the viewing of an online live, the real-life experience that can occur at a live venue of making eye contact with the performer P. Therefore, the added value of the online live is improved.

[0059] Also, according to the image transmission system 1, a seat Ci is assigned to the viewer Vi who views the online live. Therefore, the viewer Vi can view the first image PC1 that cannot be viewed by other viewers Vi+n (n is an arbitrary integer), and can obtain an experience similar to actually going to the live venue.

[0060] Further, in the image transmission system 1, when the image transmitted to the terminal device 40-i is switched from the second image PC2 to the first image PC1, an alert is transmitted to the terminal device 40-i. Therefore, the possibility that the viewer Vi misses the first image PC1 is reduced, and the satisfaction with the online live is further improved.

[0061] In the first embodiment, one first imaging device 10 moves between a plurality of seats C1 to C7 to capture a first image PC1. Therefore, compared with arranging a plurality of imaging devices, the system cost of the image transmission system 1 is reduced.

[0062] In the first embodiment, the direction L of the line of sight of the performer P is detected based on the second image PC2 captured by the second imaging device 20. Therefore, compared with separately arranging an imaging device for tracking the line of sight of the performer P, the system cost of the image transmission system 1 is reduced.

[0063] B. Second Embodiment Hereinafter, a second embodiment of the present invention will be described. In the following description, for the sake of simplicity of explanation, the same reference numerals are used for the same components as in the first embodiment, and the description of their functions may be omitted. In the following description, for the sake of simplicity of explanation, mainly the differences between the second embodiment and the first embodiment will be described.

[0064] B-1. System Configuration FIG. 9 is a block diagram showing the configuration of an image transmission system 2 according to the second embodiment. The image transmission system 2 includes a plurality of first imaging devices 10-1 to 10-7, a second imaging device 20, and an image transmission device 30B. Hereinafter, when representing any one of the first imaging devices 10-1 to 10-7, it is denoted as the first imaging device 10-i (i is an arbitrary integer from 1 to 7). The image transmission device 30B is connected to a plurality of terminal devices 40 (40-1, 40-2... 40-7) via a communication network N. Since the second imaging device 20 and the terminal devices 40-1 to 40-7 are the same as those in the first embodiment, the description thereof is omitted.

[0065] FIG. 10 is an explanatory diagram schematically showing the arrangement of the first imaging devices 10-1 to 10-7 in the second embodiment. FIG. 10 is a view of the theater G seen from above. The first imaging devices 10-1 to 10-7 are respectively installed at the positions of the seats C1 to C7 in the theater G. Each of the first imaging devices 10-1 to 10-7 is fixed at the positions of the seats C1 to C7. For example, the first imaging device 10-1 is fixed at the position of the seat C1, and the first imaging device 10-2 is fixed at the position of the seat C2. Note that being fixed means not moving in the direction of other seats Ci in the theater G. For example, the first imaging device 10-i may be capable of panning, tilting, such as swinging the neck, and moving up and down.

[0066] The first imaging device 10-i captures an image of the stage S seen from the seat Ci as the first image PC1-i. The imaging range of the first imaging device 10-i may be set, for example, so that the entire stage S is reflected, or may be set so that the face of the performer P is reflected close-up. When it is set so that the face of the performer P is reflected close-up, it is preferable that the first imaging device 10-i is provided with an auxiliary mechanism for remotely operating, for example, panning, tilting, zooming, etc.

[0067] B-2. Image transmission device 30B FIG. 11 is a block diagram showing the configuration of the image transmission device 30B. The image transmission device 30B includes an input device 301, a display device 302, a communication device 303, an image input interface 304, a storage device 305, a processing device 306B, and a bus 307. Since the input device 301, the display device 302, the communication device 303, the image input interface 304, and the bus 307 are the same as those of the image transmission device 30A in the first embodiment, the description thereof is omitted.

[0068] The storage device 305 stores a program PG2. The program PG2 is a program for operating the image transmission device 30B.

[0069] By executing program PG2, the processing device 306B functions as an image acquisition unit 310, a gaze detection unit 312B, a transmission control unit 316B, and a designation reception unit 320. At least one of the image acquisition unit 310, the gaze detection unit 312B, the transmission control unit 316B, and the designation reception unit 320 may be constituted by circuits such as DSP, ASIC, PLD, and FPGA.

[0070] The image acquisition unit 310 functions in the same manner as in the first embodiment.

[0071] The designation reception unit 320 receives from the viewer Vi a designation as to whether the image to be transmitted to the terminal device 40-i is the first image PC1 or the second image PC2. Here, in the first embodiment, one first imaging device 10 moved between a plurality of seats C1 to C7. In contrast, in the second embodiment, the first imaging devices 10-1 to 10-7 are installed at each of the seats C1 to C7, and an image of the stage S seen from an arbitrary seat Ci can always be acquired. By receiving the designation of the image to be transmitted to the terminal device 40-i, the viewer Vi can view the performance from the seat Ci associated with himself / herself.

[0072] FIGS. 12A and 12B are explanatory diagrams showing an example of a viewing screen of an online live event on the terminal device 40-1. In FIGS. 12A and 12B, as an example of the terminal device 40-i, the terminal device 40-1 associated with the seat C1 will be described as an example. The terminal device 40-1 is, for example, a smartphone. For example, as shown in FIG. 10, consider the case where the performer P is facing the seat C4.

[0073] When viewing the second image PC2 on the terminal device 40-1, an image captured from the front of the stage S and showing the entire body of the performer P is displayed on the display 400 of the terminal device 40-1 as shown in FIG. 12A. Also, a first button B1 for image switching is displayed on the display 400. The first button B1 is a display switching button to the first image PC1-1 captured by the first imaging device 10-1 installed at the seat C1 associated with the terminal device 40-1.

[0074] When viewer V1 presses the first button B1, as shown in FIG. 12B, a first image PC1-1 is displayed on the display 400. The first image PC1-1 shows an image of the performer P looking in the direction of seat C4 taken from seat C1. Also, a second button B2 for image switching is displayed on the display 400. The second button B2 is a display switching button to the second image PC2 captured by the second imaging device 20. When viewer V1 presses the second button B2, a second image PC2 as shown in FIG. 12A is displayed on the display 400.

[0075] The line-of-sight detection unit 312B detects the direction L of the line of sight of the performer P. The line-of-sight detection unit 312B is an example of a detection unit. In the second embodiment, the line-of-sight detection unit 312B detects the direction L of the line of sight of the performer P based on the first image PC1. The line-of-sight detection unit 312B performs image analysis on each of the first images PC1-1 to PC1-7 to detect the position of the performer P on the stage S and the orientation of the face of the performer P. The line-of-sight detection unit 31, for example, estimates that the line of sight of the performer P is directed toward the direction of the first imaging device 10-i that has captured an image of the performer P with the face facing forward. Alternatively, the line-of-sight detection unit 312B may further detect the position of the black eyes of the performer P from the first images PC1-1 to PC1-7 to more precisely detect the direction L of the line of sight of the performer P.

[0076] Note that also in the second embodiment, similar to the first embodiment, the direction L of the line of sight of the performer P may be detected based on the second image PC2.

[0077] The transmission control unit 316B transmits an image of the performer P to the terminal devices 40-1 to 40-7. The image of the performer P is the first image PC1 or the second image PC2. The transmission control unit 316B switches the image to be transmitted to the terminal devices 40-1 to 40-7 between the first image PC1 and the second image PC2 based on the designation result received by the designation reception unit 320. That is, the images transmitted to the terminal devices 40-1 to 40-7 are different from each other for each of the terminal devices 40-1 to 40-7.

[0078] During normal times, the transmission control unit 316B transmits the image specified by the viewer Vi from among the first image PC1 or the second image PC2 to the terminal device 40-i. Normal times refer to when the direction L of the gaze of the performer P is not directed at the seat Ci associated with the terminal device 40-i. On the other hand, when the direction L of the gaze of the performer P is directed at the seat Ci corresponding to the terminal device 40-i, the transmission control unit 316B transmits the first image PC1-i to the terminal device 40-i regardless of the designation made by the viewer Vi.

[0079] FIG. 13 is an explanatory diagram schematically showing the switching of the transmission image performed by the transmission control unit 316B. FIG. 13 shows, for example, the transmission image to the terminal device 40-1. Hereinafter, the case where the performer P directs the line of sight to the seat C1 associated with the terminal device 40-1 from time t11 to t12 will be considered.

[0080] For example, "when the second image is specified" in the figure indicates the case where the viewer V1 specifies the second image PC2 immediately before time t11. The transmission control unit 316B transmits the second image PC2 to the terminal device 40-1 until time t11. The transmission control unit 316B transmits the first image PC1-1 to the terminal device 40-1 during the period from time t11 to t12 when the gaze of the performer P is directed at the seat C1. The transmission control unit 316B may transmit an alert (reference numeral N in the figure) before the display image on the terminal device 40-1 is switched to the first image PC1-1. After time t12 when the gaze of the performer P moves away from the seat C1, the transmission control unit 316B returns the image transmitted to the terminal device 40-1 to the second image PC2.

[0081] In addition, "when the first image is specified" in the figure indicates a case where the viewer V1 specifies the first image PC1 immediately before the time t11. The transmission control unit 316B transmits the first image PC1-1 to the terminal device 40-1 at the time t11. Also, the transmission control unit 316B continues to transmit the first image PC1-1 to the terminal device 40-1 during the time period from t11 to t12 when the line of sight of the performer P is directed at the seat C1. Further, the transmission control unit 316B transmits the first image PC1-1 to the terminal device 40-1 even after the time t12 when the line of sight of the performer P has moved away from the seat C1. Note that it is preferable for the transmission control unit 316B to transmit an alert to the terminal device 40-1 before the time t11 when the line of sight of the performer P is directed at the seat C1 (reference numeral N in the figure). Therefore, the viewer V1 can recognize that the line of sight of the performer P is directed at the seat C1 and can prevent missing the image of the camera line of sight.

[0082] B-3. Operation of the processing device 306B FIG. 14 and FIG. 14B are flowcharts showing the operation of the processing device 306B. In the flowcharts of FIG. 14 and FIG. 14B, for convenience of explanation, attention is paid to the image transmission to one terminal device 40-i. The processing device 306B performs the following processing in parallel for each of the terminal devices 40-1 to 40-7.

[0083] The processing device 306B functions as an image acquisition unit 310 and acquires the first image PC1 from the first imaging device 10 and the second image PC2 from the second imaging device 20, respectively (step S200). The acquisition of the first image PC1 and the second image PC2 is continuously performed during the subsequent processing.

[0084] The processing device 306B functions as a designation reception unit 320 and receives a designation as to whether the image to be transmitted from the viewer Vi to the terminal device 40-i is the first image PC1 or the second image PC2 (step S202). When the first image PC1 is selected (step S204: YES), the processing device 306B functions as a transmission control unit 316B and transmits the first image PC1 to the terminal device 40-i (step S206). Also, when the first image PC1 is not selected (step S204: NO, that is, when the second image PC2 is selected), the processing device 306B functions as a transmission control unit 316B and transmits the second image PC2 to the terminal device 40 (step S208).

[0085] The processing device 306B functions as a gaze detection unit 312B and detects the direction L of the gaze of the performer P based on the first image PC1 (step S210). The processing device 306B functions as a transmission control unit 316B and determines whether there is a seat Ci on the extension line of the gaze of the performer P, that is, whether the gaze of the performer P is directed at the seat Ci (step S212). When the gaze of the performer P is not directed at the seat Ci (step S212: NO), the processing device 306B returns the process to step S202.

[0086] When the gaze of the performer P is directed at the seat Ci (step S212: YES), the processing device 306B functions as a transmission control unit 316B and determines whether the image being transmitted to the terminal device 40-i corresponding to the seat Ci is the second image PC2 (step S214). When the image being transmitted to the terminal device 40-i is the second image PC2 (step S214: YES), the processing device 306B transmits the first image PC1 to the terminal device 40-i (step S216). That is, the transmitted image to the terminal device 40-i is switched from the second image PC2 to the first image PC1. At this time, the processing device 306B may transmit an alert to the terminal device 40-i.

[0087] The processing device 306B waits until the line of sight of the performer P deviates from the seat Ci (step S218: NO). During this period, the first image PC1 is transmitted to the terminal device 40-i. When the line of sight of the performer P deviates from the seat Ci (step S218: YES), the processing device 306B functions as a transmission control unit 316B and transmits the second image PC2 to the terminal device 40-i (step S220). Thereafter, the processing device 306B returns the process to step S200.

[0088] Also, in step S214, when the image being transmitted to the terminal device 40-i is not the second image PC2 (step S214: NO), that is, when it is the first image PC1, the processing device 306B transmits an alert to the terminal device 40-i (step S222). Thereafter, the processing device 306B returns the process to step S200.

[0089] As described above, in the second embodiment, the first imaging devices 10-1 to 10-7 are arranged corresponding to the respective seats C1 to C7. Therefore, the viewer Vi can view the captured image (first image PC1-i) from the seat Ci associated with himself / herself at an arbitrary timing, and can obtain a viewing experience close to an actual live performance. Also, due to the arrangement of the first imaging devices 10-1 to 10-7 corresponding to the respective seats C1 to C7, it is possible to more reliably capture the image toward which the line of sight of the performer P is directed. Therefore, the satisfaction of the viewer Vi with respect to the online live performance is improved.

[0090] Also, in the second embodiment, the direction L of the line of sight of the performer P is detected based on the first images PC1-1 to PC1-7 captured by the first imaging devices 10-1 to 10-7. Therefore, compared with tracking the line of sight of the performer P based on the image captured by a single imaging device, the direction L of the line of sight of the performer P is detected with higher accuracy.

[0091] C. Modification The modification modes in the above-described embodiments are shown below. Two or more modes arbitrarily selected from the following modification modes may be appropriately combined within a range that does not conflict with each other.

[0092] C1. First Variant Example The first variant example adopts the configuration of the second embodiment shown in FIGS. 9 to 11. In the above-described embodiment, the case where there is one performer P has been described. On the other hand, when the performer P is a member of a band or a group, etc., there may be a case where a plurality of performers P go up on the stage S. In this case, depending on the number of performers P, it may frequently happen that the line of sight of any one of the performers P is directed toward the seat Ci. Also, there may be a case where the viewer Vi is interested only in a specific member among a plurality of members. That is, if the above-described control is performed for all of the plurality of performers P, it may conversely become difficult to watch the actual performance or the satisfaction of the viewer Vi may decrease.

[0093] For this reason, when there are a plurality of performers P on the stage S, the viewer Vi may be able to specify the performer P who switches to the first image PC1 when looking at the seat Ci. For example, the first performer P1 and the second performer P2 who performs together with the first performer P1 are on the stage S. The image transmission device 30B shown in FIG. 11 receives the designation of the first performer P1 or the second performer P2 from the viewer Vi by the designation reception unit 320. The designation reception unit 320 is an example of a reception unit. When the first performer P1 is designated from the viewer Vi, the first imaging device 10-i tracks the first performer P1 and captures a first image in which the first performer P1 appears. Also, when the second performer P2 is designated from the viewer Vi, the first imaging device 10-i tracks the second performer P2 and captures a first image in which the second performer P2 appears.

[0094] The line-of-sight detection unit 312B detects the direction L of the line of sight of the first performer P1 and the direction L of the line of sight of the second performer. The transmission control unit 316B transmits the first image PC1 to the terminal device 40-i based on the fact that the first performer P1 is designated from the viewer Vi and the line of sight of the first performer P1 is directed toward the seat Ci. The transmission control unit 316B transmits the first image PC1 to the terminal device 40-i based on the fact that the second performer P2 is designated from the viewer Vi and the line of sight of the second performer P2 is directed toward the virtual seat Ci.

[0095] In other words, when the speaker P designated by the viewer Vi (hereinafter referred to as the designated speaker PX) looks at the seat Ci, the transmission control unit 316B switches the image to be transmitted to the terminal device 40-i to the first image PC1. On the other hand, when a speaker P other than the designated speaker PX looks at the seat Ci, the transmission control unit 316B does not switch the image to be transmitted to the terminal device 40-i.

[0096] According to the first modification example, the switching to the first image PC1 is performed only when the speaker P designated by the viewer Vi directs his / her line of sight to the seat Ci. Therefore, it is possible to prevent the image transmitted to the terminal device 40-i from being frequently switched. In addition, the viewer Vi can watch without missing the actual performance of the speaker P he / she wants to focus on.

[0097] Note that in the first modification example, the configuration of the second embodiment is adopted. However, for example, when there is one viewer Vi, or when all viewers Vi designate the same speaker P (for example, the first speaker P1), the configuration of the first embodiment may be adopted.

[0098] C2. Second modification example In the above-described embodiment, the transmission control unit 316A switches the image to be transmitted to the terminal device 40-i to the first image PC1 at the timing when the line of sight of the speaker P faces the seat Ci. However, not limited to this, the image may be switched to the first image PC1 before the line of sight of the speaker P faces the seat Ci.

[0099] FIG. 15 is an explanatory diagram schematically showing a second modification. In FIG. 15, illustration of the first imaging device 10 and the second imaging device 20 is omitted. Image switching ranges H1 to H7 are set for the seats C1 to C7, respectively. The widths of the image switching ranges H1 to H7 are set wider left and right than the ranges of the seats C1 to C7. For example, when the line of sight of the performer P enters the image switching range H1, the first image PC1 is transmitted to the terminal device 40-1. As described above, since the widths of the image switching ranges H1 to H7 are set wider than the ranges of the seats C1 to C7, the transmission of the first image PC1 to the terminal device 40-1 starts at a timing earlier than when the performer P faces the seat C1. Also, the transmission of the first image PC1 to the terminal device 40-1 ends after a certain time has elapsed since the line of sight of the performer P leaves the seat C1. For example, as shown in FIG. 15, when the performer P is looking at the overlapping range of the image switching ranges H3 and H4, the first image PC1 is transmitted to the terminal device 40-3 corresponding to the seat C3 and the terminal device 40-4 corresponding to the seat C4.

[0100] According to the second modification, the viewer Vi can see the state in which the performer P looks at the seat Ci corresponding to himself / herself, and can more realistically experience the feeling of making eye contact with the performer P.

[0101] C3. Third Modification In the above-described embodiment, the direction L of the line of sight of the performer P is detected based on the first image PC1 captured by the first imaging device 10 or the second image PC2 captured by the second imaging device 20. Not limited to this, a third imaging device for capturing an image for detecting the direction L of the line of sight of the performer P may be provided. The third imaging device may be, for example, an imaging device arranged at a position closer to the stage S than the first imaging device 10. Also, the third imaging device may be, for example, an imaging device mounted on XR (Extended Reality) glasses worn by the performer P during the actual performance. Also, the third imaging device may be, for example, an imaging device mounted on an eye tracking device specialized for tracking the line of sight of the performer P.

[0102] According to such a modification example, since the direction L of the line of sight of the performer P is detected based on an image specialized for detecting the direction L of the line of sight of the performer P, the direction L of the line of sight of the performer P can be detected with higher accuracy. Further, since the first imaging device 10 or the second imaging device 20 does not have to always image the performer P, for example, it is possible to image others in accordance with the performance during the actual performance, and the degree of freedom of the visual expression of the online live can be improved.

[0103] Further, instead of detecting the direction L of the line of sight of the performer P from an image, for example, the direction L of the line of sight of the performer P may be calculated based on the detection value of a sensor attached to the head of the performer P. As the sensor in this case, for example, an inertial measurement unit (IMU) or a geomagnetic sensor can be used.

[0104] According to such a modification example, the processing load on the processing device 306A or 306B can be reduced as compared with performing image processing.

[0105] D: Others (1) Each function illustrated in FIGS. 5 and 11 is realized by an arbitrary combination of hardware and software. The method of realizing each function is not particularly limited. Each function may be realized using one physically or logically combined device, or may be realized using a device configured by directly or indirectly (for example, using wired, wireless, etc.) connecting two or more physically or logically separated devices. Each function may be realized by combining software with the above one device or the above plurality of devices.

[0106] (2) In this specification, the term "device" may be read as other terms such as a circuit, a device, or a unit.

[0107] (3) In each of the first embodiment, the second embodiment, and the first to third modification examples, the storage device 305 may be configured by at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray (registered trademark) disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy (registered trademark) disk, a magnetic strip, etc. Further, the program may be transmitted from a network via a telecommunication line.

[0108] (4) Each of the first embodiment, the second embodiment, and the first to third modification examples may be applied to at least one of systems using LTE (Long Term Evolution), LTE-A (LTE-Advanced), SUPER 3G, IMT-Advanced, 4G (4th generation mobile communication system), 5G (5th generation mobile communication system), 6th generation mobile communication system (6G), xth generation mobile communication system (xG) (x is, for example, an integer or a decimal), FRA (Future Radio Access), NR (new Radio), New radio access (NX), Future generation radio access (FX), W-CDMA (registered trademark), GSM (registered trademark), CDMA2000, UMB (Ultra Mobile Broadband), IEEE 802.11 (Wi-Fi (registered trademark)), IEEE 802.16 (WiMAX (registered trademark)), IEEE 802.20, UWB (Ultra-WideBand), Bluetooth (registered trademark), other appropriate systems, and next-generation systems extended, modified, created, and defined based on these. Also, a combination of multiple systems (for example, a combination of at least one of LTE and LTE-A and 5G, etc.) may be applied.

[0109] (5) The processing procedures, sequences, flowcharts, etc. exemplified in each of the first embodiment, the second embodiment, and the first to third modification examples may be reordered as long as there is no contradiction. For example, for the methods described in this specification, the elements of various steps are presented in an exemplary order and are not limited to the specific order presented.

[0110] (6) In each of the first embodiment, the second embodiment, and the first to third modification examples, the input / output information etc. may be stored in a specific location (for example, a memory), or may be managed using a management table. The input / output information etc. may be overwritten, updated, or appended. The output information etc. may be deleted. The input information etc. may be transmitted to other devices.

[0111] (7) In each of the first embodiment, the second embodiment, and the first to third modification examples, the determination may be made based on a value represented by 1 bit (0 or 1), may be made based on a truth value (Boolean: true or false), or may be made based on a numerical comparison (for example, comparison with a predetermined value).

[0112] (8) The program exemplified in each of the first embodiment, the second embodiment, and the first to third modification examples should be broadly interpreted to mean an instruction, an instruction set, code, a code segment, program code, a subprogram, a software module, an application, a software application, a software package, a routine, a subroutine, an object, an executable file, an execution thread, a procedure, a function, etc., regardless of whether it is called a software, firmware, middleware, microcode, or a hardware description language, or by another name. Also, software, instructions, etc. may be transmitted and received via a transmission medium. For example, when software is transmitted from a website, a server, or another remote source using at least one of wired technologies (such as coaxial cables, optical fiber cables, twisted pairs, and digital subscriber lines (DSL)) and wireless technologies (such as infrared rays, microwaves), at least one of these wired technologies and wireless technologies is included within the definition of the transmission medium.

[0113] (9) The information etc. described in each of the first embodiment, the second embodiment, and the first to third modification examples may be represented using any of various different technologies. For example, the data, information, etc. that may be referred to throughout the above description may be represented by voltage, current, electromagnetic wave, magnetic field, magnetic particle, optical field, photon, or any combination thereof. In addition, the terms described in this specification and the terms necessary for understanding this specification may be replaced with terms having the same or similar meanings.

[0114] (10) In each of the first embodiment, the second embodiment, and the first to third modification examples, the terms "system" and "network" are used interchangeably.

[0115] (11) In each of the first embodiment, the second embodiment, and the first to third modification examples, the terminal device 40 may be a mobile station. A mobile station may be referred to by those skilled in the art using terms such as subscriber station, mobile unit, subscriber unit, wireless unit, remote unit, mobile device, wireless device, wireless communication device, remote device, mobile subscriber station, access terminal, mobile terminal, wireless terminal, remote terminal, handset, user agent, mobile client, client, or some other suitable terms.

[0116] (12) The mobile station may also be referred to as a transmission device, a reception device, a communication device, etc. The mobile station may be a device mounted on a mobile body, or the mobile body itself. The mobile body means a movable object. The moving speed of the mobile body is arbitrary. The mobile body can stop. The mobile body includes, for example, vehicles, transport vehicles, automobiles, motorcycles, bicycles, connected cars, excavators, bulldozers, wheel loaders, dump trucks, forklifts, trains, buses, rickshaws, human-powered vehicles, ships (ship and other watercraft), airplanes, rockets, artificial satellites, drones (registered trademark), multicopters, quadcopters, balloons, and objects mounted on these, and is not limited thereto. The mobile body may be a mobile body that autonomously travels based on an operation command. The mobile body may be a vehicle (e.g., a car, an airplane, etc.), a mobile body that moves without a driver (e.g., a drone, an autonomous driving vehicle, etc.), or a robot (humanoid or non-humanoid). The mobile station also includes devices that do not necessarily move during communication operations. For example, the mobile station may be an IoT (Internet of Things) device such as a sensor.

[0117] (13) In each of the first embodiment, the second embodiment, and the first to third modification examples, the term "determining" may encompass a wide variety of operations. "Determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up / searching / inquiring (e.g., looking up in a table, database, or another data structure), and considering something as having "determined" what has been ascertained. Also, "determining" may include considering something as having "judged" / "determined" what has been received (e.g., receiving information), transmitted (e.g., transmitting information), input, output, accessed (e.g., accessing data in a memory), etc. Further, "determining" may include considering something as having "determined" what has been resolved, selected, chosen, established, compared, etc. That is, "determining" may include considering that some operation has been "determined". Also, "determining" may be replaced by "assuming", "expecting", "considering", etc.

[0118] (14) In each of the first embodiment, the second embodiment, and the first to third modification examples, the term "connected", or any variation thereof, means any direct or indirect connection or coupling between two or more elements, and can include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements can be physical, logical, or a combination thereof. For example, "connected" may be read as "accessed". As used in this disclosure, two elements can be considered to be "connected" or "coupled" to each other using at least one of one or more wires, cables, and printed electrical connections, and, as some non-limiting and non-inclusive examples, electromagnetic energy having wavelengths in the radio frequency region, microwave region, and optical (both visible and invisible) region.

[0119] (15) In each of the first embodiment, the second embodiment, and the first to third modification examples, the description "based on" does not mean "only based on" unless otherwise specified. In other words, the description "based on" means both "only based on" and "at least based on".

[0120] (16) Any reference to an element using designations such as "first" and "second" used in this specification does not generally limit the quantity or order of those elements. These designations can be used in this specification as a convenient way to distinguish between two or more elements. Thus, a reference to a first and a second element does not mean that only two elements can be employed or that the first element must precede the second element in any form.

[0121] (17) In each of the first embodiment, the second embodiment, and the first to third modification examples, when the terms "include", "including", and their variants are used in this specification or the claims, these terms are intended to be inclusive, similar to the term "comprising". Further, the term "or" used in this specification or the claims is intended not to be an exclusive disjunction.

[0122] (18) Throughout this application, for example, when articles are added by translation, such as a, an, and the in English, this disclosure may include that the nouns following these articles are in the plural form.

[0123] (19) It is obvious to those skilled in the art that the present invention is not limited to the embodiments described in this specification. The present invention can be implemented in modified and changed forms without departing from the spirit and scope of the present invention determined based on the description of the claims. Therefore, the description in this specification is for illustrative purposes and has no restrictive meaning for the present invention. Also, a plurality of aspects selected from the aspects illustrated in this specification may be combined.

Explanation of Reference Numerals

[0124] 1, 2... Image transmission system, 10, 10-1 to 10-7 (10-i)... First imaging device, 20... Second imaging device, 30A, 30B... Image transmission device, 40-1 to 40-7 (40-i)... Terminal device, 50... Moving mechanism, 301... Input device, 302... Display device, 303... Communication device, 304... Image input interface, 305... Storage device, 306A, 306B... Processing device, 307... Bus, 310... Image acquisition unit, 312A, 312B... Line-of-sight detection unit, 314... Movement control unit, 316A, 316B... Transmission control unit, 320... Designation reception unit, C1 to C7 (Ci)... Seats, G... Theater, L... Direction of line of sight, N... Communication network, P... Performer, PC1... First image, PC2... Second image, S... Stage, V1 to V7 (Vi)... Viewers.

Claims

1. A first imaging device that captures a first image of a performer in the real space from a position of a virtual first seat set in the real space; A second imaging device that captures a second image including the performer from a position different from the position of the first imaging device; A transmission control unit that transmits the second image to a terminal device of a viewer associated with the virtual first seat; A detection unit that detects the direction of the performer's line of sight, and The transmission control unit transmits the first image to the terminal device based on the fact that the direction of the performer's line of sight is directed toward the virtual first seat. An image transmission system.

2. A moving mechanism that moves the first imaging device; and A movement control unit that controls the moving mechanism, and The real space has a virtual second seat at a position different from the position of the virtual first seat, The movement control unit moves the first imaging device to the moving mechanism between the virtual first seat and the virtual second seat based on the direction of the performer's line of sight detected by the detection unit. The image transmission system according to claim 1.

3. The detection unit detects the direction of the performer's line of sight based on the second image. The image transmission system according to claim 2.

4. The first imaging device is fixed at the position of the virtual first seat. The image transmission system according to claim 1.

5. The detection unit detects the performer's line of sight based on the first image. The image transmission system according to claim 4.

6. The transmission control unit transmits an alert to the terminal device substantially at the same time as the time when the first image is transmitted to the terminal device, or before the first image is transmitted to the terminal device. The image transmission system according to any one of claims 1 to 5.

7. Further comprising a reception unit, The performer is either a first performer or a second performer who performs in concert with the first performer, The reception unit receives a designation of the first performer or the second performer from the viewer, The first imaging device, When the first performer is designated by the viewer, captures the first image in which the first performer appears, When the second performer is designated by the viewer, captures the first image in which the second performer appears, The detection unit detects the line of sight of the first performer and the line of sight of the second performer, The transmission control unit, Based on the fact that the first performer is designated by the viewer and the line of sight of the first performer is directed towards the virtual first seat, the first image is transmitted to the terminal device. Based on the fact that the second performer is designated by the viewer and the line of sight of the second performer is directed towards the virtual first seat, the first image is transmitted to the terminal device. The image transmission system according to any one of claims 1 to 6.

8. A first acquisition unit that acquires a first image of a performer in the real space, which is captured by a first imaging device from the position of a virtual first seat set in the real space. A second acquisition unit that acquires a second image including the performer, which is captured by a second imaging device from a position different from that of the first imaging device. A transmission control unit that transmits the second image to a terminal device of a viewer associated with the virtual first seat. A detection unit that detects the direction of the line of sight of the performer, and is provided with: The transmission control unit transmits the first image to the terminal device based on the fact that the line of sight of the performer is directed towards the virtual first seat. Image transmission device.

Images