Information processing apparatus, information processing method, information processing program, terminal device, control method for terminal device, and control program

Abstract

This information processing device is provided with a control unit that: determines, on the basis of first identification information received from a terminal device and second identification information received from an imaging device, whether or not the terminal device and the imaging device are associated with each other; and sets, on the basis of the determined result, the terminal device as a transmission destination of a return video.

Description

Technical Field

[0001] The present technology relates to an information processing apparatus, an information processing method, an information processing program, a terminal device, a control method for a terminal device, and a control program.

Background Art

[0002] In an imaging device that performs imaging for applications such as live relay reporting and real-time streaming, there is a desire to check how the captured video is currently being broadcast and distributed.

[0003] Therefore, for example, Patent Document 1 discloses a technique of distributing a currently broadcast video as a return video to a terminal possessed by a reporter (Patent Document 1).

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, there was a risk of distributing the return video to a terminal not associated with the imaging device used by the reporter.

[0006] The present technology has been made in view of such points, and an object thereof is to provide an information processing apparatus, an information processing method, an information processing program, a terminal device, a control method for a terminal device, and a control program that enable the distribution of a return video to a terminal associated with an imaging device.

Means for Solving the Problems

[0007] In order to solve the above-described problems, a first technique is an information processing apparatus including a control unit that determines whether a terminal device and an imaging device are associated based on first identification information received from the terminal device and second identification information received from the imaging device, and sets the terminal device as a transmission target of a return video based on the determination result.

[0008] Further, a second technique is an information processing method that determines whether a terminal device and an imaging device are associated based on first identification information received from the terminal device and second identification information received from the imaging device, and transmits a return video to the terminal device based on the determination result.

[0009] Further, a third technique is an information processing program that determines whether a terminal device and an imaging device are associated based on first identification information received from the terminal device and second identification information received from the imaging device, and transmits a return video to the terminal device based on the determination result.

[0010] In order to solve the above-described problems, a fourth technique transmits first identification information associating imaging device identification information for identifying an external imaging device received from the external imaging device and terminal device identification information for identifying itself, and an external information processing device determines whether the terminal device and External the imaging device are associated based on the first identification information and second identification information received from the external imaging device, and includes a control unit that controls to receive a transmitted return video based on the determination result.

[0011] Further, a fifth technique transmits first identification information associating imaging device identification information for identifying an external imaging device received from the external imaging device and terminal device identification information for identifying itself, and an external information processing device determines whether the terminal device and External the imaging device are associated based on the first identification information and second identification information received from the external imaging device, and is a control method for the terminal device that controls to receive a transmitted return video based on the determination result.

[0012] Furthermore, the sixth technology transmits first identification information associating imaging device identification information for identifying an external imaging device received from an external imaging device with terminal device identification information for identifying itself, and an external information processing device determines whether a terminal device is External associated with an imaging device based on the first identification information and second identification information received from the external imaging device, and controls a computer to execute a control program for receiving a return video transmitted based on the determination result.

Brief Description of the Drawings

[0013]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

[0014] Hereinafter, embodiments of the present technology will be described with reference to the drawings. The description will be made in the following order. <1. Embodiment> [1-1. Configuration of Information Processing System 10] [1-2. Configuration of Imaging Device 100] [1-3. Configuration of Terminal Device 200] [1-4. Configuration of Information Processing Device 300] [1-5. Processing in Information Processing System 10] <2. Application Examples of the Present Technology> [2-1. Configuration of MOJO] [2-2. First Aspect of Coverage in MOJO] [2-3. Second Aspect of Coverage in MOJO] <3. Modification Examples>

[0015] <1. Embodiment> [1-1. Configuration of Information Processing System 10] First, the configuration of the information processing system 10 will be described with reference to FIG. 1. The information processing system 10 is composed of an imaging device 100, a terminal device 200, and an information processing device 300. The terminal device 200 and the information processing device 300, and the imaging device 100 and the information processing device 300 are connected via a network such as the Internet. Further, the terminal device 200 and the imaging device 100 are connected and associated with each other by, for example, short-range wireless communication.

[0016] The imaging device 100 captures video and images and transmits them to the information processing device 300 as video data and image data. In this embodiment, the case of transmitting video data from the imaging device 100 will be described as an example.

[0017] The terminal device 200 receives return video data, which is video being broadcast, from the information processing device 300 and displays the return video data for the user (reporter) to view.

[0018] The information processing apparatus 300 receives the video data transmitted from the imaging apparatus 100, performs predetermined broadcast editing processing on the video data, and transmits the processed data to a switch of an external video processing apparatus that generates return video data. Further, the information processing apparatus 300 transmits the return video data generated by the external video processing apparatus and transmitted from the switch to the terminal device 200. Note that both the video data and the return video data may or may not include audio data.

[0019] This information processing system 10 is used, for example, in a system that performs a broadcast using video captured by the imaging apparatus 100. When the video captured by the imaging apparatus 100 is transmitted to the information processing apparatus 300 and the video data is being broadcast, the video data processed for broadcast is transmitted to the terminal device 200 as return video data.

[0020] [1-2. Configuration of Imaging Apparatus 100] Next, the configuration of the imaging apparatus 100 will be described with reference to the block diagram of FIG. 2. The imaging apparatus 100 includes a control unit 101, an optical imaging system 102, a lens drive driver 103, an imaging element 104, an image signal processing unit 105, an image memory 106, a storage unit 107, a communication unit 108, an input unit 109, a display unit 110, and a microphone 111.

[0021] The control unit 101 is composed of a CPU (Central Processing Unit), a RAM (Random Access Memory), a ROM (Read Only Memory), and the like. The CPU executes various processes according to the programs stored in the ROM and issues commands to control the entire imaging device 100 and each part thereof. In addition, the control unit 101 stores imaging device identification information for identifying the imaging device 100, which is required when connecting to the terminal device 200 and the information processing device 300. The control unit 101 controls the connection to the terminal device 200 and the information processing device 300 via the communication unit 108. Furthermore, the control unit 101 stores imaging device function information indicating the functions of the imaging device 100, and the imaging device function information is transmitted to the information processing device 300 via the communication unit 108 after the connection between the imaging device 100 and the information processing device 300 is established. The functions of the imaging device 100 indicated by the imaging device function information include a video data transmission function, a video display function, a voice call function, a tally signal presentation function, and the like.

[0022] The optical imaging system 102 is composed of an imaging lens for condensing light from a subject onto the imaging element 104, a drive mechanism for moving the imaging lens for focusing and zooming, a shutter mechanism, an iris mechanism, and the like. These are driven based on control signals from the control unit 101 and the lens drive driver 103. The optical image of the subject obtained through the optical imaging system 102 is formed on the imaging element 104.

[0023] The lens drive driver 103 is composed of, for example, a microcomputer, and performs autofocusing so as to focus on a target subject by moving the imaging lens along the optical axis direction by a predetermined amount according to the control of the control unit 101. In addition, according to the control from the control unit 101, it controls the operations of the drive mechanism, shutter mechanism, iris mechanism, etc. of the optical imaging system 102. Thereby, adjustments such as the exposure time (shutter speed) and the aperture value (F value) are made.

[0024] The imaging device 104 photoelectrically converts the incident light from the subject obtained through the imaging lens, converts it into an electric charge amount, and outputs an imaging signal. Then, the imaging device 104 outputs a pixel signal to the image signal processing unit 105. As the imaging device 104, a CCD (Charge Coupled Device), a CMOS (Complementary Metal Oxide Semiconductor), etc. are used.

[0025] The image signal processing unit 105 performs sample hold, AGC (Auto Gain Control) processing, A / D (Analog / Digital) conversion, etc. on the imaging signal output from the imaging device 104 by CDS (Correlated Double Sampling) processing to maintain a good S / N (Signal / Noise) ratio and create an image signal. In addition, the image signal processing unit 105 performs recording processing on the recording image signal and display processing on the display image signal.

[0026] The image memory 106 is a buffer memory composed of a volatile memory, for example, a DRAM (Dynamic Random Access Memory). The image memory 106 temporarily stores the image data that has been subjected to predetermined processing by the image signal processing unit 105.

[0027] The storage unit 107 is a large-capacity storage medium such as a hard disk or a flash memory, for example. The image signal subjected to recording image processing by the image signal processing unit 105 is stored in a compressed state or an uncompressed state based on a standard such as JPEG (Joint Photographic Experts Group). Also, EXIF (Exchangeable Image File Format) data including additional information such as information about the stored image, imaging position information indicating the imaging position, and imaging time information indicating the imaging date and time is stored in association with the image.

[0028] The communication unit 108 is a communication module for transmitting and receiving various data and information with the terminal device 200 and the information processing device 300. The communication can use any method that can connect to the Internet and other devices, such as wireless LAN (Local Area Network), WAN (Wide Area Network), WiFi (Wireless Fidelity), 4G (the fourth-generation mobile communication system), 5G (the fifth-generation mobile communication system), broadband, etc. Note that the communication between the imaging device 100 and the terminal device 200 can also be a short-range wireless communication such as NFC (Near Field Communication) or ZigBee (registered trademark), or a tethering connection such as WiFi tethering, USB (Universal Serial Bus) tethering, or Bluetooth (registered trademark) tethering.

[0029] The input unit 109 is for the user to give various instructions to the imaging device 100. When an input is made by the user to the input unit 109, a control signal corresponding to the input is generated and supplied to the control unit 101. Then, the control unit 101 performs various processes corresponding to the control signal. Examples of the input unit 204 include a shutter button for shutter input, physical buttons for various operations, and a touch panel, a touch screen integrated with the display as the display unit 110, etc.

[0030] The display unit 110 is a display device such as a display for displaying a through-image which is a display image signal subjected to display processing by the image signal processing unit 105, an image / video stored in the storage unit 107 after being subjected to recording image processing by the image signal processing unit 105, a GUI (Graphical User Interface), etc.

[0031] The microphone 111 is for recording sound during video shooting.

[0032] The imaging device 100 is configured as described above. The imaging device 100 may be a device specialized for camera functions such as a digital camera, a single-lens reflex camera, a camcorder, a professional-use camera, or other imaging equipment, or it may also be a smartphone, a tablet terminal, a wearable device, a portable game machine, etc. that has a camera function.

[0033] [1-3. Configuration of the terminal device 200] Next, the configuration of the terminal device 200 will be described with reference to FIG. 3. The terminal device 200 includes a control unit 201, a storage unit 202, a communication unit 203, an input unit 204, a display unit 205, a microphone 206, a speaker 207, and a camera 208.

[0034] The control unit 201 is composed of a CPU, a RAM, a ROM, etc. The CPU executes various processes according to the programs stored in the ROM and issues commands to control the entire terminal device 200 and each part thereof. In addition, the control unit 201 stores the terminal device identification information required for connecting to the imaging device 100 and the information processing device 300, and the control unit 201 controls the connection to the imaging device 100 and the information processing device 300 via the communication unit 203.

[0035] Also, the control unit 201 stores the terminal device function information indicating the functions of the terminal device 200, and the terminal device function information is transmitted to the information processing device 300 via the communication unit 203 after the connection between the terminal device 200 and the information processing device 300 is established. The functions of the terminal device 200 indicated by the terminal device function information include a return video data display function, a video display function, a voice call function, a tally signal presentation function, etc. The return video display function information in the claims is included in the terminal device function information. In addition, when the control unit 201 receives the imaging device identification information from the imaging device 100, the control unit 201 stores the imaging device identification information in itself. Furthermore, the control unit 201 controls the reception of the return video data and the tally signal transmitted from the information processing device 300 via the communication unit 203.

[0036] The storage unit 202 is a large-capacity storage medium such as a hard disk or a flash memory. Various applications and data used in the terminal device 200 are stored in the storage unit 202.

[0037] The communication unit 203 is a communication module for transmitting and receiving data and various information between the terminal device 200 and the information processing device 300. The communication can use any method that can connect to the Internet and other devices, such as wireless LAN, WAN, WiFi, 4G, 5G, and broadband. Note that the communication between the terminal device 200 and the imaging device 100 may also be short-range wireless communication such as NFC or ZigBee (registered trademark), or tethering connections such as WiFi tethering, USB tethering, or Bluetooth (registered trademark) tethering.

[0038] The input unit 204 is for the user to input information, messages, and various instructions to the terminal device 200. When an input is made from the user to the input unit 204, a control signal corresponding to the input is generated and supplied to the control unit 201. Then, the control unit 201 performs various processes corresponding to the control signal. In addition to physical buttons, the input unit 204 includes a touch panel and a touch screen integrally configured with the display as the display unit 205.

[0039] The display unit 205 is a display device such as a display for displaying videos, images, GUIs, etc. The return video data transmitted from the information processing device 300 is displayed on the display unit 205. Thereby, the user can view the return video.

[0040] The microphone 206 is used for the user to input voice to the terminal device 200. The microphone 206 is also used as an audio input device in voice calls and video calls between the terminal device 200 and the information processing device 300.

[0041] Speaker 207 is an audio output device that outputs voice messages, the audio of videos, etc. Speaker 207 is also used as an audio output device in voice calls and video calls between the terminal device 200 and the information processing device 300.

[0042] Camera 208 is composed of a lens, an image sensor, a video signal processing circuit, etc., and is for shooting videos and images.

[0043] The terminal device 200 is configured as described above. Specific examples of the terminal device 200 include, for example, smartphones, tablet terminals, personal computers, wearable devices, portable game machines, etc. The configuration for performing the processing in the terminal device 200 may be realized by a program.

[0044] [1-4. Configuration of Information Processing Device 300] Next, the configuration of the information processing device 300 will be described with reference to the block diagram of FIG. 4. The information processing device 300 is configured to include a control unit 301, a communication unit 302, a data storage unit 303, and an audio communication unit 304.

[0045] The control unit 301 is composed of a CPU, a RAM, a ROM, etc., and the CPU executes various processes according to the programs stored in the ROM and issues commands to control the entire information processing device 300 and each part thereof. Also, the control unit 301 performs matching determination processing between the terminal device 200 and the information processing device 300, and matching processing between the imaging device 100 and the information processing device 300. Details of the matching determination processing will be described later.

[0046] Also, the control unit 301 performs reception processing for receiving information transmitted from the imaging device 100 and / or the terminal device 200 and video data transmitted from the imaging device 100 via the communication unit 302.

[0047] Furthermore, the control unit 301 confirms that the terminal device 200 and the imaging device 100 are paired (associated) based on the imaging device identification information and the terminal device identification information received from the terminal device 200 and the imaging device identification information received from the imaging device 100, and sets the terminal device 100 as the transmission target for the return video data and the tally signal. Then, the control unit 301 performs a transmission process of transmitting the return video data and the tally signal to the terminal device 200 via the communication unit 302.

[0048] The reception process is a process of receiving the imaging device identification information for identifying the imaging device 100 and / or the terminal device identification information for identifying the terminal device 200 and storing them in the control unit 301. Examples of the identification information include the serial number of the device, the name of the model, etc. The identification information may be a device-specific identification number such as IMEI (International Mobile Equipment Identifier), or any information that can identify an individual device, such as ID information dedicated to device identification. Note that the first identification information in the claims includes the terminal device identification information for identifying the terminal device 200 and the imaging device identification information for identifying the imaging device 100, and the second identification information includes the imaging device identification information.

[0049] Also, in the reception process, a process of receiving the imaging device function information indicating the functions of the imaging device 100 transmitted from the imaging device 100 and / or the terminal device function information indicating the functions of the terminal device 200 transmitted from the terminal device 200 and storing them in the control unit 301 is also performed. Examples of the functions of the imaging device 100 indicated by the imaging device function information include a video data transmission function, a video display function, a voice call function, a tally signal presentation function, etc. The imaging device function information is stored in the control unit 101 of the imaging device 100. Examples of the functions of the terminal device 200 indicated by the terminal device function information include a return video data display function, a video display function, a voice call function, a tally signal presentation function, etc. The terminal device function information is stored in the control unit 201 of the terminal device 200.

[0050] Furthermore, in the reception process, the video data transmitted from the imaging device 100 is received. The received video data is transmitted to a video processing device that performs predetermined broadcast editing processing.

[0051] The transmission process is a process of transmitting, via the communication unit 302, the return video data generated by performing predetermined broadcast editing processing on the video data by the video processing device to the terminal device 200.

[0052] Furthermore, in the transmission process, a tally signal is transmitted to the imaging device 100 and / or the terminal device 200. The tally signal is a signal for transmitting information such as which video is currently being broadcast and which imaging device 100 will broadcast the next video (who the cameraman is), and is generated, for example, by an external video processing device and transmitted from the switcher of the video processing device to the communication unit 302.

[0053] The communication unit 302 is a communication module for transmitting and receiving various data and various information between the imaging device 100 and the terminal device 200. Communication methods include wireless LAN, WAN, WiFi, 4G, 5G, etc.

[0054] The data storage unit 303 stores and manages the received video data, the identification information and function information of the imaging device 100 and the terminal device 200, etc. In addition to the video data, the data storage unit 303 may store and manage the return video data in association with the video data. The information processing device 300 may transmit the video data and the return video data stored in the data storage unit 303 to the imaging device 100 and the terminal device 200 as an archive according to needs and requests.

[0055] The voice communication unit 304 performs transmission and reception processing of voice data for making a voice call with the imaging device 100 and / or the terminal device 200 via the communication unit 302.

[0056] The information processing apparatus 300 is configured as described above. The information processing apparatus 300 is composed of a program, and the program may be installed in a server or the like in advance, or may be distributed via download, a storage medium, etc., and the user may install it in a server or the like by himself / herself. Also, the information processing apparatus 300 may be configured on the cloud. Furthermore, the information processing apparatus 300 may be realized not only by a program, but also by combining dedicated devices, circuits, etc. with hardware having its functions.

[0057] As shown in FIG. 5, the information processing apparatus 300 is connected to an external video processing apparatus 400 that performs broadcast editing processing on video data. The video processing apparatus 400 includes at least a control unit 401, a communication unit 402, and a switch 403, and operates, for example, in a broadcasting station 500. Also, a studio 600 is provided in the broadcasting station 500.

[0058] In the studio 600, staff such as producers, directors, assistant directors, cameramen, etc. (hereinafter collectively referred to as staff) of the broadcasting station 500 perform work for broadcasting. Also, the studio 600 includes at least a CCU 601 and a camera 602. The CCU 601 processes and outputs the video captured by the camera 602, and transmits video data and the like. The camera 602 captures a caster or the like who conducts reporting in the studio 600.

[0059] The control unit 401 is composed of a CPU, a RAM, a ROM, etc., and the CPU executes various processes according to the program stored in the ROM and issues commands to control the entire video processing apparatus 400 and each part thereof. Also, the control unit 401 performs broadcast video editing processing. Examples of the editing processing include video splitting and combining, cropping, trimming, editing, captioning, CG synthesis, and effect application. However, the broadcast editing processing is not limited to these and may be any processing performed for broadcasting video data.

[0060] The communication unit 402 is a communication module for transmitting and receiving various data, various information, etc. with an information processing device 300 or the like. Examples of the communication method include wireless LAN, WAN, WiFi, 4G, 5G, etc.

[0061] The switcher 403 is a device that switches or mixes a plurality of input signals. When there are a plurality of imaging devices 100, the video data captured by the plurality of imaging devices 100 and transmitted to the video processing device 400 via the information processing device 300 gathers at the switcher 403. Then, the switcher 403 selects which imaging device 100's video data to use for broadcasting by switching. Further, when the video processing device 400 performs editing processing for broadcasting on the video data to generate return video data, the switcher 403 transmits the return video data to the information processing device 300. The return video data is transmitted to the terminal device 200 via the information processing device 300.

[0062] The video processing device 400 may be configured by hardware, or may be configured by a program, and the program may be installed in a server having the functions of the information processing device 300, or may be installed in another server, or may be configured on the cloud.

[0063] Note that as shown in FIG. 6A, the video processing device 400 may have the functions as the information processing device 300, or as shown in FIG. 6B, the information processing device 300 may have the functions as the video processing device 400. In FIG. 6, the broadcasting station and the studio are omitted, but similar to FIG. 5, the information processing device 300 and the video processing device 400 may operate at the broadcasting station.

[0064] [1-5. Processing in the information processing system 10] Next, a first example of the processing in the information processing system 10 will be described with reference to the sequence diagram of FIG. 7. The first example in FIG. 7 is a case where the connection between the terminal device 200 and the information processing device 300 is established prior to the imaging device 100.

[0065] First, as shown in step S101, pairing (association processing) and information transmission and reception are performed between the imaging device 100 and the terminal device 200. The pairing can be performed, for example, when using NFC, by bringing devices having the NFC function into contact with each other. In the case of various tetherings, pairing is performed by exchanging necessary information such as authentication codes, session keys, SSID (Service Set Identifier), identification information such as serial numbers, etc. between the devices. As a result, the terminal device 200 and the imaging device 100 are paired and short-range wireless communication is established. In this pairing, the imaging device identification information of the imaging device 100 is transmitted to the terminal device 200, and in step S102, the terminal device 200 receives the imaging device identification information. The imaging device identification information received by the terminal device 200 is stored in the control unit 201.

[0066] Next, as shown in step S103, the terminal device 200 makes a connection request to the information processing device 300 and transmits the imaging device identification information, the terminal device identification information, and the terminal device function information to the information processing device 300. The information processing device 300 receives the imaging device identification information, the terminal device identification information, and the terminal device function information in step S104. The terminal device 200 has grasped in advance the information processing device 300 to be connected, and based on this grasp, a connection request is made from the terminal device 200 to the information processing device 300. That the terminal device 200 and the information processing device 300 are connected means that the information processing device 300 has identified the terminal device 200, that is, has received the terminal device identification information.

[0067] Next, as shown in step S105, the information processing device 300 starts a matching determination. The matching determination is a process of confirming whether the imaging device 100 and the terminal device 200 that are connected to the information processing device 300 are paired (in an associated relationship) based on the identification information transmitted from the terminal device 200. As a result, it can be confirmed that the transmission destination of the return video data based on the video data transmitted from the imaging device 100 is the terminal device 200 paired with the imaging device 100.

[0068] The matching determination can be performed by checking whether the imaging device identification information of the imaging device 100 received from the terminal device 200 is the same as the imaging device identification information received from the imaging device 100. The fact that this imaging device identification information is the same is the association condition in the claims. When the imaging device identification information of the imaging device 100 received from the terminal device 200 is the same as the imaging device identification information received from the imaging device 100, the association condition is satisfied.

[0069] In addition, when the imaging device 100 is not connected to the information processing device 300, or when the imaging device 100 has not yet transmitted the imaging device identification information to the information processing device 300, the result is "no matching". When the result of the matching determination is no matching, the result is transmitted to the imaging device 100 and / or the terminal device 200 and notified to the user by display or the like.

[0070] Then, when the connection between the imaging device 100 and the information processing device 300 is established as shown in step S106, next, as shown in step S107, the imaging device 100 transmits the imaging device identification information and the imaging device function information to the information processing device 300. Then, in step S108, the information processing device 300 receives the imaging device identification information and the imaging device function information.

[0071] Next, matching confirmation is performed as shown in step S109. When the imaging device 100 and the terminal device 200 are paired, the result is "matching". Note that the information processing device 300 may recognize the paired terminal device 200 and the imaging device 100 as one device and display them on the UI.

[0072] When the connection between the information processing device 300 and the imaging device 100 is established, next, as shown in step S110, the information processing device 300 issues a video data transmission request to the imaging device 100. Note that the video data transmission request can be issued at any timing as long as the connection between the information processing device 300 and the imaging device 100 is established, and thus the video data transmission request is not limited to the timing described in this embodiment.

[0073] Upon receiving the video data transmission request, the imaging device 100 transmits the video data generated by shooting to the information processing device 300 as shown in step S111. Then, in step S112, the information processing device 300 receives the video data. The transmission and reception of this video data continue as long as the imaging device 100 performs shooting to generate video data and the video data is required in the broadcast. Note that the transmission of the video data may be performed at the stage when the connection between the information processing device 300 and the imaging device 100 is established without a video data transmission request from the information processing device 300.

[0074] Next, in step S113, when the result of the matching determination is "matching exists" and the control unit 301 has received return video display function information (included in the terminal device function information) indicating that the terminal device 200 has a return video display function, the control unit 301 sets the terminal device 200 as the transmission target for the return video data. Receiving the return video display function information indicating that the terminal device 200 has a return video display function satisfies the return video condition in the claims.

[0075] Furthermore, when the result of the matching determination is "matching exists" and the control unit 301 has received tally signal presentation function information indicating that the terminal device 200 has a function of presenting a tally signal, the control unit 301 sets the terminal device 200 as the transmission target for the tally signal. Receiving the tally signal presentation function information indicating that the terminal device 200 has a tally signal presentation function satisfies the tally signal condition in the claims. Since this setting of the transmission target is based on the result of the matching determination, it may be performed at any time after step S109 and before transmitting to the terminal device 200.

[0076] When the return video data is generated by the video processing device 400, next, as shown in step S114, the information processing device 300 transmits the return video data to the terminal device 200. Since it is confirmed that the terminal device 200 paired with the imaging device 100 can be identified by the matching determination, it is confirmed that the transmission destination of the return video data is the terminal device 200. Then, in step S115, the terminal device 200 receives the return video data.

[0077] The transmission and reception of this return video data are continuously performed as long as the imaging device 100 continues to take pictures and generate video data and the return video data is generated. However, the transmission may be stopped when a request to stop the transmission of the return video data is received from the terminal device 200 to the information processing device 300. Note that the transmission of this return video data may be immediately performed regardless of the presence or absence of a request when the video data is processed to generate the return video data, or may be performed after a transmission request for the return video data is received from the terminal device 200.

[0078] In parallel with the transmission of the return video data, as shown in step S116, the information processing device 300 transmits a tally signal to the terminal device 200, and in step S117, the terminal device 200 receives the tally signal. Note that the transmission of the tally signal is not essential and may be performed only when necessary.

[0079] Note that after the connection between the terminal device 200 and the information processing device 300 is established, a voice call connection may be established between the terminal device 200 and the information processing device 300. Thereby, communication, instructions, and communication of intentions by voice call can be performed between the person on the information processing device 300 side and the person using the terminal device 200.

[0080] The process shown in FIG. 7 is performed as described above.

[0081] Next, a second example of the processing in the information processing system 10 will be described with reference to the sequence diagram of FIG. 8. The second example in FIG. 8 is a case where the connection between the imaging device 100 and the information processing device 300 is established prior to the terminal device 200. Note that the pairing and the transmission and reception of the imaging device identification information in steps S101 and S102 are the same as those in FIG. 7.

[0082] When the connection between the imaging device 100 and the information processing device 300 is established as shown in step S201, the imaging device 100 transmits the imaging device identification information and the imaging device function information to the information processing device 300 as shown in step S202, and the information processing device 300 receives them in step S203.

[0083] Next, as shown in step S204, the information processing device 300 starts the matching determination. The matching determination is the same as that described in FIG. 7. If the terminal device 200 is not connected to the information processing device 300 at this time, the result is "no matching".

[0084] Next, as shown in step S205, the information processing device 300 issues a video data transmission request to the imaging device 100 for which the connection is established.

[0085] Next, as shown in step S206, the imaging device 100 that has received the video data transmission request transmits the video data generated by shooting to the information processing device 300. Then, in step S207, the information processing device 300 receives the video data. The transmission and reception of this video data are continued as long as the imaging device 100 performs shooting to continuously generate video data and the video data is required in the broadcast. Note that the transmission of the video data may be performed at the stage where the information processing device 300 and the imaging device 100 are matched without a video data transmission request from the information processing device 300.

[0086] Then, as shown in step S208, the terminal device 200 makes a connection request to the information processing device 300 and transmits imaging device identification information, terminal device identification information, and terminal device function information to the information processing device 300. The information processing device 300 receives the imaging device identification information, terminal device identification information, and terminal device function information in step S209. The terminal device 200 knows in advance the information processing device 300 to be connected, and based on this knowledge, a connection request is made from the terminal device 200 to the information processing device 300. The fact that the terminal device 200 and the information processing device 300 are connected means that the information processing device 300 has identified the terminal device 200, that is, it has received the terminal device identification information.

[0087] Next, as shown in step S210, a matching check is performed. The information processing device 300 confirms that the imaging device identification information received from the terminal device 200 is the same as the imaging device identification information received from the imaging device 100, thereby confirming that the imaging device 100 and the terminal device 200 are paired. If the imaging device 100 and the terminal device 200 are paired, the result is "matching exists".

[0088] Next, in step S211, based on the result of "matching exists" in the above step S109, the control unit 301 sets the terminal device 200 as the target for transmitting return video data. Also, the control unit 301 sets the terminal device 200 as the target for transmitting tally signals.

[0089] Then, as shown in steps S212 to 215, similar to FIG. 7, return video data is transmitted and received between the information processing device 300 and the terminal device 200, and further, tally signals are transmitted and received in parallel with the transmission of the return video data.

[0090] Note that after the connection between the terminal device 200 and the information processing device 300 is established, an audio call connection may be established between the terminal device 200 and the information processing device 300.

[0091] The process shown in FIG. 8 is performed as described above.

[0092] Next, with reference to the sequence diagrams of FIGS. 9 and 10, a third example of the processing in the information processing system 10 will be described. The third example in FIGS. 9 and 10 is a case where there are a plurality of terminal devices and the connection between the terminal devices and the information processing device 300 is established prior to the imaging device 100. In the description of FIGS. 9 and 10, there are two terminal devices, which are the terminal device 200A and the terminal device 200B, respectively. Also, regarding the identification information and the function information, those of the terminal device 200A are the terminal device identification information A and the terminal device function information A, and those of the terminal device 200B are the terminal device identification information B and the terminal device function information B.

[0093] First, as shown in step S301, pairing (association processing) and information transmission and reception are performed between the imaging device 100 and the terminal device 200A, and short-range wireless communication is established by the pairing. In this pairing, the imaging device identification information of the imaging device 100 is transmitted to the terminal device 200A, and in step S302, the terminal device 200A receives the imaging device identification information.

[0094] Next, as shown in step S303, the terminal device 200A makes a connection request to the information processing device 300 and transmits the imaging device identification information, the terminal device identification information A, and the terminal device function information A to the information processing device 300. In step S304, the information processing device 300 receives the imaging device identification information, the terminal device identification information A, and the terminal device function information A.

[0095] Next, as shown in step S305, the information processing device 300 starts a matching determination. If the imaging device 100 is not connected to the information processing device 300 or the imaging device 100 has not yet transmitted the imaging device identification information to the information processing device 300, the result is "no matching".

[0096] Next, as shown in step S306, when pairing is performed between the imaging device 100 and the terminal device 200B and short-range wireless communication is established, the imaging device 100 transmits the imaging device identification information to the terminal device 200B. In step S307, the terminal device 200B receives the imaging device identification information of the imaging device 100.

[0097] When the connection between the terminal device 200B and the information processing device 300 is established as shown in step S308, the terminal device 200B transmits the imaging device identification information of the imaging device 100, its own terminal device identification information B, and the terminal device function information B to the information processing device 300. The information processing device 300 receives the imaging device identification information, the terminal device identification information B, and the terminal device function information B in step S309.

[0098] Next, as shown in step S310, the information processing device 300 performs a matching determination. If the imaging device 100 is not connected to the information processing device 300, or if the imaging device 100 has not yet transmitted the imaging device identification information to the information processing device 300, the result is "no matching".

[0099] When the connection between the imaging device 100 and the information processing device 300 is established as shown in step S311, next, in step S312, the imaging device 100 transmits its own imaging device identification information and the imaging device function information to the information processing device 300. Then, in step S313, the information processing device 300 receives the imaging device identification information and the imaging device function information.

[0100] Next, in step S314, by confirming that the imaging device identification information of the imaging device 100 received from the terminal device 200A by the information processing device 300 in the matching determination is the same as the imaging device identification information received from the imaging device 100, it is confirmed that the terminal device 200A and the imaging device 100 are paired.

[0101] Also, in the matching determination, the information processing device 300 confirms that the terminal device 200B and the imaging device 100 are paired by confirming that the imaging device identification information of the imaging device 100 received from the terminal device 200B is the same as the imaging device identification information received from the imaging device 100.

[0102] When the connection between the information processing apparatus 300 and the imaging apparatus 100 is established, next, as shown in step S315, the information processing apparatus 300 issues a video data transmission request to the imaging apparatus 100.

[0103] Next, as shown in step S316, the imaging apparatus 100 that has received the video data transmission request transmits the video data generated by shooting to the information processing apparatus 300. Then, in step S317, the information processing apparatus 300 receives the video data. The transmission of this video data continues as long as the imaging apparatus 100 performs shooting to generate video data and video data is required in the broadcast.

[0104] Next, in step S318, based on the result of "matching" in step S314 described above, the control unit 301 of the information processing apparatus 300 sets the terminal devices 200A and 200B as the transmission targets of the return video data. Further, the control unit 301 sets the terminal devices 200A and 200B as the transmission targets of the tally signal.

[0105] When the return video data is generated by the external video processing apparatus 400, next, as shown in step S319, the information processing apparatus 300 transmits the return video data to the terminal device 200A. Then, in step S320, the terminal device 200A receives the return video data. Further, in parallel with the transmission of the return video data, as shown in step S321, a tally signal is transmitted from the information processing apparatus 300 to the terminal device 200A by the tally signal transmission unit 308, and in step S322, the terminal device 200A receives the tally signal.

[0106] Furthermore, as shown in step S323, the information processing apparatus 300 transmits return video data to the terminal device 200B, and in step S324, the terminal device 200B receives the return video data. Further, in parallel with the transmission of the return video data, as shown in step S325, the information processing apparatus 300 transmits a tally signal to the terminal device 200B, and in step S326, the terminal device 200B receives the tally signal. Note that the transmission may be stopped when a request to stop the transmission of the return video data is received from the terminal device 200A and / or the terminal device 200B to the information processing apparatus 300.

[0107] After the connection between the terminal device 200A and the information processing apparatus 300 is established, a voice call connection may be established between the terminal device 200A and the information processing apparatus 300. The same applies to the terminal device 200B.

[0108] The processes shown in FIGS. 9 and 10 are performed as described above. In FIG. 10, the return video data is transmitted to the terminal device 200B after being transmitted to the terminal device 200A, but the transmission may be performed simultaneously, or the transmission to the terminal device 200B may be performed first.

[0109] The number of terminal devices is not limited to two and may be any number. The example in FIGS. 9 and 10 where there are a plurality of terminal devices 200 is useful when there are a plurality of users involved in shooting and a plurality of users want to view and confirm the return video data on different terminal devices respectively.

[0110] The processing of the information processing system 10 in the present technology is performed as described above. By this processing, for example, a user who shoots with the imaging device 100 and transmits video data to the information processing apparatus 300 can receive and display the return video data on the terminal device 200, and thus can confirm the return video in a highly visible state. That is, the return video can be confirmed in a more visible state than when both the through video and the return video during shooting are displayed on the display unit 110 of the imaging device 100 using only the imaging device 100.

[0111] <2. Application Examples of This Technology> [2-1. Configuration of MOJO] Next, specific application examples of this technology will be described. This technology can be applied to a new reporting mechanism called MOJO (Mobile Journalism).

[0112] MOJO refers to journalism in which journalists, ordinary citizens, etc. conduct interviews and reports using terminal devices. With the spread of the Internet and terminal devices such as smartphones, even ordinary citizens or freelance journalists without dedicated imaging devices or video editing devices can easily shoot, edit, and provide reporting materials by using applications in the terminal device, and MOJO enables more timely reporting than before.

[0113] First, with reference to FIG. 11, an example of the configuration in the case of conducting interviews and reports in the MOJO format will be described. Interviews and reports in the MOJO format are composed of, for example, journalist 1000, broadcasting station 2000, and server 3000. The broadcasting station 2000, the server 3000, and the imaging device 100 and terminal device 200 used by the journalist 1000 are connected via a network such as the Internet. Note that the imaging device 100, the terminal device 200, and the information processing device 300 are the same as those described in the embodiment.

[0114] The journalist 1000 is a person who goes to the interview location, which is the scene of the incident, conducts an interview, and provides the interview content (video data, image data, audio data, text, etc.) to the broadcasting station 2000. The journalist 1000 conducts an interview using the imaging device 100 and / or the terminal device 200.

[0115] The broadcasting station 2000 is an entity and / or facility that gives an interview instruction to the journalist 1000 and performs predetermined broadcast editing processing on the interview content provided by the journalist 1000 and then broadcasts it.

[0116] Server 3000 is the information processing apparatus 300 according to the present technology, and is also responsible for transmitting and receiving information, data, instructions, etc. between the broadcasting station 2000 and the journalist 1000. Server 3000 may be managed and operated by the broadcasting station 2000 itself, or by an affiliated company or organization of the broadcasting station 2000, or may be managed and operated by a company other than the broadcasting station 2000 and used by the broadcasting station 2000.

[0117] Server 3000 (information processing apparatus 300) includes a control unit 301, a communication unit 302, and a data storage unit 303.

[0118] The control unit 301 is composed of a CPU, a RAM, a ROM, etc., and the CPU executes various processes according to the programs stored in the ROM and issues commands to control the entire information processing apparatus 300 and each part thereof.

[0119] The communication unit 302 is a communication module for transmitting and receiving various data and information between the imaging device 100 and the terminal device 200. Communication methods include wireless LAN, WAN, WiFi, 4G, 5G, etc.

[0120] The data storage unit 303 stores and manages the received video data, identification information and function information of the imaging device 100 and the terminal device 200. In addition to the video data, the data storage unit 303 may store and manage the return video data in association with the video data. The information processing apparatus 300 may transmit the video data and the return video data stored in the data storage unit 303 to the imaging device 100 and the terminal device 200 as needed or required as an archive.

[0121] The broadcasting station 2000 has at least a control room 2100 and a studio 2200. In the control room 2100 and the studio 2200, staff such as producers, directors, assistant directors, cameramen, etc. of the broadcasting station 2000 (hereinafter collectively referred to as staff) perform operations for program broadcasting.

[0122] The control room 2100 includes at least a control unit 2101, a switcher 2102, a monitor 2103, and a communication unit 2104. The control unit 2101 controls each part and the whole of the control room 2100. The control unit 2101 further performs transmission and reception of video data and the like between the switcher 2102 and the monitor 2103, and also performs transmission and reception of video data and the like between the control room 2100 and the CCU 2201 (Camera Control Unit) of the studio 2200.

[0123] The switcher 2102 is a device that switches and mixes a plurality of input signals, and is used, for example, by the staff in charge of switching the video to be displayed on the monitor 2103. The monitor 2103 is a display device that displays various videos such as pre-broadcast videos, broadcast videos, and videos being processed for broadcast.

[0124] Also, the studio 2200 includes at least a CCU 2201 and a camera 2202. The CCU 2201 processes and outputs the video captured by the camera 2202, and transmits video data and the like. The camera 2202 captures a caster or the like who conducts reporting in the studio 2200.

[0125] As shown in FIG. 11, for example, a newsgathering instruction and newsgathering information are transmitted from the broadcasting station 2000 to the journalist 1000 via the server 3000. The newsgathering information includes, for example, the date, the journalist's ID, the newsgathering ID, the newsgathering location, the title, the setting information of the imaging device, and the like.

[0126] When the journalist 1000 conducts newsgathering based on the newsgathering instruction and newsgathering information, the journalist 1000 transmits the newsgathering content to the broadcasting station 2000 via the server 3000. The newsgathering content includes, for example, videos, still images, voices, texts, and the like obtained through newsgathering.

[0127] In the broadcasting station 2000, the staff configures, records, edits, etc. the program based on the newsgathering content transmitted from the journalist 1000, and broadcasts the newsgathering content within the program.

[0128] [The First Mode of Interviewing in MOJO] Next, referring to FIG. 12, the first mode of interviewing in MOJO will be described.

[0129] Journalists in MOJO include citizen journalists (ordinary citizens who are journalists), video journalists, and broadcast journalists.

[0130] A citizen journalist is an ordinary citizen who provides the filmed video to broadcasting station 2000 or posts it on the Internet. It emerged due to the popularization of terminal devices such as smartphones, which enables ordinary citizens to easily film events. When encountering the scene of an event by chance, it can conduct reporting with higher immediacy than video journalists and broadcast journalists.

[0131] A video journalist is a staff member or freelance journalist of broadcasting station 2000, etc., who takes it as their profession to provide the video filmed with a device that is more portable than professional shooting equipment such as smartphones and digital cameras to broadcasting station 2000. Therefore, video journalists can conduct reporting more quickly than broadcast journalists.

[0132] A broadcast journalist is a staff member or freelance journalist of broadcasting station 2000, etc., who takes it as their profession to provide the video filmed using professional shooting equipment to broadcasting station 2000. Since professional shooting equipment is heavier and more elaborate than terminal devices 200 such as smartphones and imaging devices 100 such as ordinary digital cameras, it takes more time to arrive at the scene of the event and start the interview compared to citizen journalists and video journalists. However, because it is filmed with professional shooting equipment, it can conduct reporting with higher image quality and more details than the videos filmed by citizen journalists and video journalists.

[0133] Figure 12 shows the first interview mode of MOJO in which a broadcast is conducted based on an interview with a video journalist using the terminal device 200 and a broadcast journalist using the imaging device 100 as professional filming equipment.

[0134] In this first interview mode, first, when the broadcasting station 2000 learns of the occurrence of an event, it requests an interview from the video journalist and the broadcast journalist via the server 3000 or directly.

[0135] The video journalist who has received the interview request uses the terminal device 200 to perform shooting, editing, etc. as an interview. In this example, since the terminal device 200 is superior in portability and mobility compared to professional-specification filming equipment, the video journalist can arrive at the event site earlier than the broadcast journalist who uses professional-specification filming equipment and conduct an interview. Also, the broadcasting station 2000 can request an interview directly from the video journalist who has the terminal device 200 and is near the event site. On the other hand, since professional-specification filming equipment takes more time to prepare than the terminal device 200, it is considered that it may be slower than the broadcast journalist to arrive at the event site and conduct an interview.

[0136] When a video journalist using the terminal device 200 has completed shooting and editing at the scene of an incident, the journalist transmits video data, which is the content of the interview, to the broadcasting station 2000 via the server 3000. The broadcasting station 2000 that has received the video data conducts first reporting using the video shot by the terminal device 200. This enables reporting with higher immediacy than when only conventional professional shooting equipment was used. Such an interview method is suitable, for example, when an incident occurs during a live broadcast of a news program broadcast during the day, and the content of the incident needs to be reported in real time. Note that the video journalist using the terminal device 200 can also continue shooting at the scene of the incident as it is. Also, although the case where the terminal device 200 transmits video data whose editing as the content of the interview has been completed to the broadcasting station 2000 has been shown, for example, the editing performed by the terminal device 200 may be processing for broadcasting, or the video journalist may only transmit text data such as interview notes to the broadcasting station 2000 as the content of the interview together with the video, and the broadcasting station 2000 may perform editing such as processing for broadcasting.

[0137] When a broadcast journalist using professional-specification shooting equipment has completed shooting and editing at the scene of an incident, the journalist transmits the video data to the broadcasting station 2000 via the server 3000. The broadcasting station 2000 that has received the video data conducts further reporting after the first report using the video data. Since the professional-specification shooting equipment is more high-performance than the terminal device 200, it is possible to conduct reporting with higher image quality and more details than the video shot by the terminal device 200. Such an interview method is suitable, for example, when conducting additional interviews on an incident covered by a video journalist during the day and broadcasting more detailed content in an evening or night news program. However, it may be the unique interview content of a broadcast journalist different from the video journalist. Note that the broadcast journalist can also continue shooting at the scene of the incident as it is.

[0138] [The Second Aspect of Interviews in 2-3.MOJO] FIG. 13 shows a second interview mode of MOJO in which a broadcast is made based on an interview with a citizen journalist using the terminal device 200 and a broadcast journalist using the imaging device 100 as professional shooting equipment.

[0139] In this second interview mode, there is an external server 4000 for providing services on the Internet different from the server 3000 etc. used by the broadcasting station 2000. Services on the Internet include various SNSs (Social Network Services), video distribution sites, blog sites, Internet TVs, etc.

[0140] Since a citizen journalist is an ordinary citizen who has no employment relationship with the broadcasting station 2000, in this second interview mode, the broadcasting station 2000 and the citizen journalist will know about the occurrence of an event separately. Therefore, a citizen journalist can conduct an interview at his / her own will without a request for an interview from the broadcasting station 2000. When a citizen journalist accidentally discovers an event or encounters an event, he / she can immediately conduct shooting and editing as an interview.

[0141] A citizen journalist who has encountered an event uses the terminal device 200 to conduct shooting for an interview, etc. Generally, since the terminal device 200 is superior in portability, mobility, and communication function compared to professional-specification shooting equipment, it is considered that a citizen journalist can post and provide the interview content earlier than a broadcast journalist. A citizen journalist may also attach text to the captured video. In addition, editing work may be performed on the video captured by a citizen journalist within the server of the broadcasting station 2000 or the broadcasting station 2000.

[0142] After shooting and editing are completed, a citizen journalist using the terminal device 200 posts the interview content on an SNS or a video distribution site on the Internet, etc. This becomes the first report without going through the broadcasting station 2000. As a result, it is possible to conduct a report with higher immediacy than the conventional report conducted by the broadcasting station 2000.

[0143] Broadcast station 2000 periodically searches SNSs, video distribution sites, blog sites, etc. When it grasps that a citizen journalist has posted something about an incident on the Internet, it requests information from that citizen journalist and acquires the posted content.

[0144] Broadcast station 2000 that has acquired the posted content conducts reporting using that posted content. This enables reporting with higher immediacy than the coverage and reporting by video journalists and broadcast journalists.

[0145] Also, broadcast station 2000 may request a citizen journalist to supply materials such as video data in real time. This enables real-time reporting (live broadcast) using the video shot by the citizen journalist.

[0146] Also, when a broadcast journalist using professional-specification filming equipment has completed filming and editing, the journalist transmits the video data to broadcast station 2000 via server 3000. Broadcast station 2000 that has received the video data conducts further reporting after the first report using the video shot with a camera. Since professional-specification filming equipment is more high-performance than terminal device 200, it is possible to conduct reporting with higher image quality and more detail than the video shot by terminal device 200. Also, broadcast station 2000 can also conduct real-time reporting (live broadcast) using the video transmitted from the broadcast journalist.

[0147] The example of the MOJO's material-gathering mode is as above. Note that the material-gathering modes in FIGS. 12 and 13 are merely examples.

[0148] In such a reporting method, live reporting or real-time video streaming may be conducted. In such live reporting or real-time video streaming, journalists may want to check what videos are being broadcast by the 2,000 broadcasting stations at present and how the videos they are shooting are being broadcast. This technology is useful in such cases.

[0149] Specifically, journalists 1,000 (citizen journalists, video journalists, and broadcast journalists) possess an imaging device 100 and a terminal device 200, and perform shooting and editing with the imaging device 100 at the scene of the incident. Then, the shooting data is transmitted to the broadcasting station 2,000 via the server 3,000. While conducting broadcasting using the shooting data, the broadcasting station 2,000 transmits the broadcast content as return video data to the terminal device 200 of the journalist 1,000. By displaying the return video data on the terminal device 200, the journalist 1,000 can check the broadcast content on the terminal device 200 while shooting with the imaging device 100. It is also possible to give instructions, consult, and hold meetings, etc. between the broadcasting station 2,000 and the journalist 1,000 via voice calls.

[0150] <3. Modification Example> As described above, the embodiments of the present technology have been specifically described, but the present technology is not limited to the above-described embodiments, and various modifications based on the technical idea of the present technology are possible.

[0151] In the embodiment, video data was used as a specific example for explanation, but the present technology is applicable to all data such as image data, audio data, text data, statistical data, measurement data, programs, and application data.

[0152] In the embodiment, it was described that the information processing device 300 transmits return video data to the terminal device 200 and further transmits a tally signal, but the information processing device 300 may transmit only the tally signal.

[0153] In the embodiment, the imaging device 100 transmits the identification information of the imaging device 100 itself to the terminal device 200, and the terminal device 200 transmits the identification information of the terminal device 200 itself and the imaging device 100 to the information processing device 300. However, the terminal device 200 may transmit the identification information to the imaging device 100, and the imaging device 100 may transmit the identification information of the imaging device 100 itself and the terminal device 200 to the information processing device 300.

[0154] In the embodiment, the case where there are a plurality of terminal devices 200 has been described, but there may be a plurality of imaging devices 100.

[0155] In the embodiment, it has been described that the control unit 301 of the information processing device 300 performs a transmission process of transmitting return video data to the terminal device 200 via the communication unit 302. However, the control unit 301 may further perform different transmission processes according to the communication environment of the terminal device 200. Here, the communication environment of the terminal device 200 indicates, for example, the communication method (such as 5G, 4G / LTE, 3G, etc.) available for communication between the terminal device 200 and the base station at the location where the terminal device 200 is located.

[0156] The control unit 301 of the information processing device 300 according to the modification example determines whether the communication environment between the terminal device 200 and the base station is a low-speed communication method (such as 4G / LTE, 3G, etc.) or a high-speed high-capacity communication method (for example, 5G) that enables communication that is faster and has a higher capacity than the low-speed low-capacity communication method, based on the information indicating the communication environment. Here, the information indicating the communication environment is, for example, the information indicating the communication method established between the terminal device 200 and the base station, and it may be transmitted from the terminal device 200 to the information processing device 300 via the base station. In this case, the information indicating the communication environment may be included in other information transmitted from the terminal device 200 to the information processing device 300. Also, the information indicating the communication environment may be the position information indicating the position of the terminal device 200, and the information processing device 300 may acquire the position information of the terminal device 200. In this case, the control unit 301 of the information processing device 300 determines the communication environment at the position of the terminal device 200 indicated by the acquired position information by referring to the pre-stored communication area map.

[0157] When the control unit 301 of the information processing apparatus 300 determines that the communication environment is a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), it controls the communication unit to transmit at least the return video to the terminal device 200. In this way, in an environment where a low-speed and low-capacity communication method can be used, by transmitting the minimum necessary video, the amount of information can be suppressed, and stable communication can be achieved.

[0158] On the other hand, when the control unit 301 of the information processing apparatus 300 determines that the communication environment is a high-speed and high-capacity communication method (for example, 5G), it uses the technology of network slicing to virtualize the network, divide (slice) network resources, and selectively use slices according to the usage and priority of the signals to be communicated. By doing so, in addition to the return video, it controls the communication unit to transmit a plurality of other signals (including the video of the imaging device 100 paired with the destination terminal device 200, the video signals of other imaging devices 100, and tally signals).

[0159] Here, the control unit 301 of the information processing apparatus 300 generates information for specifying the slice to be allocated according to the usage and characteristics of the signals to be transmitted, and associates the signal with the information (metadata) for specifying the slice to which the signal should be allocated, and controls the communication unit to transmit it to the base station. The base station allocates each received signal to the slice specified by the respective metadata, and transmits each signal to the terminal device 200 using the allocated slice.

[0160] For example, since the return video is the signal with the highest priority and needs to be transmitted with low latency, the control unit 301 of the information processing apparatus 300 generates information (metadata) for specifying a slice that can ensure transmission with low latency for the return video, associates it with the return video, and generates information (metadata) for specifying a slice different from the above slice for other signals as signals with a lower priority than the return video, and associates them with the respective signals. Here, for each type of signal, it may be assigned to different slices, or several types of signals may be assigned to a common slice. In this way, slices can be assigned and transmitted according to the characteristics required for each signal, and the user can confirm the high-priority return video in a low-latency state and also confirm the video being shot by the paired imaging device 100 or other imaging devices 100.

[0161] Here, an example has been described in which the control unit 301 of the information processing apparatus 300 generates information for specifying the slice to which the signal to be transmitted is to be assigned. However, the control unit 301 of the information processing apparatus 300 may simply associate information indicating the type of the signal to be transmitted (such as return video, other signals, etc.) with the signal. In this case, the base station refers to the relationship between the type of signal stored in advance and the slice to which the signal of that type should be assigned based on the information indicating the type associated with each received signal, and assigns each received signal to the slice specified based on each type, and can transmit each signal to the terminal device 200 using the assigned slice.

[0162] Furthermore, the information processing apparatus 300 according to another modification example may perform different video processing on the video signal transmitted to the terminal device 200 by the video processing apparatus 400 according to the communication environment of the terminal device 200.

[0163] Furthermore, the information processing apparatus 300 according to another modification example may include a video processing unit (not shown) that performs different video processing on the video signal transmitted to the terminal device 200 according to the communication environment of the terminal device 200.

[0164] The video processing unit of the information processing apparatus 300 according to another modification example performs resolution conversion processing on the video according to the communication environment of the terminal apparatus 200.

[0165] When the communication environment determined by the control unit 301 is a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), the video processing unit of the information processing apparatus 300 performs resolution conversion processing to reduce the resolution of the return video (the resolution of the video captured by the imaging apparatus 100, for example, a high resolution such as 4K or 8K) to convert the resolution to a low-resolution video (for example, a low resolution such as HD). The control unit 301 controls the communication unit to transmit the video signal resolution-converted by the video processing unit to the base station. Thereby, although the resolution is low, the user can check the return video without causing reception delay or the like. Even when the communication environment is determined to be a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), when transmitting, in addition to the return video, the video of the imaging apparatus 100 paired with the destination terminal apparatus 200 and the video of other imaging apparatuses 100, the video processing unit can perform resolution conversion on any or all of those videos.

[0166] Note that when the communication environment determined by the control unit 301 is a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), the video processing unit does not perform resolution conversion processing to reduce (down-convert) the resolution of the video. Instead, when the communication environment determined by the control unit 301 is a high-speed and high-capacity communication method (for example, 5G), the video processing unit can also perform resolution conversion processing to improve (up-convert) the resolution of the video. That is, when the communication environment determined by the control unit 301 is a high-speed and high-capacity communication method (for example, 5G), the video processing unit of the information processing device 300 performs resolution conversion processing to improve the resolution of the return video (the resolution of the video captured by the imaging device 100, for example, a low resolution such as HD) to convert the resolution to a high-resolution video (for example, a high resolution such as 4K or 8K). The control unit 301 controls the communication unit to transmit the video signal resolution-converted by the video processing unit to the base station. Thereby, the user can confirm a clearer return video without causing reception delay or the like. In this way, by adjusting the resolution of the return video according to the communication environment, it becomes possible for the user to confirm the return video in a form that achieves both the image quality of the return video and the stability of transmission and reception.

[0167] Furthermore, the video processing unit of the information processing device 300 according to another modification example performs frame rate conversion processing on the video according to the communication environment of the terminal device 200.

[0168] When the communication environment determined by the control unit 301 is a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), the video processing unit of the information processing apparatus 300 performs frame rate conversion processing to reduce the frame rate of the return video (the frame rate of the video captured by the imaging apparatus 100, for example, a high frame rate such as 120p) to convert it to a low frame rate video (for example, a low frame rate such as 30p). The control unit 301 controls the communication unit to transmit the video whose frame rate has been converted by the video processing unit to the base station. Thereby, the user can view the return video without causing reception delay or the like even though the frame rate is low. Even when the communication environment is determined to be a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), when transmitting, in addition to the return video, the video of the imaging apparatus 100 paired with the destination terminal device 200 and the videos of other imaging apparatuses 100, the video processing unit can perform frame rate conversion on any or all of those videos.

[0169] Note that, instead of performing frame rate conversion processing to lower (down-convert) the frame rate of the video when the communication environment determined by the control unit 301 is a low-speed and low-capacity communication method (such as 4G / LTE, 3G, etc.), the video processing unit of the information processing apparatus 300 can also perform frame rate conversion processing to increase (up-convert) the frame rate of the video when the communication environment determined by the control unit 301 is a high-speed and high-capacity communication method (for example, 5G). That is, when the communication environment determined by the control unit 301 is a high-speed and high-capacity communication method (for example, 5G), the video processing unit of the information processing apparatus 300 performs frame rate conversion processing to increase the frame rate of the return (the frame rate of the video captured by the imaging apparatus 100, for example, a low frame rate such as 30p) to convert it to a high frame rate video (for example, a high frame rate such as 120p). The control unit 301 controls the communication unit to transmit the video signal whose frame rate has been converted by the video processing unit to the base station. Thereby, the user can view a higher frame rate return video without causing reception delay or the like. In the above example, the video processing unit of the information processing apparatus 300 has been described as performing resolution conversion processing and frame rate conversion processing separately. However, it is also possible to perform resolution conversion processing and frame rate conversion processing together.

[0170] This technology is applicable not only to reports such as MOJO but also to all broadcasts such as live drama and live variety shows.

[0171] This technology can also adopt the following configuration. (1) A control unit that determines whether the terminal device and the imaging device are associated based on the first identification information received from the terminal device and the second identification information received from the imaging device, and sets the terminal device as the transmission target of the return video based on the determination result. Information processing apparatus. (2) The control unit sets the terminal device as the transmission target of the return video when the determination result satisfies the association condition, which is a condition indicating that the terminal device and the imaging device are associated, in the information processing apparatus according to (1). (3) The first identification information includes terminal device identification information for identifying the terminal device and imaging device identification information for identifying the imaging device, and the second identification information includes the imaging device identification information in the information processing apparatus according to (1) or (2). (4) The control unit sets the terminal device as the transmission target of the return video when the determination result indicates that the imaging device identification information included in the first identification information matches the imaging device identification information included in the second identification information in the information processing apparatus according to (3). (5) The control unit sets the terminal device as a transmission target of the return video when the determination result satisfies both the return video condition, which is a condition indicating that the control unit has received return video display function information indicating that the control unit has a function of displaying the return video, from the terminal device, and the association condition. The information processing apparatus according to any one of (1) to (4). (6) The control unit further sets the terminal device as a transmission target of a tally signal when the determination result satisfies the association condition. The information processing apparatus according to (2). (7) The control unit sets the terminal device as a transmission target of the tally signal when the determination result satisfies both the tally signal condition, which is a condition indicating that the control unit has received tally signal presentation function information indicating that the control unit has a function of presenting a tally signal, from the terminal device, and the association condition. The information processing apparatus according to (6). (8) The terminal device, the imaging device, and the information processing apparatus communicate with each other in a wireless manner. The information processing apparatus according to any one of (1) to (7). (9) Determine whether the terminal device and the imaging device are associated based on the first identification information received from the terminal device and the second identification information received from the imaging device, An information processing method for transmitting a return video to the terminal device based on the determination result. (10) Determine whether the terminal device and the imaging device are associated based on the first identification information received from the terminal device and the second identification information received from the imaging device, An information processing program for transmitting a return video to the terminal device based on the determination result. (11) Transmit first identification information in which imaging device identification information for identifying the external imaging device received from the external imaging device and terminal device identification information for identifying itself are associated, A terminal device comprising a control unit that controls to receive a return video transmitted based on a result of determining whether the terminal device and the imaging device are associated based on the first identification information and the second identification information received from the external imaging device. (12) The control unit controls to receive the return video transmitted based on a result of determining whether the external information processing device satisfies an association condition which is a condition indicating that the terminal device and the imaging device are associated (11) The terminal device according to. (13) The first identification information includes terminal device identification information for identifying the terminal device and imaging device identification information for identifying the imaging device, and the second identification information includes the imaging device identification information (11) or (12) The terminal device according to. (14) The control unit controls to receive the return video transmitted based on a result of determining whether the imaging device identification information included in the first identification information and the imaging device identification information included in the second identification information match in the external information processing device (13) The terminal device according to. (15) The control unit determines whether both a return video condition which is a condition indicating that the external information processing device has received return video display function information indicating that the external information processing device has a function of displaying the return video from the terminal device and the association condition are satisfied, and controls to receive the return video transmitted based on the result (11) to (14) The terminal device according to any one of. (16) The control unit controls to receive a tally signal transmitted based on a result of determining whether the external information processing device further satisfies the association condition (12) The terminal device according to. (17) The control unit controls to receive the tally signal transmitted based on the result of determining whether both the tally signal condition, which is a condition indicating that the external information processing apparatus has a function of presenting a tally signal, is received from the terminal device, and the association condition are satisfied (16). The terminal device according to the above. (18) The imaging device, the information processing apparatus, and the terminal device perform communication by a wireless method. The terminal device according to any one of (11) to (17). (19) Transmit first identification information associating the imaging device identification information for identifying the external imaging device received from the external imaging device and the terminal device identification information for identifying itself, A control method for a terminal device that controls to receive a return video transmitted based on the result of determining whether the terminal device and the imaging device are associated based on the first identification information and the second identification information received from the external information processing apparatus from the external imaging device. (20) Transmit first identification information associating the imaging device identification information for identifying the external imaging device received from the external imaging device and the terminal device identification information for identifying itself, A control program that causes a computer to execute a control method for a terminal device that controls to receive a return video transmitted based on the result of determining whether the terminal device and the imaging device are associated based on the first identification information and the second identification information received from the external imaging device.

Explanation of Signs

[0172] 10 ··· Information processing system 100 ··· Imaging device 200 ··· Terminal device 300 ··· Information processing apparatus 303 ··· Information receiving unit 304 ··· Data receiving unit 305 ··· Data transmitting unit

Claims

1. A control unit that determines whether the terminal device and the imaging device are associated based on first identification information received from the terminal device and second identification information received from the imaging device, and sets the terminal device as a transmission target for return video based on the determination result. An information processing apparatus.

2. The control unit sets the terminal device as a transmission target for the return video when the determination result satisfies an association condition that is a condition indicating that the terminal device and the imaging device are associated. The information processing apparatus according to claim 1.

3. The first identification information includes terminal device identification information for identifying the terminal device and imaging device identification information for identifying the imaging device, and the second identification information includes the imaging device identification information. The information processing apparatus according to claim 1.

4. The control unit sets the terminal device as a transmission target for the return video when the determination result indicates that the imaging device identification information included in the first identification information matches the imaging device identification information included in the second identification information. The information processing apparatus according to claim 3.

5. The control unit sets the terminal device as a transmission target for the return video when the determination result satisfies both a return video condition that is a condition indicating that the control unit has received return video display function information indicating that the control unit has a function of displaying the return video, and the association condition. The information processing apparatus according to claim 2.

6. The control unit further sets the terminal device as a transmission target for a tally signal when the determination result satisfies the association condition. The information processing apparatus according to claim 2.

7. The control unit sets the terminal device as a transmission target for the tally signal when the determination result satisfies both a tally signal condition that is a condition indicating that the control unit has received tally signal presentation function information indicating that the control unit has a function of presenting a tally signal, and the association condition. The information processing apparatus according to claim 6.

8. The terminal device, the imaging device, and the information processing apparatus communicate wirelessly. The information processing apparatus according to claim 1.

9. Determine whether the terminal device and the imaging device are associated based on first identification information received from the terminal device and second identification information received from the imaging device. Transmit return video to the terminal device based on the determination result. An information processing method.

10. Determine whether the terminal device and the imaging device are associated based on the first identification information received from the terminal device and the second identification information received from the imaging device, and transmit a return video to the terminal device based on the determination result. An information processing program.

11. Transmit first identification information in which imaging device identification information for identifying the external imaging device received from the external imaging device and terminal device identification information for identifying itself are associated, and control a control unit to receive a transmitted return video based on a result of determining whether the terminal device and the imaging device are associated based on the first identification information and second identification information received from the external imaging device. A terminal device.

12. The control unit controls to receive the transmitted return video based on a result of determining whether a matching condition, which is a condition indicating that the external information processing device associates the terminal device and the imaging device, is satisfied. The terminal device according to Claim 11.

13. The first identification information includes terminal device identification information for identifying the terminal device and imaging device identification information for identifying the imaging device, and the second identification information includes the imaging device identification information. The terminal device according to Claim 11.

14. The control unit controls to receive the transmitted return video based on a result of determining whether the imaging device identification information included in the first identification information and the imaging device identification information included in the second identification information match. The terminal device according to Claim 13.

15. The control unit controls to receive the transmitted return video based on a result of determining whether both a return video condition, which is a condition indicating that the external information processing device has received return video display function information indicating that the external information processing device has a function of displaying the return video, from the terminal device, and the matching condition are satisfied. The terminal device according to Claim 12.

16. The control unit controls to receive a tally signal transmitted based on a result of determining whether the external information processing device further satisfies the matching condition. The terminal device according to Claim 12.

17. The control unit controls to receive the tally signal transmitted based on the result of determining whether both the tally signal condition, which is a condition indicating that the external information processing apparatus has received tally signal presentation function information indicating that the external information processing apparatus has a function of presenting a tally signal, from the terminal device, and the association condition are satisfied. The terminal device according to claim 16.

18. The imaging device, the information processing device, and the terminal device communicate with each other by a wireless method. The terminal device according to claim 11.

19. transmits first identification information associating the imaging device identification information for identifying the external imaging device received from the external imaging device and the terminal device identification information for identifying itself, controls to receive the return video transmitted based on the result of determining whether the terminal device and the imaging device are associated based on the first identification information and the second identification information received from the external imaging device by the external information processing device. A control method for a terminal device.

20. transmits first identification information associating the imaging device identification information for identifying the external imaging device received from the external imaging device and the terminal device identification information for identifying itself, controls to receive the return video transmitted based on the result of determining whether the terminal device and the imaging device are associated based on the first identification information and the second identification information received from the external imaging device by the external information processing device. A control program for causing a computer to execute a control method for a terminal device.

Images