Information processing system, information processing device, and information processing method

Abstract

To provide an information processing system, an information processing device, an information processing method, and an information processing program that can stably provide video based on video data transmitted from an autonomously moving mobile device.SOLUTION: An information processing system according to the present disclosure includes an acquisition unit (601) that acquires radio wave conditions in an environment, and a creation unit (602) that creates an operation plan based on the radio wave conditions and radio wave strength required to transmit video data used for distribution.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present disclosure relates to an information processing system, an information processing device, an information processing method, and an information processing program. [Background technology]

[0002] Live streaming is a well-known technique for filming live music performances and distributing the video and audio data in real time. Cost reduction and content expansion are required for live streaming. In response to this background, the use of autonomous mobile robots for filming live streaming has been proposed.

[0003] When using a mobile robot to film and distribute video, it is desirable to transmit the filmed video and audio data from the robot via wireless communication. This is because it is desirable to be able to film while moving freely around the stage where the performance is taking place, and to minimize the impact on the performers and equipment on stage. Patent Document 1 discloses a mobile robot equipped with wireless communication means that can be remotely controlled. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-260769 Summary of the Invention [Problem to be solved by the invention]

[0005] However, when transmitting video data wirelessly, depending on the radio wave conditions, there is a risk that the quality of the video data being transmitted may be degraded or the video may be interrupted. Such degradation of video quality and interruptions in the video stream can be extremely serious problems in live streaming.

[0006] An object of the present disclosure is to provide an information processing system, an information processing device, an information processing method, and an information processing program that can stably provide video based on video data transmitted from an autonomously moving mobile device. [Means for solving the problem]

[0007] The information processing system according to the present disclosure includes an acquisition unit that acquires radio wave conditions in an environment, and a creation unit that creates an operation plan based on the radio wave conditions and the radio wave strength required to transmit video data to be distributed. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a schematic diagram for explaining a video distribution system related to the present disclosure. [Figure 2] 1 is a schematic diagram illustrating a configuration of an example of an information processing system as a video distribution system according to an embodiment. [Figure 3] FIG. 2 is a functional block diagram illustrating an example of a function of the information processing device according to the embodiment. [Figure 4] FIG. 1 is a block diagram illustrating a hardware configuration of an example of an information processing apparatus according to an embodiment. [Figure 5] FIG. 2 is a functional block diagram illustrating an example of a function of a video selection device applicable to an embodiment. [Figure 6] 10 is a schematic diagram showing an example of a video selection screen displayed on a display by a display unit of a video selection device, which is applicable to an embodiment. FIG. [Figure 7A] 1A and 1B are schematic diagrams illustrating an example of the appearance of a moving device applicable to an embodiment. [Figure 7B] 1A and 1B are schematic diagrams illustrating an example of the appearance of a moving device applicable to an embodiment. [Figure 8] FIG. 2 is a block diagram illustrating a hardware configuration of an example of a mobile device that can be applied to an embodiment. [Figure 9] FIG. 2 is a functional block diagram illustrating an example of functions of a mobile device applicable to the embodiment. [Figure 10]10 is a schematic diagram showing an example of a visualization screen that displays a visualization of a radio wave condition and a movement route of a mobile device according to an operation plan according to the first embodiment. FIG. [Figure 11] FIG. 2 is a schematic diagram showing an example in which the visualization screen according to the first embodiment is applied to a screen that represents a three-dimensional space. [Figure 12] FIG. 10 is a schematic diagram showing an example of a visualization screen according to a first modified example of the first embodiment. [Figure 13] FIG. 10 is a schematic diagram showing an example of an operation on a slider on a visualization screen according to a first modified example of the first embodiment. [Figure 14] FIG. 10 is a schematic diagram for explaining generation of a field intensity map and creation of an operation plan according to a first modified example of the first embodiment. [Figure 15] 10 is a diagram for explaining that the state of radio waves transmitted from a mobile device differs depending on the orientation of the mobile device. FIG. [Figure 16] 10 is a flowchart illustrating an example of a process for creating an operation plan according to a second modified example of the first embodiment. [Figure 17A] 10 is a diagram for explaining division of a route and taking over of the operation of the mobile device 10 according to the second embodiment. FIG. [Figure 17B] 10 is a diagram for explaining division of a route and taking over of the operation of the mobile device 10 according to the second embodiment. FIG. [Figure 18] 10 is a flowchart illustrating an example of a process for creating an operation plan according to the second embodiment. [Figure 19] FIG. 11 is a schematic diagram showing an example of a video selection screen of a video selection device according to a third embodiment. [Figure 20] FIG. 10 is an enlarged schematic view showing a display area according to a third embodiment. [Figure 21] FIG. 10 is a schematic diagram illustrating a process for determining the risk of video being interrupted during distribution. [Figure 22] 13 is a flowchart illustrating an example of a video switching process according to a risk level in a video selection device according to a third embodiment. [Figure 23]FIG. 13 is a schematic diagram illustrating a process for determining the risk of video being distributed being interrupted, according to a modification of the third embodiment. [Figure 24] FIG. 11 is a schematic diagram showing an example in which the position where the disturbance of the image is detected is reflected in the radio wave condition map according to a modification of the third embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are denoted by the same reference numerals, and redundant description will be omitted.

[0010] Hereinafter, embodiments of the present disclosure will be described in the following order. 1. Video distribution system related to this disclosure 2. Configurations Applicable to Embodiments of the Present Disclosure 3. First embodiment 3-1. First Modification of the First Embodiment 3-2. Second Modification of the First Embodiment 4. Second embodiment 5. Third Embodiment 5-1. Modification of the third embodiment

[0011] [1. Video distribution system related to this disclosure] Prior to describing each embodiment of the present disclosure, a video distribution system related to the present disclosure will be briefly described to facilitate understanding. Fig. 1 is a schematic diagram for explaining a video distribution system related to the present disclosure. In the example of Fig. 1, the video distribution system includes a plurality of mobile devices 10, a receiver 20 for receiving video data transmitted from each mobile device 10, a video selection device 40 for selecting video data to be used for distribution from the video data received by the receiver 20, and a robot operation device 30 for remotely operating each mobile device 10.

[0012] The mobile device 10 is equipped with a camera 100 and has a transmitter (not shown) and an antenna 101 for transmitting video data to the receiver 20. Although not shown, the mobile device 10 also has a receiver and an antenna for receiving control signals from the robot operation device 30. Furthermore, the mobile device 10 is equipped with a drive mechanism for rotating and moving the housing of the mobile device 10, and can change the orientation and move the camera 100 in response to control signals from the robot operation device 30.

[0013] Each mobile device 10 is placed on, for example, a stage 50. On the stage 50, for example, performers 51 who will perform, stage equipment, and equipment (collectively referred to here as equipment, etc. 52) are placed. The mobile device 10 moves on the stage 50 in accordance with control signals transmitted from the robot operation device 30 and pre-programmed movements, and takes pictures with the camera 100. In each mobile device 10, video data captured by the camera 100 is transmitted by the antenna 101 and received by the receiver 20.

[0014] The receiver 20 transfers each piece of video data received from each mobile device 10 to the video selection device 40. The video selection device 40 may also transfer video data via wired or wireless communication from cameras other than the camera 100 of the mobile device 10, such as a camera carried by a person to take pictures or a camera fixedly or movably mounted on a predetermined pedestal. The video selection device 40 is a so-called switcher that displays images from each piece of video data on a display and selects video data to be output from each piece of video data in real time in accordance with user operations corresponding to, for example, the performance of a performer 51. The video data selected by the video selection device 40 is transferred to, for example, a distribution server 3. The distribution server 3 transmits the transferred video data to a network 2, such as the Internet.

[0015] The video data transmitted to the network 2 is received by each user terminal 4 connected to the network 2. Each user operating each user terminal 4 can view the video data received via the network 2 by displaying it on the display of each user terminal 4.

[0016] In the existing technology, in such a configuration, depending on the relative positions of the receiver 20, the mobile device 10, the performer 51 on the stage, and the equipment 52, it may be difficult for the video data transmitted from the mobile device 10 to reach the receiver 20. In such radio wave conditions, there is a risk that the distributed video may be interrupted. In video distribution, interruptions in the distributed video must be avoided.

[0017] In the embodiment, the video distribution system acquires radio wave conditions indicating the state of radio waves in the environment in which the mobile devices 10 are operating (for example, on the stage 50). Based on the acquired radio wave conditions and information on the radio wave strength necessary and sufficient for transmitting video data, the video distribution system creates an operation plan for each mobile device 10 to capture images while autonomously moving. This makes it possible to avoid interruptions in the video being distributed when capturing images while moving with the mobile devices 10. Therefore, according to the embodiment, it is possible to stably provide video based on video data transmitted from autonomously moving mobile devices.

[0018] The radio wave intensity refers to the signal intensity at the receiver 20 when a signal transmitted from the mobile device 10 at a predetermined signal intensity is received by the receiver 20. Also, for example, a map of radio wave intensity on the stage 50 can be generated based on the signal intensity of each signal transmitted from the mobile device 10 at multiple positions on the stage 50 and position information indicating the position from which the signal was transmitted.

[0019] 2. Configurations Applicable to Embodiments of the Present Disclosure Next, a configuration applicable to an embodiment of the present disclosure will be described. Fig. 2 is a schematic diagram showing a configuration of an example of an information processing system as a video distribution system according to an embodiment.

[0020] 2, the information processing system 1 includes a mobile device 10, a receiver 20, a video selection device 40, an information processing device 60, and a transmitter 22. Note that the environment in which the mobile device 10 operates can be the same as the stage 50 described using FIG. 1, and therefore further description thereof will be omitted here.

[0021] The mobile device 10 has a camera 100 mounted on a housing 105, as well as antennas 101t and 101r, a transmitter 102, a receiver 103, and a mobile mechanism 104. The transmitter 102 of the mobile device 10 transmits a signal containing video data captured by the camera 100 from the antenna 101t at a predetermined signal strength. The mobile device 10 can acquire location information indicating its own position and transmit the acquired location information together with the signal. The transmitted signal is received by the antenna 21 and passed to the receiver 20. Furthermore, a signal transmitted from the antenna 23 of the transmitter 22 is received by the antenna 101r and passed to the receiver 103.

[0022] Note that, for example, Wi-Fi (Wireless Fidelity) (registered trademark) can be applied as a communication method between the mobile device 10 (transmitter 102 and receiver 103) and the receiver 20 and transmitter 22. The communication method between the mobile device 10 and the receiver 20 and transmitter 22 is not limited to Wi-Fi. Data is transmitted and received between the receiver 20 and the video selection device 40 and the information processing device 60 using wired or wireless communication. For wireless communication, Wi-Fi can be used.

[0023] Also, although the receiver 103 and the antenna 101r have been described here as only receiving signals transmitted from the transmitter 22, this is not limited to this example. For example, the receiver 103 may have a signal receiving function as well as a signal transmitting function, and may enable two-way communication with the transmitter 22 using a predetermined communication method such as Wi-Fi.

[0024] In the moving device 10, the moving mechanism 104 is provided on the bottom surface of the housing 105, and is driven by a driving unit (not shown) to rotate and move the moving device 10.

[0025] For the sake of explanation, the mobile device 10 is assumed to have an antenna 101t for transmitting signals and an antenna 101r for receiving signals, but this is not limiting. For example, the mobile device 10 may transmit and receive signals using a single antenna.

[0026] The receiver 20 transfers the video data received from the mobile device 10 to the video selection device 40. The receiver 20 also measures the strength of the signal containing the video data received from the mobile device 10 and acquires signal strength information. The receiver 20 transfers the acquired signal strength information to the information processing device 60. Here, the signal strength information may include identification information that identifies the mobile device 10 that transmitted the signal and location information that indicates the location of the mobile device 10.

[0027] In the following, unless otherwise specified, "transmitting a signal containing video data" will be described as "transmitting video data," and "receiving a signal containing video data" will be described as "receiving video data," etc.

[0028] The information processing device 60 includes the functions of a radio wave condition acquisition unit 601 and an operation plan creation unit 602 .

[0029] The signal strength information transferred from the receiver 20 is passed to the radio wave condition acquisition unit 601 as the radio wave condition in the environment including the mobile device 10 and the receiver 20. The radio wave condition acquisition unit 601 analyzes the radio wave condition passed from the receiver 20. The radio wave condition includes, for example, signal strength and location information indicating the location from which the signal was transmitted. In other words, the radio wave condition acquisition unit functions as an acquisition unit that acquires the radio wave condition in the environment. The radio wave condition acquisition unit 601 passes information indicating the analyzed radio wave condition to the operation plan creation unit 602.

[0030] The operation plan creation unit 602 creates an operation plan for controlling the movement path and orientation of the mobile device 10 based on the information indicating the radio wave conditions passed from the radio wave condition acquisition unit 601. That is, the operation plan creation unit 602 functions as a creation unit that creates an operation plan based on the radio wave conditions and the radio wave strength required to transmit the video data used for distribution. The operation plan creation unit 602 transfers information indicating the created operation plan to the transmitter 22.

[0031] The transmitter 22 transmits a signal including information indicating the operation plan transferred from the information processing device 60 from the antenna 73. This signal is received by the antenna 101r in the mobile device 10 and is taken into the mobile device 10 via the receiver 103. The movement and rotational movements of the mobile device 10 are controlled in accordance with the information indicating the taken-in operation plan, and in accordance with the movement path and orientation indicated in the operation plan.

[0032] 3 is a functional block diagram illustrating an example of functions of the information processing device 60 according to the embodiment. In FIG. 3, the information processing device 60 according to the embodiment includes a radio wave condition acquisition unit 601, an operation plan creation unit 602, a map generation unit 603, a display unit 604, an input unit 605, and a communication unit 606.

[0033] The radio wave condition acquisition unit 601, the operation plan creation unit 602, the map generation unit 603, the display unit 604, the input unit 605, and the communication unit 606 are configured by an information processing program according to the embodiment running on, for example, a CPU (Central Processing Unit). However, without being limited to this, some or all of the radio wave condition acquisition unit 601, the operation plan creation unit 602, the map generation unit 603, the display unit 604, the input unit 605, and the communication unit 606 may be configured by hardware circuits that operate in cooperation with each other.

[0034] 3, the display unit 604 generates screen information and displays a screen according to the generated screen information on a display included in or connected to the information processing device 60. The input unit 605 accepts input from a user. The communication unit 606 controls communication between the information processing device 60 and external devices. For example, the communication unit 606 controls communication between the information processing device 60 and the video selection device 40, the receiver 20, and the transmitter 22. The communication unit 606 also controls communication over a network.

[0035] The radio wave condition acquisition unit 601 acquires the radio wave condition in the environment (for example, on the stage 50) transferred from the receiver 20. The radio wave condition acquisition unit 601 can, for example, obtain the radio wave intensity distribution in the environment based on the radio wave conditions transmitted from each mobile device 10, and obtain the obtained radio wave intensity distribution as the radio wave condition in the environment.

[0036] The motion plan creation unit 602 creates a motion plan for controlling the operation of each mobile device 10 based on the position information of the performers 51 and each piece of equipment 52 on the stage 50 and the radio wave conditions acquired by the radio wave condition acquisition unit 601. Here, the position information of the performers 51 and each piece of equipment 52 on the stage 50 is created in advance as two-dimensional or three-dimensional map information by, for example, a director who directs the performance on the stage 50, and is input to the motion plan creation unit 602.

[0037] The map generation unit 603 generates a radio wave condition map that visualizes the radio wave condition acquired by the radio wave condition acquisition unit 601. The map generation unit 603 generates a map showing the distribution of radio wave strength as a radio wave condition map based on, for example, the signal strength and location information included in the radio wave condition. The generated radio wave condition map is displayed on a display by the display unit 604 as a map screen.

[0038] Fig. 4 is a block diagram showing an example of a hardware configuration of an information processing device 60 according to an embodiment. In Fig. 4, the information processing device 60 includes a CPU 6000, a read-only memory (ROM) 6001, a random access memory (RAM) 6002, a display control unit 6003, a storage device 6004, a data I / F 6005, and a communication I / F 6006, which are communicably connected to each other via a bus 6010.

[0039] The storage device 6004 is a non-volatile storage medium such as a hard disk drive, flash memory, etc. The CPU 6000 operates in accordance with programs stored in the ROM 6001 or the storage device 6004, using the RAM 6002 as a work memory, and controls the overall operation of this information processing device 60.

[0040] The display control unit 6003 generates a display signal that can be displayed by the display 6020, based on a display control signal generated by the CPU 6000 in accordance with a program. The display control unit 6003 supplies the generated display signal to the display 6020. The display 6020 displays a screen in accordance with the supplied display signal.

[0041] The data interface (I / F) 6005 is an interface for inputting and outputting data to and from external devices. In this example, an input device 6021 including a pointing device such as a mouse and a keyboard is connected to the data I / F 6005. However, the input device 6021 may be a device built into the information processing device 60. The communication I / F 6006 controls communication with the network.

[0042] Communications between the information processing device 60 and the video selection device 40, receiver 20, and transmitter 22 are performed using the data I / F 6005 and communication I / F 6006 as appropriate.

[0043] In the information processing device 60, the CPU 6000 executes the information processing program according to the embodiment, thereby configuring the above-mentioned radio wave condition acquisition unit 601, operation plan creation unit 602, map generation unit 603, display unit 604, input unit 605 and communication unit 606, each as, for example, a module, on the main memory area of ​​the RAM 6002.

[0044] The information processing program can be acquired from an external device (e.g., a server) via a network such as a LAN or the Internet by communication via the communication I / F 6006, and can be installed on the information processing device 60. However, the information processing program may be provided by being stored in a removable storage medium such as a CD (Compact Disk), a DVD (Digital Versatile Disk), or a USB (Universal Serial Bus) memory.

[0045] In the above description, it has been explained that one information processing device 60 includes the functions of the radio wave condition acquisition unit 601 and the operation plan creation unit 602, but this is not limited to this example. For example, the functions of the radio wave condition acquisition unit 601 and the operation plan creation unit 602 may be configured to be included in separate information processing devices that are communicatively connected to each other.

[0046] 5 is a functional block diagram illustrating an example of the functions of a video selection device 40 applicable to the embodiment. In FIG. 5, the video selection device 40 includes a video processing unit 400, a display unit 401, and an operation unit 402.

[0047] The display unit 401 displays a screen on a display in accordance with a display signal output from the video processing unit 400. The operation unit 402 accepts a user operation and passes a control signal to the video processing unit 400 in accordance with the user operation.

[0048] A plurality of pieces of video data are input to the video processing unit 400. The video processing unit 400 performs a synthesis process to display on a single screen each input video formed by the plurality of pieces of video data that have been input and an output video to be output from the video processing unit 400. The video processing unit 400 transfers to the display unit 401 a display signal to display on a display the screen synthesized by the synthesis process.

[0049] Furthermore, the video processing unit 400 outputs the video data of the video selected from each input video by a user operation on the operation unit 402 as output video data from the video selection device 40. Furthermore, the video processing unit 400 can apply a predetermined effect processing to the video data of the video selected from each input video in accordance with the user operation on the operation unit 402.

[0050] FIG. 6 is a schematic diagram showing an example of a video selection screen displayed on a display by the display unit 401 of the video selection device 40, which is applicable to the embodiment.

[0051] 6, video selection screen 403 includes output video display area 410 and input video display area 420. In this example, output video display area 410 includes display area 411 where the video currently being output is displayed, and display area 412 where output candidate videos are displayed.

[0052] Meanwhile, input video display area 420 includes multiple display areas 421, each displaying an input video captured by a different camera. In this example, input video display area 420 includes multiple (eight in this example) display areas 421, allowing multiple input videos to be viewed simultaneously. In the example of FIG. 6, input video display area 420 includes eight display areas 421, allowing eight input videos to be viewed simultaneously. Furthermore, nine or more input videos can be displayed by, for example, switching the display of input video display area 420. The number of display areas 421 included in input video display area 420 is not limited to eight, and may be seven or less, or nine or more. Furthermore, in this example, videos from seven cameras identified by camera IDs [1c] to [7c] are displayed as input videos in each display area 421.

[0053] The user can specify an image to be output from each input image displayed in input image display area 420 by operating operation unit 402. In the example shown in the figure, the input image from camera ID [2c] is specified as the image to be output, as indicated by the bold frame in display area 421. In output image display area 410, the image from camera ID [2c] specified in input image display area 420 is displayed in display area 411 as the output image being output.

[0054] Display area 412 displays the video that is to be output next after the output video displayed in display area 411. Furthermore, video processing unit 400 can apply a predetermined effect to the output video displayed in display area 411, for example, in response to a user operation on operation unit 402. Furthermore, when switching the output video from the video displayed in display area 411 to the video displayed in display area 412, video processing unit 400 can apply a transition effect associated with switching of the output video, such as cross-fade processing.

[0055] The video selection device 40 may be configured as a hardware configuration including a CPU, a DSP (Digital Signal Processor), a RAM, a storage device, an interface compatible with a plurality of video data, and the like.

[0056] 7A and 7B are schematic diagrams showing an example of the appearance of the moving device 10 applicable to the embodiment. Fig. 7A is a perspective view showing the appearance of the moving device 10, and Fig. 7B is a diagram of the moving device 10 as seen from the bottom side.

[0057] The mobile device 10 illustrated in Fig. 7A has an exterior shape that is close to a cylinder. The outer housing 105 includes an upper housing 105a and a lower housing 105b. A gap is provided between the upper housing 105a and the lower housing 105b. A sensor that recognizes the surrounding environment is provided at the same height as the gap. The gap is formed so as not to interfere with the sensor's recognition.

[0058] An opening is formed in the top of upper housing 105a so as not to interfere with imaging by camera 100. Camera 100 is provided so as to be rotatable horizontally relative to housing 105. Antenna 101 is provided at a predetermined position on upper housing 105a so as not to interfere with imaging by camera 100. Here, antenna 101 combines the functions of antennas 101t and 101r described above. Of antennas 101t and 101r, at least antenna 101t is provided, for example, on a portion of the periphery of housing 105.

[0059] In FIG. 7B, the moving mechanism 104 moves the moving device 10 in any direction on the stage 50 and also rotates the moving device 10. In the example shown in FIG. 7B, the moving mechanism 104 is configured to include a plurality of rollers. The rollers are incorporated into, for example, a dolly or the like (not shown) and abut against the stage 50 through openings formed below the lower housing 105b. The moving mechanism 104 may be configured to include, for example, Mecanum wheels, enabling omnidirectional movement and rotation of the moving device 10. This allows the moving device 10 to escape from a situation where it is trapped by a person without unnecessary rotation, or to freely position itself among complicated equipment.

[0060] The mobile device 10 is capable of independently controlling the orientation of the housing 105 (the direction in which the antenna 101 is provided is defined as the rear) and the shooting direction of the camera 100. In other words, the mobile device 10 is capable of moving and rotating the housing 105 while maintaining the direction in which the camera 100 shoots.

[0061] Fig. 8 is a block diagram showing an example of the hardware configuration of a mobile device 10 applicable to the embodiment. In Fig. 8, the mobile device 10 includes a CPU 1000, a ROM 1001, a RAM 1002, a mobile mechanism drive unit 1003, a storage device 1004, a sensor I / F 1005, a camera I / F 1006, and a communication I / F 1007, which are communicatively connected to each other via a bus 1010.

[0062] The storage device 1004 is a non-volatile storage medium such as a hard disk drive or flash memory. The CPU 1000 controls the overall operation of the mobile device 10 in accordance with programs stored in the ROM 1001 or the storage device 1004, using the RAM 1002 as a work memory.

[0063] The moving mechanism driving unit 1003 includes a power source such as a motor and a driving circuit for driving the power source, and drives the moving mechanism 104 according to instructions from the CPU 1000. The sensor I / F 1005 is an interface for the sensor 120, and transfers a sensor output signal output from the sensor 120 to the bus 1010. The sensor 120 includes at least a position sensor that acquires the current position of the moving device 10. The sensor 120 may acquire the current position by combining a gyro sensor, an acceleration sensor, etc., or may acquire the current position by communication via Wi-Fi (registered trademark), a predetermined beacon, etc.

[0064] The camera I / F 1006 is an interface for the camera 100, and transmits and receives various data such as video data, shooting control data, and status information to and from the camera 100. The communication I / F 1007 controls communication between the transmitter 102 and the receiver 103.

[0065] FIG. 9 is a functional block diagram of an example for explaining the functions of the mobile device 10 applicable to the embodiment.

[0066] 9, the mobile device 10 includes an imaging unit 110, a sensor information acquisition unit 111, a communication unit 112, and a drive control unit 113. The imaging unit 110, the sensor information acquisition unit 111, the communication unit 112, and the drive control unit 113 are configured by, for example, executing a program on a CPU. However, the imaging unit 110, the sensor information acquisition unit 111, the communication unit 112, and the drive control unit 113 may be configured by hardware circuits that operate in cooperation with each other.

[0067] The imaging unit 110 controls the imaging operation of the camera 100. The sensor information acquisition unit 111 acquires sensor information based on a sensor output signal from the sensor 120. For example, the sensor information acquisition unit 111 acquires position information indicating at least the current position of the mobile device 10 based on the sensor output signal.

[0068] The communication unit 112 controls the transmission process by the transmitter 102 and the reception process by the receiver 103 via the communication I / F 1007 .

[0069] The drive control unit 113 generates a control signal for driving the moving mechanism 104 and passes it to the moving mechanism driving unit 1003. For example, the drive control unit 113 receives, from the communication unit 112, motion plan data indicating a motion plan created by the motion plan creating unit 602 in the information processing device 60. The drive control unit 113 generates a drive signal for driving the moving mechanism 104 in accordance with the received motion plan data. In this way, the moving device 10 moves autonomously based on the motion plan.

[0070] The mobile device 10 stores the received operation plan data in the storage device 1004 or the RAM 1002. When the mobile device 10 receives new operation plan data, the mobile device 10 may overwrite the already stored operation plan data with the newly received operation plan data.

[0071] 3. First Embodiment Next, a first embodiment of the present disclosure will be described.

[0072] In the first embodiment, the information processing device 60 generates an operation plan for the mobile device 10 by the operation plan creation unit 602 based on the radio wave conditions acquired by the radio wave condition acquisition unit 601. At this time, the operation plan creation unit 602 creates an operation plan for the mobile device 10 based on the radio wave conditions so that the receiver 20 can receive video data transmitted from the mobile device 10 without interruption and the mobile device 10 moves while avoiding areas where the radio wave strength is below a threshold.

[0073] Furthermore, the information processing device 60 visualizes the radio wave conditions and presents them to the user using the map generation unit 603. At this time, the map generation unit 603 explicitly presents areas where the radio wave strength is equal to or less than a threshold in the visualized radio wave conditions.

[0074] FIG. 10 is a schematic diagram showing an example of a visualization screen 70 that displays the visualization of the radio wave condition and the movement route of the mobile device 10 according to the operation plan according to the first embodiment.

[0075] As shown in Fig. 10, it is assumed that the mobile device 10 is initially positioned on the right edge of the diagram of the stage 50. An area where the radio wave intensity is below a threshold is generated by equipment 52 disposed between the mobile device 10 and the receiver 20. In the example of Fig. 10, the map generating unit 603 visualizes on the visualization screen 70 the area where the radio wave intensity is below the threshold, i.e., the area where there is a high possibility that the image of the transmitted video data will be interrupted or distorted if the video data is transmitted, as a danger area 200.

[0076] Furthermore, the map generating unit 603 displays a route A according to the operation plan created by the operation plan creating unit 602 on the visualization screen 70. As shown in FIG. 10 , the route A is set so as to avoid the danger area 200.

[0077] The map generating unit 603 sequentially updates the position of the mobile device 10 on the visualization screen 70 as the mobile device 10 moves on the stage 50. The map generating unit 603 also sequentially updates the display of the danger area 200 when the radio wave conditions change.

[0078] Furthermore, the map generating unit 603 may display on the visualization screen 70 a danger area 200 based on radio wave conditions estimated from the structural relationship of the environment, such as the positional relationship between the equipment 52 arranged on the stage 50 and the antenna 20. As an example, it can be assumed that an area where there is no obstruction to the radio waves transmitted from the antenna 20 has good radio wave conditions. On the other hand, it can be assumed that an area where there is obstruction to the radio waves transmitted from the antenna 20 has slightly poor radio wave conditions, and the map generating unit 603 can display such an area as the danger area 200.

[0079] 11 is a schematic diagram showing an example in which the visualization screen according to the first embodiment is applied to a screen that represents a three-dimensional space. For example, when a user wears a so-called see-through head-mounted display (AR (Augmented Reality) glasses) that transmits an image of the real space, a display indicating a danger area 200 can be superimposed on an image of a stage 50 that can be seen through the AR glasses.

[0080] For convenience, the glass portion of the AR glasses will be referred to as the screen, and the image projected through the glass portion to the user's eye will be described as the image displayed on the screen. The camera (not shown) is assumed to be mounted on a mobile device 10drn (e.g., a drone) that can move freely up, down, left, right, forward, and backward in real space and can stay stationary within the real space.

[0081] 11, the visualization screen 71 displayed on the AR glasses includes a performer 51 and equipment 52, which are images in real space, as well as a display of a danger area 200 generated by the map generation unit 603. In this example, the performer 51 on the right side of the visualization screen 71 and the equipment 52 on the left side create areas where the radio wave intensity is below the threshold, and these are displayed as danger areas 200.

[0082] Furthermore, the map generating unit 603 displays a route B including height information in the three-dimensional space in accordance with the operation plan created by the operation plan creating unit 602 on the visualization screen 71. In this example, the route B is also set so as to avoid the danger area 200, as shown in FIG.

[0083] In this way, the information processing device 60 according to the embodiment explicitly presents, based on the radio wave intensity, the danger area 200 where the image of the transmitted video data is likely to be interrupted on the visualization screen 70 or 71. This allows the user to easily confirm that the mobile device 10 or 10drn is moving in a manner that does not cause interruptions or disturbances in the image of the transmitted video data.

[0084] It is also conceivable that the user may manually operate the mobile device 10 or 10'. Even in this case, since the danger area 200 is clearly displayed on the visualization screen 70 or 71', it is easy to operate the mobile device 10 or 10' without causing interruptions or disturbances in the image due to the transmitted image data.

[0085] (3-1. First Modification of the First Embodiment) Next, a first modified example of the first embodiment will be described. An information processing device 60 according to the first modified example of the first embodiment allows a user to specify an acceptable level of radio wave intensity, which is evaluated as an absolute value, on a visualization screen including visualization information based on radio wave conditions. In the information processing device 60, an operation plan creation unit 602 determines a route for the mobile device 10 according to the level of radio wave intensity specified by the user, and creates an operation plan.

[0086] FIG. 12 is a schematic diagram showing an example of a visualization screen 80 according to a first modified example of the first embodiment.

[0087] 12, the visualization screen 80 includes a radio wave condition map 800. In the example of Fig. 12, the radio wave condition map 800 visualizes the distribution of radio wave intensity on the stage 50 using contour lines.

[0088] As an example, before starting filming using the mobile device 10, a user measures radio wave intensity at each position while moving the mobile device 10 across the stage 50 with performers 51 and equipment 52 arranged on the stage 50. The measured radio wave intensity at each position is associated with the measurement position and input to the information processing device 60, and is passed to the radio wave status acquisition unit 601, for example.

[0089] The radio wave condition acquisition unit 601 normalizes the received radio wave intensity at each location using a predetermined method and classifies the normalized radio wave intensity at each location into levels. For example, the radio wave condition acquisition unit 601 classifies each normalized radio wave intensity into five groups according to its value. Of the five groups, the radio wave condition acquisition unit 601 classifies the group with the highest radio wave intensity value as level Lv1, the group with the lowest radio wave intensity value as level Lv5, and the groups with intermediate values ​​as levels Lv4, Lv3, and Lv2, in ascending order.

[0090] Based on the radio wave intensity at each location classified into levels Lv1 to Lv5 by the radio wave condition acquisition unit 601, the map generation unit 603 generates a radio wave condition map 800 shown in Fig. 12, which is displayed with contour lines corresponding to each level. In the example of Fig. 12, each area corresponding to each of levels Lv1 to Lv5 is painted with a different shade of color according to the level. For example, in the example of the radio wave condition map 800 in Fig. 12, the area painted with the darkest color is the area at level Lv5, and the area that is not painted is the area at level Lv1.

[0091] In the example of FIG. 12, the radio wave condition map 800 displays the mobile device 10 and a route R according to an operation plan for the mobile device 10 superimposed on the radio wave intensity display.

[0092] 12, a slider 801 located on the right side can be used to specify an allowable level for radio wave strength by operating a knob 802. The allowable level for radio wave strength indicates the maximum allowable level for transmission of video data by the mobile device 10. For example, the operation plan creation unit 602 creates an operation plan for the mobile device 10, with the specified allowable level of radio wave strength set as the lower limit of radio wave strength when transmitting video data.

[0093] Here, the radio wave strength is classified into five discrete values ​​to generate the radio wave condition map 800, but this is not limited to this example. For example, the radio wave strength may be classified into finer levels to generate the radio wave condition map 800, or the radio wave strength may be treated as analog information, and the electric field strength may be treated as a continuous value to generate the radio wave condition map 800 using gradation or the like.

[0094] 13 is a schematic diagram showing an example of an operation on a slider 801 on a visualization screen 80 according to a first modified example of the first embodiment. In FIG. 13, sections (a), (b), and (c) show examples in which the knob 802 is moved to positions of level Lv1, level Lv3, and level Lv4, respectively. It can be seen that in each radio wave condition map 800, the radio wave condition map 800 is updated according to the position of the knob 802, and the filled-in area is changed. In the radio wave condition map 800, the area of ​​the level specified by the knob 802, i.e., the unfilled area, is the area to which the mobile device 10 is moved according to the operation plan.

[0095] In the example of section (a) in Fig. 13, level Lv1 is set as the allowable level, and only the area of ​​the radio wave intensity corresponding to level Lv1 is left unfilled. On the other hand, the route R of the mobile device 10 extends beyond the area of ​​level Lv1, which is the allowable level. For example, the operation plan creation unit 602 can avoid creating an operation plan for such a route R that extends beyond the area indicated by the allowable level.

[0096] 13, level Lv3 is set as the acceptable level, and the area corresponding to the acceptable level is expanded compared to the example of section (a). The operation plan creation unit 602 can create an operation plan for the route R of the mobile device 10 within the area indicated by this expanded acceptable level. However, in this case, the route R according to the operation plan is set to include an area where the radio wave intensity is level Lv3, which is lower than level Lv1, and therefore there is a risk that interruptions or disturbances may occur in the video data transmitted from the mobile device 10.

[0097] In the example of section (c) in Figure 13, level Lv4 is set as the acceptable level, and the area corresponding to the education level is further expanded compared to the example of section (b). The operation plan creation unit 602 can create an operation plan for a route R for the mobile device 10 within the area indicated by this expanded acceptable level, and can also create a new operation plan for, for example, a route R' for another mobile device 10'. In this case, the route R' according to the new operation plan is set to include an area where the radio wave intensity is level Lv4, which is even lower than level Lv3. Therefore, the risk of interruptions or distortions occurring in the video data transmitted from the mobile device 10 is even greater compared to the example of section (b).

[0098] 14 is a schematic diagram for explaining generation of a radio wave condition map 800 and creation of an operation plan according to a first modified example of the first embodiment. The map generation unit 603 generates a radio wave condition map based on the radio wave intensity at each position on the stage 50 acquired prior to image capture by the mobile device 10, for example, as described above (step S10). The radio wave condition map generated here is a static radio wave condition map in an initial state.

[0099] The map generating unit 603 also generates a radio wave condition map based on the location information and radio wave intensity acquired from the mobile device 10 during image capture and movement (step S11). This radio wave condition map is updated sequentially in accordance with the image capture and movement of the mobile device 10.

[0100] Furthermore, the map generating unit 603 acquires the tolerance level designated by the user using the slider 801 on the visualization screen 80 (step S12).

[0101] The map generating unit 603 generates and updates the radio wave condition map 800 based on these static radio wave condition maps, the successively updated radio wave condition map, and the tolerance level designated by the user (step S13). Here, the map generating unit 603 presents, on the radio wave condition map 800, an area in which the operation of the mobile device 10 is permitted.

[0102] The operation plan creating unit 602 acquires an operation plan that has been automatically or manually created (step S14). The operation plan creating unit 602 modifies the operation plan acquired in step S14 based on the radio wave condition map 800 generated by the map generating unit 603 in step S13. For example, the map generating unit 603 reflects the operation plan modified by the operation plan creating unit 602 in the radio wave condition map 800, for example, as a route for the mobile device 10, and presents it to the user.

[0103] The revised motion plan is different from the original motion plan obtained in step S14 in that the movement range of the mobile device 10 is restricted in accordance with the tolerance level in the radio wave condition map 800. As an example, when it is desired to move the mobile device 10 back and forth equally to the left and right in front of the performer 51 to photograph the performer 51, it is possible to restrict the mobile device 10 so that it does not go outside the area based on the tolerance level.

[0104] As described above, according to the first modification of the first embodiment, the area in which the mobile device 10 moves can be set based on the radio wave condition map 800. This allows the user to more freely set the area in which the mobile device 10 moves while reducing the risk of video interruption.

[0105] (3-2. Second Modification of the First Embodiment) Next, a second modified example of the first embodiment will be described. In the second modified example of the first embodiment, the orientation of the mobile device 10 is further controlled.

[0106] Fig. 15 is a diagram illustrating that the state of radio waves transmitted from the mobile device 10 differs depending on the orientation of the mobile device 10. In Fig. 15, the mobile devices 10a, 10b, and 10c are equipped with antennas 101ta, 101tb, and 101tc, respectively, for transmitting video data. In the example of Fig. 15, in the mobile devices 10a and 10b, there are no obstructions to radio waves between the antennas 101ta and 101tb and the receiver 20. Therefore, the receiver 20 can receive signals transmitted from the mobile devices 10a and 10b without any problems.

[0107] On the other hand, the antenna 101tc of the mobile device 10c faces in the opposite direction to the receiver 20. Therefore, the mobile device 10c itself acts as a shield, and there is a possibility that the signal transmitted from the antenna 101tc will not be received with sufficient signal strength by the receiver 20. Therefore, in the second modified example of the first embodiment, the autonomous movement of the mobile device 10 is optimized so that the antenna 101t faces the receiver 20 as much as possible.

[0108] 16 is a flowchart illustrating an example of a process for creating an operation plan according to the second modified example of the first embodiment. In step S100, the operation plan creation unit 602 automatically or manually creates an operation plan. In the next step S101, the operation plan creation unit 602 compares the position of the receiver 20 for receiving video data, which is installed at a known site (for example, the stage 50), with the position of the mobile device 10 based on the operation plan created in step S100.

[0109] In the next step S102, the operation plan creation unit 602 creates an operation plan in which the orientation of the mobile device 10 is corrected, based on the result of comparing the position of the receiver 20 in step S101 with the position of the mobile device 10 according to the operation plan. More specifically, the operation plan creation unit 602 corrects the orientation of the mobile device 10 (housing 105) in the operation plan created in step S100 so that the antenna 101t of the transmitter 102 that transmits video data in the mobile device 10 faces the receiver 20.

[0110] As described above, in the second modified example of the first embodiment, the operation plan creation unit 602 creates an operation plan for the mobile device 10 so that, when the mobile device 10 moves while capturing images, the antenna 101t for transmitting video data always faces the receiver 20. This prevents the mobile device 10 itself from becoming a blockage of the signal for transmitting video data, and enables stable transmission of video data.

[0111] 4. Second Embodiment Next, a second embodiment of the present disclosure will be described. In the second embodiment, the route of the mobile device 10 according to the operation plan is divided according to the radio wave conditions, and the operation is handed over from one mobile device 10 to another mobile device 10 at the dividing position.

[0112] 17A and 17B are diagrams illustrating the division of a route and the handover of the operation of the mobile device 10 according to the second embodiment. In FIG. 17A, three performers 51a, 51b, and 51c are positioned on a stage 50, and a case is considered in which performers 51a, 51b, and 51c are photographed in sequence from right to left in the figure. In this case, if there are no obstacles to the movement of the mobile device 10d and the reception of video data transmitted from the mobile device 10d, the operation plan will be, for example, a route R along which the mobile device 10d moves from right to left on the stage 50.

[0113] 17A, it is assumed that a dangerous area 200 is close to the portion of route R where mobile device 10d moves from performer 51a to performer 51b. In this case, when mobile device 10d passes through space 210 between dangerous area 200 and the edge of stage 50 (the lower edge in the figure), there is a risk that the video image transmitted from mobile device 10d will be interrupted. Furthermore, if mobile device 10d moves autonomously, passing through space 210 at the edge of stage 50 will pose a risk to the operation of mobile device 10d itself.

[0114] 17B, ​​the motion plan creation unit 602 creates a motion plan for the mobile device 10d according to a route Ra along which the mobile device 10d moves to just before the dangerous area 200. Furthermore, the motion plan creation unit 602 creates a motion plan for a mobile device 10e, which is different from the mobile device 10d, according to a route Rb from a position beyond the dangerous area 200.

[0115] 17A at a position corresponding to a space 210 that is close to the dangerous area 200 and too narrow for the mobile device 10d to pass through, and creates motion plans for routes Ra and Rb obtained by dividing the route R. The motion of the mobile device 10d is taken over by the mobile device 10e according to the motion plan for the route Ra and the motion plan for the route Rb.

[0116] In this way, even when the mobile device 10d crosses over the dangerous area 200 or approaches the dangerous area 200, it is possible to avoid a situation in which the video data of the performers 51a to 51c is interrupted by having the operation of the mobile device 10d taken over by the mobile device 10e. Furthermore, even when the route R includes an area that poses a risk to the passage of the mobile device 10d, it is possible to reduce the risk associated with movement by having the operation of the mobile device 10d taken over by the mobile device 10e across that area.

[0117] Furthermore, when the operation of mobile device 10d is handed over to mobile device 10e as described above, it is also possible to control video selection device 40 to connect the video data transmitted from mobile device 10d with the video data transmitted from mobile device 10e.

[0118] That is, the action plan creation unit 602 can communicate with the mobile device 10 to know the current location of the mobile device 10, and can therefore accurately grasp the timing at which the action will be handed over from the mobile device 10d to the mobile device 10e. Using information indicating the timing of this handover, the action plan creation unit 602 can instruct the video selection device 40 to automatically switch between the input video from the mobile device 10d and the input video from the mobile device 10e.

[0119] Furthermore, for example, the action plan creation unit 60 can use the information indicating the timing to virtually integrate the video data transmitted from the mobile devices 10d and 10e as if it were video data captured by a single camera, for example, at a stage preceding the video selection device 40. This integrated video data of the video data transmitted from the mobile devices 10d and 10e is input to the video selection device 40. In this way, for example, a user operating the video selection device 40 can switch input videos and perform editing work without being aware of the transfer of operations from the mobile device 10d to the mobile device 10e.

[0120] 18 is a flowchart illustrating an example of an operation plan creation process according to the second embodiment. In step S200, the operation plan creation unit 602 automatically or manually creates an operation plan. In the next step S201, the operation plan creation unit 602 compares the route R according to the operation plan created in step S200 with the radio wave conditions acquired by the radio wave condition acquisition unit 601.

[0121] In the next step S202, the operation plan creation unit 602 determines whether or not it is necessary to divide the route R based on the result of comparing the route R according to the operation plan in step S201 with the radio wave conditions. The operation plan creation unit 602 can determine that it is necessary to divide the route R, for example, when the route R crosses the dangerous area 200 based on the radio wave conditions or approaches the dangerous area 200 within a predetermined distance, or when the route R includes a portion with a width equal to or less than a predetermined width.

[0122] If the motion plan creation unit 602 determines in step S202 that it is necessary to divide the route R (step S202, "Yes"), it proceeds to step S203. In step S203, the motion plan creation unit 602 divides the route R into routes Ra and Rb, and creates motion plans for each of the mobile devices 10d and 10e to transfer the motion from the mobile device 10d to the mobile device 10e.

[0123] On the other hand, if the operation plan creation unit 602 determines in step S202 that it is not necessary to divide the route R (step S202, "No"), it proceeds to step S204. In step S204, the operation plan creation unit 602 creates an operation plan using a detour route as necessary. For example, when there is a route that is farther away from the danger area 200 or that is wider than the original route R without changing the subject to be photographed, the operation plan creation unit 602 may create an operation plan using a detour route.

[0124] After the processing of step S203 or step S204, the motion plan creation unit 602 shifts the processing to step S205. In step S205, the motion plan creation unit 602 presents the motion plan created in step S203 or step S204 to the user by, for example, displaying it on a screen. When the presented motion plan is approved by the user, the motion plan creation unit 602 shifts the processing to step S206 and executes the motion plan created in step S203 or step S204.

[0125] 5. Third Embodiment Next, a third embodiment of the present disclosure will be described. In the third embodiment, the signal strength of video data transmitted from each mobile device 10 and information indicating a transition of the signal strength are displayed on the video selection screen 403 of the video selection device 40.

[0126] Fig. 19 is a schematic diagram showing an example of a video selection screen 403 by a video selection device 40 according to the third embodiment. In the example of Fig. 19, an indicator 434 indicating the signal strength of video data transmitted from the corresponding mobile device 10 and a transition display 430 including information indicating the transition of the signal strength of the video data are arranged for each display area 421 on the video selection screen 403. The transition display 430 is overlaid on the video displayed in the display area 421, for example, at a predetermined transparency.

[0127] Fig. 20 is an enlarged schematic diagram of a display area 421 according to the third embodiment. In Fig. 20, an indicator 434 arranged in the upper left of the display area 421 indicates the current signal strength of the video data transmitted from the mobile device 10 corresponding to the display area 421 (in this example, the mobile device 10 equipped with the camera 100 with camera ID [2c]).

[0128] In the transition display 430 arranged at the bottom of the display area 421, the vertical axis indicates signal strength, and the horizontal axis indicates the passage of time toward the right. Furthermore, a bar 433 indicates the current time, and a level indication line 432 indicates the allowable level for signal strength. A characteristic line 431 indicates the transition of the signal strength of the video data transmitted from the mobile device 10 corresponding to the display area 421. On the characteristic line 431, the part to the right of the bar 433 indicating the current time indicates a predicted value of the signal strength. This predicted value can be calculated, for example, based on an operation plan for the mobile device 10 and radio wave conditions (radio wave condition map).

[0129] By checking this transition display 430, the user can determine whether or not there is a possibility that the video displayed in the corresponding display area 421 will be interrupted. For example, if there is a portion of the characteristic line 431 to the right of the bar 433 that exceeds the level indication line 432, the user can determine that there is a possibility that the video displayed in the display area 421 will be interrupted in the near future (for example, in a few seconds).

[0130] As an example, if the user determines based on the transition display 430 that there is a possibility that the video displayed in the display area 421 designated as the video to be output may be interrupted, the user can determine that there is a high risk of an accident in which the video being distributed is interrupted. In this case, the user can switch the video to be output to video from the camera 100 with a different camera ID. This can prevent an accident in which the video being distributed is interrupted.

[0131] In addition to this, it is also possible for the information processing device 60 to instruct the video selection device 40 to automatically switch the video to be output to a video from a camera 100 with a different camera ID, for example.

[0132] FIG. 21 is a schematic diagram for explaining the process of calculating the risk of video being distributed being interrupted according to the third embodiment.

[0133] The mobile device 10 executes the operation plan previously passed by the operation plan creating unit 602 and estimates its current location (step S300). The mobile device 10 transmits the operation plan and location information indicating the estimated current location. The operation plan and location information transmitted from the mobile device 10 are received by the information processing device 60. The mobile device 10 moves and captures images according to the operation plan (step S301), and transmits the captured video data (step S302).

[0134] It is assumed that the video selected by the video selection device 40 is the video based on the video data transmitted in step S302 as the video to be output.

[0135] The video data transmitted from the mobile device 10 is received by the receiver 20. The receiver 20 transfers the received video data to the video selection device 40 (step S303). The receiver 20 also passes signal strength information based on the video data transmitted from the mobile device 10 to the information processing device 60 as the radio wave conditions in the environment including the mobile device 10 and the receiver 20 (step S304).

[0136] The information processing device 60 predicts the risk of the video being interrupted by the video data transmitted from the mobile device 10 based on prior information (e.g., the location of the receiver 20, the placement of the performer 51 and equipment, etc. 52) and the radio wave conditions transmitted from the receiver 20 (step S305).

[0137] For example, information processing device 60 predicts the probability that the video will be interrupted if mobile device 10 moves from the current point in time based on the operation plan of mobile device 10, location information indicating the location of mobile device 10, and a radio wave condition map based on the radio wave conditions. As an example, based on the operation plan, location information, and radio wave condition map, if mobile device 10 moves to an area with weak radio wave strength, it is predicted that the probability that the video will be interrupted is high depending on the radio wave strength at the destination. Information processing device 60 transfers a value indicating this probability to video selection device 40 as a risk prediction result.

[0138] The video selection device 40 can automatically switch the video being output to a video captured by a different camera 100, depending on the risk prediction result transferred from the information processing device 60. Furthermore, the video selection device 40 may display a message that there is a risk that the video being output may be interrupted, and urge the user to switch the video.

[0139] FIG. 22 is a flowchart illustrating an example of video switching processing in accordance with the risk level in the video selection device 40 according to the third embodiment.

[0140] In step S400, the video selection device 40 acquires the predicted risk level for the video being output from the information processing device 60. In the next step S401, the video selection device 40 checks the predicted risk level N seconds later (N is a positive value) based on the predicted result acquired in step S400. Note that N seconds may be a value preset in the video selection device 40, or may be a value set by the user.

[0141] In the next step S402, the video selection device 40 determines whether the risk level confirmed in step S401 exceeds a threshold. If the video selection device 40 determines that the risk level is equal to or lower than a preset threshold level (step S402, "No"), the process returns to step S400. On the other hand, if the video selection device 40 determines that the risk level exceeds the threshold level (step S402, "Yes"), the process proceeds to step S403.

[0142] In step S403, the video selection device 40 switches the video being output to a video from another video input. More specifically, the video selection device 40 switches the mobile device 10 that transmits the video to be used for output from the current mobile device 10 to another mobile device 10. Alternatively, the video selection device 40 notifies the user that there is a risk that the video being output may be interrupted.

[0143] There are various possible methods for selecting a switching destination video when video selection device 40 switches the video being output to a video from another video input. For example, it is possible to register the most frequently used video in advance as a switching video (by displaying it in display area 412 where output candidate videos are displayed), and then switch the video being output to this registered video. The video registered for switching is not limited to the most frequently used video. For example, it is also possible to register a highly versatile video, such as a video of stage 50 shot from a wide angle that shows the entire stage 50, as a switching video.

[0144] For example, the image to be switched to may be an image based on image data transmitted under good radio wave conditions. Furthermore, for example, the image to be switched to may be an image captured with a similar composition to the image being output. Whether the compositions are similar or not can be determined based on the position and orientation of the mobile device 10 capturing the image.

[0145] As described above, according to the third embodiment, the video selection device 40 switches the video to be output depending on the risk level of the video currently being output, which is determined by the information processing device 60. Alternatively, the video selection device 40 prompts the user to switch the video to be output depending on the risk level. This makes it possible to prevent accidents such as interruptions or disruptions to the video currently being distributed.

[0146] (5-1. Modification of the third embodiment) Next, a modified example of the third embodiment will be described. In the third embodiment described above, the risk level for the video being output is calculated based on the radio wave conditions. In contrast, in the modified example of the third embodiment, the risk level is calculated based on the video being output.

[0147] Fig. 23 is a schematic diagram for explaining a process for determining the risk of video being distributed being interrupted, according to a modification of the third embodiment. In Fig. 23, a received video determination process shown in step S310 is added to the configuration shown in Fig. 21. In Fig. 21, the information processing device 60 determines whether or not there is video distortion in the received video data, based on the radio wave conditions passed from the receiver 20 (step S310).

[0148] The information processing device 60 predicts whether or not there is video distortion based on, for example, signal strength information included in the radio wave conditions. For example, the information processing device 60 can digitize the signal strength indicated by the signal strength information, and predict that video distortion will occur if the value indicating the signal strength is smaller than a predetermined value. Without being limited to this, the information processing device 60 may detect whether or not there is video distortion directly from the video data. For example, the information processing device 60 can detect whether or not there is video distortion based on the continuity of the video. In this case, the video selection device 40 can also detect whether or not there is video distortion.

[0149] For example, when an unusual device is used as the receiver 20 at the filming location, a decision based on the video may be required. One example is when the receiver 20 does not have the function of measuring the signal strength. The modified example of the third embodiment is suitable for use in such a case.

[0150] In addition, if a disturbance in the video is detected during the distribution of video data captured by the mobile device 10, the location (coordinates) where the disturbance was detected can be acquired and recorded, and the location can be reflected in the radio wave condition map 800.

[0151] Fig. 24 is a schematic diagram showing an example of a modification of the third embodiment in which the position where image disturbance is detected is reflected on a radio wave condition map 800. In Fig. 24, the position where image disturbance is detected is indicated by a mark 810 on the radio wave condition map 800. The user can set a route for the mobile device 10 by referring to each mark 810 on the radio wave condition map 800. The information indicated by the marks 810 is closer to the actual situation at the shooting site, and is therefore expected to provide useful hints for the route for the mobile device 10.

[0152] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0153] The present technology can also be configured as follows. (1) an acquisition unit that acquires radio wave conditions in the environment; a creation unit that creates an operation plan based on the radio wave condition and radio wave strength required to transmit video data to be distributed; Equipped with Information processing system. (2) The camera further includes a moving device having a photographing function, a transmitting function for transmitting video data photographed by the photographing function, and a moving mechanism for rotating and moving the housing, The acquisition unit acquiring the radio wave conditions in the environment in which the mobile device operates; The creation unit creating the operation plan indicating a plan for the operation of the mobile device based on the radio wave conditions and the radio wave strength required for the mobile device to transmit the video data to be distributed; The information processing system according to (1) above. (3) the mobile devices include a first mobile device and a second mobile device; The creation unit creating the operation plan for handing over the operation from the first mobile device to the second mobile device based on the radio wave conditions and the radio wave strength; The information processing system according to (2) above. (4) The creation unit generating the motion plan further based on a movement risk of the first mobile device; The information processing system according to (3) above. (5) a map generation unit that generates a map that visualizes the radio wave condition; a display unit that displays the map; Further provided with The information processing system according to any one of (2) to (4) above. (6) The creation unit creating the operation plan further based on an allowable level for the radio wave strength in the radio wave conditions set for the map; The information processing system according to (5) above. (7) The creation unit creating the operation plan based on the tolerance level set in response to a user operation based on the map displayed on the display unit; The information processing system according to (6) above. (8) The map generation unit updating the map based on the tolerance level set in response to the user operation; The information processing system according to (7) above. (9) a video display unit that receives the video data transmitted from the mobile device and displays a video based on the received video data; a video selection unit that selects a video to be distributed from the videos displayed on the video display unit; Furthermore, The creation unit predicting a transition of the radio wave condition for the mobile device based on the radio wave condition and the operation plan, and displaying information indicating the predicted transition of the radio wave condition on the video display unit; The information processing system according to any one of (2) to (8) above. (10) The creation unit selecting video data to be distributed from the video data transmitted from each of the mobile devices based on information indicating the predicted transition of the radio wave conditions for each of the mobile devices; The information processing system according to (9) above. (11) a determination unit that determines whether or not there is distortion in the video image based on the video data transmitted from the mobile device; Furthermore, selecting video data to be used for the distribution further based on the presence or absence of the disturbance; The information processing system according to (10) above. (12) a map generation unit that generates a map that visualizes the radio wave condition; a display unit that displays the map; Furthermore, The creation unit acquiring a position of the mobile device when the video data determined to have been disturbed was transmitted, and displaying information indicating the acquired position on the map; The information processing system according to (11) above. (13) The moving device is an antenna for transmitting the video data by the transmission function, the antenna being disposed on a part of the periphery of the housing of the mobile device; The creation unit creating the motion plan including information for controlling the orientation of the mobile device based on the position of the antenna; The information processing system according to any one of (2) to (12) above. (14) an acquisition unit that acquires radio wave conditions in the environment; a creating unit that creates an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; Equipped with Information processing device. (15) Executed by a processor, an acquisition step of acquiring radio wave conditions in the environment; a creating step of creating an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; having Information processing methods. (16) an acquisition step of acquiring radio wave conditions in the environment; a creating step of creating an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; An information processing program that causes a computer to execute the above. [Explanation of symbols]

[0154] 1. Information Processing Systems 10,10',10a,10b,10c,10d,10e,10drn Mobile device 20,103 receivers 21, 23, 101, 101t, 101ta, 101tb, 101tc, 101r antennas 22,102 transmitters 40 Video selection device 50 stages 51 Performers 52 Equipment, etc. 60 Information processing equipment 70,71,80 Visualization screen 100 cameras 104 Moving mechanism 105 Case 110 Imaging unit 111 Sensor information acquisition unit 112 Communications Department 113 Drive control unit 200 Danger Zone 403 Video selection screen 411,412,421 display area 430 Progress display 434 indicator 601 Radio wave status acquisition unit 602 Motion Planning Unit 603 Map Generation Unit 604 Display section 605 Input section 606 Communications Department 800 Signal Condition Map 801 Slider 802 Nobu 810 marks

Claims

1. an acquisition unit that acquires radio wave conditions in the environment; a creation unit that creates an operation plan based on the radio wave condition and radio wave strength required to transmit video data to be distributed; Equipped with Information processing system.

2. The camera further includes a moving device having a photographing function, a transmitting function for transmitting video data photographed by the photographing function, and a moving mechanism for rotating and moving the housing, The acquisition unit acquiring the radio wave conditions in the environment in which the mobile device operates; The creation unit creating the operation plan indicating a plan for the operation of the mobile device based on the radio wave conditions and the radio wave strength required for the mobile device to transmit the video data to be distributed; The information processing system according to claim 1 .

3. the mobile devices include a first mobile device and a second mobile device; The creation unit creating the operation plan for handing over the operation from the first mobile device to the second mobile device based on the radio wave conditions and the radio wave strength; The information processing system according to claim 2 .

4. The creation unit generating the operation plan further based on a movement risk of the first mobile device; The information processing system according to claim 3 .

5. a map generation unit that generates a map that visualizes the radio wave condition; a display unit that displays the map; Further provided with The information processing system according to claim 2 .

6. The creation unit creating the operation plan further based on an allowable level for the radio wave strength in the radio wave conditions set for the map; The information processing system according to claim 5 .

7. The creation unit creating the operation plan based on the tolerance level set in response to a user operation based on the map displayed on the display unit; The information processing system according to claim 6 .

8. The map generation unit updating the map based on the tolerance level set in response to the user operation; The information processing system according to claim 7 .

9. a video display unit that receives the video data transmitted from the mobile device and displays a video based on the received video data; a video selection unit that selects a video to be distributed from the videos displayed on the video display unit; Furthermore, The creation unit predicting a transition of the radio wave condition for the mobile device based on the radio wave condition and the operation plan, and displaying information indicating the predicted transition of the radio wave condition on the video display unit; The information processing system according to claim 2 .

10. The creation unit selecting video data to be distributed from the video data transmitted from each of the mobile devices based on information indicating the predicted transition of the radio wave conditions for each of the mobile devices; The information processing system according to claim 9 .

11. a determination unit that determines whether or not there is distortion in the video image based on the video data transmitted from the mobile device; Furthermore, selecting video data to be used for the distribution further based on the presence or absence of the disturbance; The information processing system according to claim 10.

12. a map generation unit that generates a map that visualizes the radio wave condition; a display unit that displays the map; Furthermore, The creation unit acquiring a position of the mobile device when the video data determined to have been disturbed was transmitted, and displaying information indicating the acquired position on the map; The information processing system according to claim 11.

13. The moving device is an antenna for transmitting the video data by the transmission function, the antenna being disposed on a part of the periphery of the housing of the mobile device; The creation unit creating the motion plan including information for controlling the orientation of the mobile device based on the position of the antenna; The information processing system according to claim 2 .

14. an acquisition unit that acquires radio wave conditions in the environment; a creating unit that creates an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; Equipped with Information processing device.

15. Executed by a processor, an acquisition step of acquiring radio wave conditions in the environment; a creating step of creating an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; having Information processing methods.

16. an acquisition step of acquiring radio wave conditions in the environment; a creating step of creating an operation plan based on the radio wave condition and the radio wave strength required to transmit the video data; An information processing program that causes a computer to execute the above.

Images