Performance device, information processing device, performance system, and control method

Abstract

To improve interaction performance. [Solution] The autonomously mobile performance device comprises a moving mechanism, a light-emitting unit positioned on the outside of the housing, a sensor for recognizing performers, and a control unit that controls the movement of the moving mechanism and the light-emitting expression of the light-emitting unit in accordance with the movement of the performer recognized by the sensor, so as to form part of the performance together with the performer.

Description

Technical Field

[0001] The present disclosure relates to a presentation device, an information processing device, a presentation system, and a control method.

Background Art

[0002] Devices for interactive robots that perform interactions in language have been proposed (see, for example, Patent Document 1).

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using a device for shooting a music event or the like, an interaction for moving while avoiding a collision with the surrounding environment is required. In an environment such as a music event, it is difficult to have a dialogue, and an interaction in language is not sufficient.

[0005] One aspect of the present disclosure improves interaction performance.

Means for Solving the Problems

[0006] A presentation device according to one aspect of the present disclosure is an autonomously movable presentation device, including a moving mechanism, a light-emitting unit disposed outside the housing, a sensor for recognizing a performer, and a control unit for controlling the movement by the moving mechanism and the light-emitting expression by the light-emitting unit in accordance with the movement of the performer recognized by the sensor so as to form a part of a performance together with the performer. Furthermore, the information processing device relating to one aspect of this disclosure includes a UI unit that displays a first timeline representing the performance of a performer and a second timeline representing the operation of an autonomously mobile performance device different from the performer, in a manner that allows the user to select any timing on each timeline, and accepts user operations to select any timing on at least one of the first timeline and the second timeline, and a processor that sets a plan for the operation of a performance device so as to link the operation of the performance device to the performance in response to the user operation.

[0007] A performance system relating to one aspect of this disclosure comprises the performance device described above and the information processing device described above, wherein the control unit of the performance device controls the movement by the moving mechanism and the light emission expression by the light emission unit based on a plan set by the processor of the information processing device and the recognition results by the sensors of the performance device.

[0008] One aspect of the present disclosure relates to a control method for an autonomously mobile performance device, the performance device comprising a moving mechanism, a light-emitting unit located on the outside of the housing, and a sensor for recognizing a performer, and the control method includes controlling the movement by the moving mechanism and the light-emitting unit in accordance with the movement of the performer recognized by the sensor, so as to form part of the performance together with the performer. [Brief explanation of the drawing]

[0009] [Figure 1] This figure shows an example of a schematic configuration of the imaging system according to the embodiment. [Figure 2] This figure shows an example of a schematic configuration of a mobile device. [Figure 3] This diagram shows an example of a functional block in an imaging system. [Figure 4] This figure shows examples of movement and / or imaging operations. [Figure 5] This figure shows examples of movement and / or imaging operations. [Figure 6] This figure shows examples of movement and / or imaging operations. [Figure 7] A diagram showing examples of movement operations and / or imaging operations. [Figure 8] A diagram showing examples of movement operations and / or imaging operations. [Figure 9] A diagram showing examples of movement operations and / or imaging operations. [Figure 10] A diagram showing examples of the imaging ranges of multiple mobile devices. [Figure 11] A diagram showing an example of a light emission operation. [Figure 12] A diagram showing an example of a light emission operation. [Figure 13] A diagram showing an example of a light emission operation. [Figure 14] A diagram showing an example of a light emission operation. [Figure 15] A diagram showing an example of a light emission operation. [Figure 16] A diagram showing an example of a light emission operation. [Figure 17] A diagram showing an example of a light emission operation. [Figure 18] A diagram showing an example of a light emission operation. [Figure 19] A diagram showing an example of a lighting operation. [Figure 20] A diagram showing an example of a lighting operation. [Figure 21] A diagram showing an example of a lighting operation. [Figure 22] A diagram showing an example of a lighting operation. [Figure 23] A flowchart showing an example of a process (control method for a mobile device) executed in an imaging system. [Figure 24] A front view showing an example of the appearance of a mobile device when viewed from the front. [Figure 25] A rear view showing an example of the appearance of a mobile device when viewed from the rear. [Figure 26] A right side view showing an example of the appearance of a mobile device when viewed from the right side. [Figure 27] A left side view showing an example of the appearance of a mobile device when viewed from the left side. [Figure 28] A plan view showing an example of the appearance of a mobile device when viewed from above. [Figure 29] This is a bottom view showing an example of the appearance of the mobile device when viewed from below. [Figure 30] This figure shows an example of a schematic configuration of a mobile device. [Figure 31] This figure shows an example of a schematic configuration of a mobile device. [Figure 32] This is a front view showing an example of the appearance of the mobile device when viewed from the front. [Figure 33] This is a rear view showing an example of the appearance of the mobile device when viewed from the rear. [Figure 34] This is a right-side view showing an example of the appearance of the mobile device when viewed from the right side. [Figure 35] This is a left side view showing an example of the appearance of the mobile device when viewed from the left side. [Figure 36] This is a plan view showing an example of the appearance of a mobile device when viewed from above. [Figure 37] This is a bottom view showing an example of the appearance of the mobile device when viewed from below. [Figure 38] This figure shows an example of the hardware configuration of an operation instruction device, etc. [Figure 39] This figure shows an example of light emission operation. [Figure 40] This figure shows an example of light emission operation. [Figure 41] This figure shows an example of a cross-section. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described in detail below with reference to the drawings. In the following embodiments, the same elements are denoted by the same reference numerals to avoid redundant descriptions.

[0011] This disclosure will be explained in the order of the items shown below. 1. Embodiments 1.1 Example of a general configuration 1.2 Examples of movement and imaging operations 1.3 Examples of Light Emission Operation 1.4 Examples of lighting operation 1.5 Example of processing 1.6 Example of the appearance of a mobile device 1.7 Example Hardware Configuration 2. Effects

[0012] 1. Embodiments 1.1 Example of a general configuration Figure 1 shows an example of a schematic configuration of an imaging system according to an embodiment. In this example, the imaging system is used for streaming a live music event. Multiple performers 5 are performing on a stage ST. To distinguish each performer 5, they are shown as performer 5-1, etc.

[0013] The imaging system 1 includes a mobile device 10, a tag T, an environmental sensor 20, and an operation instruction device 30. The mobile device 10 may consist of multiple mobile devices 10, and in this example, the same number of mobile devices 10 as the number of performers 5 are used. To distinguish each mobile device 10, they are shown as mobile device 10-1, etc.

[0014] The mobile device 10 operates to photograph a subject. The subject is mainly the performer 5, but other objects can also be subjects. Since there are multiple mobile devices 10, various combinations of mobile devices 10 and subjects are possible. The mobile device 10 images the performer 5 while moving. At least some of the operations of the mobile device 10 are autonomous operations that automatically perform collision avoidance with the surrounding environment (obstacle avoidance, etc.) without requiring real-time manual operation. The mobile device 10 transmits the imaging results (video) to the operation instruction device 30 in real time. Further details of the mobile device 10 will be described later.

[0015] Tag T is used to acquire the location information of Performer 5. Each Performer 5 has a Tag T. An example of a Tag T is a transmitter that emits pulse signals, beacon signals, etc. The Tag T that each Performer 5 has is referred to as Tag T-1, etc., and is illustrated in the diagram.

[0016] The environmental sensor 20 recognizes (detects, etc.) the environment in which the mobile device 10 is located. Recognition of the environment includes recognition of the position of the performer 5. Any sensor capable of recognizing the position of the performer 5 may be included in the environmental sensor 20. In this example, the environmental sensor 20 recognizes the position of the performer 5 using the sensor group 21. The position information of the performer 5 recognized by the environmental sensor 20 is transmitted to the operation instruction device 30 in real time. Further details of the environmental sensor 20 will be described later.

[0017] The control device 30 remotely controls the mobile device 10 and also distributes video in real time. The control device 30 may be implemented using a general-purpose computer, which in this example is a laptop. A tablet device may also be used. Further details of the control device 30 will be described later.

[0018] Figure 2 shows an example of the schematic configuration of a mobile device. Among the components of the mobile device 10, the housing 11, housing 12, imaging unit 13, sensor 14, light-emitting unit 15, illumination unit 16, and mobile mechanism 17 are shown with reference numerals. The part hidden behind the housing 12 is drawn with a dashed line.

[0019] The housing 11 is an inner housing located inside the housing 12. In this example, the imaging unit 13, illumination unit 16, and moving mechanism 17 of the illustrated mobile device 10 are housed in the housing 11. The inside of the housing 11 houses a communication device, a control unit (control board, etc.), wiring, a battery, etc. (not shown).

[0020] The housing 12 is an outer housing located outside the housing 11. In this example, the light-emitting unit 15, one of the components of the illustrated mobile device 10, is provided in the housing 12. The housing 12 may be flexible, for example, to mitigate impact during collisions. The material, thickness, etc., of the housing 12 are selected as appropriate. An opening is formed in the upper part of the housing 12 so as not to interfere with imaging by the imaging unit 13 and recognition by the sensor 14. The part that forms the opening (the edge of the opening) is referred to as the opening 12a and is shown in the figure. The opening 12a has an annular shape (ring shape), more specifically a circular shape. The housing 12 also has an opening at the bottom.

[0021] The imaging unit 13 is positioned above the housing 11. The imaging unit 13 is mounted on the housing 11 via a neck rotation axis 13a and a three-axis gimbal 13b so that its position and orientation can be changed. The imaging unit 13 is, for example, a CCD (Charge Coupled Device) camera and is configured to be parameter-controllable. Examples of parameters include zoom magnification, depth of field, and angle of view.

[0022] Sensor 14 is positioned on the top of the housing 11. Sensor 14 recognizes the surrounding environment of the mobile device 10. Recognition of the surrounding environment includes recognition of objects present in the surroundings. Object recognition includes recognition of the type of object, recognition of the position of the object, etc. Examples of object types include performer 5, other mobile devices 10, and obstacles. Examples of object positions include the relative position of the object to the mobile device 10 (distance and direction, etc.), the position of the object on the stage ST (coordinates, etc.), etc.

[0023] Any sensor for recognizing an object may be included in sensor 14. Examples of such sensors include image sensors and distance measuring sensors. The imaging unit 13 may also serve as the image sensor.

[0024] Sensor 14 also recognizes the position of the mobile device 10. Any sensor capable of recognizing the position of the mobile device 10 may be included in sensor 14. Examples of such sensors include indoor positioning sensors.

[0025] In addition, the sensor 14 may include various other sensors for recognizing the distance traveled, speed, direction of movement, and orientation of the mobile device 10. Examples of such sensors include IMU (Inertial Measurement Unit) sensors, acceleration sensors, and angular velocity sensors.

[0026] The light-emitting section 15 is provided along the opening 12a of the housing 12. For example, the light-emitting section 15 is provided on the inner surface (e.g., the cut surface) of the opening 12a. If the opening 12a has an annular shape as described above, the light-emitting section 15 is arranged in an annular shape on the housing 12. The light-emitting section 15 may be provided in parts of the housing 12 other than the opening 12a, or it may be provided on the entire surface of the housing 12. An example will be explained later with reference to Figures 39 to 41. An example of a light source for the light-emitting section 15 is a plurality of point light sources arranged along the opening 12a. An example of a point light source is an LED (light-emitting diode). Multiple point light sources emitting light of different colors may be provided. The light sources may be covered with a smoke cover. The light-emitting section 15 performs various light emission expressions. Details of the light emission expressions will be described later.

[0027] The lighting unit 16 is located at the bottom of the housing 11. The lighting unit 16 includes lighting for projection, etc. For example, the lighting unit 16 illuminates the area in front and below the mobile device 10 through an opening formed at the bottom of the housing 12, and performs projection.

[0028] The moving mechanism 17 is located at the bottom of the housing 11. The moving mechanism 17 moves the mobile device 10 in any direction on the stage ST. In the example shown in Figure 2, the moving mechanism 17 consists of multiple rollers. The rollers are incorporated into, for example, a trolley (not shown) and contact the stage ST through an opening formed at the bottom of the housing 12. The moving mechanism 17 may be configured to include, for example, Mecanum wheels, enabling the mobile device 10 to move in all directions. This allows the mobile device 10 to escape from being trapped by a person without unnecessary rotation, or to freely position itself among intricate equipment.

[0029] Figure 3 shows an example of the functional blocks of the imaging system. Several typical functional blocks in the mobile device 10, the environmental sensor 20, and the operation instruction device 30 are illustrated.

[0030] The mobile device 10 includes an imaging unit 13, a sensor 14, a light-emitting unit 15, an illumination unit 16, a mobile mechanism 17, a communication unit 18, and a control unit 19. The housings 11 and 12 shown in Figure 2 are omitted from Figure 3. The imaging unit 13, sensor 14, light-emitting unit 15, illumination unit 16, and mobile mechanism 17 have been described earlier with reference to Figure 2. To reiterate regarding the sensor 14, the sensor 14 recognizes the position of the performer 5 and the mobile device 10, etc., and acquires this information as position information of the performer 5 and the mobile device 10, etc.

[0031] The communication unit 18 transmits and receives information with the operation instruction device 30. Short-range wireless communication such as wireless LAN may be used for communication. The communication unit 18 transmits video to the operation instruction device 30 in real time. The communication unit 18 also transmits the recognition results of the sensor 14, in particular the position information of the performer 5 and the position information of the mobile device 10, to the operation instruction device 30 in real time.

[0032] The control unit 19 controls the operation of the mobile device 10 by controlling each element of the mobile device 10. The operation of the mobile device 10 includes movement, imaging, light emission, and illumination. Control of movement includes controlling the direction and speed of movement of the mobile device 10. Control of imaging includes controlling the orientation (camera orientation) and parameters of the imaging unit 13. Control of light emission includes controlling the light emission expression performed by the light emission unit 15. Control of illumination includes controlling the projection performed by the illumination unit 16.

[0033] In one embodiment, the control unit 19 controls the operation of the mobile device 10 based on a set shooting plan. The shooting plan defines the operation of the mobile device 10. The shooting plan may be set before the performance of the performer 5 begins, or it may be set while the performance is in progress. In the latter case, the shooting plan may be changed (updated, etc.) at any time. The shooting plan is set by the operation instruction device 30, as will be described later.

[0034] The shooting plan may specify the operation of multiple mobile devices 10. By specifying the operation of each mobile device 10 so that they are linked or otherwise synchronized, the multiple mobile devices 10 will operate in coordination.

[0035] The control unit 19 controls the operation of the mobile device 10 (movement and imaging) so that the moving mechanism 17 moves the mobile device 10 and the imaging unit 13 captures images of the performer 5 recognized by the sensor 14, based on the set imaging plan. At least part of the operation of the mobile device 10 is autonomous. The control unit 19 automatically controls the operation of the mobile device 10 to achieve the imaging plan.

[0036] Based on or in lieu of the shooting plan, the control unit 19 may control the movement, imaging, light emission, and illumination operations in accordance with other operations. For example, the control unit 19 may cause the light emission unit 15 to perform light emission expressions in accordance with the movement, imaging, and illumination operations, or cause the illumination unit 16 to perform projections in accordance with the movement, imaging, and light emission operations.

[0037] A specific example of how the control unit 19 controls the operation of the mobile device 10 will be explained later with reference to Figure 4 and subsequent figures.

[0038] As explained earlier with reference to Figure 1, the tag T will not be explained again here.

[0039] The environmental sensor 20 includes a sensor group 21, a location information acquisition unit 22, and a communication unit 23.

[0040] The sensor group 21 consists of multiple sensors, each synchronized in timing by cable connections or the like. A sensor may be a receiver that receives the transmitted signal of tag T.

[0041] The position information acquisition unit 22 recognizes the distance and direction to tag T from the transmission signal of tag T received by each sensor in the sensor group 21. Since the positions of the sensor group 21 are known, the position of tag T and, consequently, the position of performer 5 possessing it are recognized. Various known methods may be used for recognition, and a detailed explanation is omitted here. The position information acquisition unit 22 acquires the recognized position of performer 5 as performer 5 position information.

[0042] The communication unit 23 transmits the location information acquired by the location information acquisition unit 22 to the operation instruction device 30 in real time.

[0043] The location information acquired by the location information acquisition unit 22 may be transmitted to the mobile device 10 via the operation instruction device 30 or directly from the environmental sensor 20. For example, if the mobile device 10 has not yet recognized the position of the performer 5, the location information from the environmental sensor 20 may be used as a reference.

[0044] The operation instruction device 30 includes a map generation unit 31, a UI unit 32, a distribution unit 33, and a communication unit 34.

[0045] The map generation unit 31 generates a map that shows the positions of the mobile device 10 and the performer 5 in real time. The map also includes terrain information (stage ST in the example of Figure 1). The position of the mobile device 10 is obtained as position information transmitted by the mobile device 10. The position of the performer 5 is obtained as position information transmitted by the mobile device 10 and the environmental sensor 20. Even when the mobile device 10 does not recognize the position of the performer 5, the position information of the performer 5 can be reliably obtained because of the position information transmitted by the environmental sensor 20. The map generated by integrating the position information from the mobile device 10 and the environmental sensor 20 can also be called an integrated map.

[0046] The UI unit 32 provides a user interface for presenting information to the user and accepting user operations. These operations may be performed while displaying the map generated by the map generation unit 31. The UI unit 32 is composed of, for example, a display and an operation panel (which may be a touch panel, etc.). Other components such as operation levers and faders may also be included in the UI unit 32. Several examples of how a user can use the UI unit 32 are described below.

[0047] For example, the user sets a shooting plan using the UI unit 32 that displays a map. The set shooting plan is transmitted to the mobile device 10, and the mobile device 10 operates based on the shooting plan. As described above, the map is an integrated map generated by integrating position information from the mobile device 10 and the environment-side sensor 20. By setting a shooting plan using the UI unit 32 that displays such a map, the mobile device 10 operates based on the recognition results of the sensor 14 of the mobile device 10 and the recognition results of the environment-side sensor 20. This makes it possible to improve the operational accuracy of the mobile device 10 compared to, for example, when the mobile device 10 operates based only on the recognition results of the sensor 14 of the mobile device 10.

[0048] As mentioned above, the shooting plan may be updated, and in that case, information regarding the update (for example, the updated shooting plan) is transmitted to the mobile device 10 so that the update of the shooting plan is reflected in the operation of the mobile device 10 in real time.

[0049] For example, the user can manually instruct the operation of the mobile device 10 using the UI unit 32. This enables manual operation of the mobile device 10. Examples of instructions include panning, tilting, and zooming of the imaging unit 13 of the mobile device 10, and adjusting the movement speed of the mobile device 10. The instructions are transmitted to the mobile device 10, and the mobile device 10 operates in real time according to the instructions.

[0050] For example, the user uses the UI unit 32 to select video footage. From the video footage obtained from each of the multiple mobile devices 10, the video footage to be used for distribution is selected.

[0051] The distribution unit 33 distributes video. The distribution may be live streaming. For example, the distribution unit 33 distributes video by sending it to a distribution server (not shown).

[0052] The communication unit 34 transmits and receives information with the mobile device 10, the environmental sensor 20, etc. Examples of information include location information, shooting plans, manual operation instructions, and video, as previously described.

[0053] 1.2 Examples of movement and imaging operations Let's explain the mobile device 10 again. As mentioned above, the control unit 19 of the mobile device 10 controls the operation of the mobile device 10. Several examples of the movement operation and imaging operation (movement by the movement mechanism 17 and imaging by the imaging unit 13) will be explained with reference to Figures 4 to 9, along with examples of setting up the imaging plan that defines the operation.

[0054] In the example shown in Figure 4, the control unit 19 moves the mobile device 10 along a set virtual path (virtual rail). Figure 4 shows an example of setting up a shooting plan that defines the operation of such a mobile device 10. The user sets up a virtual path on the stage ST using the UI unit 32 which displays a map as shown in the figure. For example, multiple virtual paths with different directions and lengths, as indicated by dashed arrows, can be set up. Multiple mobile devices 10 and multiple virtual paths may be set up in various combinations.

[0055] Along with the virtual path, the movement speed and movement pattern may also be set. The movement speed may be set to vary depending on the position on the virtual path. Examples of movement patterns include one-way movement, round-trip movement, etc. A default setting such as round-trip movement at a constant speed may also be provided.

[0056] Speaking of manual operation, while the mobile device 10 is in operation, it is also possible to specify a position on the virtual path to immediately move the mobile device 10 or to have it wait at the destination.

[0057] In the example shown in Figure 5, the control unit 19 changes the imaging parameters according to the movement of the mobile device 10. Figure 5 shows an example of setting an imaging plan that defines the operation of such a mobile device 10. Zoom magnification is exemplified as an imaging parameter. The user uses the UI unit 32, which displays a map as shown, to set the zoom magnification according to the position of the mobile device 10 on the virtual path. In this example, the zoom magnification when the mobile device 10 is furthest from the performer 5 is set to the reference value Z. f0 The zoom magnification is at its maximum value Z when the mobile device 10 is closest to the performer 5. fmax The zoom magnification increases as the moving device 10 approaches the performer 5. However, the manner in which the zoom magnification is changed is not limited to this example. For example, the zoom magnification may be continuously increased or decreased as the moving device 10 moves from one end of the virtual path to the other.

[0058] In the example shown in Figure 6, the control unit 19 causes the mobile device 10 to image the designated performer 5. Figure 6 shows an example of setting up an imaging plan that defines the operation of such a mobile device 10. The user uses the UI unit 32, which displays a map as shown, to specify performer 5-2 from among the multiple performers 5 using the cursor C. Each mobile device 10 rotates to image the designated performer 5-2, for example, so that it is centered in the field of view. The imaging direction of each mobile device 10 is schematically shown by a dashed-dotted arrow. Note that any location on the stage ST may be specified, not limited to performer 5.

[0059] In the example shown in Figure 7, the control unit 19 operates the mobile device 10 in accordance with the performance (song, music, etc.). Figure 7 shows an example of setting a shooting plan that defines the operation of such a mobile device 10. The user uses the UI unit 32, which displays a map as shown (a 3D map in this example), to set the operation of the mobile device 10 in relation to the performance of a virtual performer 5H. Performer 5H performs the planned performance of Performer 5. In this example, the performance of Performer 5H is represented by timeline TL1. The operation of the mobile device 10 is represented by timeline TL2. Timelines TL1 and TL2 are WayPoint columns representing time, position, speed, and direction, etc. Any timing in timeline TL1 can be selected by bar B1. Any timing in timeline TL2 can be selected by bar B2. By defining the operation of the mobile device 10 represented by timeline TL2 in accordance with the performance of Performer 5H represented by timeline TL1, it is linked to the performance (synchronized with the song, staging, etc.).

[0060] If a user attempts to set an action that is impossible to perform, the UI unit 32 may display an error message or suggest a modified action that is possible to perform. Examples of impossible actions include actions that require the mobile device 10 to pass through a location where it cannot move, or actions that require an operating speed that exceeds the maximum moving speed of the mobile device 10.

[0061] Note that the actual operation of Performer 5 during shooting may differ from the operation of Performer 5H. In such cases, the mobile device 10 may perform an operation different from the operation defined in Timeline TL2, for example, by the amount of the deviation in operation.

[0062] In the example shown in Figure 8, the control unit 19 rotates (spins) the mobile device 10 relative to the performer 5. Figure 8 shows an example of setting an imaging plan that defines the operation of such a mobile device 10. The user uses the UI unit 32, which displays a map as shown, to specify the performer 5 to be imaged and set the distance and rotation range (angle range) relative to the performer 5. The mobile device 10 images the performer 5 while rotating around the specified performer 5 within the set distance and rotation range. The distance and rotation range may be set to default values.

[0063] If a user attempts to set an impossible rotational movement, the UI unit 32 may display an error message or suggest a modified rotational movement that is feasible. An example of an impossible rotational movement is a rotational movement for Performer 5 playing drums on a step.

[0064] In the example shown in Figure 9, the control unit 19 causes the mobile device 10 to track and photograph the performer 5. Figure 9 shows an example of setting up a shooting plan that defines the operation of such a mobile device 10. The user specifies the performer 5 using the UI unit 32 which displays a map as shown. The mobile device 10 follows the specified performer 5 and continues to photograph it. For example, when the performer 5 moves to a position indicated by the white circle with a dashed line, the mobile device 10 moves with it. The arrows schematically indicate the imaging direction of each mobile device 10.

[0065] In tracking photography, the mobile device 10 may move in a manner that maintains the distance and direction (relative position) to the performer 5. The mobile device 10 may also move in a manner that changes direction while maintaining the distance to the performer 5. By changing direction, effects such as Paring noise can be obtained. The composition of the image, such as which part of the performer 5 is being photographed, may also be changed as appropriate during tracking photography. The mobile device 10 may photograph the performer 5 while moving alongside the performer 5 or while moving ahead of the performer 5. Such parallel and leading movement is also a form of tracking photography.

[0066] Any shooting plan can be set that combines any operation, including the movement and imaging operations described above. For example, a shooting plan (programmable rail shooting) can be set in which the moving device 10 changes direction to track the performer 5 while moving along a virtual path at a specified speed and then takes an image.

[0067] When multiple mobile devices 10 perform imaging simultaneously, the control unit 19 may control the operation of the mobile devices 10 so as not to image other mobile devices 10.

[0068] Figure 10 shows an example of the imaging range of multiple mobile devices. In this example, mobile devices 10-1 and 10-2 are performing imaging simultaneously. Mobile device 10-1 operates so that mobile device 10-2 is not included in the imaging range A1 of the imaging unit 13 by recognizing the position of mobile device 10-2. Mobile device 10-2 operates so that mobile device 10-1 is not included in the imaging range A2 of the imaging unit 13 by recognizing the position of mobile device 10-1.

[0069] With the mobile device 10 that performs the movement and imaging operations as described above, movement and imaging can be performed even without actual rails like those of conventional dolly equipment, thus reducing introduction costs.

[0070] Since the mobile device 10 operates (including autonomous operation) simply by setting the shooting plan, the burden of manual operation is reduced. For example, the user only needs to manually adjust the movement speed of the mobile device 10 and specify the target to be tracked and photographed at the start of the performance. Manual adjustment of the movement speed can be done by simply registering and recalling a movement speed that has been confirmed during rehearsals, etc.

[0071] In addition to the manual operations described above at the start of the performance, manual operations may be introduced during the performance. For example, the mobile device 10 may be manually positioned to wait at the end of a virtual path. The mobile device 10 may be moved when the performance reaches a relatively exciting point (such as the chorus of a song). The mobile device 10 may be slowed down when the performance calms down. The mobile device 10 may be accelerated when the performance reaches an exciting point again. In addition, the nuances of the image composition, including manual operations such as zooming and panning, may also be formed manually.

[0072] In addition, manual operation is also possible, such as having the mobile device 10 wait at the ST end of the stage and performing imaging with the mobile device 10 only at desired timings.

[0073] 1.3 Examples of Light Emission Operation As mentioned earlier, the light emission operation of the mobile device 10 (the light emission expression performed by the light emission unit 15) is performed according to the shooting plan, or according to the movement operation, imaging operation, lighting operation, etc. Some examples of these will be explained with reference to Figures 11 to 18.

[0074] In the examples shown in Figures 11 and 12, the control unit 19 changes the direction of light emission from the light-emitting unit 15. As shown in Figure 11, when the mobile device 10 moves forward in the direction of the arrow, the front portion of the light-emitting unit 15 mainly emits light. As shown in Figure 12, when the mobile device 10 moves sideways, the side portion of the light-emitting unit 15 mainly emits light. The change in the direction of light emission may be expressed not only by the presence or absence of light emission, but also by changing the color of the emitted light, changing the amount of emitted light (luminance), etc.

[0075] In the examples shown in Figures 13 and 14, the control unit 19 causes the light-emitting unit 15 to blink. Specifically, part 15a of the light-emitting unit 15 alternates between being illuminated and not emitting light. For example, blinking red light may indicate that the mobile device 10 is performing imaging (REC state).

[0076] In the examples shown in Figures 15 and 16, the control unit 19 causes the light-emitting unit 15 to perform a countdown. The countdown light emission includes a change in light emission from just before the start or end of a predetermined operation of the mobile device 10 (movement operation, imaging operation, etc.) to the start or end of the operation. Specifically, as shown in Figure 15, several parts 15b of the light-emitting unit 15 and the parts in between emit light, and as time progresses, the amount of light emitted decreases as shown in Figure 16.

[0077] In the example shown in Figure 17, the control unit 19 causes the entire light-emitting unit 15 to emit light. Specifically, the entire light-emitting unit 15 emits light, while some parts 15c emit light in a different manner (different emission color, emission intensity, etc.) than the other parts. During overall illumination, the emission intensity may be increased at the timing when a part is highlighted, and decreased at the timing when it is not highlighted (when it becomes dark).

[0078] In the example shown in Figure 18, the control unit 19 causes the light-emitting unit 15 to emit light in coordination with the light emitted by the other moving devices 10. Specifically, as indicated by the arrows, moving devices 10-1, 10-2, 10-3, and 10-4 emit light in sequence in coordination (interlocking), realizing a wave effect.

[0079] The aforementioned light emission operation may not only be performed in conjunction with the movement and imaging of the mobile device 10, but may also be incorporated into the performance. For example, the mobile device 10 may perform a light emission while moving in accordance with the movement of the performer 5, thereby forming part of the performance. The mobile device 10 may also perform imaging while forming part of the performance.

[0080] The shooting plan for defining the aforementioned light emission operation may also be set, for example, using the UI unit 32 that displays a map.

[0081] 1.4 Examples of lighting operation As mentioned earlier, the lighting operation of the mobile device 10 (projection performed by the lighting unit 16) is performed according to the shooting plan, or according to the movement operation, imaging operation, light emission operation, etc. Some examples of these will be explained with reference to Figures 19 to 22.

[0082] In the examples shown in Figures 19 and 20, the control unit 19 causes the lighting unit 16 to perform projection according to the movement speed of the mobile device 10. Specifically, the lighting unit 16 illuminates and projects an area having an area corresponding to the movement speed of the mobile device 10. As shown in Figure 19, when the mobile device 10 is moving forward at a relatively fast speed, an area with a relatively large area along the direction of movement is illuminated. As shown in Figure 20, when the mobile device 10 is moving forward at a relatively slow speed, an area with a relatively small area along the direction of movement is illuminated.

[0083] In the examples shown in Figures 21 and 22, the control unit 19 causes the illumination unit 16 to perform color-coded projection. Specifically, the illumination unit 16 illuminates a region R1 near the mobile device 10 and a region R2 on the opposite side of region R1 from the mobile device 10 with different colors. For example, region R1 represents the imaging zone, which is the area to be imaged, and region R2 represents the movement zone, which is the area within that zone that can be moved without being imaged. The sizes of regions R1 and R2 are determined according to the imaging operation of the mobile device 10, etc.

[0084] The lighting operations described above may not only be performed in conjunction with the movement of the mobile device 10 and imaging, but may also be incorporated into the performance. Furthermore, the shooting plan for defining the lighting operations may be set, for example, using the UI unit 32 that displays a map.

[0085] Furthermore, at least a portion of the shooting plan may be set automatically. For example, multiple combinations of movement actions (virtual paths, etc.) and imaging actions (imaging composition, etc.) may be registered in advance, and a shooting plan that executes them sequentially over a predetermined period may be automatically generated. Not limited to movement actions and imaging actions, predetermined actions (default actions) may be automatically incorporated into the shooting plan for various actions. Multiple combinations of default actions may be automatically incorporated into the shooting plan. Actions that connect to user-defined actions may be automatically incorporated into the shooting plan before and after those actions.

[0086] 1.5 Example of processing Figure 23 is a flowchart showing an example of a process (control method for the moving device) performed in the imaging system.

[0087] In step S1, the imaging plan is set. The imaging plan may be set by the user, as described above, or it may be set automatically. The set imaging plan is transmitted to the mobile device 10.

[0088] In step S2, shooting and distribution are performed. The control unit 19 controls the operation of the mobile device 10 based on the shooting plan set in step S1. For example, the mobile device 10 is controlled so that as it moves, the performer 5 recognized by the sensor 14 is captured by the sensor 14, and the light-emitting unit 15 performs a light emission. The video is transmitted and distributed by the operation instruction device 30.

[0089] In step S3, the operation instruction device is operated as needed. For example, the user can use the UI unit 32 to update the shooting plan or manually operate the mobile device 10. This allows the mobile device 10 to perform actions that were not specified in the shooting plan set in step S1.

[0090] For example, the mobile device 10 is controlled in the manner described above.

[0091] 1.6 Example of the appearance of a mobile device Examples of the appearance of the mobile device 10 are shown in Figures 24 to 29. Reference numerals are omitted in the illustration. Figure 24 is a front view showing an example of the appearance of the mobile device when viewed from the front. When viewed from the front, the mobile device has, for example, a height of approximately 480 mm and a width of approximately 650 mm. Figure 25 is a rear view showing an example of the appearance of the mobile device when viewed from the rear. Figure 26 is a right side view showing an example of the appearance of the mobile device when viewed from the right side. Figure 27 is a left side view showing an example of the appearance of the mobile device when viewed from the left side. Figure 28 is a top view showing an example of the appearance of the mobile device when viewed from above. Figure 29 is a bottom view showing an example of the appearance of the mobile device when viewed from below.

[0092] The above describes an example in which the moving device has an external shape close to a sphere. However, the moving device may have various shapes other than a sphere. An example of another shape will be explained with reference to Figures 30 to 37.

[0093] Figure 30 shows an example of the schematic configuration of a mobile device. The mobile device 10A illustrated in Figure 30 has an external shape that is close to a cylindrical shape. The outer housing, housing 12A, includes an upper housing 121 and a lower housing 122. A gap is provided between the upper housing 121 and the lower housing 122. A sensor (corresponding to sensor 14 in Figure 2) that recognizes the surrounding environment is provided at the same height as the gap. The gap is formed so as not to interfere with recognition by the sensor.

[0094] An opening is formed in the upper part of the upper housing 121 so as not to obstruct imaging by the imaging unit 13A. The imaging unit 13A may have the same configuration as the imaging unit 13 (Figure 2). The part that forms the opening (the edge of the opening) is referred to as the opening 121a and is shown in the figure. The light-emitting unit 15A is arranged in the opening 121a. The light-emitting unit 15A may have the same configuration as the light-emitting unit 15 (Figure 2). In this example, the opening 121a is annular in shape, and the light-emitting unit 15A is arranged in an annular shape on the upper housing 121.

[0095] The lower housing 122 has a cylindrical shape. Although not shown, the mobile device 10A includes an illumination unit and a mobile mechanism (corresponding to the illumination unit 16 and mobile mechanism 17 in Figure 2), etc.

[0096] The mobile device 10B illustrated in Figure 31 differs from the mobile device 10A in that it has a light-emitting unit 15B instead of a light-emitting unit 15A. The light-emitting unit 15B is arranged in a ring shape on the upper part of the lower housing 122.

[0097] Examples of the external appearance of the mobile devices 10A and 10B are shown in Figures 32 to 37. Reference numerals are omitted in the illustration. Figure 32 is a front view showing an example of the external appearance of the mobile device when viewed from the front. When viewed from the front, the mobile device has a height of approximately 730 mm (480 mm excluding the camera part) and a width of approximately 530 mm. Figure 33 is a rear view showing an example of the external appearance of the mobile device when viewed from the rear. Figure 34 is a right side view showing an example of the external appearance of the mobile device when viewed from the right side. Figure 35 is a left side view showing an example of the external appearance of the mobile device when viewed from the left side. Figure 36 is a top view showing an example of the external appearance of the mobile device when viewed from above. Figure 37 is a bottom view showing an example of the external appearance of the mobile device when viewed from below.

[0098] Up to this point, we have described examples in which the light-emitting parts are arranged in a ring shape, but other arrangements are also possible. The mobile device 10C illustrated in Figures 39 and 41 differs from the mobile device 10 in that it has a light-emitting part 15C instead of a light-emitting part 15. The light-emitting part 15C is arranged on the entire surface of the housing 12. The entire surface of the housing 12 here may be the surface of the part of the housing 12 that is visible from the outside.

[0099] The appearance of the mobile device 10C is similar to that of the mobile device 10, except for the light-emitting section 15C. The mobile device 1C has curved LED panels arranged on the entire surface of the housing 12, enabling more diverse light-emitting expressions through surface illumination. Figure 41 is a schematic diagram showing a cross-section. The curved LED panels are arranged facing the housing 12. The curved LEDs may be in contact with the housing 12.

[0100] The mobile device 10C can perform various lighting effects through the surface illumination of curved LED panels arranged across its entire casing, and can also function as live performance equipment. More specifically, it can display various lighting effects such as the color, intensity, and blinking patterns of the light, as well as the artist name, song title, and text such as a clock for Performer 5 and the audience. Furthermore, by emitting light in conjunction with the stage lighting, the mobile device 10C can blend seamlessly into the stage. Specifically, when a blue spotlight is shining near the mobile device 10C, it will emit blue light, and when the stage lighting is dimmed, it will emit black or dark chromatic colors to create a blended lighting effect.

[0101] As explained above, compared to the case where a part of the housing 12 (for example, an opening 12a) becomes the light-emitting part 15 and performs light emission as in the mobile device 10, the mobile device 10C uses the entire housing 12 as the light-emitting part 15C to perform light emission, which allows it to blend into the atmosphere more seamlessly and enables a wider variety of light emission expressions, such as those exemplified by the aforementioned text expression.

[0102] 1.7 Example Hardware Configuration Figure 38 shows an example of the hardware configuration of an operation instruction device, etc. For example, the operation instruction device 30 and the control unit 19 of the mobile device 10 are implemented by a computer 1000. The computer 1000 has a CPU 1100, RAM 1200, ROM (Read Only Memory) 1300, HDD (Hard Disk Drive) 1400, a communication interface 1500, and an input / output interface 1600. The various parts of the computer 1000 are connected by a bus 1050. Note that the control unit 19 of the mobile device 10 does not necessarily need an input / output interface 1600. Since the mobile device 10 is equipped with a communication unit 18, the control unit 19 does not necessarily need a communication interface 1500.

[0103] The CPU 1100 operates based on programs stored in the ROM 1300 or HDD 1400, and controls various parts. For example, the CPU 1100 loads the programs stored in the ROM 1300 or HDD 1400 into the RAM 1200 and executes processing corresponding to various programs.

[0104] ROM1300 stores boot programs such as the BIOS (Basic Input Output System) executed by CPU1100 when computer 1000 starts up, as well as programs that depend on the computer 1000's hardware.

[0105] HDD1400 is a computer-readable recording medium that non-temporarily records programs executed by CPU1100 and data used by such programs. Specifically, HDD1400 is a recording medium that records an information processing program related to this disclosure, which is an example of program data 1450.

[0106] The communication interface 1500 is an interface for the computer 1000 to connect to an external network 1550 (e.g., the Internet). For example, the CPU 1100 can receive data from other devices or transmit data it has generated to other devices via the communication interface 1500.

[0107] The input / output interface 1600 is an interface for connecting the input / output device 1650 and the computer 1000. For example, the CPU 1100 receives data from input devices such as a keyboard or mouse via the input / output interface 1600. The CPU 1100 also transmits data to output devices such as a display, speaker, or printer via the input / output interface 1600. The input / output interface 1600 may also function as a media interface for reading programs recorded on a predetermined recording medium (media). Examples of media include optical recording media such as DVDs (Digital Versatile Discs) and PDs (Phase Change Rewritable Disks), magneto-optical recording media such as MOs (Magneto-Optical Disks), tape media, magnetic recording media, or semiconductor memory.

[0108] For example, when computer 1000 functions as an operation instruction device 30, the CPU 1100 of computer 1000 implements functions such as the map generation unit 31, UI unit 32, and distribution unit 33 by executing a program loaded onto RAM 1200. The HDD 1400 stores a program that enables the computer to function as an operation instruction device 30. While the CPU 1100 reads and executes program data 1450 from the HDD 1400, another example is that these programs may be obtained from other devices via an external network 1550.

[0109] For example, if the computer 1000 functions as the control unit 19 of the mobile device 10, the CPU 1100 of the computer 1000 realizes the functions of the control unit 19 by executing a program loaded on the RAM 1200. The HDD 1400 stores a program that allows the computer to function as the control unit 19. The CPU 1100 reads and executes program data 1450 from the HDD 1400, but as an alternative, these programs may be obtained from other devices via an external network 1550.

[0110] While embodiments of this disclosure have been described above, the technical scope of this disclosure is not limited to the embodiments described above, and various modifications are possible without departing from the gist of this disclosure.

[0111] In the above embodiment, an example of the imaging system 1 being used in a music live event was described. However, the use of the imaging system 1 is not limited to music live events. For example, the imaging system 1 may be used in a variety of applications such as sports, weddings, and even surveillance.

[0112] In the above embodiment, an example was described in which the imaging system 1 is equipped with the same number of mobile devices 10 as the number of performers 5. However, the number of mobile devices 10 may be different from the number of performers 5. There may be only one mobile device 10.

[0113] In the above embodiment, an example was described in which the environmental sensor 20 recognizes the position of the performer 5 by recognizing the tag T. However, the environmental sensor 20 may recognize the performer 5 without using the tag T. In that case, the imaging system 1 does not need to be equipped with the tag T.

[0114] The arrangement of the components of the mobile device 10 is not limited to the above embodiment. The components may be arranged in various ways as long as the function of the mobile device 10 can be realized. Similarly, the external shape of the mobile device 10 is not limited to the above embodiment.

[0115] 2. Effects The mobile device 10 described above can be specified, for example, as follows. As explained with reference to Figures 1 and 2, the mobile device 10 is a mobile device that operates to photograph a subject (performer 5), and comprises an imaging unit 13, a sensor 14 that recognizes the surrounding environment, and a light-emitting unit 15 arranged in a ring shape on the housing 12. The operation of the mobile device 10 includes the mobile device 10 moving based on a set imaging plan and the imaging unit 13 imaging the subject recognized by the sensor 14. The light-emitting unit 15 performs a light emission expression according to at least one of the imaging plan and the operation of the mobile device 10. As explained with reference to Figure 39, the light-emitting unit may be a light-emitting unit 15C arranged on the entire surface of the housing 12.

[0116] The mobile device 10 described above can improve interaction performance by recognizing the surrounding environment using the sensor 14. For example, by moving and taking images while avoiding obstacles, it becomes possible to take pictures from various viewpoints at music events, etc. Interaction performance can be further improved by using light effects. The mobile device 10 has the advantage of not requiring large equipment such as cranes or manual labor for movement, and also does not require complex manual operation.

[0117] As explained with reference to Figure 4, the shooting plan may include the mobile device 10 moving along a set virtual path. As explained with reference to Figure 5, the imaging unit 13 is parameter controllable, and the shooting plan may include changing the parameters of the imaging unit 13 in accordance with the movement of the mobile device 10. As explained with reference to Figure 9, the shooting plan may include the mobile device 10 tracking and photographing a subject. As explained with reference to Figures 1 and 7, the shooting plan may include the mobile device 10 operating in sync with music. For example, by setting such a shooting plan, the mobile device 10 can be made to perform various operations.

[0118] As explained with reference to Figure 3, the mobile device 10 may include a communication unit 18 that transmits the imaging results (video) from the imaging unit 13 in real time. This allows for real-time video streaming, for example.

[0119] As explained with reference to Figures 11 and 12, the light emission expression of the light-emitting unit 15 may include changes in the emission color. As explained with reference to Figures 13 and 14, the light emission expression of the light-emitting unit 15 may include blinking. As explained with reference to Figures 15 and 16, the light emission expression of the light-emitting unit 15 may include countdown. As explained with reference to Figure 18, the light emission expression of the light-emitting unit 15 may include emitting light in coordination with the emission of other moving devices. For example, such various light emission expressions can further improve interaction performance.

[0120] As explained with reference to Figures 2, 3, and 19-22, the mobile device 10 may be equipped with a projection lighting unit 16. The lighting unit 16 may perform projection according to at least one of the shooting plan and the operation of the mobile device 10. For example, such various projections can further improve interaction performance.

[0121] The imaging system 1 described with reference to Figures 1 and 3 is also one aspect of the present disclosure. Specifically, the imaging system 1 includes a mobile device 10 that operates to photograph a subject (performer 5) and an operation instruction device 30 that remotely controls the mobile device 10. The mobile device 10 includes an imaging unit 13 and a sensor 14 that recognizes the surrounding environment. The operation of the mobile device 10 includes the mobile device 10 moving based on a set shooting plan and the imaging unit 13 photographing the subject recognized by the sensor 14. Even with such an imaging system 1, interaction performance can be improved by having the mobile device 10 recognize the surrounding environment with the sensor 14.

[0122] As explained with reference to Figures 3 to 9, the shooting plan may define the movement and / or imaging operations of the mobile device 10. For example, by setting such an shooting plan, the mobile device 10 can be made to perform various movement and / or imaging operations.

[0123] As explained with reference to Figure 6, the operation instruction device 30 may have a user interface (UI section 32) for specifying the subject. This allows the user to manually specify the subject.

[0124] As explained with reference to Figures 1 and 10, the mobile device 10 may be a plurality of mobile devices 10 that operate in coordination with each other. This makes it possible to take pictures from various viewpoints, for example, compared to when there is only one mobile device 10.

[0125] As explained with reference to Figures 1 and 3, the imaging system 1 includes an environmental sensor 20 that recognizes the position of the subject, and the mobile device 10 may operate based on the recognition result of the sensor 14 and the recognition result of the operation instruction device 30. This makes it possible to improve the operational accuracy of the mobile device 10 compared to, for example, when the mobile device 10 operates based solely on the recognition result of sensor 14.

[0126] A control method for the mobile device 10, as described with reference to Figure 23, is also one aspect of this disclosure. That is, the control method for the mobile device 10 includes an imaging unit 13, a sensor 14 for recognizing the surrounding environment, and a light-emitting unit 15 arranged in a ring shape on the housing 12, and includes controlling the mobile device 10 so that the mobile device 10 moves, the imaging unit 13 captures an image of a subject (performer 5) recognized by the sensor 14, and the light-emitting unit 15 performs a light-emitting expression (step S2). With such a control method for the mobile device 10, interaction performance can be improved by having the mobile device 10 recognize the surrounding environment with the sensor 14. Interaction performance can be further improved by performing a light-emitting expression. As described with reference to Figure 39, the light-emitting unit may be a light-emitting unit 15C arranged on the entire surface of the housing 12.

[0127] The effects described in this disclosure are illustrative and not limited to those disclosed. Other effects may also occur.

[0128] Furthermore, this technology can also be configured as follows. (1) A mobile device that operates to photograph a subject, Imaging unit, Sensors that recognize the surrounding environment, The housing has a light-emitting section arranged in a ring shape or across the entire surface of the housing, Equipped with, The operation of the mobile device includes moving the mobile device and capturing images of the subject recognized by the sensor using the imaging unit, based on a set shooting plan. The light-emitting unit performs a light emission expression corresponding to at least one of the shooting plan and the operation of the moving device. Mobile device. (2) The aforementioned shooting plan includes the mobile device moving along a set virtual path, (1) The mobile device described above. (3) The imaging unit is parameter-controllable, The aforementioned shooting plan includes changing the parameters of the imaging unit in accordance with the movement of the mobile device. The mobile device described in (1) or (2). (4) The aforementioned shooting plan includes the mobile device tracking and photographing the subject, A mobile device as described in any of (1) to (3). (5) The aforementioned filming plan includes the operation of the mobile device in time with the music. A mobile device as described in any of (1) to (4). (6) The system includes a communication unit that transmits the imaging results of the imaging unit in real time. A mobile device as described in any of (1) to (5). (7) The light emission expression of the light-emitting part includes changing the light emission color. A mobile device as described in any of (1) to (6). (8) The light emission expression of the light-emitting part includes a blinking expression. A mobile device as described in any of (1) to (7). (9) The light emission expression of the light-emitting unit includes a countdown expression. A mobile device as described in any of (1) to (8). (10) The light emission of the aforementioned light-emitting unit includes emitting light in coordination with the light emission of other moving devices. A mobile device as described in any of (1) to (9). (11) Equipped with a lighting unit for projection, A mobile device as described in any of (1) to (10). (12) The lighting unit performs projection according to at least one of the shooting plan and the operation of the moving device. (11) The mobile device described above. (13) A mobile device that operates to photograph the subject, An operation instruction device for remotely controlling the aforementioned mobile device, Equipped with, The aforementioned mobile device is Imaging unit, Sensors that recognize the surrounding environment, Includes, The operation of the mobile device includes moving the mobile device and capturing images of the subject recognized by the sensor using the imaging unit, based on a set shooting plan. Imaging system. (14) The aforementioned shooting plan defines the movement operation of the mobile device. The imaging system described in (13). (15) The aforementioned shooting plan defines the imaging operation of the mobile device. The imaging system described in (13) or (14). (16) The operation instruction device has a user interface for specifying the subject, An imaging system as described in any of (13) to (15). (17) The aforementioned moving device is a plurality of moving devices that operate in coordination with each other. An imaging system as described in any of (13) to (16). (18) The system includes an environmental sensor that recognizes the position of the subject, The mobile device operates based on the recognition results of the sensor and the recognition results of the environmental sensor. An imaging system as described in any of (13) to (17). (19) A control method for a mobile device including an imaging unit, a sensor for recognizing the surrounding environment, and a light-emitting unit arranged in an annular shape on the housing or across the entire surface of the housing, The mobile device is controlled such that it moves, the subject recognized by the sensor is captured by the imaging unit, and the light-emitting unit performs a light-emitting expression. A method for controlling a mobile device. [Explanation of symbols]

[0129] 1. Imaging System 5 Performer (Subject) 10 Mobile device 11 cabinets 12 cabinets 13 Imaging Unit 14 sensors 15 Light-emitting part 16 Lighting Section 17 Moving mechanism 18 Communications Department 19 Control Unit 20 Environmental sensors 21 Sensor Groups 22 Location information acquisition unit 23 Communications Department 30 Operation instruction device 31 Map generation unit 32 UI section 33 Distribution Department 34 Communications Department

Claims

1. An autonomously mobile performance device, Movement mechanism, The light-emitting part is located on the outside of the housing, A sensor that recognizes performers, A control unit controls the movement of the movement mechanism and the light emission expression of the light emission unit in accordance with the movement of the performer recognized by the sensor, so as to form part of the performance together with the performer, Equipped with, Performance device.

2. The control unit controls the movement by the moving mechanism and the light emission expression by the light-emitting unit based on a plan that defines the operation of the performance device. The performance device according to claim 1.

3. The plan is set up so that the operation of the performance device is linked to the performance. The performance device according to claim 2.

4. The aforementioned performance includes a performance relating to at least one of singing and music. The performance device according to any one of claims 1 to 3.

5. The control of the movement of the moving mechanism by the control unit includes capturing an image of the subject, including the performer recognized by the sensor, by the imaging unit of the performance device. The performance device according to any one of claims 1 to 4.

6. A UI unit that displays a first timeline representing the performance of a performer and a second timeline representing the operation of an autonomously mobile performance device different from the performer, in a manner that allows the user to select any timing on each timeline, and accepts user operations to select any timing on at least one of the first timeline and the second timeline, A processor that sets a plan for the operation of the performance device so as to link the operation of the performance device to the performance in response to the user operation, Equipped with, Information processing device.

7. The UI unit displays the first timeline and the second timeline in the manner described above, along with a map including the performer and the performance device. The information processing apparatus according to claim 6.

8. The system includes a communication unit that transmits the set plan to the performance device. The information processing apparatus according to claim 6 or 7.

9. The aforementioned performance includes a performance relating to at least one of singing and music. The information processing apparatus according to any one of claims 6 to 8.

10. A performance device according to any one of claims 1 to 5, An information processing device according to any one of claims 6 to 9, Equipped with, The control unit of the performance device controls the movement by the movement mechanism and the light emission expression by the light emission unit based on the plan set by the processor of the information processing device and the recognition results by the sensors of the performance device. Production system.

11. A method for controlling an autonomously moving performance device, The aforementioned performance device is Movement mechanism, The light-emitting part is located on the outside of the housing, A sensor that recognizes performers, Includes, The control method described above is To form part of the performance together with the performer, the movement of the movement mechanism and the light emission expression of the light-emitting part are controlled in accordance with the movement of the performer recognized by the sensor. including, Control method.

Images