Spatial display device, spatial display method, and spatial display program

Abstract

To provide a spatial display device capable of efficiently operating a plurality of unmanned aerial vehicles. [Solution] The spatial display device calculates the number of dots that will be illuminated when the image is displayed in a dot matrix based on image data related to an image to be displayed in the air, generates position information for each dot, generates illumination pattern information for each dot based on color data, assigns an unmanned aerial vehicle to each dot, assigns the image to one of a number of groups, transmits the position information and illumination pattern information to each unmanned aerial vehicle, and if it determines that the number of unmanned aerial vehicles is sufficient for the number of dots, it assigns an unmanned aerial vehicle belonging to the group to which the image is assigned to each dot, and if it determines that the number of unmanned aerial vehicles is not sufficient for the number of dots, it assigns an unmanned aerial vehicle of the group to which the image is assigned to each dot and also assigns unmanned aerial vehicles of other groups.

Description

[Technical field]

[0001] The present invention relates to a spatial display device, a spatial display method, and a spatial display program, and in particular to a spatial display device, a spatial display method, and a spatial display program that are characterized by illuminating a plurality of unmanned aerial vehicles to display images in the air. [Background technology]

[0002] In recent years, the use of unmanned aerial vehicles (UAVs), also known as drones, small helicopters, multicopters, etc., has become widespread. For example, the use of UAVs in amusement parks has been proposed for special effects to entertain guests participating in attractions such as rides or shows (see Patent Document 1).

[0003] In addition, the use of a spatial display device, a spatial display method, and a spatial display program that display images in the air using multiple UAVs is also beginning to spread. Such spatial display devices are used, for example, in drone shows held in various places. The UAVs of the spatial display devices used in drone shows are equipped with light-emitting units such as LEDs, and multiple UAVs fly in formation in cooperation with each other, drawing beautiful illuminations mainly in the night sky or on a stage to entertain the audience.

[0004] Generally, UAVs used in the above-mentioned spatial display devices are programmed in advance to fly in a specific trajectory in synchronization with each other and to emit light at a specific timing. Once launched, these UAVs receive GPS signals to recognize their current location and fly autonomously according to pre-created programming.

[0005] In spatial display devices using multiple UAVs, there is a need to efficiently display images in the air. In particular, as the number of unmanned aerial vehicles used increases, there is a need to efficiently operate multiple unmanned aerial vehicles. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2017-526443 Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, an object of the present invention is to provide a spatial display device, a spatial display method, and a spatial display program that can efficiently operate multiple unmanned aerial vehicles. [Means for solving the problem]

[0008] That is, the spatial display device of the first aspect is a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and displays an image in the air by causing the plurality of unmanned aerial vehicles to emit light, and the processing unit of the spatial display device includes an image data acquisition unit that acquires image data related to the image, a dot number calculation unit that calculates, based on the image data, the number of dots that will be in an illuminating state when the image is displayed in a dot matrix, a position information generation unit that generates, based on the image data, position information for each of the number of dots that will be in an illuminating state, a light emission processing unit that generates light emission pattern information that specifies the light emission mode for each dot, based on color data related to the color of the image included in the image data, an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicles to each dot, an image allocation unit that allocates the image to one of the plurality of groups, and a dot number calculation unit that calculates the number of dots that will be in an illuminating state when the image is displayed in a dot matrix format. The system comprises a transmitting unit that transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles assigned to each dot, and a number of aircraft sufficiency determination unit that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned suffices the number of dots calculated by the dot number calculation unit, and when the number of aircraft sufficiency determination unit determines that the number of unmanned aerial vehicles suffices the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned, for each dot, and when the number of aircraft sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned, and allocates unmanned aerial vehicles belonging to other groups, for each dot.

[0009] In a second aspect, in the spatial display device of the first aspect, if the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, and an unmanned aerial vehicle belonging to another group cannot be assigned to a dot, the position information generation unit generates position information for each dot, the number of dots corresponding to the number of unmanned aerial vehicles belonging to the group to which the image is assigned, and the unmanned aerial vehicle assignment unit assigns an unmanned aerial vehicle belonging to the group to which the image is assigned, to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

[0010] A third aspect is a spatial display device according to the first aspect, wherein the image includes a moving image, and the processing unit of the spatial display device includes a flight control unit that generates flight pattern information that specifies a flight route, flight timing, and flight speed for each dot that constitutes the moving image portion based on moving image data relating to the moving image portion of the image included in the image data, and the transmitting unit may transmit the flight pattern information to each of the unmanned aerial vehicles assigned to each dot that constitutes the moving image portion, together with position information and light emission pattern information corresponding to the dot.

[0011] In a fourth aspect, in the spatial display device of the first aspect, an image may be displayed by contours, and the position information generation unit may acquire contour data regarding the contours based on the image data, and generate position information for each dot that is illuminated when the contours are displayed in a dot matrix format based on the contour data.

[0012] In a fifth aspect, in the spatial display device of the first aspect, the image is a three-dimensional image composed of a plurality of layers, and the position information generation unit may generate, based on the image data, position information for each dot that is illuminated when the image divided into the plurality of layers is displayed in a dot matrix manner.

[0013] A sixth aspect is a spatial display device according to the first aspect, further comprising an external information input unit that accepts original image data that is the source of an image, and a learning model acquisition unit that acquires a learning model that has previously learned the correspondence between image data related to the image and the original image data, and the image data acquisition unit may acquire image data by inputting the original image data accepted by the external information input unit into the learning model.

[0014] A seventh aspect is a spatial display device according to the first aspect, in which there are other groups waiting in the air and other groups waiting on the ground, and the unmanned aircraft allocation unit may give priority to allocating unmanned aircraft belonging to the other groups waiting in the air over the other groups waiting on the ground.

[0015] In an eighth aspect, in the spatial display device of the first aspect, the light emission processing unit may set the light emission color included in the light emission pattern information of dots that are placed in the diagonal and curved portions of the image and form the boundary with the background of the image to an intermediate color between the color of the background and the color of the image.

[0016] In a ninth aspect, in the spatial display device of the first aspect, the unmanned aerial vehicle may be provided with an acquisition unit that acquires the position information and the light emission pattern information transmitted from the transmission unit.

[0017] In a tenth aspect, in the spatial display device of the third aspect, the unmanned aerial vehicle may be provided with an acquisition unit that acquires flight pattern information transmitted from the transmission unit together with position information and light emission pattern information.

[0018] In an eleventh aspect, in a spatial display device relating to the ninth or tenth aspect, the unmanned aerial vehicle is equipped with an abnormality detection unit that detects an abnormality in the own aircraft and an abnormality notification unit that notifies the spatial display device of abnormality information related to the abnormality in the own aircraft, the processing unit of the spatial display device is equipped with an abnormality information receiving unit that receives the abnormality information of the unmanned aerial vehicle, the unmanned aerial vehicle allocation unit assigns an unmanned aerial vehicle belonging to another group in place of the unmanned aerial vehicle that notified the abnormality information to a dot to which the unmanned aerial vehicle had been assigned, and the transmission unit transmits a return command to the unmanned aerial vehicle that notified the abnormality information to return.

[0019] As for a twelfth aspect, in the spatial display device according to the eleventh aspect, the abnormality may include a case where a remaining charge of the battery is equal to or lower than a threshold value.

[0020] In a thirteenth aspect, in a spatial display device relating to the ninth or tenth aspect, the unmanned aerial vehicle is provided with a light deficiency detection unit that detects an insufficient amount of light emitted by the aircraft itself, and a light deficiency notification unit that notifies the spatial display device of light deficiency information regarding the insufficient amount of light emitted by the aircraft itself, and the processing unit of the spatial display device is provided with a light deficiency information receiving unit that receives the light deficiency information of the unmanned aerial vehicle, and the unmanned aerial vehicle allocation unit may allocate an unmanned aerial vehicle belonging to another group together with the unmanned aerial vehicle to a dot to which the unmanned aerial vehicle that notified the light deficiency information had been assigned.

[0021] In a fourteenth aspect, in a spatial display device relating to the ninth or tenth aspect, the unmanned aerial vehicle may be provided with an illuminance sensor that measures external illuminance, and an emission luminance correction unit that corrects the emission luminance included in the emission pattern information based on the measurement result of the illuminance sensor.

[0022] A spatial display method according to a fifteenth aspect is a spatial display method used in a spatial display device which divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, and a processing unit of the spatial display device includes an image data acquisition step of acquiring image data relating to the image, a dot number calculation step of calculating, based on the image data, the number of dots which will be in an illuminating state when the image is displayed in a dot matrix, a position information generation step of generating, based on the image data, position information for each of the number of dots which will be in an illuminating state, a light emission processing step of generating light emission pattern information which specifies the light emission mode for each dot, based on color data relating to the color of the image included in the image data, a unmanned aerial vehicle allocation step of allocating a unmanned aerial vehicle to each dot, an image allocation step of allocating an image to one of a plurality of groups, and an allocation step of allocating each dot to each dot. The method includes a transmission step of transmitting position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles assigned to the image, and a number sufficiency determination step of determining whether the number of unmanned aerial vehicles included in the group to which the image is assigned is sufficient for the number of dots calculated in the dot number calculation step.If it is determined in the number sufficiency determination step that the number of unmanned aerial vehicles is sufficient for the number of dots calculated in the dot number calculation step, then in the unmanned aerial vehicle allocation step, an unmanned aerial vehicle belonging to the group to which the image is assigned is assigned to each dot, and if it is determined in the number sufficiency determination step that the number of unmanned aerial vehicles does not satisfy the number of dots calculated in the dot number calculation step, then in the unmanned aerial vehicle allocation step, an unmanned aerial vehicle belonging to the group to which the image is assigned and an unmanned aerial vehicle belonging to another group are assigned to each dot.

[0023] A spatial display program according to a sixteenth aspect is a spatial display program used in a spatial display device which divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, and the processing unit of the spatial display device includes an image data acquisition function which acquires image data relating to the image, a dot number calculation function which calculates, based on the image data, the number of dots which will be in an illuminating state when the image is displayed in a dot matrix, a position information generation function which generates, based on the image data, position information for each of the number of dots which will be in an illuminating state, a light emission processing function which generates light emission pattern information which specifies the light emission mode for each dot, based on color data relating to the color of the image contained in the image data, a unmanned aerial vehicle allocation function which allocates a unmanned aerial vehicle to each dot, an image allocation function which allocates an image to one of a plurality of groups, and an image allocation function which allocates an image to each of the plurality of groups, The system includes a transmission function that transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles assigned to the group, and a number sufficiency determination function that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated in the dot number calculation function.If the number sufficiency determination function determines that the number of unmanned aerial vehicles satisfies the number of dots calculated in the dot number calculation function, the unmanned aerial vehicle allocation function allocates unmanned aerial vehicles belonging to the group to which the image is assigned, for each dot, and if the number sufficiency determination function determines that the number of unmanned aerial vehicles does not satisfy the number of dots calculated in the dot number calculation function, the unmanned aerial vehicle allocation function allocates unmanned aerial vehicles belonging to the group to which the image is assigned, and allocates unmanned aerial vehicles belonging to other groups, for each dot. Effect of the Invention

[0024] The spatial display device of the present invention is a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and displays an image in the air by causing the plurality of unmanned aerial vehicles to emit light, and the processing unit of the spatial display device includes an image data acquisition unit that acquires image data related to the image, a dot number calculation unit that calculates, based on the image data, the number of dots that will be in an illuminating state when the image is displayed in a dot matrix, a position information generation unit that generates, based on the image data, position information for each of the number of dots that will be in an illuminating state, a light emission processing unit that generates light emission pattern information that specifies the light emission mode for each dot, based on color data related to the color of the image included in the image data, an unmanned aerial vehicle allocation unit that assigns an unmanned aerial vehicle to each dot, an image allocation unit that assigns an image to one of the plurality of groups, and an unmanned aerial vehicle assigned to each dot. The system is equipped with a transmitting unit that transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles, and an aircraft number sufficiency determination unit that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned suffices the number of dots calculated by the dot number calculation unit.If the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles suffices the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned, for each dot, and if the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned, and also allocates unmanned aerial vehicles belonging to other groups, thereby enabling efficient operation of multiple unmanned aerial vehicles. In addition, the spatial display method and the spatial display program, like the spatial display device, can efficiently operate multiple unmanned aerial vehicles. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram for explaining an overview of the spatial display device of this embodiment. [Diagram 2]FIG. 2 is a diagram showing an example of the appearance of an unmanned aerial vehicle used in the spatial display device of this embodiment. [Diagram 3] FIG. 3 is a block diagram for explaining an example of the configuration of an unmanned aerial vehicle used in the spatial display device of this embodiment. [Figure 4] FIG. 4 is a block diagram for explaining an example of a hardware configuration of the spatial display device of this embodiment. [Diagram 5] FIG. 5 is a block diagram for explaining an example of a functional configuration of the spatial display device of this embodiment. [Figure 6] FIG. 6 is a diagram for explaining a dot matrix display used in the spatial display device of this embodiment. [Figure 7] FIG. 7 is a diagram for explaining the display of a stereoscopic image by a dot matrix display used in the spatial display device of this embodiment. [Figure 8] FIG. 8 is a diagram for explaining a first example of use of the spatial display device of this embodiment. [Figure 9] FIG. 9 is a diagram for explaining a second use example of the spatial display device of this embodiment. [Figure 10] FIG. 10 is a diagram for explaining a third application example of the spatial display device of this embodiment. [Figure 11] FIG. 11 is an example of a flowchart of the spatial display program according to this embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] (Overview of spatial display device 10) 1 to 10, an embodiment of a spatial display device 10 according to the present disclosure will be described. First, with reference to FIG. 1, an overview of a spatial display device 10 of this embodiment will be described. The spatial display device 10 of this embodiment divides a plurality of unmanned aerial vehicles 11 into a plurality of groups, and displays an image in the air 13 by illuminating the plurality of unmanned aerial vehicles 11 (11a, 11b, 11c, 11d, 11e, . . . ). The spatial display device 10 may, for example, divide 1,000 unmanned aerial vehicles 11 into five groups with 200 evenly distributed in each group, or may divide multiple unmanned aerial vehicles 11 into two groups, one for primary use and the other as a secondary group. The primarily used group refers to a group to which the unmanned aerial vehicle 11 belongs that the spatial display device 10 uses with priority over other groups. The auxiliary group is a group that assists the primary group by, for example, supplying unmanned aerial vehicles 11 when the primary group does not have enough unmanned aerial vehicles 11, and by providing unmanned aerial vehicles 11 to be used as substitutes when an unmanned aerial vehicle 11 belonging to the primary group becomes unusable due to a malfunction or other reason. Furthermore, the spatial display device 10 may not only distribute the unmanned aerial vehicles 11 evenly among a number of groups, but may also reduce or increase the number of unmanned aerial vehicles 11 belonging to a particular group. The spatial display device 10 is an information processing device called a personal computer (hereinafter, PC), a notebook PC, a tablet PC, a smartphone, a server, or the like, and is connected to an information communication network 17 (see FIG. 4). The spatial display device 10 can be realized by using a general-purpose information processing device, but may also be an information processing device specialized for causing a plurality of unmanned aerial vehicles 11 to emit light and display images in the air 13. The image may be a still image or a video, or may be a character, a drawing, a mark, a code, etc. The image may be, for example, a QR code (registered trademark) 14, the character Japan 15 displayed in the alphabet, or Kanji characters 16 displayed in a calligraphy style. Specifically, images can be text, pictures, 3D images, animations, maps, buildings, large designs, etc., such as event or corporate logos, advertisements and promotions for companies, products or services, or QR codes. Videos can be animations that add movement to still images of animals, characters, robots, machines, etc., displayed in a dot matrix.

[0027] An animation is an image with movement and is created based on the original image data described below. Based on the animation, the flight pattern of the unmanned aerial vehicle 11 and the light emission pattern of the light-emitting unit 21 installed on the unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display of the animation are determined. Images are represented by a dot-matrix display. A dot-matrix display is a form of visual representation using a two-dimensional array of dots, and is used to represent characters, symbols, and images. In a dot-matrix display, straight lines and curves are represented as a series of dots. The spatial display device 10 displays a predetermined image in the air by arranging unmanned aerial vehicles 11 that emit light at each point (dot) that constitutes a dot matrix display. The spatial display device 10 can express, for example, fireworks using animation. Specifically, an unmanned aerial vehicle 11 is assigned to each of the light particles of the fireworks, and multiple unmanned aerial vehicles 11 fly along their respective predetermined routes, causing the light emitting unit 21 of the unmanned aerial vehicle 11 to express the trajectory of the light particles of the fireworks.

[0028] The spatial display device 10 may be connected to a wireless antenna 12 and perform two-way wireless communication with the unmanned aerial vehicle 11 via the wireless antenna 12. The spatial display device 10 may also exchange data with the unmanned aerial vehicle 11 via a recording medium such as an SD card or a USB memory. An unmanned aerial vehicle (UAV) 11 is also called a drone, a small helicopter, or a multicopter, and refers to an unmanned aircraft that flies by wireless remote control or autonomous control. Specifically, the unmanned aerial vehicle 11 is a small quadcopter (an aircraft equipped with four motors), but the number of motors is not limited to this, and the aircraft may be equipped with six, eight, etc. Each unmanned aerial vehicle 11 is equipped with a light-emitting unit 21 (see FIG. 2) such as an LED, and displays an image in the air by illuminating the light-emitting unit 21 at appropriate times. 1 shows five unmanned aerial vehicles 11 (11a, 11b, 11c, 11d, 11e), the number is not limited to five and may be four or less, or six or more. The spatial display device 10 uses, for example, several tens to several thousands of unmanned aerial vehicles 11.

[0029] (About the configuration of Unmanned Aerial Vehicle 11) Next, the configuration of the unmanned aerial vehicle 11 will be described with reference to Figures 2 and 3. Figure 2 is a diagram showing an example of the appearance of the unmanned aerial vehicle 11 used in the spatial display device 10, and Figure 3 is a block diagram for explaining an example of the configuration of the unmanned aerial vehicle 11 used in the spatial display device 10. As shown in Figure 2, the unmanned aerial vehicle 11 has a main body 18 in the center and four arms 19 (a first arm 19a, a second arm 19b, a third arm 19c, and a fourth arm 19d) extending from the main body 18 in all four directions. Propellers 20 are provided at the tips of the four arms 19. The propellers 20 are rotationally driven by a flight drive mechanism 30 (see FIG. 3) constituted by a motor and the like. Specifically, a first propeller 20a is provided at the tip of the first arm 19a, a second propeller 20b is provided at the tip of the second arm 19b, a third propeller 20c is provided at the tip of the third arm 19c, and a fourth propeller 20d is provided at the tip of the fourth arm 19d. The propeller 20 shown in FIG. 2 is a two-bladed propeller, but may be a three-bladed propeller or a four-bladed propeller, etc.

[0030] As shown in FIG. 3, unmanned aerial vehicle 11 includes a hardware configuration including a communication unit 22, a ROM 23, a RAM 24, a storage unit 25, a control unit 26, a battery 27, and an input / output interface 28. In addition, the unmanned aerial vehicle 11 is equipped with hardware components such as a flight drive unit 29, a flight drive mechanism 30, a GPS receiver 31, an illuminance sensor 32, a geomagnetic sensor 33, an altitude sensor 34, a gyro sensor 35, an obstacle detection camera 36, ​​and a light-emitting unit (LED) 21, which are connected to the control unit 26 via the input / output interface 28 to enable two-way data communication.

[0031] The communication unit 22 is mainly used for wireless communication between the spatial display device 10 and other unmanned aerial vehicles 11. The communication method of the communication unit 22 is not particularly limited, and may be Wi-Fi (registered trademark), LoRa (registered trademark), Bluetooth (registered trademark), Zigbee (registered trademark), infrared wireless communication, microwave wireless, broadcast wireless, satellite communication, etc. The communication unit 22 may perform wireless communication between the spatial display device 10 and other unmanned aerial vehicles 11 via the wireless antenna 12.

[0032] The ROM (Read Only Memory) 23 can be used as a recording device, and records firmware required to control the operation of each functional part of the unmanned aerial vehicle 11, various applications, and various data used therefor. The RAM (Random Access Memory) 24 is used to configure the main memory accessed by the control unit 26, and may also be used to temporarily store data obtained from various sensors such as the GPS receiver 31 and illuminance sensor 32 installed on the unmanned aerial vehicle 11. The memory unit 25 is a recording medium such as an SD card or a USB memory, or a storage device such as an SSD (Solid State Drive), and stores the light emission pattern information and flight pattern information described below. The memory unit 25 may be used to temporarily store data acquired from various sensors such as the GPS receiver 31 and the illuminance sensor 32, instead of the RAM 24. The memory unit 25 may not be mounted on the unmanned aerial vehicle 11, and the RAM 24 may perform the function of the memory unit 25.

[0033] The control unit 26 is mainly used to execute the firmware of the unmanned aerial vehicle 11, and includes a central processing unit (CPU), a micro processing unit (MPU), a graphics processing unit (GPU), etc., and is realized by logic circuits (hardware) formed by integrated circuits (IC (Integrated Circuit) chips, LSI (Large Scale Integration)), etc., or dedicated circuits. Battery 27 serves as the power source for unmanned aerial vehicle 11 and supplies power to various functional units. For example, a lithium polymer battery is used as battery 27. Battery 27 is used under an overcharge protection function to protect it from overcharging. Furthermore, the remaining battery charge of battery 27 is monitored by a battery remaining charge check unit (not shown), and when the remaining battery charge falls below a predetermined value, the user is notified of the insufficient remaining battery charge.

[0034] The input / output interface 28 is an interface for transmitting and receiving data to and from the flight drive unit 29, flight drive mechanism 30, GPS receiver 31, illuminance sensor 32, geomagnetic sensor 33, altitude sensor 34, gyro sensor 35, obstacle detection camera 36, ​​and light-emitting unit (LED) 21, etc. The input / output interface 28 may use different standards depending on the data to be handled, and may be compatible with a number of standards, such as USB 2.0, USB 3.0, RS-232C, IEEE 1394, SCSI, and SASI. The flight drive unit 29 controls the drive of the flight drive mechanism 30 based on the flight drive signal generated by the control unit 26. The flight drive mechanism 30 is a mechanism composed of a motor and the like for rotating the propeller 20.

[0035] The GPS (Global Positioning System) receiver 31 acquires location information of the latitude, longitude, and altitude of the current position of the unmanned aerial vehicle 11. The acquired location information of the current position of the unmanned aerial vehicle 11 is stored in the RAM 24 or the memory unit 25. In order to obtain more accurate location information, the current position may be corrected using radio waves from a mobile phone base station or radio waves from an access point of wireless communication (Wi-Fi (registered trademark), Bluetooth (registered trademark, etc.). In addition, the accuracy of the location information of the current position of the unmanned aerial vehicle 11 may be improved by using a quasi-zenith satellite system or a RTK-GPS (Real Time Kinematic - Global Positioning System). The Quasi-Zenith Satellite System (QZSS) is a satellite positioning system provided by Japan that provides location information similar to GPS, but with higher accuracy, especially in Japan's region. RTK-GPS improves the accuracy of location information by adding corrected location information from electronic reference points installed on the ground in addition to location information obtained from GPS. The general location error of location information obtained from GPS is said to be on the order of several meters, but the location error of location information obtained from RTK-GPS is said to be several centimeters. In drone shows, multiple unmanned aerial vehicles 11 fly in formation, so each unmanned aerial vehicle 11 needs to be positioned in the air with high accuracy, and RTK-GPS may be used.

[0036] The illuminance sensor 32 measures the illuminance outside the unmanned aerial vehicle 11. The illuminance sensor 32 is mounted on each unmanned aerial vehicle 11 and measures the external illuminance for each unmanned aerial vehicle 11. The measurement results obtained by the illuminance sensor 32 are used by a light emission luminance correction unit 26f, which will be described later. The emission luminance correction section 26f corrects the emission luminance included in the emission pattern information of the light-emitting section 21 based on the measurement result of the illuminance sensor 32. The light emission brightness correction unit 26f corrects the light emission brightness of the light-emitting unit 21 according to the external illuminance of each unmanned aerial vehicle 11, thereby making it possible to prevent a lack of brightness of the light-emitting unit 21 when displaying an image. Since the light-emitting unit 21 consumes a large amount of power, by appropriately correcting the luminance of the light-emitting unit 21 using the light emission luminance correction unit 26f, it is possible to suppress adverse effects on the capacity of the battery 27 and the flight time.

[0037] The geomagnetic sensor 33 is used to detect the direction in which the unmanned aerial vehicle 11 is flying and moving. The altitude sensor 34 is used to detect the altitude at which the unmanned aerial vehicle 11 is currently located, and includes, for example, a barometric sensor. The gyro sensor 35 detects changes in acceleration in the unmanned aerial vehicle 11, and is used to detect the direction of travel, speed, and attitude of the unmanned aerial vehicle 11. The obstacle detection camera 36 is used to detect objects that may hinder the flight movement of the unmanned aerial vehicle 11, for example, to prevent collisions with other unmanned aerial vehicles 11, and to detect the landing site of the unmanned aerial vehicle 11. The light-emitting unit 21 is used to emit light from the unmanned aerial vehicle 11, and mainly uses an LED. The LED may be a single-color LED or a multi-color LED. A multi-color LED is one that creates various colors by mixing three colors of light, RGB (red, green, blue) or four colors of light, RGBW (red, green, blue, white). The light-emitting unit 21 may also emit laser light. The LED used in the light-emitting unit 21 is preferably lightweight, highly durable, highly efficient, and highly bright.

[0038] The control unit 26 loads the firmware stored in the ROM 23 into a main memory configured with the RAM 24 or the like. The control unit 26 accesses the main memory into which the firmware has been loaded and executes the firmware. By executing the firmware, the control unit 26 is provided with functional configurations such as an acquisition unit 26a, an abnormality detection unit 26b, an abnormality notification unit 26c, an insufficient light amount detection unit 26d, an insufficient light amount notification unit 26e, and an emission brightness correction unit 26f.

[0039] The acquiring unit 26a acquires the position information and the light emission pattern information transmitted from a transmitting unit 47 of the spatial display device 10, which will be described later. Alternatively, the acquisition unit 26a acquires flight pattern information transmitted from the transmission unit 47 together with the position information and the light emission pattern information. The position information refers to the position information of the unmanned aerial vehicle 11 for each dot constituting the matrix display of the image displayed in the air, and is expressed by (x, y, z) in a three-dimensional Cartesian coordinate system. The position information may be expressed using a spherical coordinate system, a cylindrical coordinate system, or a polar coordinate system, in addition to the three-dimensional Cartesian coordinate system. The position information may also be expressed by a combination of latitude, longitude, and altitude. The position information exists uniquely for each dot constituting the matrix display of the image displayed in the air, and each dot is unique to the assigned unmanned aerial vehicle 11. The acquisition unit 26a receives and acquires the position information corresponding to the dot assigned to the unmanned aerial vehicle 11 on which the acquisition unit 26a is mounted. The light emission pattern information is control information that specifies the light emission mode for each dot that constitutes the matrix display of an image to be displayed in the air, and controls the light emission unit 21 of the unmanned aerial vehicle 11 that is assigned to each dot. The light emission mode refers to the light color, light brightness, and light emission timing of the light-emitting unit 21. When an LED is used for the light-emitting unit 21, the light emission mode refers to the light color, light brightness, and light timing of the LED. The flight pattern information, which will be described in detail later, specifies the flight route, flight timing, and flight speed of the unmanned aerial vehicle 11 assigned to each dot of the dot matrix display that makes up the moving image portion of the image displayed by multiple unmanned aerial vehicles 11.

[0040] The abnormality detection unit 26b detects an abnormality in the device itself. The abnormality detection unit 26b detects the occurrence of an abnormality in each functional unit constituting the unmanned aerial vehicle 11, which is the vehicle itself, before and during flight. Each functional unit includes the above-mentioned hardware configuration of the unmanned aerial vehicle 11 and the above-mentioned functional configuration of the unmanned aerial vehicle 11. The abnormality detection unit 26b detects an abnormality that has occurred in these hardware configurations and functional configurations of the unmanned aerial vehicle 11. The abnormality in the battery 27 includes a case where the remaining charge of the battery 27 is equal to or lower than a threshold value. When the remaining charge of battery 27 is insufficient for the planned operating time of unmanned aerial vehicle 11, abnormality detection unit 26b detects an abnormality in battery 27 by determining that the remaining charge of battery 27 is below a threshold value.

[0041] The abnormality notification unit 26c notifies the spatial display device 10 of abnormality information relating to an abnormality in the own device. When the abnormality detection unit 26b detects an abnormality in the unmanned aerial vehicle 11, which is the vehicle itself, the abnormality notification unit 26c immediately notifies the spatial display device 10 of abnormality information regarding the abnormality via wireless communication via the communication unit 22, and the spatial display device 10 receives the abnormality information via the abnormality information receiving unit 44 described below. Abnormality information regarding an abnormality in the own aircraft includes identification information unique to the own aircraft, which is the unmanned aerial vehicle 11, information such as which functional part among the functional parts that make up the unmanned aerial vehicle 11 has experienced the abnormality, and the severity of the abnormality. Abnormalities in the aircraft include, for example, malfunctions in the flight drive mechanism 30, flight drive unit 29, GPS receiver 31, obstacle detection camera 36, ​​and light-emitting unit 21. Furthermore, examples of abnormalities in the own aircraft include malfunctions of the illuminance sensor 32, the geomagnetic sensor 33, and the altitude sensor . Furthermore, examples of abnormalities in the device itself include malfunctions of the functional units (26a to 26f) of the control unit 26.

[0042] The light amount deficiency detection unit 26d detects a deficiency in the amount of light emitted by the own device. The light insufficiency detection unit 26d detects a lack of light intensity during emission of light from the light-emitting unit 21 of the unmanned aerial vehicle 11, which is the vehicle itself. The light amount deficiency detection unit 26d may use an illuminance sensor 32 to detect a deficiency in the amount of light emitted by the own device. An insufficient amount of light emitted by the aircraft itself refers to an insufficient amount of light emitted by the light-emitting unit 21 of the unmanned aerial vehicle 11 relative to the illuminance of the background of the image displayed in the air. Even if the light-emitting unit 21 of the unmanned aerial vehicle 11 is operating normally, the light insufficiency detection unit 26d detects an insufficient amount of light emitted by the aircraft itself when the amount of light emitted by the light-emitting unit 21 is insufficient relative to the illuminance of the background of the image displayed in the air.

[0043] The light amount deficiency notification unit 26e notifies the spatial display device 10 of light amount deficiency information related to the deficiency of the light amount emitted by the own device. When the light deficiency notification unit 26e detects a deficiency in the amount of light emitted by the light-emitting unit 21 of the unmanned aerial vehicle 11, which is the vehicle itself, it notifies the spatial display device 10 of the light deficiency information regarding the deficiency by wireless communication via the communication unit 22, and the spatial display device 10 receives the light deficiency information via the light deficiency information receiving unit 45 described below. The light deficiency information regarding the insufficient light intensity of the aircraft's own light includes identification information unique to the unmanned aerial vehicle 11, which is the aircraft's own light intensity, the light intensity of the light-emitting element 21 of the unmanned aerial vehicle 11, and the degree of the light deficiency.

[0044] The emission luminance correction unit 26f corrects the emission luminance included in the emission pattern information based on the measurement result of the illuminance sensor 32, as described above. The light emission luminance refers to the light emission luminance of light-emitting unit 21, and in the case where an LED is adopted for light-emitting unit 21, the light emission luminance refers to the light emission luminance of the LED.

[0045] (Hardware configuration of spatial display device 10) Next, an example of a hardware configuration of the spatial display device 10 will be described with reference to Fig. 4. Fig. 4 is a block diagram for explaining an example of a hardware configuration of the spatial display device 10. The spatial display device 10 includes a communication unit 10a, a ROM 10b, a RAM 10c, a storage unit 10d, a processing unit 10e, and an input / output interface 10f. Furthermore, the spatial display device 10 includes a display unit 10g, an operation input unit 10h, and an external information input unit 10i, which input and output data via an input / output interface 10f.

[0046] The communication unit 10a has a function of performing two-way communication between the unmanned aerial vehicle 11 and other information processing devices. When communication unit 10a performs two-way communication with unmanned aerial vehicle 11, the communication unit 10a performs the two-way communication by wireless communication, and the wireless communication may be performed via wireless antenna 12 (see FIG. 1), or via a wireless antenna (not shown) connected to information and communication network 17. The wireless communication method is not particularly limited, and may be Wi-Fi (registered trademark), LoRa (registered trademark), Bluetooth (registered trademark), Zigbee (registered trademark), infrared wireless communication, microwave wireless, broadcast wireless, satellite communication, or the like. The other information processing device may be another spatial display device 10 or another information processing device. When the communication unit 10a performs two-way communication with another information processing device, the communication unit 10a may perform the two-way communication via the information communication network 17, or may directly connect to the other information processing device to perform the two-way communication. The communication unit 10a may use wired communication or wireless communication for communication with the other information processing device. An external information input unit 10i, which will be described later, may receive original image data from another information processing device via the communication unit 10a.

[0047] The ROM 10b can be used as a recording device, and stores a BIOS (Basic Input Output System) required for controlling the operation of each functional unit of the spatial display device 10, various data used by the BIOS, and the like. The BIOS is a program that manages the basic input / output functions of the spatial display device 10 as an information processing device. It is the first program that runs when the spatial display device 10 is turned on, controlling hardware such as the communication unit 10a, ROM 10b, RAM 10c, memory unit 10d, processing unit 10e, and input / output interface 10f, and preparing to start up the OS (Operating System).

[0048] The RAM 10c is used to configure a main memory accessed by the processing unit 10e, and is further used to temporarily store various data acquired or generated by the spatial display device 10 before the data is stored in the storage unit 10d.

[0049] The storage unit 10d is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), an online storage, etc., and stores an OS, a spatial display program described later, other application software, various data used by these programs, etc. The storage unit 10d also stores various data acquired or generated by the spatial display device 10.

[0050] The processing unit 10e includes a CPU, an MPU, a GPU, and the like, and is realized by a logic circuit or a dedicated circuit formed by an integrated circuit (IC chip, LSI), and the like. The input / output interface 10f is an interface for transmitting and receiving data to and from the display unit 10g, the operation input unit 10h, and the external information input unit 10i. The input / output interface 10f may use different standards depending on the data to be handled, and may be compatible with a number of standards, such as HDMI (registered trademark), USB 2.0, USB 3.0, RS-232C, IEEE 1394, SCSI, and SASI.

[0051] The display unit 10g is a monitor of the spatial display device 10 and displays various data. The operation input unit 10h accepts input of operations by a user for the spatial display device 10. The operations are performed using a keyboard, a mouse, or the like, and output signals from the keyboard, mouse, etc. are input to the operation input unit 10h. The external information input unit 10i receives original image data that is the source of the image displayed by the spatial display device 10. The original image data is image data that is the material of the image displayed by spatial display device 10, and an image expressed as a dot matrix display is obtained based on the original image data. Furthermore, the original image data is used by a user of the spatial display device 10 to derive an image that he or she wishes to display in the air using multiple unmanned aerial vehicles 11 based on the idea of ​​the original image data.

[0052] The image data includes shape data relating to the size and shape of the image to be displayed in the air, color data relating to the color of the image, and video data relating to the video portion of the image. The image data also includes contour data relating to the contour of the image to be displayed in the air. The shape data defines the size and shape of the image to be displayed in the air, and also defines the display mode of the image, which can be a display using the outline of the image, a two-dimensional display of the image, or a three-dimensional display of the image. The contour data defines the shape of the boundary between an image to be displayed in the air and the background of the image, and may be included in the shape data. A dot number calculation unit 39, which will be described later, calculates the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on shape data included in the image data. A position information generating unit 40, which will be described later, generates position information for each of the dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the shape data included in the image data. The color data defines the color of the image to be displayed in the air. A light emission processing unit 42, which will be described later, generates light emission pattern information that defines the light emission state of each dot that will be in a light-emitting state when the image is displayed in a dot matrix, based on color data included in the image data. The light emission mode refers to the light color, light brightness, and light timing of each dot that is in an illuminated state when the image is displayed in a dot matrix, as well as the light color, light brightness, and light timing of the light-emitting section 21 of the unmanned aerial vehicle 11. The video data defines the video portion of the image to be displayed in the air, and more specifically defines the duration of the video portion, the position of the video portion within the image to be displayed in the air, and the manner and speed of movement of the position of the video portion. The flight control unit 41, described later, generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that constitutes the video portion and that will be illuminated when the image is displayed in a dot matrix, based on the video data contained in the image data.

[0053] The image data that is the source of the image may be obtained based on the original image data received by the external information input unit 10i, using a learning model that has previously learned the correspondence between image data and original image data regarding the image that the spatial display device 10 displays in the air. The learning model may be trained through machine learning and perform image recognition on the original image data to classify which category the original image data belongs to, or it may perform object detection on objects appearing in the original image data. The learning model may, for example, classify the original image data into a predetermined category and output image data related to the category to which the original image data belongs. Specifically, if the original image data is "a video of a baseball player hitting a home run," the learning model to which this original image data is input classifies the original image data as a video related to a "home run" and outputs image data of the characters "home run." The image data acquisition unit 38 described below acquires the image data of the characters "home run." Furthermore, for example, when a specific object is included in the original image data, the learning model may detect the specific object and output image data related to the detected object. Specifically, when the original image data includes the "7 of hearts of a playing card," the learning model to which this original image data is input detects the object included in this original image data as the "7 of hearts of a playing card," and outputs image data of a heart shape and image data of the number "7." The image data acquisition unit 38, which will be described later, acquires image data of the heart shape and image data of the number "7."

[0054] The original image data may be a still image or a video, or may be text, music, symbols, codes, etc. The external information input unit 10i receives the original image data and inputs it to the spatial display device 10. The external information input unit 10i may be connected to external input devices such as a camera, a video camera, a microphone, and a scanner, and may receive output data from these input devices as input data.

[0055] The camera may be an ultra-high-speed camera, and may capture high-speed phenomena such as magician's tricks, animal movements, machine movements, images of natural phenomena, and celestial movements by high-speed photography, and may cut out that moment and input it to the external information input unit 10i. In this case, the spatial display device 10 can use multiple unmanned aerial vehicles 11 to project in the air a dot matrix display of an image that cuts out a moment of the high-speed phenomenon. The camera may be an infrared camera, and a thermal distribution image obtained by capturing infrared rays radiated from an object may be input to the spatial display device 10 via the external information input unit 10i. The camera may be an X-ray camera, and an image of the inside of the body obtained by, for example, an X-ray may be input to the spatial display device 10 via the external information input unit 10i. Furthermore, the external information input unit 10i may receive, as the original image data, the movement of an object detected by a tracker used in motion capture. Motion capture is a technology that digitally records the movements of a real person, animal, or object that is the subject of detection. There are various types of motion capture, such as optical, inertial sensor, and magnetic. In the case of optical motion capture, the movement of the subject of detection is recorded by combining a tracker and a marker. For example, in optical motion capture, multiple markers are attached to the body of the person to be detected, and the movement of the markers is detected by a tracker, so that the person's movements can be digitally recorded.

[0056] The external information input unit 10i may receive input of images acquired in a medical field, such as CT scan images, MRI images, ultrasound tomographic images, nuclear medicine examination images, and angiographic images, and input the images to the spatial display device 10. Furthermore, the external information input unit 10i may accept input of real-time video showing the current situation captured by fixed observation cameras, such as security cameras, live video from the Meteorological Agency, live video from a concert hall, live video from a stadium, live video from a theater hall, live video from a zoo or botanical garden, and live video from an astronomical observation, and input the video to the spatial display device 10. The spatial display device 10 can use multiple unmanned aerial vehicles 11 to project a dot matrix display of the real-time video in the air. Furthermore, the external information input unit 10i may receive input of video captured by a camera mounted on a vehicle such as an automobile, motorcycle, or bicycle, and input the video to the spatial display device 10. The spatial display device 10 can project a dot matrix display of video viewed from a vehicle such as an automobile, motorcycle, or bicycle in the air using a plurality of unmanned aerial vehicles 11. The spatial display device 10 can display live video of a race of an automobile, motorcycle, bicycle, or the like in the air using a plurality of unmanned aerial vehicles 11, and can show a realistic video to many spectators.

[0057] When the image that the spatial display device 10 displays in the air is, for example, an animation for a drone show, the animation is created by the following procedure. (Step 1) First, an animation is created based on the original image data. To create an animation, the people and objects that will appear, as well as the background to be used, are decided. People that will appear include people, animals, mascots, etc. Objects that will appear include vehicles, buildings, supplies, tools, as well as letters, illustrations, marks, codes, etc. Codes may include QR codes, bar codes, ciphers, etc. Backgrounds include, for example, mountains, forests, rivers, night skies, snowy scenery, fireworks, or scenery. In animation, once the people and objects that will appear have been decided, the way they will move is then determined.

[0058] (Step 2) Next, the type and number of unmanned aerial vehicles 11 are determined. The type and number of unmanned aerial vehicles 11 to be used for animation are determined. Once the animation is created, the animation is converted to a dot matrix display to determine the number of dots required to display the animation, which will determine the number of unmanned aerial vehicles 11 to be used. The type of unmanned aerial vehicle 11 is determined by the possible continuous flight time and the number of light colors. The possible continuous flight time of unmanned aerial vehicle 11 must be at least longer than the running time of the animation. The more light colors the unmanned aerial vehicle 11 has, the more colorful the animation can be.

[0059] (Step 3) Create flight pattern information for the unmanned aerial vehicle 11. Flight pattern information is created that specifies the flight route, flight timing, and flight speed of the unmanned aerial vehicle 11 assigned to each dot of the dot matrix display that makes up the animation. The flight pattern information is different for each dot of the dot matrix display. A particular pattern or design is created by multiple unmanned aerial vehicles 11 flying according to their respective flight pattern information. The flight pattern information is used by multiple unmanned aerial vehicles 11 flying simultaneously and synchronizing them in the air to create the animation. In addition, if the animation does not involve movement, that is, if the image displayed in the air by illuminating the multiple unmanned aerial vehicles 11 is a still image, the flight pattern information includes a flight route to the position information of each dot constituting the dot matrix display of the still image. In addition, the flight pattern information may or may not include a return route of the unmanned aerial vehicle 11. If the flight pattern information does not include a return route, the unmanned aerial vehicle 11 selects a flight route by autonomous control while checking its current position by GPS signals toward a predetermined return location.

[0060] (Step 4) Light emission pattern information for the unmanned aerial vehicle 11 is created. Based on the colors of the original image data, light emission pattern information is created that specifies the light emission mode of the unmanned aerial vehicle 11 assigned to each dot of the dot matrix display that displays the animation. The light emission mode refers to the light emission color, light emission brightness, and light emission timing of the LED of the light-emitting unit 21 of the unmanned aerial vehicle 11. The color and blinking pattern of the LED are set to draw designs such as characters, codes, and figures.

[0061] (Functional configuration of spatial display device 10) Next, an example of the functional configuration of the spatial display device 10 will be described with reference to Fig. 5. Fig. 5 is a block diagram for explaining an example of the functional configuration of the spatial display device 10. The spatial display device 10 loads a spatial display program, which will be described later and is stored in the storage unit 10d, into a main memory constituted by a RAM 10c, etc. The processing unit 10e accesses the main memory into which the spatial display program has been loaded, and executes the spatial display program. By executing the spatial display program, the spatial display device 10 is provided with functional units in the processing unit 10e, such as a learning model acquisition unit 37, an image data acquisition unit 38, a dot number calculation unit 39, a position information generation unit 40, a flight control unit 41, a light emission processing unit 42, an unmanned aerial vehicle allocation unit 43, an abnormality information receiving unit 44, a light insufficiency information receiving unit 45, an image allocation unit 46, a transmission unit 47, and an aircraft number adequacy determination unit 48.

[0062] The learning model acquisition unit 37 acquires a learning model that has previously learned the correspondence between image data and original image data relating to an image. In detail, the learning model acquisition unit 37 acquires a learning model that has previously learned the correspondence between image data related to an image to be displayed in the air and the original image data. The learning model may be a machine learning model learned by machine learning, or may be a table that specifies the correspondence between image data regarding an image to be displayed in the air and the original image data. The table may be, for example, a table that associates metadata that was added to the original image data when the original image data was acquired with image data related to an image to be displayed in the air. The metadata may be, for example, character information for classifying the original image data, character information indicating attributes of the original image data, or character information indicating an object included in the original image data. An image data acquisition unit 38 described below may refer to the table and acquire image data based on the original image data.

[0063] The image data acquisition unit 38 acquires image data relating to an image. In detail, the image data acquisition unit 38 acquires image data relating to images that are displayed in the air by illuminating multiple unmanned aerial vehicles 11. Image data relating to an image includes shape data, color data, and video data of the image, and is acquired based on the original image data. The image data acquisition unit 38 may acquire image data input by a user to the external information input unit 10i. The image data acquisition unit 38 may acquire image data by inputting the original image data accepted by the external information input unit 10i into a learning model.

[0064] The dot number calculation unit 39 calculates, based on the image data, the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix. Based on the shape data contained in the image data, the dot number calculation unit 39 calculates the number of dots that will be illuminated when an image in which multiple unmanned aerial vehicles 11 are illuminated and displayed in the air is displayed in a dot matrix format. The light emitting state includes such states as lighting, blinking, and flashing, and also includes a state in which light is emitted at least once while an image is displayed in the air. Dot number calculation unit 39 does not include dots that are not lit when the image to be displayed in the air is displayed in a dot matrix in the number of dots. Therefore, unmanned aerial vehicles 11 are not assigned to dots that are not lit when the image to be displayed in the air is displayed in a dot matrix, and unmanned aerial vehicles 11 that do not contribute to the display of the image are not used, so that unmanned aerial vehicles 11 can be operated efficiently.

[0065] The position information generating unit 40 generates position information for each of the dots that are in a light-emitting state based on the image data. Specifically, the position information generating section 40 generates position information for each of the dots that are in a light-emitting state, based on the shape information included in the image data. As mentioned above, location information exists uniquely for each dot that constitutes the matrix display of the image displayed in the air, and is unique to the unmanned aerial vehicle 11 assigned to each dot. When an image to be displayed in the air is displayed by a contour, the position information generating unit 40 acquires contour data relating to the contour based on the image data, and generates position information for each dot that will be illuminated when the contour is displayed in a dot matrix, based on the contour data. In other words, when an image to be displayed in the air is displayed by a contour, the position information generating unit 40 acquires contour data regarding the contour based on shape data included in the image data, and generates position information for each dot that will be illuminated when the contour is displayed in a dot matrix, based on the contour data.

[0066] Each dot that makes up the dot matrix display corresponds to a light-emitting unit 21 of the unmanned aerial vehicle 11 . When the image is a three-dimensional image composed of multiple layers, the position information generating unit 40 generates, based on the image data, position information for each dot that will be illuminated when the image divided into multiple layers is displayed in a dot matrix.

[0067] The dot matrix display will be described with reference to Fig. 6 and Fig. 7. Fig. 6 is a diagram for explaining the dot matrix display used in the spatial display device 10, and Fig. 7 is a diagram for explaining the display of a stereoscopic image by the dot matrix display used in the spatial display device 10. A dot matrix display displays characters or images using dots arranged in a two-dimensional array. The dot matrix display used in this embodiment uses an array of 8 dots vertically and 8 dots horizontally, but is not limited to this and may be, for example, an array of 6 dots vertically and 6 dots horizontally, or an array of 12 dots vertically and 12 dots horizontally.

[0068] An example of a dot-matrix display, ie, the display of the alphabet "R" 49, will be described with reference to Fig. 6. Fig. 6(a) shows the state of the alphabet "R" 49 before it is converted into a dot-matrix display, and Fig. 6(b) shows the state of the alphabet "R" 50 after it is converted into a dot-matrix display. The dot matrix display of the alphabet "R" 50 is displayed by 30 dots. In FIG. 6(b), 6 of the 30 dots are denoted by reference characters 51 to 56. Each of the 30 dots is assigned to the light emitting unit 21 of the unmanned aerial vehicle 11 in an illuminated state. The dot matrix display of the alphabet "R" 50 may be achieved by using 64 unmanned aerial vehicles 11 arranged in a two-dimensional array, with the light emitting units 21 of 30 of the unmanned aerial vehicles 11 illuminated, or by using 30 unmanned aerial vehicles 11 and with the light emitting units 21 of all of the unmanned aerial vehicles 11 illuminated to form the dot matrix display of the alphabet "R" 50.

[0069] Next, the display of a stereoscopic image by a dot matrix display will be described with reference to Fig. 7. Fig. 7 shows that a stereoscopic image is composed of multiple layers superimposed on top of each other. Each layer is represented by a dot matrix display in a two-dimensional array of 8 dots vertically and 8 dots horizontally. When the image displayed by the spatial display device 10 is a stereoscopic image, the stereoscopic image is expressed by overlapping seven layers, as shown in Fig. 7. Each of the seven layers shown in Fig. 7 is a two-dimensional dot matrix display, and is composed of a first layer 60, a second layer 61, a third layer 62, a fourth layer 63, a fifth layer 64, a sixth layer 65, and a seventh layer 66. Based on the shape data contained in the image data, the dot number calculation unit 39 calculates the number of dots that will be illuminated when an image is displayed in the air in a dot matrix format, in which multiple unmanned aerial vehicles 11 are illuminated, in each of multiple layers (first layer 60 to seventh layer 66). The position information generating unit 40 generates, based on image data, position information for each of a plurality of layers (first layer 60 to seventh layer 66) for each dot that will be in a light-emitting state when images divided into these layers are displayed in a dot matrix.

[0070] The flight control unit 41 generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that constitutes the moving image portion, based on video data relating to the moving image portion of the image contained in the image data. When the image to be displayed in the air is a stereoscopic image, the flight control unit 41 generates flight pattern information for each of the multiple layers (first layer 60 to seventh layer 66) based on video data relating to the video portion of the stereoscopic image contained in the image data, which specifies the flight route, flight timing, and flight speed for each dot that makes up the video portion.

[0071] The light emission processing unit 42 generates light emission pattern information that specifies the light emission state of each dot that will be in a light-emitting state when the image to be displayed in the air is displayed in a dot matrix, based on color data related to the color of the image contained in the image data. When the image to be displayed in the air is a stereoscopic image, the light emission processing unit 42 generates light emission pattern information that specifies the light emission state of each dot in each of multiple layers (first layer 60 to seventh layer 66) based on color data related to the color of the image contained in the image data.

[0072] The unmanned aerial vehicle allocation unit 43 allocates an unmanned aerial vehicle 11 to each dot that is illuminated when an image to be displayed in the air is displayed in a dot matrix.

[0073] The abnormality information receiving unit 44 receives abnormality information of the unmanned aerial vehicle 11. The abnormality information receiving unit 44 receives the abnormality information when the abnormality detection unit 26b of the unmanned aerial vehicle 11 detects an abnormality in the vehicle and the abnormality notification unit 26c notifies the spatial display device 10 of the abnormality information regarding the abnormality. The unmanned aerial vehicle allocation unit 43 of the spatial display device 10 that receives the abnormality information allocates an unmanned aerial vehicle 11 belonging to another group to the dot that was previously assigned to the unmanned aerial vehicle 11 that notified the abnormality information, in place of the unmanned aerial vehicle 11. Then, the transmitter 47 transmits a return command to the unmanned aerial vehicle 11 that has notified the abnormality information to return. Furthermore, the transmitting unit 47 transmits position information and light emission pattern information, or position information, light emission pattern information, and flight pattern information corresponding to the dot to which the unmanned aerial vehicle 11 that notified the abnormality information was assigned, to an unmanned aerial vehicle 11 belonging to another group that was assigned in place of the unmanned aerial vehicle 11 that notified the abnormality information. An unmanned aerial vehicle 11 belonging to another group assigned in place of the unmanned aerial vehicle 11 that notified the abnormality information acquires, by the acquisition unit 26a, position information and light emission pattern information, or position information, light emission pattern information, and flight pattern information corresponding to the dot to which the unmanned aerial vehicle 11 that notified the abnormality information was assigned, and is operated in place of the unmanned aerial vehicle 11 that notified the abnormality information.

[0074] The insufficient light information receiving unit 45 receives insufficient light information of the unmanned aerial vehicle 11. The light deficiency information receiving unit 45 receives the light deficiency information when the light deficiency detection unit 26d of the unmanned aerial vehicle 11 detects a deficiency in the amount of light emitted by the unmanned aerial vehicle and the light deficiency notification unit 26e notifies the spatial display device 10 of the light deficiency information regarding the deficiency. The unmanned aerial vehicle allocation unit 43 of the spatial display device 10 that receives the insufficient light information allocates an unmanned aerial vehicle 11 that belongs to another group together with the unmanned aerial vehicle 11 that notified the insufficient light information to the dot to which the unmanned aerial vehicle 11 had been assigned. In this case, the transmitting unit 47 transmits position information and light emission pattern information, or position information, light emission pattern information, and flight pattern information corresponding to the dot to which the unmanned aerial vehicle 11 that notified the insufficient light information was assigned, to unmanned aerial vehicle 11 belonging to another group that is assigned to the dot to which the unmanned aerial vehicle 11 that notified the insufficient light information was assigned. As a result, two unmanned aerial vehicles 11 are assigned to emit light for dots that were lacking in light, making it possible to provide a sufficient amount of light for the brightness of the background of the image displayed in the air.

[0075] The image allocation unit 46 allocates the image to be displayed in the air to one of a plurality of groups. The image allocation unit 46 allocates an image to be displayed in the air to one of the multiple groups to which the multiple unmanned aerial vehicles 11 are distributed. The number of groups to which the image is allocated may be one, or may be two or more groups.

[0076] The transmitter 47 transmits position information and light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles 11 assigned to each dot that will be illuminated when an image to be displayed in the air is displayed in a dot matrix. When the image to be displayed in the air includes a moving image, the transmitting unit 47 transmits flight pattern information to each of the unmanned aerial vehicles 11 assigned to each dot that constitutes the moving image portion, together with position information and light emission pattern information corresponding to that dot.

[0077] During the flight of the unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display, the transmitter 47 may transmit to each of the unmanned aerial vehicles 11 in flight position information and light emission pattern information corresponding to the dot to which the unmanned aerial vehicle 11 is assigned. In this case, the acquisition unit 26a of the unmanned aerial vehicle 11 can receive the position information and light emission pattern information transmitted from the transmission unit 47 during flight. This allows the spatial display device 10 to transmit position information and light emission pattern information generated based on newly added original image data to the unmanned aerial vehicle 11 in flight, thereby allowing the image displayed by the spatial display device 10 to be changed in real time based on the newly added original image data.

[0078] If the image is a video, the transmitter 47 may transmit flight pattern information to each of the unmanned aerial vehicles 11 assigned to each flying dot, together with position information and light emission pattern information corresponding to the dot. In other words, when the image is a video, while an unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display is flying, the transmitter 47 may transmit flight pattern information to each of the unmanned aerial vehicles 11, along with position information and light emission pattern information corresponding to the dot. The acquisition unit 26a of the unmanned aerial vehicle 11 can receive flight pattern information transmitted from the transmission unit 47 along with position information and light emission pattern information during flight. As a result, even if the image is a video, the spatial display device 10 can transmit position information, light emission pattern information, and flight pattern information generated based on newly added original image data to the unmanned aerial vehicle 11 in flight, so that the image displayed by the spatial display device 10 can be changed in real time based on the newly added original image data.

[0079] The aircraft number sufficiency determination unit 48 determines whether the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation unit 39 . The number of aircraft sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation unit 39 when the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is equal to or greater than the number of dots calculated by the dot number calculation unit 39. On the other hand, if the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is smaller than the number of dots calculated by the dot number calculation unit 39, the number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned does not suffice the number of dots calculated by the dot number calculation unit 39.

[0080] If the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation unit 39, the unmanned aerial vehicle allocation unit 43 assigns an unmanned aerial vehicle 11 belonging to the group to which the image is assigned to each dot. On the other hand, if the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 included in the group to which the image is assigned does not suffice the number of dots calculated by the dot number calculation unit 39, the unmanned aerial vehicle allocation unit 43 assigns, for each dot, unmanned aerial vehicles 11 belonging to the group to which the image is assigned, and also assigns unmanned aerial vehicles 11 belonging to other groups. In this case, the unmanned aerial vehicle allocation unit 43 assigns unmanned aerial vehicles 11 belonging to the group to which the image is assigned to dots that will light up when the image is displayed in a dot matrix, and assigns unmanned aerial vehicles 11 belonging to other groups in an insufficient number to dots that will light up when the image is displayed in a dot matrix. In addition, the other groups may be waiting in the air or on the ground, and the unmanned aircraft allocation unit 43 may give priority to allocating unmanned aircraft 11 belonging to other groups that are waiting in the air over other groups that are waiting on the ground.

[0081] If the number of aircraft sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 does not suffice the number of dots calculated by the dot number calculation unit 39, and if an unmanned aerial vehicle 11 belonging to another group cannot be assigned to a dot that will be illuminated when the image to be displayed in the air is displayed in a dot matrix, the position information generation unit 40 generates position information for each dot, the number of dots corresponding to the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned, and the unmanned aerial vehicle assignment unit 43 assigns an unmanned aerial vehicle 11 belonging to the group to which the image is assigned, to each dot, the number of dots corresponding to the number of unmanned aerial vehicles 11. In other words, if the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 does not suffice the number of dots calculated by the dot number calculation unit 39, and if it is not possible to assign an unmanned aerial vehicle 11 belonging to another group to a dot that will be illuminated when the image is displayed in a dot matrix, the number of dots that will be illuminated when the image is displayed in a dot matrix is ​​reduced, and an unmanned aerial vehicle 11 belonging to the group to which the image is assigned is assigned.

[0082] The spatial display device 10 distributes multiple unmanned aerial vehicles 11 into multiple groups, and if there are not enough unmanned aerial vehicles 11 in a group to which an image to be displayed in the air is assigned, unmanned aerial vehicles 11 belonging to other groups are assigned, thereby allowing the unmanned aerial vehicles 11 to be operated flexibly and efficiently. Furthermore, even if there are an insufficient number of unmanned aerial vehicles 11 belonging to the group to which the image to be displayed in the air is assigned, and an unmanned aerial vehicle 11 belonging to another group cannot be assigned, the spatial display device 10 can respond by reducing the number of dots in the dot matrix display of the image to be displayed in the air.

[0083] (Application Example 1 of the spatial display device 10) Next, a first example of use of the spatial display device 10 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining a first example of use of the spatial display device 10. A first example of use of the spatial display device 10 is a live broadcast of a card guessing trick. In the spatial display device 10, the original image data is a video of a live broadcast of a magic trick. The card guessing trick involves having a spectator draw a card from a well-shuffled deck of cards, memorizing the suit and number of the card without showing it to the magician, and then returning the card to the original deck. The magician then draws the same card as the spectator from the deck.

[0084] A video camera is prepared to capture the area around the magician's hands. The video camera captures the card 70 drawn by the audience. The image of the card 70 drawn by the audience is received by the external information input unit 10i and input to the spatial display device 10. When the card 70 drawn by the customer is, for example, the "7 of hearts," the spatial display device 10 stores an image of the "heart mark and the number 7" as the original image data in the storage unit 10d. Based on the original image data, ie, the image of the "heart mark and the number 7", the position information generating unit 40 generates position information of the unmanned aerial vehicle 11 for each dot constituting the dot matrix display of the "heart mark and the number 7". The light emission processing unit 42 generates light emission pattern information for each dot constituting a dot matrix display of the "heart mark and the number 7" based on the image of the "heart mark and the number 7" which is the original image data. The unmanned aerial vehicle allocation unit 43 allocates an unmanned aerial vehicle 11 to each of the dots that make up the dot matrix display of the "heart symbol and the number 7." The transmitter 47 transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles 11 assigned to each dot in flight. The unmanned aerial vehicle 11 can immediately display a dot matrix display of a heart mark and the number 7 in the air 13 by receiving the location information and light emission pattern information transmitted from the transmitter 47 during flight using the acquisition unit 26a.

[0085] First, as shown in Fig. 8(a), the spatial display device 10 displays the "7 of hearts" card 70 drawn by the customer in the air 13. Next, the spatial display device 10 receives an image of the card 71 drawn by the magician via the external information input unit 10i, and displays the "7 of hearts" card 71 drawn by the magician in the air 13, similar to the card 70 drawn by the customer, as shown in Fig. 8(b). The spatial display device 10 displays a card 70 drawn by a spectator and a card 71 drawn by the magician side by side in the air 13, thereby indicating in the air 13 that the magician has succeeded in his magic trick. As shown in FIG. 8, by displaying the card 70 drawn by the spectator and the card 71 drawn by the magician side by side in the air 13, not only the spectator taking part in the magic trick but also the many spectators watching the air 13 can be given a sense of realism as if they were taking part in the magic trick.

[0086] (Use Example 2 of the spatial display device 10) Next, a second example of use of the spatial display device 10 will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining a second example of use of the spatial display device 10. The second application example of the spatial display device 10 is an example in which the device is used for live broadcasting of a card guessing magic trick, similar to the first application example. In the spatial display device 10 of the second usage example, a plurality of unmanned aerial vehicles 11 (256 vehicles) are arranged in advance in a two-dimensional array of eight dots vertically and eight dots horizontally in four locations in the air 13. Then, based on the original image data received by the external information input unit 10i and input to the spatial display device 10, light emission pattern information for each of the light-emitting units 21 of the 256 unmanned aerial vehicles 11 is generated, and the light emission pattern information corresponding to each dot is transmitted from the transmitting unit 47 to the unmanned aerial vehicle 11 assigned to that dot. The unmanned aerial vehicle 11 causes the light-emitting unit 21 to emit light based on the light emission pattern information received by the acquiring unit 26a. The spatial display device 10 can immediately generate light emission pattern information based on the original image data received by the external information input unit 10i, and transmit the information to the unmanned aerial vehicle 11 by the transmission unit 47. Then, the unmanned aerial vehicle 11 can immediately cause the light emitting unit 21 to emit light in accordance with the light emission pattern information received by the acquisition unit 26a. Since the spatial display device 10 does not need to change the position of the unmanned aerial vehicle 11 in the air due to changes in the original image data, it can change the aerial image in real time in response to changes in the original image data.

[0087] 9(a), the position information generating unit 40 generates the position information of the unmanned aerial vehicle 11 for each dot constituting a two-dimensional array of eight dots vertically and eight dots horizontally in the four dot matrix display. The required number of unmanned aerial vehicles 11 is 256. In Figure 9, a white circle indicates the unmanned aerial vehicle 11's light-emitting unit 21 in an off state, and a black circle indicates the unmanned aerial vehicle 11's light-emitting unit 21 in an on state. The unmanned aerial vehicle allocation unit 43 allocates an unmanned aerial vehicle 11 to each of the dots constituting the two-dimensional array of the four dot matrix display. The external information input unit 10i receives an image of the card 70 drawn by the customer, and the image is input to the spatial display device 10. When the card 70 drawn by the customer is, for example, the "seven of hearts," the spatial display device 10 stores an image of the "heart mark and the number seven" as the original image data in the storage unit 10d. The light emission processing unit 42 generates light emission pattern information for each dot constituting a dot matrix display of the "heart mark and the number 7" based on the image of the "heart mark and the number 7" which is the original image data. The transmitter 47 transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles 11 assigned to each dot in flight. During flight, unmanned aerial vehicle 11 receives, by acquisition unit 26a, the position information and light emission pattern information transmitted from transmission unit 47. Because there is no change in the received position information, unmanned aerial vehicle 11 causes light-emitting unit 21 to emit light in accordance with the light emission pattern information without changing its position in air 13, and can immediately display a dot matrix display of a "heart mark and the number 7" in air 13 (see FIG. 9(b)). Next, the spatial display device 10 receives an image of the card 71 drawn by the magician via the external information input unit 10i, and displays the card 71 drawn by the magician, a "7 of hearts", in the air 13, similar to the card 70 drawn by the customer, as shown in Figure 9(c). The spatial display device 10 can show in the air 13 that the magician has succeeded in the magic trick by arranging and displaying in the air 13 the card 70 drawn by the guest and the card 71 drawn by the magician. As shown in FIG. 9, by preparing in the air 13 a two-dimensional array of four dot matrix displays in the air 13, it is not necessary to change the position of the unmanned aircraft 11, so that the image in the air 13 can be changed in response to the change in the original image data received by the external information input unit 10i quickly.

[0088] (Regarding Usage Example 3 of the Spatial Display Device 10) Referring to FIG. 10, as a usage example 3 of the spatial display device 10, an example of the case of smoothly showing the image displayed in the air will be described. FIG. 10 is a diagram for explaining Usage Example 3 of the spatial display device 10. FIG. 10(a) is a diagram for explaining the grayscale font of the Chinese character "big", and FIG. 10(b) is a diagram for explaining the dots that will be in the light-emitting state when the Chinese character "big" is displayed in dot matrix. The grayscale font refers to a font with anti-aliasing applied. When a character is displayed in dot matrix, it refers to a font that looks smooth by inserting intermediate colors so that the boundaries of the slanted and curved parts of the character do not look jagged. When the spatial display device 10 displays not only characters but also other images in the air, the image can be smoothly displayed by applying anti-aliasing processing to the image. Aliasing refers to the phenomenon that the diagonal and curved parts of an image look jagged, and anti-aliasing refers to making this jaggedness less noticeable. The intermediate color refers to the intermediate color between the color of the image displayed in the air and the color of the background of the image. The intermediate color has 4 gradations, 8 gradations, 16 gradations, etc. The gradations such as 4 gradations, 8 gradations, 16 gradations, etc. are determined by the size and luminance of one dot of the image displayed by the spatial display device 10. When one dot of the image is relatively large and has a high luminance, it is preferable to use low gradations such as 4 gradations and 8 gradations because a large number of intermediate gradation levels are not required. When the pixels of the image are fine, one dot is small, and the luminance is low, it is preferable to use multi-gradations such as 16 gradations and 32 gradations because a large number of intermediate gradation levels are used.

[0089] The light emission processing unit 42 of the spatial display device 10 in Use Example 3 sets the light emission color included in the light emission pattern information of the dots arranged in the slanted line portion and the curved line portion included in the image displayed in the air and constituting the boundary with the background of the image to be the intermediate color between the color of the background and the color of the image displayed in the air. Referring to FIG. 10(a), the gradation font will be described by taking the Chinese character "大" as an example. As shown in FIG. 10(a), the black Chinese character "大" displayed on a white background of 16 dots in the vertical direction and 16 dots in the horizontal direction has 80 black dots constituting the inside, and the dots constituting the outline are dark gray dots 81 or light gray dots 82. Among the dots constituting the outline of the Chinese character "大", the dots that are in contact with more black dots become dark gray dots 81, and the other dots become light gray dots 82.

[0090] When the Chinese character "大" is displayed in the air using the gradation font, as shown in FIG. 10(b), 85 unmanned aerial vehicles 11 are required. The black dots 80 shown in FIG. 10(a) correspond to the bright dots 84 in FIG. 10(b), and 39 unmanned aerial vehicles 11 are assigned to the bright dots 84 and emit light with the largest light emission amount. The dark gray dots 81 shown in FIG. 10(a) correspond to the slightly dark dots 85 in FIG. 10(b), and 11 unmanned aerial vehicles 11 are assigned to the slightly dark dots 85 and emit light with a medium light emission amount. The light gray dots 82 shown in FIG. 10(a) correspond to the dark dots 86 in FIG. 10(b), and the 35 unmanned aerial vehicles 11 assigned to the dark dots 86 emit a small amount of light.

[0091] (Spatial display method and spatial display program) Next, a spatial display program according to an embodiment of the present invention will be described together with a spatial display method with reference to Fig. 11. Fig. 11 is an example of a flowchart of the spatial display program according to this embodiment. The spatial display method is executed by the processing unit 10e of the spatial display device 10 based on the spatial display program. The spatial display program includes a learning model acquisition step S37, an image data acquisition step S38, a dot number calculation step S39, a position information generation step S40, a flight control step S41, a light emission processing step S42, an unmanned aerial vehicle allocation step S43, an abnormality information reception step S44, a light insufficiency information reception step S45, an image allocation step S46, a transmission step S47, and an aircraft adequacy determination step S48. The spatial display program causes the processing unit 10e of the spatial display device 10 to realize a learning model acquisition function, an image data acquisition function, a dot number calculation function, a position information generation function, a flight control function, a light emission processing function, an unmanned aerial vehicle allocation function, an abnormality information reception function, a light amount deficiency information reception function, an image allocation function, a transmission function, and a number of aircraft sufficiency determination function. These functions are executed in the order shown in the flowchart of FIG. 11, but the order can be changed as appropriate. Note that each function overlaps with the description of the various functional units of the spatial display device 10 described above, and therefore detailed description thereof will be omitted.

[0092] The learning model acquisition function acquires a learning model that has previously learned the correspondence between image data and original image data relating to an image (S37: learning model acquisition step). The image data acquisition function acquires image data relating to the image (S38: image data acquisition step). The dot number calculation function calculates, based on the image data, the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix (S39: dot number calculation step). The position information generating function generates position information for each of the dots that are in a light-emitting state based on the image data (S40: position information generating step). The flight control function generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that constitutes the video portion, based on video data regarding the video portion of the image included in the image data (S41: flight control step). The light emission processing function generates light emission pattern information that defines the light emission mode for each dot based on color data relating to the color of the image contained in the image data (S42: light emission processing step). The unmanned aerial vehicle allocation function allocates an unmanned aerial vehicle 11 to each dot (S43: unmanned aerial vehicle allocation step). The abnormality information receiving function receives abnormality information of the unmanned aerial vehicle 11 (S44: abnormality information receiving step). The insufficient light information receiving function receives insufficient light information from the unmanned aerial vehicle 11 (S45: insufficient light information receiving step). The image allocation function allocates the image to one of a plurality of groups (S46: image allocation step). The transmission function transmits position information and light emission pattern information corresponding to the dot to each of the unmanned aerial vehicles 11 assigned to each dot (S47: transmission step). The number of aircraft sufficiency determination function determines whether the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation function (S48: number of aircraft sufficiency determination step).

[0093] According to the spatial display device 10 of the above embodiment, the dot number calculation unit 39 does not include in the number of dots that are not lit when the image to be displayed in the air is displayed in a dot matrix. Therefore, the unmanned aerial vehicle 11 is not assigned to the dots that are not lit when the image to be displayed in the air is displayed in a dot matrix, and the unmanned aerial vehicle 11 that does not contribute to the display of the image is not used, so that the unmanned aerial vehicle 11 can be operated efficiently.

[0094] In addition, the spatial display device 10 of the above-mentioned embodiment distributes multiple unmanned aerial vehicles 11 into multiple groups, and if there are an insufficient number of unmanned aerial vehicles 11 belonging to a group to which an image to be displayed in the air is assigned, unmanned aerial vehicles 11 belonging to other groups are assigned, thereby allowing the operation of the unmanned aerial vehicles 11 to be flexibly and efficiently performed.

[0095] Furthermore, in the above-described embodiment of the spatial display device 10, even if there are an insufficient number of unmanned aerial vehicles 11 belonging to the group to which the image to be displayed in the air is assigned, and an unmanned aerial vehicle 11 belonging to another group cannot be assigned, the spatial display device 10 can deal with the situation by reducing the number of dots in the dot matrix display of the image to be displayed in the air.

[0096] In addition, according to the above-mentioned embodiment of the spatial display device 10, position information, light emission pattern information, and flight pattern information are transmitted to the unmanned aerial vehicle 11 in flight, and the unmanned aerial vehicle 11 can receive the position information, light emission pattern information, and flight pattern information transmitted from the spatial display device 10 while in flight, so that the unmanned aerial vehicle 11 can easily display an image different from a predetermined image in the air even while in flight.

[0097] Furthermore, according to the above-described embodiment of the spatial display device 10, the unmanned aerial vehicle 11 is equipped with an illuminance sensor 32 and has a light emission brightness correction unit 26f, so that the unmanned aerial vehicle 11 can adjust the light emission brightness of the light-emitting unit 21 according to the external illuminance, thereby preventing the image of the air 13 from being difficult to see due to insufficient brightness, or preventing the light-emitting unit 21 from emitting light with excessive light emission brightness, and therefore preventing more consumption of the battery 27 than necessary.

[0098] Furthermore, according to the above-described embodiment of the spatial display device 10, two unmanned aerial vehicles 11 are assigned to emit light for dots that were lacking in light, making it possible to provide a sufficient amount of light for the brightness of the background of the image displayed in the air.

[0099] Furthermore, according to the spatial display device 10 of the above-described embodiment, a stereoscopic image can be displayed in the air 13.

[0100] Furthermore, according to the spatial display device 10 of the above-described embodiment, by displaying fireworks in the air 13 using multiple unmanned aerial vehicles 11, it is possible to reduce carbon dioxide emissions, unlike actual fireworks, and thus to put the concept of carbon neutrality into practice.

[0101] Furthermore, according to the spatial display device 10 of the above-mentioned embodiment, images acquired at the medical site can be input from the external information input unit 10i and the images can be displayed in the air 13, so that the images acquired at the medical site can be shown to many people.

[0102] Furthermore, according to the spatial display device 10 of the above-described embodiment, images captured by an ultra-high speed camera can be displayed in the air 13, so that an image capturing a moment of a high-speed phenomenon can be projected in the air, impressing many people.

[0103] Furthermore, according to the spatial display device 10 of the above-described embodiment, real-time images showing the current situation captured by a fixed observation camera can be displayed in the air 13, so that the real-time images can be shown to many people.

[0104] The present invention is not limited to the spatial display device 10, the spatial display method, and the spatial display program according to the above-described embodiments, and can be embodied in various other modified or applied examples without departing from the gist of the present invention described in the claims. In addition, although the term "data" is used in the above-described embodiments, the term "data" can be replaced with "information", and the term "information" can be replaced with "data". [Explanation of symbols]

[0105] 10 Spatial display device 10a Communications Department 10b ROM 10c RAM 10d storage section 10e Processing section 10f Input / Output Interface 10g display 10h Operation input section 10i External information input section 11 Unmanned aerial vehicles (drones) 11a Unmanned aerial vehicles (drones) 11b Unmanned aerial vehicles (drones) 11c Unmanned Aerial Vehicles (Drones) 11d Unmanned Aerial Vehicles (Drones) 11e Unmanned Aerial Vehicles (Drones) 12 Radio Antenna 13 Air 14 Marks 15 characters 16 QR Code 17 Information and Communication Networks 18 Main body 19 Arm 19a 1st Arm 19b 2nd Arm 19c 3rd Arm 19d 4th Arm 20 Propeller 20a No. 1 propeller 20b No. 2 propeller 20c 3rd propeller 20d 4th propeller 21 Light emitting part 22 Communications Department 23 ROM 24 RAM 25 Memory section 26 Control Unit 26a Acquisition Department 26b Anomaly detection section 26c Abnormality notification section 26d Light shortage detection unit 26e Insufficient light notification section 26f Luminance correction section 27 Battery 28 Input / Output Interface 29 Flight drive unit (motor) 30 Flight drive mechanism (rotor) 31 GPS receiver 32 Illuminance sensor 33 Geomagnetic Sensor 34 Altitude Sensor 35 Gyro sensor 36 Obstacle Detection Camera 37 Learning model acquisition unit 38 Image data acquisition section 39 Dot Count Calculation Unit 40 Location information generation section 41 Flight control unit 42 Light emission processing section 43 Unmanned Aerial Vehicle Allocation Division 44 Abnormality Information Reception Department 45. Light Deficiency Information Reception Unit 46 Image Allocation Section 47 Transmitter 48 Aircraft Sufficiency Determination Department 49 "R" 50 "R" 51 dots 52 dots 53 dots 54 dots 55 dots 56 dots 60 1st Layer 61 Second Layer 62 3rd Layer 63 4th Layer 64 5th Layer 65 6th Layer 66 7th Layer 69 No display 70 Cards drawn by customers 71 The Magician's Card 80 black dots 81 dark gray dots 82 Light Gray Dots 83 None 84 bright dots 85 slightly dark dot 86 dark dots

Claims

1. A spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups, causes the plurality of unmanned aerial vehicles to emit light and displays an image in the air, The processing unit of the spatial display device includes: an image data acquisition unit that acquires image data relating to the image; a dot number calculation unit that calculates the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the image data; a position information generating unit that generates position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing unit that generates light emission pattern information that defines a light emission mode for each of the dots based on color data related to a color of the image included in the image data; an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicle to each of the dots; an image allocation unit that allocates the image to one of the plurality of groups; A transmitter that transmits the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number-of-aircraft sufficiency determination unit that determines whether the number of the unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit; Equipped with When the aircraft number sufficiency determination unit determines that the number of the unmanned aerial vehicles is sufficient for the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is allocated, When the aircraft number sufficiency determination unit determines that the number of the unmanned aerial vehicles does not suffice the number of the dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, the unmanned aerial vehicles belonging to the group to which the image is allocated, and allocates the unmanned aerial vehicles belonging to other groups; When the number of aircraft sufficiency determination unit determines that the number of the unmanned aerial vehicles does not satisfy the number of the dots calculated by the dot number calculation unit, and the unmanned aerial vehicles belonging to another group cannot be assigned to the dots, The position information generation unit generates position information for each dot, the number of dots corresponding to the number of unmanned aerial vehicles belonging to the group to which the image is assigned, A spatial display device characterized in that the unmanned aerial vehicle allocation unit allocates the unmanned aerial vehicle belonging to the group to which the image is assigned to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

2. A spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups, causes the plurality of unmanned aerial vehicles to emit light and displays an image in the air, The processing unit of the spatial display device includes: an image data acquisition unit that acquires image data relating to the image; a dot number calculation unit that calculates the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the image data; a position information generating unit that generates position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing unit that generates light emission pattern information that defines a light emission mode for each of the dots based on color data related to a color of the image included in the image data; an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicle to each of the dots; an image allocation unit that allocates the image to one of the plurality of groups; A transmitter that transmits the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number-of-aircraft sufficiency determination unit that determines whether the number of the unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit; Equipped with When the aircraft number sufficiency determination unit determines that the number of the unmanned aerial vehicles is sufficient for the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is allocated, When the aircraft number sufficiency determination unit determines that the number of the unmanned aerial vehicles does not suffice the number of the dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, the unmanned aerial vehicles belonging to the group to which the image is allocated, and allocates the unmanned aerial vehicles belonging to other groups; There are other groups waiting in the air and other groups waiting on the ground, A spatial display device characterized in that the unmanned aerial vehicle allocation unit prioritizes allocation of unmanned aerial vehicles belonging to other groups that are waiting in the air over unmanned aerial vehicles belonging to other groups that are waiting on the ground.

3. the image includes a video; The processing unit of the spatial display device includes: A flight control unit that generates flight pattern information that specifies a flight route, a flight timing, and a flight speed for each of the dots constituting the moving image portion based on video data related to the moving image portion of the image included in the image data, The spatial display device described in claim 1, characterized in that the transmitting unit transmits the flight pattern information together with the position information and the illumination pattern information corresponding to the dots to each of the unmanned aerial vehicles assigned to each of the dots that constitute the video portion.

4. The image is represented by contours, The position information generating unit The spatial display device according to claim 1, characterized in that contour data regarding the contour is obtained based on the image data, and position information for each dot that will be illuminated when the contour is displayed in a dot matrix is ​​generated based on the contour data.

5. The image is a three-dimensional image composed of a plurality of layers, The position information generating unit The spatial display device according to claim 1, characterized in that, based on the image data, position information for each dot that will be in an illuminated state when the image divided into the multiple layers is displayed in a dot matrix is ​​generated.

6. an external information input unit that receives original image data that is the source of the image; a learning model acquisition unit that acquires a learning model that has previously learned a correspondence relationship between image data related to the image and the original image data; Equipped with The image data acquisition unit The spatial display device according to claim 1, characterized in that the image data is acquired by inputting the original image data accepted by the external information input unit into the learning model.

7. The spatial display device according to claim 1, characterized in that the light-emitting processing unit sets the light-emitting color included in the light-emitting pattern information of dots that are located in the diagonal and curved portions of the image and form a boundary with the background of the image to an intermediate color between the color of the background and the color of the image.

8. The unmanned aerial vehicle includes: The spatial display device according to claim 1 , further comprising an acquisition unit for acquiring the position information and the light emission pattern information transmitted from the transmission unit.

9. The unmanned aerial vehicle includes: The spatial display device according to claim 3 , further comprising an acquisition unit for acquiring flight pattern information transmitted from the transmission unit together with the position information and the light emission pattern information.

10. The unmanned aerial vehicle includes: an abnormality detection unit that detects an abnormality in the device itself; an abnormality notification unit that notifies the spatial display device of abnormality information regarding an abnormality of the own device; Equipped with The processing unit of the spatial display device includes: an abnormality information receiving unit that receives abnormality information of the unmanned aerial vehicle; the unmanned aerial vehicle allocation unit allocates another unmanned aerial vehicle belonging to the group to the dot to which the unmanned aerial vehicle that notified the abnormality information had been allocated, in place of the unmanned aerial vehicle; The spatial display device according to claim 8 or 9, characterized in that the transmission unit transmits a return command to the unmanned aerial vehicle that has notified the abnormality information to return.

11. The spatial display device according to claim 10 , wherein the abnormality includes a case where a remaining charge of a battery is equal to or lower than a threshold value.

12. The unmanned aerial vehicle includes: a light quantity deficiency detection unit that detects a deficiency in the quantity of light emitted by the own device; a light amount deficiency notification unit that notifies the spatial display device of light amount deficiency information regarding a deficiency in the amount of light emitted by the own device; Equipped with The processing unit of the spatial display device includes: a light quantity deficiency information receiving unit that receives light quantity deficiency information of the unmanned aerial vehicle; The spatial display device described in claim 8 or 9, characterized in that the unmanned aerial vehicle allocation unit allocates other unmanned aerial vehicles belonging to the group together with the unmanned aerial vehicle that notified the light insufficiency information to the dot to which the unmanned aerial vehicle was assigned.

13. The unmanned aerial vehicle includes: an illuminance sensor for measuring external illuminance; a light emission luminance correction unit that corrects the light emission luminance included in the light emission pattern information based on a measurement result of the illuminance sensor; 10. The spatial display device according to claim 8 or 9, comprising:

14. A spatial display method used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and displays an image in the air by illuminating the plurality of unmanned aerial vehicles, The processing unit of the spatial display device includes: an image data acquisition step of acquiring image data relating to the image; a dot number calculation step of calculating the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the image data; a position information generating step of generating position information for each of the dots that are in a light-emitting state based on the image data; a light emission processing step of generating light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; an unmanned aerial vehicle allocation step of allocating the unmanned aerial vehicle to each of the dots; an image allocation step of allocating the image to one of the plurality of groups; A transmitting step of transmitting the position information and the light emission pattern information corresponding to the dot to each of the unmanned aerial vehicles assigned to each of the dots; a number-of-unmanned aerial vehicles sufficiency determination step of determining whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated in the dot number calculation step; Run When it is determined in the aircraft number sufficiency determination step that the number of the unmanned aerial vehicles is sufficient for the number of dots calculated in the dot number calculation step, the unmanned aerial vehicle allocation step allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned; If it is determined in the aircraft number sufficiency determination step that the number of unmanned aerial vehicles does not satisfy the number of dots calculated in the dot number calculation step, the unmanned aerial vehicle allocation step allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned, and allocates the unmanned aerial vehicle belonging to another group; When it is determined in the aircraft number sufficiency determination step that the number of the unmanned aerial vehicles does not satisfy the number of the dots calculated in the dot number calculation step, and the unmanned aerial vehicles belonging to another group cannot be assigned to the dots, In the position information generating step, position information for each dot is generated, the number of dots corresponding to the number of unmanned aerial vehicles belonging to the group to which the image is assigned; A spatial display method characterized in that, in the unmanned aerial vehicle allocation step, an unmanned aerial vehicle belonging to the group to which the image is assigned is assigned to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

15. A spatial display method used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and displays an image in the air by illuminating the plurality of unmanned aerial vehicles, The processing unit of the spatial display device includes: an image data acquisition step of acquiring image data relating to the image; a dot number calculation step of calculating the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the image data; a position information generating step of generating position information for each of the dots that are in a light-emitting state based on the image data; a light emission processing step of generating light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; an unmanned aerial vehicle allocation step of allocating the unmanned aerial vehicle to each of the dots; an image allocation step of allocating the image to one of the plurality of groups; A transmitting step of transmitting the position information and the light emission pattern information corresponding to the dot to each of the unmanned aerial vehicles assigned to each of the dots; a number-of-unmanned aerial vehicles sufficiency determination step of determining whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated in the dot number calculation step; Run When it is determined in the aircraft number sufficiency determination step that the number of the unmanned aerial vehicles is sufficient for the number of dots calculated in the dot number calculation step, the unmanned aerial vehicle allocation step allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned; If it is determined in the aircraft number sufficiency determination step that the number of unmanned aerial vehicles does not satisfy the number of dots calculated in the dot number calculation step, the unmanned aerial vehicle allocation step allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned, and allocates the unmanned aerial vehicle belonging to another group; There are other groups waiting in the air and other groups waiting on the ground, A spatial display method characterized in that, in the unmanned aerial vehicle allocation step, unmanned aerial vehicles belonging to other groups that are waiting in the air are allocated preferentially over unmanned aerial vehicles belonging to other groups that are waiting on the ground.

16. A spatial display program for use in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, The processing unit of the spatial display device includes: an image data acquisition function for acquiring image data relating to the image; a dot number calculation function for calculating the number of dots that will be lit when the image is displayed in a dot matrix based on the image data; a position information generating function for generating position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing function for generating light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; An unmanned aerial vehicle allocation function for allocating the unmanned aerial vehicle to each of the dots; an image allocation function for allocating the image to one of the plurality of groups; A transmission function of transmitting the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number of unmanned aerial vehicles sufficiency determination function that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation function; Realize this, When the number of unmanned aerial vehicles sufficiency determination function determines that the number of unmanned aerial vehicles suffices the number of dots calculated by the dot number calculation function, the unmanned aerial vehicle allocation function allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is assigned; When it is determined in the aircraft number sufficiency determination function that the number of the unmanned aerial vehicles does not suffice the number of the dots calculated in the dot number calculation function, the unmanned aerial vehicle allocation function allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned, and allocates the unmanned aerial vehicles belonging to other groups; When the number of unmanned aerial vehicles is determined not to be sufficient for the number of dots calculated by the dot number calculation function in the aircraft number sufficiency determination function, and when the unmanned aerial vehicles belonging to another group cannot be assigned to the dots, In the position information generating function, position information for each dot is generated, the number of dots being in accordance with the number of the unmanned aerial vehicles belonging to the group to which the image is assigned; A spatial display program characterized in that, in the unmanned aerial vehicle allocation function, an unmanned aerial vehicle belonging to the group to which the image is assigned is assigned to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

17. A spatial display program for use in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, The processing unit of the spatial display device includes: an image data acquisition function for acquiring image data relating to the image; a dot number calculation function for calculating the number of dots that will be lit when the image is displayed in a dot matrix based on the image data; a position information generating function for generating position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing function for generating light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; An unmanned aerial vehicle allocation function for allocating the unmanned aerial vehicle to each of the dots; an image allocation function for allocating the image to one of the plurality of groups; A transmission function of transmitting the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number of unmanned aerial vehicles sufficiency determination function that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation function; Realize this, When the number of unmanned aerial vehicles sufficiency determination function determines that the number of unmanned aerial vehicles suffices the number of dots calculated by the dot number calculation function, the unmanned aerial vehicle allocation function allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is assigned; When it is determined in the aircraft number sufficiency determination function that the number of the unmanned aerial vehicles does not suffice the number of the dots calculated in the dot number calculation function, the unmanned aerial vehicle allocation function allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned, and allocates the unmanned aerial vehicles belonging to other groups; There are other groups waiting in the air and other groups waiting on the ground, A spatial display program characterized in that, in the unmanned aerial vehicle allocation function, unmanned aerial vehicles belonging to other groups that are waiting in the air are allocated preferentially over unmanned aerial vehicles belonging to other groups that are waiting on the ground.

Images