Spatial display device, spatial display method, and spatial display program

Abstract

To provide a spatial display device and the like that can efficiently operate a plurality of unmanned aerial vehicles.SOLUTION: A spatial display device calculates, based on image data relating to an image to be displayed in the air, the number of dots that are to be in a light-emitting state when the image is displayed in a dot matrix manner, generates position information for each dot, generates light-emission pattern information for each dot based on color data, assigns an unmanned aerial vehicle to each dot, assigns the image to any one of multiple groups, and transmits the position information and the light-emission pattern information to each of the unmanned aerial vehicles. When it is determined that the number of unmanned aerial vehicles satisfies the number of dots, the spatial display device assigns, to each dot, an unmanned aerial vehicle belonging to a group to which the image is assigned. When it is determined that the number of unmanned aerial vehicles does not satisfy the number of dots, the spatial display device assigns, to each dot, unmanned aerial vehicles of the group to which the image is assigned, and also unmanned aerial vehicles of other groups.SELECTED DRAWING: Figure 5

Description

[Technical Field]

[0001] The present invention relates to a spatial display device, a spatial display method, and a spatial display program, and in particular to a spatial display device, a spatial display method, and a spatial display program that are characterized by illuminating multiple unmanned aerial vehicles to display images in the air. [Background technology]

[0002] In recent years, the use of unmanned aerial vehicles (UAVs), also known as drones, small helicopters, multicopters, etc., has been expanding. For example, the use of UAVs in amusement parks has been proposed for special effects to entertain guests participating in attractions such as rides or shows (see Patent Document 1).

[0003] In addition, the use of spatial display devices, spatial display methods, and spatial display programs that display images in the air using multiple UAVs is also beginning to spread. Such spatial display devices are used, for example, in drone shows held all over the country. The UAVs in the spatial display devices used in drone shows are equipped with light-emitting devices such as LEDs, and multiple UAVs fly in formation in coordination with each other, creating beautiful illuminations, mainly in the night sky or on a stage, to entertain audiences.

[0004] Generally, UAVs used in spatial display devices such as those described above are preprogrammed to fly in a specific trajectory in synchronization with one another and to emit light at specific times. Once launched, these UAVs receive GPS signals to identify their current location and fly autonomously according to preprogrammed controls.

[0005] In spatial display devices using multiple UAVs, there is a need to efficiently display images in the air. In particular, as the number of unmanned aerial vehicles used increases, there is a growing need to efficiently operate multiple unmanned aerial vehicles. [Prior art documents] [Patent documents]

[0006] [Patent Document 1] Special Publication No. 2017-526443 Summary of the Invention [Problem to be solved by the invention]

[0007] Therefore, an object of the present invention is to provide a spatial display device, a spatial display method, and a spatial display program that can efficiently operate multiple unmanned aerial vehicles. [Means for solving the problem]

[0008] That is, the spatial display device of the first aspect is a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, and the processing unit of the spatial display device includes an image data acquisition unit that acquires image data related to the image, a dot number calculation unit that calculates the number of dots that will be in an emitting state when the image is displayed in a dot matrix based on the image data, a position information generation unit that generates position information for each of the number of dots that will be in an emitting state based on the image data, a light emission processing unit that generates light emission pattern information that specifies the light emission mode for each dot based on color data related to the color of the image included in the image data, an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicle to each dot, an image allocation unit that allocates the image to one of the plurality of groups, and a dot The image capturing apparatus is provided with a transmitting unit that transmits position information and light emission pattern information corresponding to the dots to each unmanned aerial vehicle assigned to each dot, and a number sufficiency determination unit that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit, and if the number sufficiency determination unit determines that the number of unmanned aerial vehicles satisfies the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned to each dot, and if the number sufficiency determination unit determines that the number of unmanned aerial vehicles does not satisfy the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned and unmanned aerial vehicles belonging to other groups to each dot.

[0009] In a second aspect, in the spatial display device of the first aspect, if the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, and an unmanned aerial vehicle belonging to another group cannot be assigned to a dot, the position information generation unit may generate position information for each dot, the number of dots corresponding to the number of unmanned aerial vehicles belonging to the group to which the image is assigned, and the unmanned aerial vehicle assignment unit may assign an unmanned aerial vehicle belonging to the group to which the image is assigned, to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

[0010] In a third aspect, in the spatial display device of the first aspect, the image includes a moving image, and the processing unit of the spatial display device includes a flight control unit that generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that constitutes the moving image portion based on moving image data related to the moving image portion of the image included in the image data, and the transmitting unit may transmit the flight pattern information, along with position information and light emission pattern information corresponding to the dot, to each of the unmanned aerial vehicles assigned to each dot that constitutes the moving image portion.

[0011] In a fourth aspect, in the spatial display device according to the first aspect, an image may be displayed by contours, and the position information generation unit may acquire contour data relating to the contours based on the image data, and generate position information for each dot that will be illuminated when the contours are displayed in a dot matrix based on the contour data.

[0012] In a fifth aspect, in the spatial display device according to the first aspect, the image is a three-dimensional image composed of multiple layers, and the position information generation unit may generate, based on image data, position information for each dot that is illuminated when the image divided into multiple layers is displayed in a dot matrix.

[0013] A sixth aspect is a spatial display device according to the first aspect, which is provided with an external information input unit that accepts original image data that is the basis of an image, and a learning model acquisition unit that acquires a learning model that has previously learned the correspondence between image data related to the image and the original image data, and the image data acquisition unit may acquire the image data by inputting the original image data accepted by the external information input unit into the learning model.

[0014] In a seventh aspect, in the spatial display device of the first aspect, there are other groups waiting in the air and other groups waiting on the ground, and the unmanned aircraft allocation unit may allocate unmanned aircraft belonging to the other groups waiting in the air preferentially over the other groups waiting on the ground.

[0015] In an eighth aspect, in the spatial display device of the first aspect, the light-emitting processing unit may set the light-emitting color included in the light-emitting pattern information of dots that are arranged in the diagonal and curved portions of the image and form the boundary with the background of the image to an intermediate color between the color of the background and the color of the image.

[0016] In a ninth aspect, in the spatial display device according to the first aspect, the unmanned aerial vehicle may be provided with an acquisition unit that acquires the position information and light emission pattern information transmitted from the transmission unit.

[0017] In a tenth aspect, in the spatial display device of the third aspect, the unmanned aerial vehicle may be provided with an acquisition unit that acquires flight pattern information transmitted from the transmission unit along with position information and light emission pattern information.

[0018] In an eleventh aspect, in a spatial display device according to the ninth or tenth aspect, the unmanned aerial vehicle is provided with an abnormality detection unit that detects an abnormality in the vehicle itself and an abnormality notification unit that notifies the spatial display device of abnormality information related to the abnormality in the vehicle itself, the processing unit of the spatial display device is provided with an abnormality information receiving unit that receives the abnormality information of the unmanned aerial vehicle, the unmanned aerial vehicle allocation unit allocates an unmanned aerial vehicle belonging to another group in place of the unmanned aerial vehicle that notified the abnormality information to the dot to which the unmanned aerial vehicle had been assigned, and the transmission unit transmits a return command to the unmanned aerial vehicle that notified the abnormality information to return.

[0019] According to a twelfth aspect, in the spatial display device according to the eleventh aspect, the abnormality may include a case where the remaining charge of the battery is equal to or less than a threshold value.

[0020] In a thirteenth aspect, in a spatial display device according to the ninth or tenth aspect, the unmanned aerial vehicle is provided with a light deficiency detection unit that detects insufficient light intensity of the aircraft's own light emission, and a light deficiency notification unit that notifies the spatial display device of light deficiency information regarding the insufficient light intensity of the aircraft's own light emission, and the processing unit of the spatial display device is provided with a light deficiency information receiving unit that receives the light deficiency information of the unmanned aerial vehicle, and the unmanned aerial vehicle allocation unit may allocate an unmanned aerial vehicle that belongs to another group together with the unmanned aerial vehicle to a dot to which the unmanned aerial vehicle that notified the light deficiency information had been assigned.

[0021] In a fourteenth aspect, in the spatial display device of the ninth or tenth aspect, the unmanned aerial vehicle may be provided with an illuminance sensor that measures external illuminance, and an emission brightness correction unit that corrects the emission brightness included in the emission pattern information based on the measurement result of the illuminance sensor.

[0022] A spatial display method according to a fifteenth aspect is a spatial display method used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, wherein the processing unit of the spatial display device includes an image data acquisition step that acquires image data related to the image, a dot number calculation step that calculates, based on the image data, the number of dots that will be in an emitting state when the image is displayed in a dot matrix, a position information generation step that generates, based on the image data, position information for each of the number of dots that will be in an emitting state, a light emission processing step that generates light emission pattern information that specifies the light emission mode for each dot, based on color data related to the color of the image included in the image data, a unmanned aerial vehicle allocation step that allocates the unmanned aerial vehicle to each dot, an image allocation step that allocates the image to one of a plurality of groups, and an image allocation step that allocates each dot to one of the plurality of groups. The method includes a transmission step of transmitting position information and light emission pattern information corresponding to the dots to each of the assigned unmanned aerial vehicles, and a number sufficiency determination step of determining whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated in the dot number calculation step, and if it is determined in the number sufficiency determination step that the number of unmanned aerial vehicles satisfies the number of dots calculated in the dot number calculation step, then in the unmanned aerial vehicle allocation step, an unmanned aerial vehicle belonging to the group to which the image is assigned is allocated for each dot, and if it is determined in the number sufficiency determination step that the number of unmanned aerial vehicles does not satisfy the number of dots calculated in the dot number calculation step, then in the unmanned aerial vehicle allocation step, an unmanned aerial vehicle belonging to the group to which the image is assigned and an unmanned aerial vehicle belonging to another group are allocated for each dot.

[0023] A spatial display program according to a sixteenth aspect is a spatial display program used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, and the processing unit of the spatial display device has an image data acquisition function that acquires image data related to the image, a dot number calculation function that calculates the number of dots that will be in an illuminating state when the image is displayed in a dot matrix based on the image data, a position information generation function that generates position information for each of the number of dots that will be in an illuminating state based on the image data, a light emission processing function that generates light emission pattern information that specifies the light emission mode for each dot based on color data related to the color of the image included in the image data, an unmanned aerial vehicle allocation function that allocates the unmanned aerial vehicle to each dot, an image allocation function that allocates the image to one of a plurality of groups, and an image allocation function that allocates the image to one of a plurality of groups based on the position information for each dot. and a number sufficiency determination function that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation function. If the number sufficiency determination function determines that the number of unmanned aerial vehicles satisfies the number of dots calculated by the dot number calculation function, the unmanned aerial vehicle allocation function allocates unmanned aerial vehicles belonging to the group to which the image is assigned for each dot, and if the number sufficiency determination function determines that the number of unmanned aerial vehicles does not satisfy the number of dots calculated by the dot number calculation function, the unmanned aerial vehicle allocation function allocates unmanned aerial vehicles belonging to the group to which the image is assigned and unmanned aerial vehicles belonging to other groups for each dot. [Effects of the Invention]

[0024] The spatial display device according to the present invention is a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, and the processing unit of the spatial display device includes an image data acquisition unit that acquires image data related to the image, a dot number calculation unit that calculates the number of dots that will be in an emitting state when the image is displayed in a dot matrix based on the image data, a position information generation unit that generates position information for each of the number of dots that will be in an emitting state based on the image data, a light emission processing unit that generates light emission pattern information that specifies the emitting state of each dot based on color data related to the color of the image included in the image data, an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicles to each dot, an image allocation unit that allocates the image to one of the plurality of groups, and an image allocation unit that allocates the unmanned aerial vehicles allocated to each dot. The system is equipped with a transmitting unit that transmits position information and light emission pattern information corresponding to the dots to each unmanned aerial vehicle, and a number sufficiency determination unit that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit.If the number sufficiency determination unit determines that the number of unmanned aerial vehicles satisfies the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned for each dot.If the number sufficiency determination unit determines that the number of unmanned aerial vehicles does not satisfy the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to the group to which the image is assigned and unmanned aerial vehicles belonging to other groups for each dot.This allows for efficient operation of multiple unmanned aerial vehicles. Furthermore, the spatial display method and the spatial display program, like the spatial display device, can efficiently operate multiple unmanned aerial vehicles. [Brief explanation of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram for explaining an outline of the spatial display device of this embodiment. [Figure 2]FIG. 2 is a diagram showing an example of the appearance of an unmanned aerial vehicle used in the spatial display device of this embodiment. [Figure 3] FIG. 3 is a block diagram for explaining an example of the configuration of an unmanned aerial vehicle used in the spatial display device of this embodiment. [Figure 4] FIG. 4 is a block diagram for explaining an example of the hardware configuration of the spatial display device of this embodiment. [Figure 5] FIG. 5 is a block diagram for explaining an example of the functional configuration of the spatial display device of this embodiment. [Figure 6] FIG. 6 is a diagram for explaining a dot matrix display used in the spatial display device of this embodiment. [Figure 7] FIG. 7 is a diagram for explaining the display of a stereoscopic image by a dot matrix display used in the spatial display device of this embodiment. [Figure 8] FIG. 8 is a diagram for explaining a first example of use of the spatial display device of this embodiment. [Figure 9] FIG. 9 is a diagram for explaining a second example of use of the spatial display device of this embodiment. [Figure 10] FIG. 10 is a diagram for explaining a third example of use of the spatial display device of this embodiment. [Figure 11] FIG. 11 is an example of a flowchart of the spatial display program according to this embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0026] (Overview of the spatial display device 10) 1 to 10, one embodiment of a spatial display device 10 according to the present disclosure will be described. First, with reference to FIG. 1, an overview of a spatial display device 10 of this embodiment will be described. The spatial display device 10 of this embodiment divides a plurality of unmanned aerial vehicles 11 into a plurality of groups, and displays an image in the air 13 by illuminating the plurality of unmanned aerial vehicles 11 (11a, 11b, 11c, 11d, 11e, ...). The spatial display device 10 may, for example, divide 1,000 unmanned aerial vehicles 11 into five groups with 200 evenly distributed to each group, or may divide multiple unmanned aerial vehicles 11 into two groups: one group for primary use and one group for secondary use. The group to be primarily used refers to a group to which the unmanned aerial vehicle 11 that the spatial display device 10 uses with priority over other groups belongs. The auxiliary group is a group that assists the primary group by, for example, supplying unmanned aerial vehicles 11 to make up for any shortages in the number of unmanned aerial vehicles 11 that the primary group does not have, and by providing substitute unmanned aerial vehicles 11 in the event that an unmanned aerial vehicle 11 belonging to the primary group becomes unusable due to a malfunction or other reason. Furthermore, the spatial display device 10 may not only distribute the unmanned aerial vehicles 11 evenly among a plurality of groups, but may also reduce or increase the number of unmanned aerial vehicles 11 belonging to a particular group. The spatial display device 10 is an information processing device such as a personal computer (hereinafter referred to as a PC), a notebook PC, a tablet PC, a smartphone, or a server, and is connected to an information communication network 17 (see FIG. 4). The spatial display device 10 can be realized using a general-purpose information processing device, but may also be an information processing device specialized for illuminating a plurality of unmanned aerial vehicles 11 and displaying images in the air 13. The image may be a still image or a video, or may be text, a drawing, a mark, a code, etc. The image may be, for example, a QR code (registered trademark) 14, the Japanese alphabet 15, or Kanji characters 16 written in a calligraphy style. Specifically, images can be text, pictures, 3D video, animations, maps, buildings, large designs, etc., such as event or company logos, advertisements for companies, products, or services, or QR codes. Videos can be animations that add movement to still images of animals, characters, robots, machines, etc., displayed in a dot matrix format.

[0027] An animation is an image with movement, and is created based on the original image data described below. Based on the animation, the flight pattern of the unmanned aerial vehicle 11 and the light emission pattern of the light-emitting unit 21 installed on the unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display of the animation are determined. Images are represented by dot-matrix displays. A dot-matrix display is a form of visual representation using a two-dimensional array of dots, and is used to represent characters, symbols, and images. In a dot-matrix display, lines and curves are represented as a series of dots. The spatial display device 10 displays a predetermined image in the air by arranging unmanned aerial vehicles 11 that emit light at each point (dot) that constitutes a dot matrix display. Spatial display device 10 can express, for example, fireworks using animation. Specifically, an unmanned aerial vehicle 11 is assigned to each particle of light in the fireworks, and multiple unmanned aerial vehicles 11 fly along their respective predetermined routes, causing light-emitting units 21 of unmanned aerial vehicles 11 to express the trajectories of the particles of light in the fireworks.

[0028] The spatial display device 10 may be connected to a wireless antenna 12 and may perform two-way wireless communication with the unmanned aerial vehicle 11 via the wireless antenna 12. The spatial display device 10 may also exchange data with the unmanned aerial vehicle 11 via a recording medium such as an SD card or a USB memory. An unmanned aerial vehicle (UAV) 11, also known as a drone, small helicopter, or multicopter, is an unmanned aircraft that flies by wireless remote control or autonomous control. Specifically, the unmanned aerial vehicle 11 is a small quadcopter (an aircraft equipped with four motors), but the number of motors is not limited to this and may be an aircraft equipped with six, eight, etc. Each unmanned aerial vehicle 11 is provided with a light emitting unit 21 (see FIG. 2) such as an LED, and displays an image in the air by illuminating light emitting unit 21 at appropriate times. 1 shows five unmanned aerial vehicles 11 (11a, 11b, 11c, 11d, and 11e), the number is not limited to five and may be four or less, or six or more. Spatial display device 10 uses, for example, several tens to several thousand unmanned aerial vehicles 11.

[0029] (Configuration of Unmanned Aerial Vehicle 11) Next, the configuration of the unmanned aerial vehicle 11 will be described with reference to Figures 2 and 3. Figure 2 is a diagram showing an example of the appearance of the unmanned aerial vehicle 11 used in the spatial display device 10, and Figure 3 is a block diagram for explaining an example of the configuration of the unmanned aerial vehicle 11 used in the spatial display device 10. As shown in Figure 2, the unmanned aerial vehicle 11 has a main body 18 in the center and four arms 19 (first arm 19a, second arm 19b, third arm 19c, and fourth arm 19d) extending from the main body 18 in all directions. Propellers 20 are provided at the tips of the four arms 19. The propellers 20 are rotationally driven by a flight drive mechanism 30 (see FIG. 3) comprised of a motor or the like. Specifically, a first propeller 20a is provided at the tip of the first arm 19a, a second propeller 20b is provided at the tip of the second arm 19b, a third propeller 20c is provided at the tip of the third arm 19c, and a fourth propeller 20d is provided at the tip of the fourth arm 19d. The propeller 20 shown in FIG. 2 is a two-blade propeller, but may be a three-blade propeller or a four-blade propeller.

[0030] As shown in FIG. 3, unmanned aerial vehicle 11 includes a hardware configuration including communication unit 22, ROM 23, RAM 24, storage unit 25, control unit 26, battery 27, input / output interface 28, and the like. The unmanned aerial vehicle 11 also has hardware components such as a flight drive unit 29, a flight drive mechanism 30, a GPS receiver 31, an illuminance sensor 32, a geomagnetic sensor 33, an altitude sensor 34, a gyro sensor 35, an obstacle detection camera 36, ​​and a light-emitting unit (LED) 21, which are connected to the control unit 26 via an input / output interface 28 to enable two-way data communication.

[0031] The communication unit 22 is mainly used for wireless communication between the spatial display device 10 and other unmanned aerial vehicles 11. The communication method of the communication unit 22 is not particularly limited, and may be Wi-Fi (registered trademark), LoRa (registered trademark), Bluetooth (registered trademark), Zigbee (registered trademark), infrared wireless communication, microwave wireless, broadcast wireless, satellite communication, etc. The communication unit 22 may perform wireless communication between the spatial display device 10 and other unmanned aerial vehicles 11 via the wireless antenna 12.

[0032] The ROM (Read Only Memory) 23 can be used as a recording device, and stores firmware, various applications, and various data used for controlling the operation of each functional part of the unmanned aerial vehicle 11. The RAM (Random Access Memory) 24 is used to configure the main memory accessed by the control unit 26, and may also be used to temporarily store data obtained from various sensors such as the GPS receiver 31 and illuminance sensor 32 installed on the unmanned aerial vehicle 11. The memory unit 25 is a storage medium such as an SD card or USB memory, or a storage device such as an SSD (Solid State Drive), and stores the light emission pattern information and flight pattern information described below. The memory unit 25 may also be used to temporarily store data acquired from various sensors such as the GPS receiver 31 and the illuminance sensor 32, instead of the RAM 24. The memory unit 25 may not be mounted on the unmanned aerial vehicle 11, and the RAM 24 may perform the functions of the memory unit 25.

[0033] The control unit 26 is mainly used to execute the firmware of the unmanned aerial vehicle 11, and includes a central processing unit (CPU), MPU (Micro Processing Unit), GPU (Graphics Processing Unit), etc., and is realized by logic circuits (hardware) formed by integrated circuits (IC (Integrated Circuit) chips, LSI (Large Scale Integration)), etc., or dedicated circuits. Battery 27 serves as the power source for unmanned aerial vehicle 11 and supplies power to various functional units. For example, a lithium polymer battery is used as battery 27. Battery 27 is used with an overcharge protection function to protect it from overcharging. Furthermore, the remaining battery charge of battery 27 is monitored by a battery remaining charge check unit (not shown), and if the remaining battery charge falls below a predetermined value, the user is notified of the low remaining battery charge.

[0034] The input / output interface 28 is an interface for sending and receiving data to and from the flight drive unit 29, flight drive mechanism 30, GPS receiver 31, illuminance sensor 32, geomagnetic sensor 33, altitude sensor 34, gyro sensor 35, obstacle detection camera 36, ​​and light-emitting unit (LED) 21, etc. The input / output interface 28 may use different standards depending on the data being handled, and may be compatible with multiple standards, such as USB 2.0, USB 3.0, RS-232C, IEEE 1394, SCSI, and SASI. The flight drive unit 29 controls the drive of the flight drive mechanism 30 based on the flight drive signal generated by the control unit 26. The flight drive mechanism 30 is a mechanism that is composed of a motor and the like and that rotates the propeller 20.

[0035] The GPS (Global Positioning System) receiver 31 acquires location information of the latitude, longitude, and altitude of the current location of the unmanned aerial vehicle 11. The acquired location information of the current location of the unmanned aerial vehicle 11 is stored in the RAM 24 or the memory unit 25. Note that, to obtain more accurate location information, the current location may be corrected using radio waves from mobile phone base stations or radio waves from access points of wireless communication (Wi-Fi (registered trademark), Bluetooth (registered trademark), etc.). Furthermore, the accuracy of the location information of the current location of the unmanned aerial vehicle 11 may be improved by using a quasi-zenith satellite system or an RTK-GPS (Real Time Kinematic - Global Positioning System). The Quasi-Zenith Satellite System (QZSS) is a satellite positioning system provided by Japan that provides location information similar to GPS, but with higher accuracy, especially in Japan. RTK-GPS improves the accuracy of location information by adding corrected location information from electronic reference points installed on the ground to the location information obtained from GPS. While the general position error of location information obtained from GPS is said to be on the order of several meters, the position error of location information obtained from RTK-GPS is said to be several centimeters. At drone shows, multiple unmanned aerial vehicles 11 fly in formation, so each unmanned aerial vehicle 11 needs to be positioned in the air with high precision, and RTK-GPS may be used.

[0036] The illuminance sensor 32 measures the illuminance outside the unmanned aerial vehicle 11. The illuminance sensor 32 is mounted on each unmanned aerial vehicle 11 and measures the external illuminance for each unmanned aerial vehicle 11. The measurement results obtained by the illuminance sensor 32 are used by the light emission luminance correction unit 26f, which will be described later. The light emission luminance correcting section 26f corrects the light emission luminance included in the light emission pattern information of the light emitting section 21 based on the measurement result of the illuminance sensor 32. The light emission brightness correction unit 26f corrects the light emission brightness of the light emitting unit 21 according to the external illuminance of each unmanned aerial vehicle 11, thereby making it possible to prevent a lack of brightness of the light emitting unit 21 when displaying an image. Since light emitting unit 21 consumes a large amount of power, by appropriately correcting the brightness of light emitting unit 21 using light emission brightness correcting unit 26f, it is possible to suppress adverse effects on the capacity of battery 27 and flight time.

[0037] The geomagnetic sensor 33 is used to detect the direction in which the unmanned aerial vehicle 11 is flying and moving. The altitude sensor 34 is used to detect the altitude at which the unmanned aerial vehicle 11 is currently located, and includes, for example, a barometric pressure sensor. The gyro sensor 35 detects changes in acceleration in the unmanned aerial vehicle 11, and is used to detect the direction of travel, speed, and attitude of the unmanned aerial vehicle 11. The obstacle detection camera 36 is used to detect objects that may hinder the flight and movement of the unmanned aerial vehicle 11, for example, to prevent collisions with other unmanned aerial vehicles 11 and to detect landing points for the unmanned aerial vehicle 11. The light emitting unit 21 is used to emit light from the unmanned aerial vehicle 11, and mainly uses an LED. The LED may be a single-color LED or a multi-color LED. A multi-color LED is one that produces a variety of colors by mixing light of three colors, RGB (red, green, blue), or four colors, RGBW (red, green, blue, white). Furthermore, the light-emitting unit 21 may emit laser light. The LED used in the light-emitting unit 21 is preferably lightweight, highly durable, highly efficient, and highly bright.

[0038] The control unit 26 loads the firmware stored in the ROM 23 into a main memory configured with the RAM 24 or the like. The control unit 26 accesses the main memory into which the firmware has been loaded and executes the firmware. By executing the firmware, the control unit 26 is provided with functional components such as an acquisition unit 26a, an abnormality detection unit 26b, an abnormality notification unit 26c, an insufficient light amount detection unit 26d, an insufficient light amount notification unit 26e, and an emission brightness correction unit 26f.

[0039] The acquisition unit 26a acquires the position information and the light emission pattern information transmitted from a transmission unit 47 of the spatial display device 10, which will be described later. Alternatively, the acquisition unit 26a acquires flight pattern information transmitted from the transmission unit 47 together with the position information and light emission pattern information. The position information refers to the position information of the unmanned aerial vehicle 11 for each dot constituting the matrix display of the image displayed in the air, and is expressed as (x, y, z) in a three-dimensional Cartesian coordinate system. In addition to the three-dimensional Cartesian coordinate system, the position information may also be expressed using a spherical coordinate system, a cylindrical coordinate system, or a polar coordinate system. The position information may also be expressed as a combination of latitude, longitude, and altitude. The position information exists uniquely for each dot constituting the matrix display of the image displayed in the air, and each dot is unique to the assigned unmanned aerial vehicle 11. The acquisition unit 26a receives and acquires the position information corresponding to the dot assigned to the unmanned aerial vehicle 11 equipped with the acquisition unit 26a. The light emission pattern information is control information that specifies the light emission mode for each dot that constitutes the matrix display of the image displayed in the air, and controls the light emission unit 21 of the unmanned aerial vehicle 11 that is assigned to each dot. The light emission mode refers to the light emission color, light emission brightness, and light emission timing of the light-emitting unit 21, and when an LED is used for the light-emitting unit 21, the light emission mode refers to the light emission color, light emission brightness, and light emission timing of the LED. The flight pattern information, which will be described in detail later, specifies the flight route, flight timing, and flight speed of the unmanned aerial vehicle 11 assigned to each dot in the dot matrix display that makes up the video portion of the image displayed by multiple unmanned aerial vehicles 11.

[0040] The abnormality detection unit 26b detects abnormalities in the device itself. The abnormality detection unit 26b detects the occurrence of an abnormality in each functional unit constituting the unmanned aerial vehicle 11 before and during flight. Each functional unit includes the hardware configuration of the unmanned aerial vehicle 11 described above and the functional configuration of the unmanned aerial vehicle 11 described above. The abnormality detection unit 26b detects an abnormality that has occurred in the hardware configuration and functional configuration of the unmanned aerial vehicle 11. The abnormality of the battery 27 includes a case where the remaining charge of the battery 27 is equal to or less than a threshold value. If the remaining charge of battery 27 is insufficient for the planned operating time of unmanned aerial vehicle 11, abnormality detection unit 26b determines that the remaining charge of battery 27 is below a threshold and detects an abnormality in battery 27.

[0041] The abnormality notification unit 26c notifies the spatial display device 10 of abnormality information relating to an abnormality in the device itself. When the abnormality detection unit 26b detects an abnormality in the unmanned aerial vehicle 11, which is the vehicle itself, the abnormality notification unit 26c immediately notifies the spatial display device 10 of the abnormality information regarding the abnormality via wireless communication through the communication unit 22, and the spatial display device 10 receives the abnormality information via the abnormality information receiving unit 44 described below. Abnormality information regarding an abnormality in the own aircraft includes identification information unique to the own aircraft, which is the unmanned aircraft 11, information such as which functional part of the unmanned aircraft 11 is experiencing the abnormality, and the extent of the abnormality. Abnormalities in the aircraft itself include, for example, malfunctions in the flight drive mechanism 30, flight drive unit 29, GPS receiver 31, obstacle detection camera 36, ​​and light-emitting unit 21. Furthermore, the abnormality of the own device may be, for example, malfunction of the illuminance sensor 32, the geomagnetic sensor 33, and the altitude sensor . Furthermore, an abnormality in the device itself may be, for example, a malfunction of any of the functional units (26a to 26f) of the control unit 26.

[0042] The light amount deficiency detection unit 26d detects a deficiency in the amount of light emitted by the own device. The light intensity deficiency detection unit 26d detects a lack of light intensity during emission of light from the light emitting unit 21 of the unmanned aerial vehicle 11, which is the vehicle itself. The light amount deficiency detection unit 26d may use the illuminance sensor 32 to detect a lack of light amount emitted by the device itself. An insufficient amount of light emitted by the unmanned aerial vehicle refers to an insufficient amount of light emitted by light-emitting unit 21 of unmanned aerial vehicle 11 relative to the illuminance of the background of the image displayed in the air. Even if light-emitting unit 21 of unmanned aerial vehicle 11 is operating normally, light-insufficiency detection unit 26d detects an insufficient amount of light emitted by the unmanned aerial vehicle when the amount of light emitted by light-emitting unit 21 is insufficient relative to the illuminance of the background of the image displayed in the air.

[0043] The light amount deficiency notification unit 26e notifies the spatial display device 10 of light amount deficiency information relating to the deficiency in the light amount of the light emitted by the device itself. When the light deficiency notification unit 26e detects a deficiency in the amount of light emitted by the light emitting unit 21 of the unmanned aerial vehicle 11, which is the vehicle itself, it notifies the spatial display device 10 of the light deficiency information regarding the deficiency via wireless communication via the communication unit 22, and the spatial display device 10 receives the light deficiency information via the light deficiency information receiving unit 45 described below. The light deficiency information regarding the insufficient light intensity of the unmanned aerial vehicle's own light includes information such as identification information unique to the unmanned aerial vehicle 11, the light intensity of the light-emitting unit 21 of the unmanned aerial vehicle 11, and the degree of light deficiency.

[0044] The light emission luminance correcting section 26f corrects the light emission luminance included in the light emission pattern information based on the measurement result of the illuminance sensor 32, as described above. The light emission luminance refers to the light emission luminance of the light-emitting unit 21, and in the case where an LED is used for the light-emitting unit 21, it refers to the light emission luminance of the LED.

[0045] (Hardware configuration of spatial display device 10) Next, an example of the hardware configuration of the spatial display device 10 will be described with reference to Fig. 4. Fig. 4 is a block diagram for explaining an example of the hardware configuration of the spatial display device 10. The spatial display device 10 includes a communication unit 10a, a ROM 10b, a RAM 10c, a storage unit 10d, a processing unit 10e, and an input / output interface 10f. Furthermore, the spatial display device 10 includes a display unit 10g, an operation input unit 10h, and an external information input unit 10i, which input and output data via an input / output interface 10f.

[0046] The communication unit 10a has a function of performing two-way communication between the unmanned aerial vehicle 11 and other information processing devices. When communication unit 10a performs two-way communication with unmanned aerial vehicle 11, communication unit 10a may perform the communication wirelessly via wireless antenna 12 (see FIG. 1) or via a wireless antenna (not shown) connected to information and communication network 17. The wireless communication method is not particularly limited, and may be Wi-Fi (registered trademark), LoRa (registered trademark), Bluetooth (registered trademark), Zigbee (registered trademark), infrared wireless communication, microwave wireless, broadcast wireless, satellite communication, or the like. The other information processing device may be another spatial display device 10 or another information processing device. When the communication unit 10a performs two-way communication with another information processing device, the communication unit 10a may perform the two-way communication via the information communication network 17, or may perform the two-way communication by directly connecting to the other information processing device. The communication unit 10a may use wired communication or wireless communication for communication with the other information processing device. An external information input unit 10i, which will be described later, may receive original image data from another information processing device via the communication unit 10a.

[0047] The ROM 10b can be used as a recording device, and stores a BIOS (Basic Input Output System) required for controlling the operation of each functional unit of the spatial display device 10, various data used by the BIOS, and the like. BIOS is a program that manages the basic input / output functions of the spatial display device 10 as an information processing device. It is the first program to run when the spatial display device 10 is turned on, controlling hardware such as the communication unit 10a, ROM 10b, RAM 10c, memory unit 10d, processing unit 10e, and input / output interface 10f, and preparing to start up the OS (Operating System).

[0048] The RAM 10c is used to configure the main memory accessed by the processing unit 10e, and is also used to temporarily store various data acquired or generated by the spatial display device 10 before storing it in the storage unit 10d.

[0049] The storage unit 10d is realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), online storage, etc., and stores the OS, a spatial display program (described later), other application software, various data used by these programs, etc. The storage unit 10d also stores various data acquired or generated by the spatial display device 10.

[0050] The processing unit 10e includes a CPU, an MPU, a GPU, and the like, and is realized by a logic circuit formed by an integrated circuit (IC chip, LSI), or a dedicated circuit. The input / output interface 10f is an interface for transmitting and receiving data to and from the display unit 10g, the operation input unit 10h, and the external information input unit 10i. The input / output interface 10f may use different standards depending on the data being handled, and may be compatible with multiple standards, such as HDMI (registered trademark), USB 2.0, USB 3.0, RS-232C, IEEE 1394, SCSI, and SASI.

[0051] The display unit 10g is a monitor of the spatial display device 10 and displays various data. The operation input unit 10h accepts inputs of operations by a user for the spatial display device 10. The operations are performed using a keyboard, a mouse, or the like, and output signals from the keyboard, mouse, or the like are input to the operation input unit 10h. The external information input unit 10i receives original image data that is the source of the image displayed by the spatial display device 10. The original image data is image data that is the material for the image displayed by the spatial display device 10, and an image displayed as a dot matrix display is obtained based on the original image data. The original image data is used by the user of the spatial display device 10 to derive an image that he or she wishes to display in the air using the multiple unmanned aerial vehicles 11, inspired by the original image data.

[0052] The image data includes shape data relating to the size and shape of the image to be displayed in the air, color data relating to the color of the image, and video data relating to the video portion of the image. The image data also includes contour data relating to the contour of the image to be displayed in the air. The shape data defines the size and shape of the image to be displayed in the air, as well as the display mode of the image, which can be a display using the image's outline, a two-dimensional display, or a three-dimensional display. The contour data defines the shape of the boundary line between the image displayed in the air and the background of the image, and may be included in the shape data. A dot number calculation unit 39, which will be described later, calculates the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the shape data included in the image data. A position information generating unit 40, which will be described later, generates position information for each of the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the shape data included in the image data. The color data defines the color of the image to be displayed in the air. A light emission processing unit 42, which will be described later, generates light emission pattern information that defines the light emission state of each dot that will be in a light-emitting state when the image is displayed in a dot matrix, based on color data included in the image data. The light emission mode refers to the light emission color, light emission brightness, and light emission timing of each dot that will be in an illuminating state when the image is displayed in a dot matrix, as well as the light emission color, light emission brightness, and light emission timing of the light emission unit 21 of the unmanned aerial vehicle 11. The video data specifies the video portion of the image to be displayed in the air, specifically the duration of the video portion, the position of the video portion within the image to be displayed in the air, and the manner and speed of movement of the position of the video portion. The flight control unit 41, which will be described later, generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that constitutes the video portion and that will be illuminated when the image is displayed in a dot matrix, based on the video data contained in the image data.

[0053] The spatial display device 10 may use a learning model that has previously learned the correspondence between image data relating to an image to be displayed in the air and the original image data to obtain the image data that is the source of the image based on the original image data received by the external information input unit 10i. The learning model may be trained using machine learning and may perform image classification to classify the original image data into which category it belongs by performing image recognition on the original image data, or may perform object detection of objects appearing in the original image data. The learning model may, for example, classify the original image data into predetermined categories and output image data related to the category to which the original image data belongs. Specifically, if the original image data is "video of a baseball player hitting a home run," the learning model to which this original image data is input classifies the original image data as video related to "home run" and outputs image data of the characters "home run." The image data acquisition unit 38, described below, acquires the image data of the characters "home run." Furthermore, for example, if a predetermined object is included in the original image data, the learning model may detect the predetermined object and output image data related to the detected object. Specifically, if the original image data includes the "seven of hearts" playing card, the learning model to which this original image data is input detects the object included in this original image data as the "seven of hearts" playing card and outputs image data of a heart shape and image data of the number "7." The image data acquisition unit 38, which will be described later, acquires the image data of the heart shape and the image data of the number "7."

[0054] The original image data may be a still image or a video, or may be text, music, symbols, codes, etc. The external information input unit 10i accepts these original image data and inputs them to the spatial display device 10. The external information input unit 10i may be connected to external input devices such as a camera, a video camera, a microphone, a scanner, etc., and may receive output data from these input devices as input data.

[0055] The camera may be an ultra-high-speed camera that captures high-speed phenomena such as magician's tricks, animal movements, machine movements, images of natural phenomena, and celestial movements by high-speed photography, and then cuts out that moment and inputs it to the external information input unit 10i. In this case, the spatial display device 10 can use multiple unmanned aerial vehicles 11 to project a dot matrix display of an image of a moment of a high-speed phenomenon in the air. The camera may be an infrared camera, and a thermal distribution image obtained by capturing infrared rays emitted from an object may be input to the spatial display device 10 via the external information input unit 10i. The camera may also be an X-ray camera, and an image of the inside of the body obtained by, for example, an X-ray may be input to the spatial display device 10 via the external information input unit 10i. Furthermore, the external information input unit 10i may receive, as original image data, the movement of a detected object detected by a tracker used for motion capture. Motion capture is a technology that digitally records the movements of a real person, animal, or object. Motion capture methods include optical, inertial sensor, and magnetic. Optical motion capture records the movements of the object using a combination of trackers and markers. For example, in optical motion capture, multiple markers are attached to the body of the person to be detected, and the movements of these markers are detected by a tracker, allowing the person's movements to be digitally recorded.

[0056] The external information input unit 10i may receive input of images acquired in medical settings, such as CT scan images, MRI images, ultrasound tomographic images, nuclear medicine examination images, and angiography images, and input them to the spatial display device 10. Furthermore, the external information input unit 10i may accept input of real-time video showing the current situation captured by fixed observation cameras, such as security cameras, live video from the Japan Meteorological Agency, live video from a concert hall, live video from a stadium, live video from a theater hall, live video from a zoo or botanical garden, or live video from an astronomical observation, and input the video to the spatial display device 10. The spatial display device 10 can use multiple unmanned aerial vehicles 11 to project a dot matrix display of the real-time video in the air. Furthermore, the external information input unit 10i may receive input of video captured by a camera mounted on a vehicle such as an automobile, motorcycle, or bicycle, and input the video to the spatial display device 10. The spatial display device 10 can use a plurality of unmanned aerial vehicles 11 to project a dot matrix display of video viewed from a vehicle such as an automobile, motorcycle, or bicycle in the air. The spatial display device 10 can display live video of a race of an automobile, motorcycle, bicycle, or the like in the air using a plurality of unmanned aerial vehicles 11, allowing many spectators to see the video with a sense of presence.

[0057] When the image displayed in the air by the spatial display device 10 is, for example, an animation for a drone show, the animation is created by the following procedure. (Step 1) First, an animation is created based on the original image data. To create an animation, you decide on the people and objects that will appear, as well as the background to be used. People include people, animals, mascots, etc. Objects include vehicles, buildings, supplies, tools, and may also include letters, illustrations, marks, codes, etc. Codes may include QR codes, barcodes, and ciphers. Backgrounds include, for example, mountains, forests, rivers, night skies, snowy landscapes, fireworks, or other landscapes. In animation, once the people and objects that will appear have been decided, it is then decided how they will move.

[0058] (Step 2) Next, the type and number of unmanned aerial vehicles 11 are determined. The type and number of unmanned aerial vehicles 11 to be used for animation are determined. Once the animation is created, the animation is converted to a dot matrix display to determine the number of dots required to display the animation, which will determine the number of unmanned aerial vehicles 11 to be used. The type of unmanned aerial vehicle 11 is determined by factors such as the possible continuous flight time and the number of light colors. The possible continuous flight time of unmanned aerial vehicle 11 must be at least longer than the running time of the animation. The more light colors the unmanned aerial vehicle 11 has, the more colorful the animation can be.

[0059] (Step 3) Flight pattern information for the unmanned aerial vehicle 11 is created. Flight pattern information is created that defines the flight route, flight timing, and flight speed of the unmanned aerial vehicle 11 assigned to each dot in the dot matrix display that makes up the animation. The flight pattern information is different for each dot in the dot matrix display. A specific pattern or design is created by multiple unmanned aerial vehicles 11 flying according to their respective flight pattern information. The flight pattern information is used by multiple unmanned aerial vehicles 11 flying simultaneously and synchronizing them in the air to create the animation. Note that if the animation does not involve movement, i.e., if the image displayed in the air by illuminating multiple unmanned aerial vehicles 11 is a still image, the flight pattern information will include the flight route to the position information for each dot that makes up the dot matrix display of the still image. Furthermore, the flight pattern information may or may not include the return route of the unmanned aerial vehicle 11. If the flight pattern information does not include a return route, the unmanned aerial vehicle 11 will autonomously select a flight route toward a predetermined return location while confirming its current position using a GPS signal.

[0060] (Step 4) Light emission pattern information for the unmanned aerial vehicle 11 is created. Based on the colors of the original image data, light emission pattern information is created that specifies the light emission mode of the unmanned aerial vehicle 11 assigned to each dot of the dot matrix display that displays the animation. The light emission mode refers to the light emission color, brightness, and timing of the LED of the light-emitting unit 21 of the unmanned aerial vehicle 11. The color and blinking pattern of the LED are set to draw designs such as characters, codes, and figures.

[0061] (Functional Configuration of Spatial Display Device 10) Next, an example of the functional configuration of the spatial display device 10 will be described with reference to Fig. 5. Fig. 5 is a block diagram for explaining an example of the functional configuration of the spatial display device 10. The spatial display device 10 loads a spatial display program, which will be described later and is stored in the storage unit 10d, into a main memory configured with a RAM 10c, etc. The processing unit 10e accesses the main memory into which the spatial display program has been loaded and executes the spatial display program. By executing the spatial display program, the spatial display device 10 has functional units in the processing unit 10e, such as a learning model acquisition unit 37, an image data acquisition unit 38, a dot number calculation unit 39, a position information generation unit 40, a flight control unit 41, a light emission processing unit 42, an unmanned aerial vehicle allocation unit 43, an abnormality information reception unit 44, a light insufficiency information reception unit 45, an image allocation unit 46, a transmission unit 47, and an aircraft number sufficiency determination unit 48.

[0062] The learning model acquisition unit 37 acquires a learning model that has previously learned the correspondence between image data and original image data relating to an image. In detail, the learning model acquisition unit 37 acquires a learning model that has previously learned the correspondence between image data relating to an image to be displayed in the air and the original image data. The learning model may be a machine learning model learned by machine learning, or may be a table that defines the correspondence between image data related to an image to be displayed in the air and the original image data. The table may be, for example, a table that associates metadata assigned to original image data when the original image data was acquired with image data related to an image to be displayed in the air. The metadata may be, for example, character information for classifying the original image data, character information indicating attributes of the original image data, or character information indicating objects included in the original image data. The image data acquisition unit 38, which will be described later, may refer to the table and acquire image data based on the original image data.

[0063] The image data acquisition unit 38 acquires image data relating to an image. In detail, the image data acquisition unit 38 acquires image data relating to images that are displayed in the air by making multiple unmanned aerial vehicles 11 emit light. Image data relating to an image includes shape data, color data, and video data of the image, and is acquired based on the original image data. The image data acquisition unit 38 may acquire image data input by the user to the external information input unit 10i. The image data acquisition unit 38 may acquire image data by inputting the original image data received by the external information input unit 10i into a learning model.

[0064] The dot number calculation unit 39 calculates the number of dots that will be in a light-emitting state when an image is displayed in a dot matrix, based on the image data. The dot number calculation unit 39 calculates the number of dots that will be illuminated when an image in which multiple unmanned aerial vehicles 11 are illuminated and displayed in the air is displayed in a dot matrix format, based on the shape data contained in the image data. The light emitting state includes states such as lighting, blinking, and flashing, and also includes a state in which light is emitted at least once while the image is displayed in the air. Dot number calculation unit 39 does not count dots that do not light up when the image to be displayed in the air is displayed in a dot matrix. Therefore, unmanned aerial vehicles 11 are not assigned to dots that do not light up when the image to be displayed in the air is displayed in a dot matrix, and unmanned aerial vehicles 11 that do not contribute to the display of the image are not used, allowing unmanned aerial vehicles 11 to be operated efficiently.

[0065] The position information generating unit 40 generates position information for each of the dots that are in a light-emitting state based on the image data. Specifically, the position information generating section 40 generates position information for each of the number of dots that are in a light-emitting state, based on the shape information included in the image data. As mentioned above, the location information is unique to each dot that makes up the matrix display of the image displayed in the air, and is unique to the unmanned aerial vehicle 11 assigned to each dot. When an image to be displayed in the air is displayed using contours, the position information generating unit 40 acquires contour data relating to the contours based on the image data, and generates position information for each dot that will be illuminated when the contours are displayed in a dot matrix, based on the contour data. In other words, when an image to be displayed in the air is displayed using contours, the position information generation unit 40 acquires contour data relating to the contours based on shape data included in the image data, and generates position information for each dot that will be illuminated when the contours are displayed in a dot matrix, based on the contour data.

[0066] Each dot that makes up the dot matrix display corresponds to a light-emitting unit 21 of the unmanned aerial vehicle 11. If the image is a three-dimensional image composed of multiple layers, the position information generation unit 40 generates, based on the image data, position information for each dot that will be in an illuminated state when the image divided into multiple layers is displayed in a dot matrix.

[0067] The dot matrix display will be described with reference to Figures 6 and 7. Figure 6 is a diagram for explaining the dot matrix display used in the spatial display device 10, and Figure 7 is a diagram for explaining the display of a stereoscopic image by the dot matrix display used in the spatial display device 10. A dot matrix display displays characters or images using dots arranged in a two-dimensional array. The dot matrix display used in this embodiment uses an array of 8 dots vertically and 8 dots horizontally, but is not limited to this and may also be an array of 6 dots vertically and 6 dots horizontally, or 12 dots vertically and 12 dots horizontally, for example.

[0068] An example of a dot matrix display, the display of the alphabet "R" 49, will be described with reference to Figure 6. Figure 6(a) shows the state of the alphabet "R" 49 before conversion to a dot matrix display, and Figure 6(b) shows the state of the alphabet "R" 50 after conversion to a dot matrix display. The dot matrix display of the letter "R" 50 is displayed using 30 dots. In FIG. 6(b), six of the 30 dots are labeled with reference numerals 51 to 56. Each of the 30 dots is assigned to a light-emitting unit 21 of an unmanned aerial vehicle 11 that is in an illuminated state. The dot matrix display of the letter "R" 50 may be achieved by using 64 unmanned aerial vehicles 11 arranged in a two-dimensional array, with the light-emitting units 21 of 30 of the unmanned aerial vehicles 11 illuminated, or by using 30 unmanned aerial vehicles 11 and with the light-emitting units 21 of all of the unmanned aerial vehicles 11 illuminated to form a dot matrix display of the letter "R" 50.

[0069] Next, we will explain how to display a 3D image using a dot matrix display, with reference to Figure 7. Figure 7 shows that a 3D image is composed of multiple layers stacked one on top of the other. Each layer is represented by a dot matrix display in a two-dimensional array of 8 dots vertically and 8 dots horizontally. When the image displayed by spatial display device 10 is a stereoscopic image, the stereoscopic image is expressed by overlapping seven layers, as shown in Fig. 7. Each of the seven layers shown in Fig. 7 is a two-dimensional dot matrix display, and is composed of a first layer 60, a second layer 61, a third layer 62, a fourth layer 63, a fifth layer 64, a sixth layer 65, and a seventh layer 66. Based on the shape data contained in the image data, the dot number calculation unit 39 calculates the number of dots that will be illuminated when an image in which multiple unmanned aerial vehicles 11 are illuminated and displayed in the air is displayed in a dot matrix format in each of multiple layers (first layer 60 to seventh layer 66). The position information generating unit 40 generates, based on image data, position information for each dot that will be in a light-emitting state when the images divided into these layers are displayed in a dot matrix, for each of the multiple layers (first layer 60 to seventh layer 66).

[0070] The flight control unit 41 generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that makes up the moving image portion, based on video data relating to the moving image portion of the image included in the image data. When the image to be displayed in the air is a three-dimensional image, the flight control unit 41 generates flight pattern information for each of the multiple layers (first layer 60 to seventh layer 66) based on video data relating to the video portion of the three-dimensional image contained in the image data, which information specifies the flight route, flight timing, and flight speed for each dot that makes up the video portion.

[0071] The light emission processing unit 42 generates light emission pattern information that specifies the light emission state of each dot that will be in an illuminating state when the image to be displayed in the air is displayed in a dot matrix, based on color data regarding the color of the image contained in the image data. When the image to be displayed in the air is a three-dimensional image, the light emission processing unit 42 generates light emission pattern information that specifies the light emission mode for each dot in each of the multiple layers (first layer 60 to seventh layer 66) based on color data related to the color of the image contained in the image data.

[0072] The unmanned aerial vehicle allocation unit 43 allocates an unmanned aerial vehicle 11 to each dot that is illuminated when an image to be displayed in the air is displayed in a dot matrix.

[0073] The abnormality information receiving unit 44 receives abnormality information of the unmanned aerial vehicle 11. The abnormality information receiving unit 44 receives the abnormality information when the abnormality detection unit 26b of the unmanned aerial vehicle 11 detects an abnormality in the vehicle and the abnormality notification unit 26c notifies the spatial display device 10 of abnormality information regarding the abnormality. The unmanned aerial vehicle allocation unit 43 of the spatial display device 10 that receives the abnormality information allocates an unmanned aerial vehicle 11 belonging to another group to the dot that was previously assigned to the unmanned aerial vehicle 11 that notified the abnormality information, in place of the unmanned aerial vehicle 11. Then, the transmitter 47 transmits a return command to the unmanned aerial vehicle 11 that has notified the abnormality information to return. Furthermore, the transmitting unit 47 transmits to the unmanned aircraft 11 belonging to another group that has been assigned in place of the unmanned aircraft 11 that notified the abnormality information, position information and light emission pattern information, or position information, light emission pattern information, and flight pattern information corresponding to the dot to which the unmanned aircraft 11 that notified the abnormality information was assigned. An unmanned aerial vehicle 11 belonging to another group assigned in place of the unmanned aerial vehicle 11 that notified the abnormality information acquires, by the acquisition unit 26a, the position information and light emission pattern information, or the position information, light emission pattern information, and flight pattern information, corresponding to the dot to which the unmanned aerial vehicle 11 that notified the abnormality information was assigned, and is operated in place of the unmanned aerial vehicle 11 that notified the abnormality information.

[0074] The light insufficiency information receiving unit 45 receives the light insufficiency information of the unmanned aerial vehicle 11 . The light deficiency information receiving unit 45 receives the light deficiency information when the light deficiency detection unit 26d of the unmanned aerial vehicle 11 detects a deficiency in the amount of light emitted by the vehicle and the light deficiency notification unit 26e notifies the spatial display device 10 of the light deficiency information regarding the deficiency in light. The unmanned aerial vehicle allocation unit 43 of the spatial display device 10 that receives the light deficiency information allocates an unmanned aerial vehicle 11 that belongs to another group together with the unmanned aerial vehicle 11 that notified the light deficiency information to the dot to which the unmanned aerial vehicle 11 was assigned. In this case, the transmitting unit 47 transmits position information and light emission pattern information, or position information, light emission pattern information, and flight pattern information, corresponding to the dot to which the unmanned aerial vehicle 11 that notified the light insufficiency information was assigned to the unmanned aerial vehicle 11 that notified the light insufficiency information, to the unmanned aerial vehicle 11 that belongs to another group that is assigned to the dot to which the unmanned aerial vehicle 11 that notified the light insufficiency information was assigned. As a result, two unmanned aerial vehicles 11 are assigned to emit light for dots that were lacking in light, making it possible to provide sufficient light for the brightness of the background of the image displayed in the air.

[0075] The image allocation unit 46 allocates the image to be displayed in the air to one of a plurality of groups. The image allocation unit 46 allocates an image to be displayed in the air to one of the multiple groups into which the multiple unmanned aerial vehicles 11 are distributed. The number of groups to which the image is allocated may be one, or two or more groups.

[0076] The transmitter 47 transmits position information and light emission pattern information corresponding to each dot to each unmanned aerial vehicle 11 assigned to each dot that will be illuminated when an image to be displayed in the air is displayed in a dot matrix. If the image to be displayed in the air includes a moving image, the transmitting unit 47 transmits flight pattern information to each of the unmanned aerial vehicles 11 assigned to each dot that makes up the moving image portion, along with position information and light emission pattern information corresponding to that dot.

[0077] During the flight of the unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display, the transmitter 47 may transmit to each of the unmanned aerial vehicles 11 in flight position information and light emission pattern information corresponding to the dot to which the unmanned aerial vehicle 11 is assigned. In this case, the acquisition unit 26a of the unmanned aerial vehicle 11 can receive the position information and light emission pattern information transmitted from the transmission unit 47 during flight. This allows the spatial display device 10 to transmit position information and light emission pattern information generated based on newly added original image data to the unmanned aerial vehicle 11 in flight, thereby allowing the image displayed by the spatial display device 10 to be changed in real time based on the newly added original image data.

[0078] If the image is a video, the transmitter 47 may transmit flight pattern information to each unmanned aerial vehicle 11 assigned to each flying dot, along with position information and light emission pattern information corresponding to the dot. In other words, if the image is a video, while the unmanned aerial vehicle 11 assigned to each dot that makes up the dot matrix display is flying, the transmitter 47 may transmit flight pattern information to each of the unmanned aerial vehicles 11, along with position information and light emission pattern information corresponding to the dot. The acquisition unit 26a of the unmanned aerial vehicle 11 can receive flight pattern information transmitted from the transmission unit 47 along with position information and light emission pattern information during flight. As a result, even if the image is a video, the spatial display device 10 can transmit position information, light emission pattern information, and flight pattern information generated based on newly added original image data to the unmanned aerial vehicle 11 in flight, so that the image displayed by the spatial display device 10 can be changed in real time based on the newly added original image data.

[0079] The number sufficiency determination unit 48 determines whether the number of unmanned aerial vehicles 11 included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit 39 . The number of aircraft sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned suffices the number of dots calculated by the dot number calculation unit 39 if the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is equal to or greater than the number of dots calculated by the dot number calculation unit 39. On the other hand, if the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is smaller than the number of dots calculated by the dot number calculation unit 39, the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned does not suffice the number of dots calculated by the dot number calculation unit 39.

[0080] If the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation unit 39, the unmanned aerial vehicle allocation unit 43 allocates, for each dot, an unmanned aerial vehicle 11 belonging to the group to which the image is assigned. On the other hand, if the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 included in the group to which the image is assigned does not suffice the number of dots calculated by the dot number calculation unit 39, the unmanned aerial vehicle allocation unit 43 allocates, for each dot, unmanned aerial vehicles 11 belonging to the group to which the image is assigned, and also allocates unmanned aerial vehicles 11 belonging to other groups. In this case, the unmanned aerial vehicle allocation unit 43 assigns unmanned aerial vehicles 11 belonging to the group to which the image is assigned to dots that will light up when the image is displayed in a dot matrix, and allocates the insufficient number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned to dots that will light up when the image is displayed in a dot matrix using unmanned aerial vehicles 11 belonging to other groups. In addition, the other groups may be waiting in the air or on the ground, and the unmanned aircraft allocation unit 43 may allocate unmanned aircraft 11 belonging to other groups waiting in the air preferentially over other groups waiting on the ground.

[0081] If the number of aircraft sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 does not suffice the number of dots calculated by the dot number calculation unit 39, and if an unmanned aerial vehicle 11 belonging to another group cannot be assigned to a dot that will light up when the image to be displayed in the air is displayed in a dot matrix, the position information generation unit 40 generates position information for each dot, the number of dots matching the number of unmanned aerial vehicles 11 belonging to the group to which the image is assigned, and the unmanned aerial vehicle assignment unit 43 assigns an unmanned aerial vehicle 11 belonging to the group to which the image is assigned, to each dot, the number of dots matching the number of unmanned aerial vehicles 11. In other words, if the aircraft number sufficiency determination unit 48 determines that the number of unmanned aerial vehicles 11 does not suffice the number of dots calculated by the dot number calculation unit 39, and if it is not possible to assign an unmanned aerial vehicle 11 belonging to another group to a dot that will light up when the image is displayed in a dot matrix, the number of dots that will light up when the image is displayed in a dot matrix is ​​reduced, and an unmanned aerial vehicle 11 belonging to the group to which the image is assigned is assigned.

[0082] The spatial display device 10 distributes a plurality of unmanned aerial vehicles 11 into a plurality of groups, and if there are not enough unmanned aerial vehicles 11 belonging to a group to which an image to be displayed in the air is assigned, an unmanned aerial vehicle 11 belonging to another group is assigned, thereby enabling flexible and efficient operation of the unmanned aerial vehicles 11. Furthermore, even if there are insufficient unmanned aerial vehicles 11 belonging to the group to which the image to be displayed in the air is assigned, and an unmanned aerial vehicle 11 belonging to another group cannot be assigned, the spatial display device 10 can deal with this by reducing the number of dots in the dot matrix display of the image to be displayed in the air.

[0083] (Regarding Example 1 of Use of Spatial Display Device 10) Next, a first example of use of the spatial display device 10 will be described with reference to Fig. 8. Fig. 8 is a diagram for explaining a first example of use of the spatial display device 10. A first example of use of the spatial display device 10 is an example of use in live broadcasting of a card guessing magic trick. In the spatial display device 10, the original image data is a video of a live broadcast of a magic trick. The card guessing trick involves having a spectator draw a card from a well-shuffled deck of cards, and having the spectator memorize the suit and number of the card without showing it to the magician, and then returning the card to the original deck. The magician then draws the same card as the spectator from the deck.

[0084] A video camera is prepared to capture the area around the magician's hands. The video camera captures the card 70 drawn by the audience. The image of the card 70 drawn by the audience is received by the external information input unit 10i and input to the spatial display device 10. If the card 70 drawn by the customer is, for example, the "7 of hearts," the spatial display device 10 stores an image of the "heart mark and the number 7" as original image data in the storage unit 10d. Based on the image of the "heart mark and the number 7" which is the original image data, the position information generating unit 40 generates the position information of the unmanned aerial vehicle 11 for each dot constituting the dot matrix display of the "heart mark and the number 7". The light emission processing unit 42 generates light emission pattern information for each dot constituting the dot matrix display of the "heart mark and number 7" based on the image of the "heart mark and number 7" which is the original image data. Unmanned aerial vehicle allocation unit 43 allocates unmanned aerial vehicles 11 to each of the dots that make up the dot matrix display of the "heart mark and the number 7." The transmitter 47 transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles 11 assigned to each dot during flight. The unmanned aerial vehicle 11 receives the location information and light emission pattern information transmitted from the transmitter 47 during flight using the acquisition unit 26a, and can immediately display a dot matrix display of a heart mark and the number 7 in the air 13.

[0085] 8(a), the spatial display device 10 first displays the "7 of hearts" card 70 drawn by the spectator in the air 13. Next, the spatial display device 10 receives an image of the card 71 drawn by the magician via the external information input unit 10i, and displays the "7 of hearts" card 71 drawn by the magician in the air 13, similar to the card 70 drawn by the spectator, as shown in FIG. The spatial display device 10 displays a card 70 drawn by a spectator and a card 71 drawn by the magician side by side in the air 13, thereby indicating in the air 13 that the magician has succeeded in his magic trick. As shown in Figure 8, by displaying the card 70 drawn by the spectator and the card 71 drawn by the magician side by side in the air 13, not only the spectator taking part in the magic trick but also the many spectators watching the magic trick in the air 13 can be given a sense of realism as if they were taking part in the magic trick.

[0086] (Use Example 2 of the Spatial Display Device 10) Next, a second example of use of the spatial display device 10 will be described with reference to Fig. 9. Fig. 9 is a diagram for explaining a second example of use of the spatial display device 10. The second use example of the spatial display device 10 is an example in which the device is used for live coverage of a card guessing game, similar to the first use example. In use example 2, spatial display device 10 preliminarily arranges a plurality of unmanned aerial vehicles 11 (256 vehicles) in a two-dimensional array of eight dots vertically and eight dots horizontally in four locations in midair 13. Then, external information input unit 10i receives and generates light emission pattern information for each of light-emitting units 21 of 256 unmanned aerial vehicles 11 based on original image data input to spatial display device 10, and transmits the light emission pattern information corresponding to each dot from transmitter 47 to the unmanned aerial vehicle 11 assigned to that dot. Unmanned aerial vehicle 11 causes light-emitting unit 21 to emit light based on the light emission pattern information received by acquisition unit 26a. The spatial display device 10 can immediately generate light emission pattern information based on the original image data received by the external information input unit 10i and transmit it to the unmanned aerial vehicle 11 by the transmission unit 47. Then, the unmanned aerial vehicle 11 can immediately cause the light emission unit 21 to emit light in accordance with the light emission pattern information received by the acquisition unit 26a. The spatial display device 10 does not need to change the position of the unmanned aerial vehicle 11 in the air due to changes in the original image data, and can change the aerial image in real time in response to changes in the original image data.

[0087] 9(a), the position information generator 40 generates the position information of the unmanned aerial vehicle 11 for each dot constituting a two-dimensional array of eight dots vertically and eight dots horizontally in the four dot matrix display. The required number of unmanned aerial vehicles 11 is 256. In Figure 9, white circles indicate the unlit state of light-emitting unit 21 of unmanned aerial vehicle 11, and black circles indicate the lit state of light-emitting unit 21 of unmanned aerial vehicle 11. The unmanned aerial vehicle allocation unit 43 allocates an unmanned aerial vehicle 11 to each of the dots that make up the two-dimensional array of the four dot matrix display. The external information input unit 10i receives an image of the card 70 drawn by the customer, and the image is input to the spatial display device 10. When the card 70 drawn by the customer is, for example, the "7 of hearts," the spatial display device 10 stores an image of the "heart mark and the number 7" as original image data in the storage unit 10d. The light emission processing unit 42 generates light emission pattern information for each dot constituting the dot matrix display of the "heart mark and number 7" based on the image of the "heart mark and number 7" which is the original image data. The transmitter 47 transmits position information and light emission pattern information corresponding to the dots to each of the unmanned aerial vehicles 11 assigned to each dot during flight. During flight, unmanned aerial vehicle 11 receives, via acquisition unit 26a, the position information and light emission pattern information transmitted from transmission unit 47. Because there is no change in the received position information, unmanned aerial vehicle 11 causes light emission unit 21 to emit light in accordance with the light emission pattern information without changing its position in air 13, and is therefore able to immediately display a dot matrix display of a "heart mark and the number 7" in air 13 (see FIG. 9(b)). Next, the spatial display device 10 receives the image of the card 71 drawn by the magician through the external information input unit 10i, and displays the "7 of hearts" card 71 drawn by the magician in the air 13, similar to the card 70 drawn by the customer, as shown in Figure 9(c). The spatial display device 10 can show in the air 13 that the magician has succeeded in the magic trick by arranging and displaying in the air 13 the card 70 drawn by the guest and the card 71 drawn by the magician. As shown in FIG. 9, by preparing a two-dimensional array of four dot matrix displays in the air 13, there is no need to change the position of the unmanned aircraft 11, so the image in the air 13 can be changed in response to the change in the original image data received by the external information input unit 10i quickly.

[0088] (Regarding Use Example 3 of the Spatial Display Device 10) Referring to FIG. 10, as a use example 3 of the spatial display device 10, an example of making the image displayed in the air look smooth will be described. FIG. 10 is a diagram for explaining Use Example 3 of the spatial display device 10. FIG. 10(a) is a diagram for explaining the grayscale font of the Chinese character "big", and FIG. 10(b) is a diagram for explaining the dots that will be in a light-emitting state when the Chinese character "big" is displayed in dot matrix. The grayscale font refers to a font with anti-aliasing applied. When a character is displayed in dot matrix, it is a font that looks smooth by inserting intermediate colors so that the boundaries of the slanted and curved parts of the character do not look jagged. When the spatial display device 10 displays not only characters but also other images in the air, the image can be displayed smoothly by applying anti-aliasing processing to the image. Aliasing refers to the phenomenon that the diagonal and curved parts of an image look jagged, and anti-aliasing refers to making this jaggedness less noticeable. The intermediate color refers to the intermediate color between the color of the image displayed in the air and the color of the background of the image. The intermediate color has 4 gradations, 8 gradations, 16 gradations, etc. The 4 gradations, 8 gradations, 16 gradations, etc. are determined by the size and luminance of one dot of the image displayed by the spatial display device 10. When one dot of the image is relatively large and has a high luminance, it is preferable to use low gradations such as 4 gradations and 8 gradations because a large number of intermediate gradation levels are not required. When the pixels of the image are fine, one dot is small, and the luminance is low, it is preferable to use multi-gradations such as 16 gradations and 32 gradations because a large number of intermediate gradation levels are used.

[0089] The light emission processing unit 42 of the spatial display device 10 in Use Example 3 sets the emission color included in the emission pattern information of the dots arranged in the slanted line portions and curved line portions included in the image displayed in the air and constituting the boundary with the background of the image to be the intermediate color between the color of the background and the color of the image displayed in the air. Referring to FIG. 10(a), the gradation font will be described by taking the Chinese character "大" as an example. As shown in FIG. 10(a), the black Chinese character "大" displayed on a white background of 16 dots in vertical and 16 dots in horizontal has 80 black dots constituting the interior, and the dots constituting the outline are dark gray dots 81 or light gray dots 82. Among the dots constituting the outline of the Chinese character "大", the dots that are in contact with more black dots become dark gray dots 81, and the other dots become light gray dots 82.

[0090] When the Chinese character "大" is displayed in the air using a gradation font, as shown in FIG. 10(b), 85 unmanned aerial vehicles 11 are required. The black dots 80 shown in FIG. 10(a) correspond to the bright dots 84 in FIG. 10(b), and 39 unmanned aerial vehicles 11 are assigned to the bright dots 84 and emit light with the largest light emission amount. The dark gray dots 81 shown in FIG. 10(a) correspond to the slightly dark dots 85 in FIG. 10(b), and 11 unmanned aerial vehicles 11 are assigned to the slightly dark dots 85 and emit light with a medium light emission amount. The light gray dots 82 shown in FIG. 10(a) correspond to the dark dots 86 in FIG. 10(b), and there are 35 unmanned aerial vehicles 11 assigned to the dark dots 86, which emit a small amount of light.

[0091] (Spatial display method and spatial display program) Next, a spatial display program according to one embodiment of the present invention will be described together with a spatial display method with reference to Fig. 11. Fig. 11 is an example of a flowchart of the spatial display program according to this embodiment. The spatial display method is executed by the processing unit 10e of the spatial display device 10 based on the spatial display program. The spatial display program includes a learning model acquisition step S37, an image data acquisition step S38, a dot number calculation step S39, a position information generation step S40, a flight control step S41, a light emission processing step S42, an unmanned aerial vehicle allocation step S43, an abnormality information reception step S44, a light insufficiency information reception step S45, an image allocation step S46, a transmission step S47, and an aircraft sufficient number determination step S48. The spatial display program causes the processing unit 10e of the spatial display device 10 to realize a learning model acquisition function, an image data acquisition function, a dot number calculation function, a position information generation function, a flight control function, a light emission processing function, an unmanned aerial vehicle allocation function, an abnormality information reception function, a light insufficiency information reception function, an image allocation function, a transmission function, and a number of aircraft sufficiency determination function. These functions are executed in the order shown in the flowchart of Figure 11, but the order can also be changed as appropriate. Note that each function overlaps with the description of the various functional units of the spatial display device 10 described above, and therefore detailed description thereof will be omitted.

[0092] The learning model acquisition function acquires a learning model that has previously learned the correspondence between image data and original image data regarding an image (S37: learning model acquisition step). The image data acquisition function acquires image data relating to the image (S38: image data acquisition step). The dot number calculation function calculates, based on the image data, the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix (S39: dot number calculation step). The position information generating function generates position information for each of the dots that are in a light-emitting state based on the image data (S40: position information generating step). The flight control function generates flight pattern information that specifies the flight route, flight timing, and flight speed for each dot that makes up the video portion based on video data regarding the video portion of the image included in the image data (S41: flight control step). The light emission processing function generates light emission pattern information that defines the light emission mode for each dot based on color data relating to the color of the image included in the image data (S42: light emission processing step). The unmanned aerial vehicle allocation function allocates an unmanned aerial vehicle 11 to each dot (S43: unmanned aerial vehicle allocation step). The abnormality information receiving function receives abnormality information of unmanned aerial vehicle 11 (S44: abnormality information receiving step). The insufficient light information receiving function receives insufficient light information from unmanned aerial vehicle 11 (S45: insufficient light information receiving step). The image allocation function allocates the image to one of a plurality of groups (S46: image allocation step). The transmission function transmits the position information and light emission pattern information corresponding to the dot to each of the unmanned aerial vehicles 11 assigned to each dot (S47: transmission step). The number of aircraft sufficiency determination function determines whether the number of unmanned aerial vehicles 11 included in the group to which the image is assigned is sufficient for the number of dots calculated by the dot number calculation function (S48: number of aircraft sufficiency determination step).

[0093] According to the spatial display device 10 of the above-described embodiment, the dot number calculation unit 39 does not count dots that do not light up when an image to be displayed in the air is displayed in a dot matrix. Therefore, unmanned aerial vehicles 11 are not assigned to dots that do not light up when an image to be displayed in the air is displayed in a dot matrix, and unmanned aerial vehicles 11 that do not contribute to the display of images are not used, thereby enabling efficient operation of the unmanned aerial vehicles 11.

[0094] Furthermore, the spatial display device 10 of the above-described embodiment distributes a plurality of unmanned aerial vehicles 11 into a plurality of groups, and if there are insufficient unmanned aerial vehicles 11 belonging to a group to which an image to be displayed in the air is assigned, an unmanned aerial vehicle 11 belonging to another group is assigned, thereby enabling flexible operation of the unmanned aerial vehicles 11 and efficient operation of the unmanned aerial vehicles 11.

[0095] Furthermore, in the above-described embodiment of the spatial display device 10, even if there are insufficient unmanned aerial vehicles 11 belonging to the group to which the image to be displayed in the air is assigned, and it is not possible to assign unmanned aerial vehicles 11 belonging to other groups, the spatial display device 10 can deal with this by reducing the number of dots in the dot matrix display of the image to be displayed in the air.

[0096] Furthermore, according to the spatial display device 10 of the above-described embodiment, position information, light emission pattern information, and flight pattern information are transmitted to the unmanned aerial vehicle 11 in flight, and the unmanned aerial vehicle 11 can receive the position information, light emission pattern information, and flight pattern information transmitted from the spatial display device 10 while in flight, so that the unmanned aerial vehicle 11 can easily display an image in the air that is different from a predetermined image even while in flight.

[0097] Furthermore, according to the spatial display device 10 of the above-described embodiment, the unmanned aerial vehicle 11 is equipped with an illuminance sensor 32 and an emission brightness correction unit 26f, so that the unmanned aerial vehicle 11 can adjust the emission brightness of the light-emitting unit 21 according to the external illuminance, and therefore, the image of the air 13 is not made difficult to see due to insufficient brightness, or the light-emitting unit 21 is not made to emit light with excessive brightness, so that the battery 27 is not consumed more than necessary.

[0098] Furthermore, according to the spatial display device 10 of the above-described embodiment, two unmanned aerial vehicles 11 are assigned to emit light for dots that were lacking in light intensity, making it possible to provide sufficient light intensity for the background brightness of the image displayed in the air.

[0099] Furthermore, according to the spatial display device 10 of the above-described embodiment, a stereoscopic image can be displayed in the air 13 .

[0100] Furthermore, according to the spatial display device 10 of the above-described embodiment, by displaying fireworks in the air 13 using multiple unmanned aerial vehicles 11, it is possible to reduce carbon dioxide emissions, unlike actual fireworks, and therefore it is possible to put the concept of carbon neutrality into practice.

[0101] Furthermore, according to the spatial display device 10 of the above-described embodiment, images acquired at a medical site can be input from the external information input unit 10i and the images can be displayed in the air 13, so that many people can see the images acquired at a medical site.

[0102] Furthermore, according to the spatial display device 10 of the above-described embodiment, images taken by an ultra-high speed camera can be displayed in the air 13, so that an image capturing a moment of a high-speed phenomenon can be projected in the air, impressing many people.

[0103] Furthermore, according to the spatial display device 10 of the above-described embodiment, real-time images showing the current situation captured by a fixed observation camera can be displayed in the air 13, so that many people can see the real-time images.

[0104] It should be noted that the present invention is not limited to the spatial display device 10, the spatial display method, and the spatial display program according to the above-described embodiments, and can be embodied in various other modified examples or application examples without departing from the gist of the present invention as set forth in the claims. Also, although the term "data" is used in the above-described embodiments, the term "data" can be replaced with "information," and the term "information" can be replaced with "data." [Explanation of symbols]

[0105] 10 Spatial display device 10a Communications Department 10b ROM 10c RAM 10d storage section 10e Processing section 10f Input / Output Interface 10g display 10h Operation input section 10i External information input section 11 Unmanned aerial vehicles (drones) 11a Unmanned Aerial Vehicles (Drones) 11b Unmanned Aerial Vehicles (Drones) 11c Unmanned Aerial Vehicles (Drones) 11d Unmanned Aerial Vehicles (Drones) 11e Unmanned Aerial Vehicles (Drones) 12 Radio Antenna 13 Air 14 marks 15 characters 16 QR Code 17 Information and Communications Networks 18 Main body 19 Arm 19a First Arm 19b Second Arm 19c Third Arm 19d 4th Arm 20 propellers 20a No. 1 propeller 20b No. 2 propeller 20c 3rd propeller 20d 4th propeller 21 Light-emitting part 22 Communications Department 23 ROM 24 RAM 25 Memory section 26 Control Unit 26a Acquisition Department 26b Abnormality detection unit 26c Abnormality notification section 26d Light shortage detection unit 26e Insufficient light notification section 26f Light emission brightness correction section 27 Battery 28 Input / Output Interface 29 Flight drive unit (motor) 30 Flight drive mechanism (rotor) 31 GPS receiver 32 Illuminance sensor 33 Geomagnetic sensor 34 Altitude Sensor 35 Gyro sensor 36 Obstacle Detection Camera 37 Learning model acquisition unit 38 Image data acquisition unit 39 Dot Count Calculation Unit 40 Location information generation section 41 Flight control unit 42 Light-emitting processing section 43 Unmanned Aerial Vehicle Allocation Department 44 Abnormality Information Reception Department 45. Light Insufficiency Information Reception Unit 46 Image allocation section 47 Transmitter 48 Aircraft sufficiency determination unit 49 "R" 50 "R" 51 dots 52 dots 53 dots 54 dots 55 dots 56 dots 60 1st Layer 61 Second Layer 62 Third Layer 63 4th Layer 64 5th Layer 65 6th Layer 66 7th Layer 69 No display 70 Card drawn by customer 71 The Magician's Card 80 black dots 81 dark gray dots 82 Light Gray Dots 83 None 84 bright dots 85 slightly dark dot 86 dark dots

Claims

1. A spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups, causes the plurality of unmanned aerial vehicles to emit light, and displays an image in the air, The processing unit of the spatial display device includes: an image data acquisition unit that acquires image data related to the image; a dot number calculation unit that calculates, based on the image data, the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix; a position information generating unit that generates position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing unit that generates light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; an unmanned aerial vehicle allocation unit that allocates the unmanned aerial vehicle to each of the dots; an image allocation unit that allocates the image to one of the plurality of groups; a transmitter that transmits the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number-of-unmanned aerial vehicles sufficiency determination unit that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation unit; Equipped with When the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles suffices the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is allocated, A spatial display device characterized in that, if the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, the unmanned aerial vehicle allocation unit allocates, for each dot, the unmanned aerial vehicle belonging to the group to which the image is assigned, and also allocates the unmanned aerial vehicle belonging to other groups.

2. When the aircraft number sufficiency determination unit determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation unit, and when the unmanned aerial vehicles belonging to other groups cannot be assigned to the dots, the position information generation unit generates position information for each dot, the number of dots corresponding to the number of unmanned aerial vehicles belonging to the group to which the image is assigned; The spatial display device according to claim 1, wherein the unmanned aerial vehicle allocation unit allocates the unmanned aerial vehicle belonging to the group to which the image is allocated to each dot, the number of dots corresponding to the number of unmanned aerial vehicles.

3. the image includes a video; The processing unit of the spatial display device includes: A flight control unit is provided that generates flight pattern information that defines a flight route, flight timing, and flight speed for each of the dots that constitute the moving image portion based on moving image data related to the moving image portion of the image included in the image data, The spatial display device described in claim 1, characterized in that the transmitting unit transmits the flight pattern information along with the position information and the illumination pattern information corresponding to the dots to each of the unmanned aerial vehicles assigned to each of the dots that constitute the video portion.

4. the image is represented by contours, The position information generation unit The spatial display device according to claim 1, characterized in that contour data relating to the contour is obtained based on the image data, and position information for each dot that will be illuminated when the contour is displayed in a dot matrix is ​​generated based on the contour data.

5. The image is a three-dimensional image composed of multiple layers, The position information generation unit The spatial display device according to claim 1, characterized in that, based on the image data, position information for each dot that will be in an illuminated state when the image divided into the plurality of layers is displayed in a dot matrix is ​​generated.

6. an external information input unit that receives original image data that is the source of the image; a learning model acquisition unit that acquires a learning model that has previously learned a correspondence relationship between image data related to the image and the original image data; Equipped with The image data acquisition unit The spatial display device according to claim 1, characterized in that the image data is acquired by inputting the original image data received by the external information input unit into the learning model.

7. There are other groups waiting in the air and other groups waiting on the ground, The spatial display device according to claim 1, characterized in that the unmanned aerial vehicle allocation unit allocates unmanned aerial vehicles belonging to other groups that are waiting in the air preferentially over other groups that are waiting on the ground.

8. The spatial display device described in claim 1, characterized in that the light-emitting processing unit sets the light-emitting color included in the light-emitting pattern information of dots that are placed in the diagonal and curved portions of the image and form the boundary with the background of the image to an intermediate color between the color of the background and the color of the image.

9. The unmanned aerial vehicle comprises: The spatial display device according to claim 1 , further comprising an acquisition unit that acquires the position information and the light emission pattern information transmitted from the transmission unit.

10. The unmanned aerial vehicle comprises: The spatial display device according to claim 3, further comprising an acquisition unit that acquires flight pattern information transmitted from the transmission unit together with the position information and the light emission pattern information.

11. The unmanned aerial vehicle comprises: an abnormality detection unit that detects abnormalities in the device itself; an abnormality notification unit that notifies the spatial display device of abnormality information regarding an abnormality of the own device; Equipped with The processing unit of the spatial display device includes: an abnormality information receiving unit that receives abnormality information about the unmanned aerial vehicle; the unmanned aerial vehicle allocation unit allocates another unmanned aerial vehicle belonging to the group to the dot to which the unmanned aerial vehicle that notified the abnormality information had been allocated, in place of the unmanned aerial vehicle; The spatial display device according to claim 9 or 10, wherein the transmitter transmits a return command to the unmanned aerial vehicle that has notified the abnormality information to return.

12. The spatial display device according to claim 11 , wherein the abnormality includes a case where the remaining charge of the battery is equal to or less than a threshold.

13. The unmanned aerial vehicle comprises: a light intensity deficiency detection unit that detects a deficiency in the light intensity of the light emitted by the device itself; a light intensity deficiency notification unit that notifies the spatial display device of light intensity deficiency information regarding a deficiency in the amount of light emitted by the device itself; Equipped with The processing unit of the spatial display device includes: a light intensity deficiency information receiving unit that receives light intensity deficiency information of the unmanned aerial vehicle; The spatial display device described in claim 9 or 10, characterized in that the unmanned aerial vehicle allocation unit allocates other unmanned aerial vehicles belonging to the group together with the unmanned aerial vehicle that notified the light deficiency information to the dot to which the unmanned aerial vehicle was assigned.

14. The unmanned aerial vehicle comprises: an illuminance sensor that measures external illuminance; a light emission luminance correction unit that corrects the light emission luminance included in the light emission pattern information based on the measurement result of the illuminance sensor; 11. The spatial display device according to claim 9, further comprising:

15. A spatial display method used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light to display an image in the air, The processing unit of the spatial display device includes: an image data acquisition step of acquiring image data relating to the image; a dot number calculation step of calculating, based on the image data, the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix; a position information generating step of generating position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing step of generating light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; an unmanned aerial vehicle allocation step of allocating the unmanned aerial vehicle to each of the dots; an image allocation step of allocating the image to one of the plurality of groups; a transmitting step of transmitting the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number-of-unmanned aerial vehicles sufficiency determination step of determining whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated in the dot number calculation step; Run If it is determined in the aircraft number sufficiency determination step that the number of unmanned aerial vehicles is sufficient for the number of dots calculated in the dot number calculation step, the unmanned aerial vehicle allocation step allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is allocated; A spatial display method characterized in that if, in the aircraft number sufficiency determination step, it is determined that the number of unmanned aerial vehicles does not suffice the number of dots calculated in the dot number calculation step, in the unmanned aerial vehicle allocation step, for each dot, an unmanned aerial vehicle belonging to the group to which the image is assigned is allocated, and an unmanned aerial vehicle belonging to another group is allocated.

16. A spatial display program used in a spatial display device that divides a plurality of unmanned aerial vehicles into a plurality of groups and causes the plurality of unmanned aerial vehicles to emit light and display images in the air, The processing unit of the spatial display device includes: an image data acquisition function for acquiring image data relating to the image; a dot number calculation function that calculates the number of dots that will be in a light-emitting state when the image is displayed in a dot matrix based on the image data; a position information generating function that generates position information for each of the dots that are in the light-emitting state based on the image data; a light emission processing function that generates light emission pattern information that defines a light emission mode for each of the dots based on color data related to the color of the image included in the image data; an unmanned aerial vehicle allocation function that allocates the unmanned aerial vehicle to each of the dots; an image allocation function for allocating the image to one of the plurality of groups; a transmission function for transmitting the position information and the light emission pattern information corresponding to each dot to each of the unmanned aerial vehicles assigned to each dot; a number-of-unmanned aerial vehicles sufficiency determination function that determines whether the number of unmanned aerial vehicles included in the group to which the image is assigned satisfies the number of dots calculated by the dot number calculation function; and When the function of determining whether the number of unmanned aerial vehicles is sufficient for the number of dots calculated by the function of calculating the number of dots, the function of allocating unmanned aerial vehicles allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is allocated; A spatial display program characterized in that, if the aircraft number sufficiency determination function determines that the number of unmanned aerial vehicles does not suffice the number of dots calculated by the dot number calculation function, the unmanned aerial vehicle allocation function allocates, for each dot, an unmanned aerial vehicle belonging to the group to which the image is assigned, and also allocates an unmanned aerial vehicle belonging to another group.

Images