Aircraft Information Sharing System

Abstract

To provide an aircraft information sharing system that can give an imaging instruction actively and accurately.SOLUTION: An aircraft information sharing system 10 comprises an airborne station X having an airframe 1 to which a camera 2 is mounted and an aeronautical station Y sending / receiving information to / from the airborne station X. The airborne station X outputs a camera image of the camera 2 and aircraft information for obtaining a latitude / longitude of the camera image to the aeronautical station Y, the aeronautical station Y calculates the latitude / longitude in a photographic range of the camera from the camera image, the aircraft information, and altitude information that is altitude data linked to the latitude / longitude to superimpose it on map information to display an instruction screen as well as specifying the latitude / longitude of an indication point to photograph from the instruction screen to send it to the airborne station X, the airborne station X displays the indication point superimposed on the camera image from the aircraft information and the altitude information.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present application relates to an aircraft information sharing system. [Background technology]

[0002] Conventionally, a system that uses helicopter video (hereinafter referred to as a Heli-Tele system) has been used to survey the damage situation as one of the means for quickly grasping the damage situation when a disaster occurs. The Heli-Tele system is a system that allows a ground station to grasp the damage situation by transmitting video images taken by a camera device mounted on a helicopter and the aircraft's position information to the ground station. Conventionally, in the Heli-Tele system, in order for a ground station to confirm the damage situation at a desired location, instructions are given using telegram messages using voice or text information while checking the video images and position information being received by the ground station.

[0003] There is an airborne station using a helicopter and a video processing system that receives and displays the video. In this helicopter telemetry system, the ground station terminal transforms the received video and transforms it into a photographed image and displays it superimposed on the corresponding location on a map, thereby identifying the shooting location (for example, see Patent Document 1). [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2007-241085 A Summary of the Invention [Problem to be solved by the invention]

[0005] In conventional systems, when a disaster occurs, users of the ground station can check on a map screen which location on the map the image was taken of, allowing them to determine the damaged area and location. However, since there is no interface for transmitting instructions to shoot from the ground station, such as the location to be photographed, instructions must be transmitted by radio voice, making it difficult to actively and accurately give instructions to shoot.

[0006] This application discloses technology for solving the above-mentioned problems, and aims to provide an aircraft information sharing system that can actively and accurately issue shooting instructions from a ground station to an on-board station. [Means for solving the problem]

[0007] The aircraft information sharing system disclosed in the present application comprises: An airborne station with an aircraft equipped with a camera An aircraft information sharing system including a ground station that transmits and receives information to and from the on-board station, The on-board station outputs the camera image of the camera and aircraft information for obtaining the latitude and longitude of the camera image to the ground station; the ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and altitude information, which is altitude data linked to latitude and longitude, and displays an instruction screen by superimposing the calculated latitude and longitude on map information, and also specifies the latitude and longitude of an instruction point for shooting from the instruction screen and transmits the calculated latitude and longitude to the on-board station; The on-board station displays the designated point by superimposing it on the camera image based on the aircraft information and the altitude information, The ground station displays, as the instruction screen, a video screen consisting of a still image of the camera image on which map information is superimposed, or a map screen displaying the shooting range superimposed on a map. Display one of the following: The on-board station calculates the latitude and longitude of the imaging range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the designated point is included in the imaging range, displays a mark indicating the designated point superimposed on the camera image, and if the latitude and longitude of the designated point is not included in the imaging range, displays a mark indicating the direction of the designated point superimposed on the camera image, The ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the indicated point are included in the shooting range, it superimposes a mark indicating the indicated point on the camera image, and if the latitude and longitude of the indicated point are not included in the shooting range, it superimposes a mark indicating the shooting direction of the indicated point on the camera image. Effect of the Invention

[0008] According to the aircraft information sharing system disclosed in the present application, shooting instructions can be given actively and accurately. [Brief description of the drawings]

[0009] [Figure 1] 1 is a diagram showing a configuration of an aircraft information sharing system according to a first embodiment. [Diagram 2] 2 is a diagram showing a configuration of an on-board station of the aircraft information sharing system shown in FIG. 1. [Diagram 3] 2 is a diagram showing a configuration of a ground station of the aircraft information sharing system shown in FIG. 1. [Figure 4] FIG. 3 is a flow chart for explaining the operation of the on-board station side shown in FIG. 2. [Diagram 5] FIG. 4 is a flow diagram illustrating the operation of the ground station shown in FIG. 3. [Figure 6] 3 is a flow diagram of data conversion by the aircraft information conversion device shown in FIG. 2. [Figure 7] 3 is an explanatory diagram showing the relationship between aircraft information, camera information, static parameter information, aircraft information, camera images, and integrated camera information on the ground station side shown in FIG. 2. [Figure 8] FIG. 4 is a data structure diagram of the additional aircraft database shown in FIG. 3. [Figure 9] 4 is a flow diagram for performing a photographing instruction operation from the aircraft information display device shown in FIG. 3. [Figure 10A] 1. FIG. 4 is an explanatory diagram of a case where a shooting instruction operation is performed from a video screen in the aircraft information sharing system shown in FIG. [Figure 10B] 1. FIG. 4 is an explanatory diagram of a case where a shooting instruction operation is performed from a video screen in the aircraft information sharing system shown in FIG. [Figure 11A] 1. FIG. 4 is an explanatory diagram of a case where a shooting instruction operation is performed from a video screen in the aircraft information sharing system shown in FIG. [Figure 11B] 1. FIG. 4 is an explanatory diagram of a case where a shooting instruction operation is performed from a video screen in the aircraft information sharing system shown in FIG. [Figure 12A]1. FIG. 4 is an explanatory diagram of a case where a photographing instruction operation is performed from a map screen in the aircraft information sharing system shown in FIG. [Figure 12B] 1. FIG. 4 is an explanatory diagram of a case where a photographing instruction operation is performed from a map screen in the aircraft information sharing system shown in FIG. [Figure 13A] 1. FIG. 4 is an explanatory diagram of a case where a photographing instruction operation is performed from a map screen in the aircraft information sharing system shown in FIG. [Figure 13B] 1. FIG. 4 is an explanatory diagram of a case where a photographing instruction operation is performed from a map screen in the aircraft information sharing system shown in FIG. [Figure 14] 4 is a flow diagram for calculating designated point information from a video screen in the aircraft information sharing system shown in FIG. 1. [Figure 15] 2 is an explanatory diagram of the relationship between a video screen and real space in the aircraft information sharing system shown in FIG. 1. [Figure 16] 2 is a data structure diagram of instruction information in the aircraft information sharing system shown in FIG. 1. [Figure 17] 2 is an explanatory diagram of an image from an indicator camera in the aircraft information sharing system shown in FIG. 1. [Figure 18] 2 is an explanatory diagram of an image from an indicator camera in the aircraft information sharing system shown in FIG. 1. [Figure 19] 3 is a flow diagram of drawing an instruction video of the shooting position calculation device shown in FIG. 2. [Figure 20] 3 is a flow diagram of drawing an instruction video of the shooting position calculation device shown in FIG. 2. [Figure 21A] 3 is an explanatory diagram of an image of the instruction camera on the operator's monitor shown in FIG. 2. [Figure 21B] 3 is an explanatory diagram of an image of the instruction camera on the operator's monitor shown in FIG. 2. [Figure 21C] 3 is an explanatory diagram of an image of the instruction camera on the operator's monitor shown in FIG. 2. [Figure 22] 2 is a block diagram of a photographing position designation device, an aircraft information output device, an aircraft information distribution device, and an aircraft information display device in the aircraft information sharing system shown in FIG. 1. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0010] The aircraft information sharing system 10 of this embodiment is intended to enable accurate and smooth information sharing and instructions between an on-board station X on an aircraft 1 flying for the purpose of grasping the damage situation when a disaster occurs, and a ground station Y that receives the information on the ground.

[0011] Embodiment 1 FIG. 1 is a diagram showing the configuration of an aircraft information sharing system according to embodiment 1. FIG. 2 is a diagram showing the configuration of an on-board station of the aircraft information sharing system shown in FIG. 1. FIG. 3 is a diagram showing the configuration of a ground station of the aircraft information sharing system shown in FIG. 1. FIG. 4 is a flow diagram explaining the operation of the on-board station side shown in FIG. 2. FIG. 5 is a flow diagram explaining the operation of the ground station side shown in FIG. 3. FIG. 6 is a flow diagram of data conversion by the aircraft information conversion device shown in FIG. 2.

[0012] Fig. 7 is an explanatory diagram showing the relationship between the aircraft information, camera information, static parameter information, aircraft information, camera video, and integrated camera information on the ground station side shown in Fig. 2. Fig. 8 is a data structure diagram of the additional aircraft database shown in Fig. 3. Fig. 9 is a flow diagram when performing a shooting instruction operation from the aircraft information display device shown in Fig. 3.

[0013] Fig. 10A, Fig. 10B, Fig. 11A, and Fig. 11B are explanatory diagrams of a case where a shooting instruction operation is performed from a video screen in the aircraft information sharing system shown in Fig. 1. Fig. 12A, Fig. 12B, Fig. 13A, and Fig. 13B are explanatory diagrams of a case where a shooting instruction operation is performed from a map screen in the aircraft information sharing system shown in Fig. 1. Fig. 14 is a flow diagram for calculating instruction point information from a video screen in the aircraft information sharing system shown in Fig. 1. Fig. 15 is an explanatory diagram of the relationship between the video screen and real space in the aircraft information sharing system shown in Fig. 1.

[0014] Fig. 16 is a data structure diagram of instruction information in the aircraft information sharing system shown in Fig. 1. Fig. 17 is an explanatory diagram of an instruction camera image in the aircraft information sharing system shown in Fig. 1. Fig. 18 is an explanatory diagram of an instruction camera image in the aircraft information sharing system shown in Fig. 1. Figs. 19 and 20 are flow diagrams of drawing an instruction image of the shooting position calculation device shown in Fig. 2. Figs. 21A, 21B, and 21C are explanatory diagrams of the instruction camera image of the photographer's monitor shown in Fig. 2. Fig. 22 is a block diagram of the shooting position instruction device, aircraft information output device, aircraft information distribution device, and aircraft information display device in the aircraft information sharing system shown in Fig. 1.

[0015] As shown in FIG. 1, the aircraft information sharing system 10 is composed of an on-board station X that operates on an aircraft 1 (see FIG. 15), and a ground station Y that transmits and receives information to and from the on-board station X via a satellite or wireless line W. The on-board station X is equipped with an aircraft information output device 110 and a photographing position indication device 120. The aircraft information output device 110 outputs to the ground station Y integrated camera information in which aircraft information is superimposed on camera footage captured by a camera 2 (see FIG. 15) mounted on the aircraft 1 that flies when a disaster occurs, etc. Furthermore, the aircraft information output device 110 receives instruction information from the ground station Y. The photographing position indication device 120 displays the instruction camera footage from the instruction information, aircraft information, altitude information, and camera footage. Details of each piece of information will be described later.

[0016] The ground station Y includes an aircraft information distribution device 210 and an aircraft information display device 220. The aircraft information distribution device 210 receives integrated camera information from the on-board station X and transmits instruction information to the on-board station X. The aircraft information display device 220 creates instruction information from the camera video, aircraft information, altitude information, and map information.

[0017] As shown in FIG. 2, the aircraft information output device 110 includes an aircraft information conversion device 111, an aircraft information output device 112, a first transmission device 116, an aircraft information superimposition device 115, a camera vibration isolation device 113, a video switcher 114, and a static parameter database (hereinafter, the database will be abbreviated as DB) 117.

[0018] The camera vibration isolation device 113 is installed on the camera 2 of the aircraft 1. The camera vibration isolation device 113 delivers the camera information of the camera 2 to the aircraft information conversion device 111, and the camera video of the camera 2 to the video switcher 114. The video switcher 114 delivers the camera video to the aircraft information superimposition device 115 and an image superimposition device 122 described later. The aircraft information output device 112 outputs the aircraft information to the aircraft information conversion device 111.

[0019] The static parameter DB 117 stores static parameter information of the aircraft 1 and the camera 2. The aircraft information conversion device 111 receives the camera information and the aircraft information, extracts the static parameter information from the static parameter DB 117, converts it to aircraft information, and distributes it to the aircraft information superimposition device 115. Furthermore, the aircraft information conversion device 111 outputs the instruction information received from the first transmission device 116 described later and the aircraft information to the photographing position calculation device 121 described later.

[0020] The aircraft information superimposing device 115 superimposes aircraft information on the camera video delivered via the video switcher 114 and transmits the superimposed aircraft information as integrated camera information to the first transmission device 116. The first transmission device 116 transmits the integrated camera information to the ground station Y via a satellite or a wireless line W. Furthermore, the first transmission device 116 receives instruction information from the ground station Y, which will be described later, and outputs it to the aircraft information conversion device 111.

[0021] The photographing position indication device 120 includes a photographing position calculation device 121, an altitude DB 124, a photographer's monitor 123, and an image superimposition device 122. The altitude DB 124 stores altitude information, which is altitude data linked to latitude and longitude. The photographing position calculation device 121 calculates the photographing range from the instruction information, aircraft information, and altitude information, and transmits an instruction video to the image superimposition device 122. The image superimposition device 122 superimposes the instruction video on the camera video from the video switcher 114, and transmits it to the photographer's monitor 123 as an instruction camera video. The photographer's monitor 123 displays the instruction camera video.

[0022] 3, the aircraft information distribution device 210 includes a second transmission device 211, a video management device 212, an altitude DB 214, a video DB 215, an aircraft information management device 213, and an additional aircraft DB 216. The second transmission device 211 transmits and receives information to and from the first transmission device 116 of the on-board station X via a satellite or a wireless line W. The altitude DB 214 stores altitude information, which is altitude data linked to latitude and longitude.

[0023] The video management device 212 manages the integrated camera information by separating it into camera video and aircraft information. The video management device 212 stores the camera video in the video DB 215 and transmits it to the aircraft information display terminal 221, which will be described later. The video management device 212 also calculates the shooting center position Go and shooting range 11 (see Figures 15 and 12A) from the aircraft information and altitude information, adds it to the aircraft information, and transmits it to the aircraft information management device 213 as additional aircraft information. The aircraft information management device 213 stores the additional aircraft information in the additional aircraft DB 216 and transmits it to the aircraft information display terminal 221, which will be described later.

[0024] The aircraft information display device 220 includes an aircraft information display terminal 221, a map DB 223, and an aircraft information display monitor 222. Map information is stored in the map DB 223. The aircraft information display terminal 221 receives camera images from the video management device 212 and additional aircraft information from the aircraft information management device 213, and transmits the camera images to the aircraft information display monitor 222.

[0025] Furthermore, the aircraft information display terminal 221 creates aircraft information with GIS (Geographic Information System) for displaying the shooting range 11 or the still image with the added map information, the aircraft position, and the shooting range 11 on the map screen from the additional aircraft information and map information, and transmits it to the aircraft information display monitor 222. The aircraft information display monitor 222 enables the camera video received by the aircraft information display terminal 221 to be displayed, and displays the shooting range 11 or the still image with the added map information, the aircraft position, and the shooting range 11 on the map screen.

[0026] Although the altitude DB 124, altitude DB 214, and map DB 223 are shown as being stored within each device, this is not limited to this, and it is also possible to access and use each database stored outside the aircraft information sharing system 10.

[0027] Next, various types of information will be described. As shown in Fig. 7, aircraft information is information output from aircraft information output device 112, and is dynamic information of aircraft 1 that changes as the aircraft 1 flies, such as the time, latitude, longitude, altitude, speed, azimuth angle, roll angle, and pitch angle of aircraft 1. Camera information is information output from camera vibration isolation device 113, and is dynamic information of camera 2, such as the pan angle, tilt angle, camera roll angle, focal length, viewing angle, and aspect ratio of camera 2.

[0028] The static parameter information is information stored in static parameter DB 117, and is static information of the aircraft 1 and camera 2 that does not change as the aircraft 1 flies, such as the aircraft number, image sensor size, and visible or infrared information. The aircraft information is information that integrates camera information, aircraft information, and static parameter information, which is output from aircraft information conversion device 111. The camera video is information on the camera video of camera 2 that is output from video switcher 114.

[0029] The integrated camera information is information output from aircraft information superimposition device 115, and is information in which aircraft information is superimposed on camera footage. The aircraft information, which is made up of camera information, aircraft information, and static parameter information, is information required other than altitude information to distinguish aircraft 1 and clarify the latitude and longitude on the camera footage.

[0030] 8, the additional aircraft information is information stored in the additional aircraft DB 216, and is composed of information such as the GPS time output on the aircraft 1 side, the reception time when the video management device 212 received the data, aircraft position information, aircraft attitude information, aircraft speed information, camera information, static parameter information, shooting center position Go, and shooting range 11. The shooting center position Go and shooting range 11 will be explained in detail in the operation of the aircraft information sharing system 10.

[0031] Also, as shown in Figure 16, the instruction information is information output from the aircraft information display terminal 221, and is composed of instruction time information, which is the time information when the instruction information was registered at ground station Y, and instruction point information, which is the latitude and longitude information of the registered instruction point.

[0032] Next, the operation of the aircraft information sharing system 10 of the first embodiment configured as above will be described. First, the aircraft information conversion device 111 acquires camera information from the camera vibration isolation device 113 and aircraft information from the aircraft information output device 112 (step S1 in FIG. 4). Next, the aircraft information conversion device 111 generates and transmits aircraft information from the camera information, aircraft information, and static parameter information (step S2 in FIG. 4).

[0033] The generation of aircraft information by the aircraft information conversion device 111 at this time will be further described with reference to Fig. 6. First, static parameter information such as the aircraft number of the aircraft 1 and the image sensor size of the camera is extracted from the static parameter DB 117 (step S101 in Fig. 6). Next, aircraft information and camera information are acquired (step S102 in Fig. 6, the same as the previous step S1). Next, the static parameter information is integrated as aircraft information in addition to the aircraft information and camera information (step S103 in Fig. 6). Next, the aircraft information is transmitted to the aircraft information superimposition device 115 and the photographing position calculation device 121 (step S104 in Fig. 6).

[0034] Next, the camera vibration isolation device 113 transmits the camera image captured by the camera 2 to the aircraft information superimposing device 115 and the image superimposing device 122 via the image switcher 114 (step S3 in FIG. 4). Next, the aircraft information superimposing device 115 superimposes the received aircraft information, for example, on the ancillary area of ​​the camera information, and transmits it to the first transmission device 116 as integrated camera information. At this time, for example, it is also possible to convert the aircraft information into audio data, superimpose it as audio data on the camera information, and transmit it as integrated camera information. The relationship between the integrated camera information transmitted in this way and the aircraft information, camera information, static parameter information, aircraft information, and camera image is as shown in FIG. 7. Next, the integrated camera information transmitted from the first transmission device 116 is transmitted to the second transmission device 211 of the ground station Y via a satellite or a wireless line W (step S4 in FIG. 4).

[0035] Then, at ground station Y, the second transmission device 211 receives the integrated camera information transmitted from the aircraft information output device 110 of the on-board station X (step S5 in FIG. 5). Next, the video management device 212 receives the integrated camera information distributed from the second transmission device 211 and separates it into camera video and aircraft information. Then, the camera video is stored in the video DB 215 (step S6 in FIG. 5). Furthermore, while storing the camera video, the video management device 212 distributes the camera video being stored to the aircraft information display terminal 221. Then, the aircraft information display terminal 221 makes it possible to display the camera video on the aircraft information display monitor 222 at any time.

[0036] Next, the video management device 212 calculates the shooting center position Go and the shooting range 11 from the aircraft information and altitude information. Then, the information on the shooting center position Go and the shooting range 11 is added to the aircraft information and transmitted to the aircraft information management device 213 as additional aircraft information, which is then stored in the additional aircraft DB 216 (step S7 in FIG. 5). The data structure of the additional aircraft information registered in this way in the additional aircraft DB 216 is as shown in FIG. 8. In addition to the aircraft information acquired from the on-board station X shown above, the time when the video management device 212 received the data, the shooting center position Go, and the shooting range 11 are added.

[0037] A method for calculating the shooting center position Go and the shooting range 11 in this case will be described with reference to Fig. 15. As shown in Fig. 15, in a coordinate system having due north, due east, and vertically downward directions as axes centered on the aircraft position O (LatO, LonO), when the reference attitude of the aircraft 1 is set to a state in which the nose is flying horizontally with due north, a nose direction vector of the aircraft 1 is calculated from information on the azimuth angle, pitch angle, and roll angle from the reference attitude. Next, when the reference attitude of the camera 2 is set to point toward the nose of the aircraft and the shooting image is taken so that it is horizontal, the nose direction vector is rotated with respect to the nose direction vector from information on the pan angle, tilt angle, and camera roll to calculate a shooting direction vector.

[0038] Next, a shooting range vector for each of the four corner points a(0,0), b(0,h), c(w,0), and d(w,h) of the shooting range 11 on the image is calculated from the shooting direction vector, the horizontal length, the vertical length, and the focal length information. The shooting direction vector passes through the shooting center point o(w / 2,h / 2) on the image. Next, the intersection point Go with the shooting center vector V1 and the ground surface is calculated from the shooting center vector V1 and the altitude information of the altitude DB 214, and the intersection point Go is the shooting center position Go, and the latitude and longitude of the shooting center position Go are determined. In addition, the shooting range 11 is determined from the latitude and longitude of the four corners of the image by calculating the intersection points Ga, Gb, Gc, and Gd (see FIG. 12A) between each shooting range vector and the ground surface from each shooting range vector and the altitude information.

[0039] Next, the aircraft information display terminal 221 generates a still image from the camera video (step S8 in FIG. 5). Then, based on the additional aircraft information and map information from the aircraft information management device 213, an image frame connecting the points Ga, Gb, Gc, and Gd on the map screen is displayed, and the still image is transformed so that the four corners a, b, c, and d of the shooting range 11 of the still image overlap with Ga, Gb, Gc, and Gd, and is displayed superimposed on the map using the map information from the map DB 223. Then, GIS-added aircraft information is created for displaying the shooting range 11 or the still image with the added map information, the aircraft position, and the shooting range 11 on the map screen (step S9 in FIG. 5).

[0040] Next, the user of the aircraft information display device 220 of ground station Y transmits instruction information to the on-board station X by operating the flow diagram of Fig. 5 and Fig. 9. First, the user checks the still image of the state of the aircraft 1 superimposed on the map displayed in the previous step on the aircraft information display monitor 222, and determines whether to perform the shooting instruction operation on the video screen or the map screen as the instruction screen (step S201 in Fig. 9).

[0041] Although an example of checking the state of the aircraft 1 using a still image has been shown, this is not limiting, and checking using a camera image received at any time is also conceivable. Also, the shooting instruction operation is performed on the image screen when, for example, the designated point can be clearly determined from the image screen. Also, when performing the operation on the map screen, for example, the designated point is clearly outside the still image, or the still image is too complex to be determined, etc.

[0042] Next, when the video screen is designated, the video screen, which is a still image as shown in FIG. 10A, is displayed, and the shooting instruction button on the video screen is clicked 1 (step S211 in FIG. 9, step S11 in FIG. 5). Next, as shown in FIG. 10B, a shooting instruction dialogue is displayed on a part of the video screen. Then, as shown in FIG. 11A, when an arbitrary designated point is clicked 2 from the still image of the shooting instruction dialogue, a white circle is displayed on the video screen at that point (step S212 in FIG. 9, step S12 in FIG. 5). Next, by clicking 3 the registration button as shown in FIG. 11A, the designated point is registered, and instruction information as shown in FIG. 16 is transmitted to the on-board station X (step S213 in FIG. 9, step S14 in FIG. 5).

[0043] A specific method for calculating the designated point when it is specified from the video screen in this way will be described with reference to the flow diagram in Fig. 14. First, additional aircraft information is obtained from the additional aircraft DB 216 shown in Fig. 8 (step S301 in Fig. 14). Next, a designated direction vector V2 is calculated from the designated point P on the screen (see Fig. 15) and the additional aircraft information (step S302 in Fig. 14). Next, an intersection point Pg in real space is calculated from the designated direction vector V2 and altitude information in the altitude DB 214 (step S303 in Fig. 14). Next, the intersection point Pg is registered as the designated point Pg, and the latitude and longitude information is registered as designation information and transmitted to the on-board station X (step S304 in Fig. 14).

[0044] In this case, the relationship between the designated point P on the image and the designated point Pg in real space is as shown in Figure 15. Using the additional aircraft information, a designated direction vector V2 for the coordinates of an arbitrary designated point P(x, y) on the image can be calculated, and the latitude and longitude information of the designated point Pg(LatP, LonP), which is the intersection with real space, can be calculated from the designated direction vector V2 and altitude information.

[0045] In addition, the calculation of the instruction direction vector V2 can be performed from the instruction point P(x, y) specified on the screen using the shooting direction vector, the length and width of the image sensor, and information on the focal length, in the same manner as the calculation of the shooting range vector described above.

[0046] Then, if the user selects to specify from the map screen in step S201, a map screen is displayed with the position of the aircraft 1 and the shooting range 11 superimposed on the map as shown in Fig. 12A, and the user clicks 4 on the shooting instruction button on the map screen (step S221 in Fig. 9, step S10 in Fig. 5). Next, as shown in Fig. 12B, when the user clicks 5 on an arbitrary designated point on the map screen, a pin mark is displayed on the map screen at that location (step S222 in Fig. 9, step S12 in Fig. 5).

[0047] Next, as shown in Fig. 13A, the designated point is registered by clicking the registration button 6, and instruction information is transmitted to the airborne station X (step S223 in Fig. 9, step S14 in Fig. 5). When registering a designated point on a map screen, the latitude and longitude of the designated point are obviously clear, and the latitude and longitude are registered as the designated point.

[0048] Next, positional relationship information with respect to the indicated point is calculated based on the camera image and additional aircraft information that change from moment to moment, and the relationship is displayed on a video screen such as that shown in FIG. 11B or a map screen such as that shown in FIG. 13B (step S202 in FIG. 9, step S13 in FIG. 5).

[0049] At this time, the instruction information transmitted to the airborne station X has a data structure as shown in Fig. 16. Specifically, the instruction information is composed of the time information at which the instruction information is registered at the ground station Y as the designated time, and the latitude and longitude information of the designated point registered in the shooting instruction as the designated point information.

[0050] 10 to 13, the instruction information transmitted from the ground station Y is received by the first transmission device 116 of the on-board station X (step S15 in FIG. 4). Next, the received instruction information is transmitted to the aircraft information conversion device 111. Next, the aircraft information conversion device 111 transmits the received instruction information to the photographing position calculation device 121 of the photographing position indication device 120.

[0051] Next, the photographing position calculation device 121 calculates the photographing range 11 currently being photographed (the calculation of the photographing range 11 is the same as that described above, and therefore the explanation will be omitted) from the aircraft information and altitude information in the altitude DB 124, and if the instruction point of the received instruction information is included in the photographing range 11, it delivers an instruction video indicating the instruction point at that position to the image superimposing device 122. If it is not included in the photographing range 11, it calculates the direction of the instruction point relative to the photographing direction, and delivers an instruction video indicating the direction information to the image superimposing device 122 (step S16 in FIG. 4).

[0052] The instruction image for drawing the instruction point of the photographing position calculation device 121 at this time will be described with reference to the flow diagrams of Fig. 19 and Fig. 20. First, it is determined whether instruction information has been received (step S411 in Fig. 19). At the same time, if instruction information has been received (step S421 in Fig. 19), the instruction information is updated (step S422 in Fig. 19). Then, if instruction information has not been received, step S411 is repeated.

[0053] Furthermore, when instruction information is received, aircraft information is acquired from the aircraft information conversion device 111 (step S412 in FIG. 19). Next, the shooting direction vectors of each of the four corners of the shooting range 11 currently being shot are calculated from the aircraft information, and compared with the altitude information in the altitude DB 124 to calculate the intersection with the ground surface (step S413 in FIG. 19). Note that this calculation method is the same as in the case described above, and therefore description thereof will be omitted. Next, it is determined whether or not the instruction point is located within the shooting range 11 (step S414 in FIG. 20).

[0054] If it is determined that the designated point is not within the shooting range 11, a designated direction vector is calculated (step S441 in FIG. 20). Next, an orthogonal projection vector onto the shooting plane is calculated, and an arrow is drawn on the designated image as a mark indicating the shooting direction of the designated point (step S442 in FIG. 20, see the arrow of the shooting position calculation device 121 in FIG. 18). If it is determined that the designated point is within the shooting range 11, a designated direction vector is calculated, and a circle is drawn on the designated image as a mark at the designated point (step S431 in FIG. 20, see the circle of the shooting position calculation device 121 in FIG. 17). Next, the designated image is transmitted to the image superimposition device (step S415 in FIG. 20).

[0055] Next, the image superimposing device 122 superimposes the instruction image received from the photographing position calculation device 121 onto the camera image delivered from the aircraft information output device 110 (step S3 in FIG. 4; see the camera image of the aircraft information output device 110 in FIGS. 17 and 18), thereby displaying the instruction camera image with the instruction information displayed on the camera image on the photographer's monitor 123. Then, the camera operator can confirm the photographing instruction (see step S17 in FIG. 4, and the photographer's monitor 123 in FIGS. 17 and 18).

[0056] The designated camera image created by the image superimposing device 122 at this time is image information to be superimposed on the camera image as shown in Figures 17 and 18. When the shooting location is included within the shooting range 11, as shown in Figure 17, a designated image of a circle mark as a mark of the designated location of the shooting position calculation device 121 shown in step S431 of Figure 20 is superimposed on the camera image of the aircraft information output device 110 and transmitted to the photographer monitor 123 for display. When the shooting location is not included within the shooting range 11, as shown in Figure 18, an arrow as a mark indicating the shooting direction of the designated location in step S121 of Figure 20 is calculated as follows.

[0057] If the instruction direction vector calculated above is v, the shooting instruction direction is expressed by the orthogonal projection vector h obtained by orthogonally projecting the instruction direction vector onto the image plane. If the shooting direction vector indicating the direction of the shooting center is Los, the orthogonal projection vector h can be calculated by the following formula.

[0058]

number

[0059] The drawing position is determined by calculating the angle formed by the dot product of vector y passing through point y(w / 2,0) with the shooting center point as the origin and orthogonal projection vector h. Then, the arrow indication image of the shooting position calculation device 121 can be superimposed on the camera image of the aircraft information output device 110 and displayed as the indication camera image on the photographer's monitor 123.

[0060] As a result, the instruction information displayed on the photographer's monitor 123 becomes as shown in Fig. 21. The camera operator can operate camera 2 while checking the screen of the instruction camera image, which actively changes from Fig. 21A to Fig. 21B or Fig. 21C in real time, and can therefore photograph the point instructed by ground station Y at any time based on the camera operator's shooting priority order. Also, as shown in step S202 of Fig. 9 above, ground station Y is constantly receiving camera images and GIS-equipped aircraft information, so that the aircraft information display monitor 222 also displays a screen such as that shown in Fig. 21, just like the photographer's monitor 123.

[0061] As a result, the image and mark displayed on the photographer's monitor 123 of the airborne station X are also displayed in the same way on the aircraft information display monitor 222 of the ground station Y, so that the airborne station X and the ground station Y can confirm the shooting position with each other, and therefore the ground station Y, which is the instructing party, can accurately instruct the desired instruction point, and the airborne station X, which is the receiving party, can accurately determine the instruction point.

[0062] According to the aircraft information sharing system of the first embodiment configured as described above, An airborne station with an aircraft equipped with a camera An aircraft information sharing system including a ground station that transmits and receives information to and from the on-board station, The on-board station outputs the camera image of the camera and aircraft information for obtaining the latitude and longitude of the camera image to the ground station; the ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and altitude information, which is altitude data linked to latitude and longitude, and displays an instruction screen by superimposing the calculated latitude and longitude on map information, and also specifies the latitude and longitude of an instruction point for shooting from the instruction screen and transmits the calculated latitude and longitude to the on-board station; The on-board station displays the designated point by superimposing it on the camera image based on the aircraft information and the altitude information. The designated location specified by the ground station can be superimposed on the camera image, allowing active and accurate confirmation. Therefore, shooting can be performed while checking the designated location.

[0063] In addition, the ground station displays the designated point on the camera image by superimposing it thereon, At the ground station, the designated point is displayed superimposed on the camera image, allowing both parties to confirm the designated point, enabling the airborne station and ground station to accurately and actively indicate and determine the designated point.

[0064] Furthermore, the on-board station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the designated point is included in the shooting range, it displays a mark indicating the designated point superimposed on the camera image, and if the latitude and longitude of the designated point is not included in the shooting range, it displays a mark indicating the direction of the designated point superimposed on the camera image. Since the on-board station can optimally display the designated point depending on whether it is within the photographing range or outside the photographing range, the designated point can be confirmed more accurately.

[0065] Furthermore, the ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the designated point is included in the shooting range, displays a mark indicating the designated point superimposed on the camera image, and if the latitude and longitude of the designated point is not included in the shooting range, displays a mark indicating the shooting direction of the designated point superimposed on the camera image. The ground station can optimally display the indicated point depending on whether it is within the shooting range or outside the shooting range, so that the indicated point can be confirmed more accurately.

[0066] In addition, the ground station can indicate the point on a video screen or map screen that displays either a video screen consisting of a still image of the camera video with map information superimposed thereon, or a map screen that displays the shooting range superimposed on a map, thereby increasing versatility.

[0067] As shown in FIG. 22, an example of hardware of the photographing position indication device 120, the aircraft information output device 110, the aircraft information distribution device 210, and the aircraft information display device 220 is composed of a processor 100 and a storage device 101. Although the storage device is not shown, it includes a volatile storage device such as a random access memory, and a non-volatile auxiliary storage device such as a flash memory. Also, instead of the flash memory, an auxiliary storage device such as a hard disk may be provided. The processor 100 executes a program input from the storage device 101. In this case, the program is input to the processor 100 from the auxiliary storage device via the volatile storage device. Also, the processor 100 may output data such as a calculation result to the volatile storage device of the storage device 101, or may store the data in the auxiliary storage device via the volatile storage device.

[0068] Although the present application describes exemplary embodiments, the various features, aspects, and functions described in the embodiments are not limited to application to a particular embodiment, but may be applied to the embodiments alone or in various combinations. Therefore, countless modifications not illustrated are expected within the scope of the technology disclosed in this application, including, for example, modifying, adding, or omitting at least one component. [Explanation of symbols]

[0069] 1 Aircraft, 2 Camera, 10 Aircraft Information Sharing System, 11 Shooting range, 100 processor, 101 storage device, 110 aircraft information output device, 111 aircraft information conversion device, 112 aircraft information output device, 113 camera vibration isolation device, 114 video switcher, 115 aircraft information superimposition device, 116 first transmission device, 117 static parameter DB, 120 shooting position indication device, 121 shooting position calculation device, 122 image superimposition device, 123 photographer monitor, 124 altitude DB, 210 aircraft information distribution device, 211 second transmission device, 212 video management device, 213 Aircraft information management device, 214 Altitude DB, 215 Video DB, 216 additional aircraft DB, 220 aircraft information display device, 221 aircraft information display terminal, 222 aircraft information display monitor, 223 map DB, X on-board station, Y ground station, W Satellite or wireless link.

Claims

1. An airborne station with an aircraft equipped with a camera An aircraft information sharing system including a ground station that transmits and receives information to and from the on-board station, The on-board station outputs the camera image of the camera and aircraft information for obtaining the latitude and longitude of the camera image to the ground station; the ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and altitude information, which is altitude data linked to latitude and longitude, and displays an instruction screen by superimposing the calculated latitude and longitude on map information, and also specifies the latitude and longitude of an instruction point for shooting from the instruction screen and transmits the calculated latitude and longitude to the on-board station; The on-board station displays the designated point by superimposing it on the camera image based on the aircraft information and the altitude information, the ground station displays, as the instruction screen, either a video screen consisting of a still image of the camera image on which map information is superimposed, or a map screen on which the shooting range is superimposed and displayed; The on-board station calculates the latitude and longitude of the imaging range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the designated point is included in the imaging range, displays a mark indicating the designated point superimposed on the camera image, and if the latitude and longitude of the designated point is not included in the imaging range, displays a mark indicating the direction of the designated point superimposed on the camera image, An aircraft information sharing system in which the ground station calculates the latitude and longitude of the shooting range of the camera image from the camera image, the aircraft information, and the altitude information, and if the latitude and longitude of the indicated point is included in the shooting range, it superimposes a mark indicating the indicated point on the camera image, and if the latitude and longitude of the indicated point is not included in the shooting range, it superimposes a mark indicating the shooting direction of the indicated point on the camera image.

2. The aircraft information sharing system according to claim 1 , wherein the ground station displays the designated point by superimposing it on the camera image.

Images