Generation apparatus, generation method, and generation program

Abstract

The process of generating an appropriate map from images taken of the Earth's surface. [Solution] A generating device having a processor that executes a program and a storage device that stores the program stores terrain data that models the terrain having location information of each point on the terrain, first image data obtained by imaging a subject which is part of the terrain from a first imaging point on the terrain, and location information of the first imaging point. The generating device performs a determination process to determine a destination location to which to assign information of a specific pixel corresponding to the line of sight vector from the pixel group constituting the first image data, based on the direction of the line of sight vector from the first imaging point to the subject, and a generation process to generate map data corresponding to the terrain by assigning the information of the specific pixel determined by the determination process to the destination location.

Description

[Technical Field]

[0001] The present invention relates to a data generation device, a data generation method, and a data generation program. [Background technology]

[0002] Imaging the natural environment (downward) using satellites or drones is difficult depending on weather conditions, making frequent data acquisition challenging. As a means to compensate for this, or as a new method, it is conceivable to generate maps representing the state of the natural environment, such as vegetation, using imaging devices installed on moving objects such as railways.

[0003] For example, Patent Document 1 below discloses an overhead wire position measuring device that performs non-contact three-dimensional position measurement of overhead wires other than trolley wires using an inspection vehicle. This overhead wire position measuring device includes first and second three-dimensional measuring devices, first and second cameras, a region of interest setting unit that sets at least one region of interest based on angle data and distance data generated by each of the first and second three-dimensional measuring devices, an image processing unit that performs image processing to mask images outside the region of interest for each of the first and second image data generated by the first and second cameras, respectively, and stores the first and second image data in memory, an overhead wire extraction unit that extracts overhead wires that satisfy predetermined conditions from two-dimensional images represented by each of the stored first and second image data, and an overhead wire position calculation unit that calculates the position of the overhead wire relative to a railway vehicle based on the coordinates of the overhead wire extracted by the overhead wire extraction unit.

[0004] Furthermore, Patent Document 2 below discloses a method for estimating illumination spectra. This estimation method calculates illumination spectra based on weather information. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2014-169939 [Patent Document 2] International Publication No. WO2012 / 86658 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] Conventional cameras installed on railway vehicles primarily have a frontal field of view, limiting the area they can cover for map generation. Furthermore, observing plants requires observing in the infrared region while also observing in the visible region with high wavelength resolution, necessitating the analysis of spectral information.

[0007] The present invention aims to generate appropriate maps from images of the Earth's surface taken from the Earth's surface. [Means for solving the problem]

[0008] A generating apparatus that represents one aspect of the invention disclosed in this application is a generating apparatus having a processor that executes a program and a storage device that stores the program, wherein the storage device stores terrain data that models the terrain having location information of each point on the terrain, first image data obtained by imaging a subject which is part of the terrain from a first imaging point on the terrain, and location information of the first imaging point, and the processor is characterized by performing a determination process to determine an assignment destination location to which to assign information of a specific pixel corresponding to the line of sight vector from a group of pixels constituting the first image data, based on the direction of the line of sight vector from the first imaging point to the subject, and a generating process to generate map data corresponding to the terrain by assigning the information of the specific pixel determined by the determination process to the assignment destination location. [Effects of the Invention]

[0009] According to a typical embodiment of the present invention, a suitable map can be generated from an image of the Earth's surface taken from the Earth's surface. Problems, configurations, and effects other than those described above will be clarified by the following description of the embodiments. [Brief explanation of the drawing]

[0010] [Figure 1] FIG. 1 is an explanatory diagram showing an example of map generation. [Figure 2] FIG. 2 is an explanatory diagram showing a configuration example of a map generation system. [Figure 3] FIG. 3 is a block diagram showing an example of the hardware configuration of a computer. [Figure 4] FIG. 4 is an explanatory diagram showing a configuration example of a moving body. [Figure 5] FIG. 5 is an explanatory diagram showing an example of planning information. [Figure 6] FIG. 6 is a flowchart showing an example of a self-position estimation processing procedure of a moving body. [Figure 7] FIG. 7 is an explanatory diagram showing the relationship between a world coordinate system and each variable. [Figure 8] FIG. 8 is an explanatory diagram showing a vehicle body tilt angle. [Figure 9] FIG. 9 is a block diagram showing a functional configuration example of a generation device. [Figure 10] FIG. 10 is a flowchart showing an example of a map data generation processing procedure by a generation device. [Figure 11] FIG. 11 is an explanatory diagram showing an example of a determination processing by a determination unit. [Figure 12] FIG. 12 is an explanatory diagram showing an example of imaging information. [Figure 13] FIG. 13 is an explanatory diagram showing an example of an input / output screen. [Figure 14] FIG. 14 is an explanatory diagram showing an example of a display screen displayed by an output unit.

MODE FOR CARRYING OUT THE INVENTION

[0011] <FIG. 1 Map Generation Example> Figure 1 is an explanatory diagram showing an example of map generation. In Figure 1, the world coordinate system 100 is a coordinate system that defines the real world. The X, Y, and Z axes intersect at the origin O and are mutually orthogonal. The XY plane corresponds to the ideal Earth surface at altitude 0, and points on the XY plane are represented by the distance from the origin O corresponding to the latitude (hereinafter referred to as the Y distance) and the distance from the origin O corresponding to the longitude (hereinafter referred to as the X distance). The Z axis indicates the vertical direction. The actual Earth surface 101 has relief in the Z-axis direction relative to the XY plane.

[0012] The moving object 102 moves along the track 103. In this example, the moving object 102 is a train, and the track 103 is a railway track. The moving object 102 has imaging devices on both sides. The optical axes of the imaging devices are perpendicular to the direction of travel of the moving object 102 and parallel to the XY plane. The imaging devices take images at predetermined timings and generate a series of image data sequences 110. The image data sequence 110 may also be video data. The imaging device may also be a camera that generates image data by spectrally separating the light emitted by the subject into wavelengths and taking images, such as a multispectral camera or a hyperspectral camera.

[0013] Map generation is a process that generates map data 112 by reconstructing terrain data 111 using image data sequences 110 generated by the imaging device while the mobile object 102 is moving. Terrain data 111 is, for example, a DEM (Digital Elevation Model) or DSM (Digital Surface Model) that digitizes the ground surface 101, and is a 3D model in which the ground surface 101 is divided into equally spaced squares called a mesh, and the elevation value of the center point of each square.

[0014] In Figure 1, the moving body 102, which moves along the track 103, is depicted as a train running on a railway track. However, the track 103 could be a suspended monorail, with the moving body 102 suspended from the monorail. Furthermore, if the route is predetermined, the track 103 could be a road (either a public road or a highway). In this case, the moving body 102 would be an automobile.

[0015] <Figure 2: Map Generation System> Figure 2 is an explanatory diagram showing an example configuration of a map generation system. The map generation system 200 includes a management device 201, a generation device 202, and a distribution device 203, at least one of which is the generation device 202, and a mobile unit 102. The management device 201, generation device 202, distribution device 203, and mobile unit 102 are connected to each other via a network such as the Internet, LAN (Local Area Network), or WAN (Wide Area Network).

[0016] The control device 201 manages the planning information 210. The planning information 210 is information that manages the operation of the mobile body 102 and the position and height of the orbit 103 in the world coordinate system 100. The control device 201 transmits the planning information 210 to the generation device 202.

[0017] The generation device 202 generates map data 112 by reconstructing terrain data 111 using the image data sequence 110 generated by the imaging device while the mobile body 102 is moving.

[0018] The distribution device 203 is, for example, an existing weather server which is an external service, and it distributes weather information 230 to the generation device 202.

[0019] The mobile body 102 has an imaging device 220 and moves along the track 103. The imaging device 220 is positioned so that its optical axis is directed horizontally from the side of the mobile body 102. The mobile body 102 transmits the image data sequence 110 to the generation device 202.

[0020] <Figure 3: Example of computer hardware configuration> Figure 3 is a block diagram showing an example of the hardware configuration of a computer. Computer 300 includes a processor 301, a storage device 302, an input device 303, an output device 304, and a communication interface (communication IF) 305. The processor 301, storage device 302, input device 303, output device 304, and communication IF 305 are connected by a bus 306. The processor 301 controls the computer 300. The storage device 302 serves as the work area for the processor 301. The storage device 302 is a non-temporary or temporary recording medium that stores various programs and data. Examples of storage devices 302 include ROM (Read Only Memory), RAM (Random Access Memory), HDD (Hard Disk Drive), and flash memory. The input device 303 takes data in. Examples of input devices 303 include a keyboard, mouse, touch panel, numeric keypad, scanner, microphone, and sensor. The output device 304 outputs data. Output devices 304 include, for example, displays, printers, and speakers. The communication IF 305 connects to the network 204 and sends and receives data.

[0021] The computer 300 is implemented in the management device 201, the generation device 202, the distribution device 203, and the mobile unit 102.

[0022] <Figure 4: Example of the configuration of the mobile unit 102> Figure 4 is an explanatory diagram showing an example configuration of the mobile body 102. (A) is a schematic partial side cross-sectional view of the mobile body 102, and (B) is a front view of the mobile body 102. The local coordinate system 400 is the coordinate system that defines the direction of the mobile body 102. The x, y, and z axes intersect at the origin o and are orthogonal to each other.

[0023] The x-axis is the normal to the front of the moving body 102 and indicates the direction of movement of the moving body 102. The y-axis is the normal to the side of the moving body 102 and defines the optical axis of the imaging device 220 (side imaging device 407). The z-axis is the normal to the top surface of the moving body 102 and indicates the height direction of the moving body 102. α is the rotation angle around the z-axis and is called the yaw angle α. Its direction is also called the yaw direction α. ​​β is the rotation angle around the y-axis and is called the pitch angle β. Its direction is also called the pitch direction β. γ is the rotation angle around the x-axis and is called the roll angle γ. Its direction is also called the roll direction γ.

[0024] The mobile unit 102 includes a control unit 401, a position and attitude calculation unit 402, a communication unit 403, a clock 404, a counter 405, a roll angle compensation mechanism 406, a side imaging device 407, a front imaging device 408, wheels 410, a rotation angle meter 411, and an on-board unit 412. In addition, a ground unit 420 is installed on the track 103.

[0025] The control unit 401 controls various processes performed by the mobile body 102. The control unit 401 is realized by causing the processor 301 to execute a program stored in the memory device 302.

[0026] The position and attitude calculation unit 402 calculates the position and attitude of the moving body 102 in the world coordinate system 100. Attitude refers to the yaw angle α, pitch angle β, and roll angle γ of the moving body 102. The position and attitude calculation unit 402 is realized by causing the processor 301 to execute a program stored in the memory device 302.

[0027] The communication unit 403 transmits data to the generation device 202 via the network 204 and receives data (for example, planning information 210) from the management device 201. The communication unit 403 is implemented by the communication IF 305.

[0028] The clock 404 measures the current time. The counter 405 counts the pulses output from the rotation angle meter 411. The roll angle compensation mechanism 406 compensates for the displacement of the lateral imaging device 407 in the roll direction γ.

[0029] The lateral imaging device 407 is an imaging device whose optical axis is set in the y-axis direction. The lateral imaging devices 407 are installed on both the right and left sides in the direction of travel. The right-side lateral imaging device 407 will be referred to as the right-side lateral imaging device 407, and the left-side lateral imaging device 407 will be referred to as the left-side lateral imaging device 407.

[0030] The forward-facing imaging device 408 is installed in front of the moving body 102. The forward-facing imaging device 408 is installed with its optical axis directed toward the ground surface at a predetermined angle from the x-axis direction. Therefore, the forward-facing imaging device 408 images the trajectory 103 in the direction of travel. The side imaging device 407 and the forward-facing imaging device 408 are collectively referred to as the imaging device 220.

[0031] The wheel 410 rotates on the track 103, moving the mobile body 102 in the direction of travel. The rotation angle meter 411 is mounted on the rotation axis of the wheel 410 and measures N during one rotation of the wheel 410. ppr Each pulse is output to counter 405.

[0032] The on-board unit 412 is capable of contactless communication with the ground unit 420. When the on-board unit 412 is positioned directly above the ground unit 420, it receives the position information of the ground unit 420 that it holds from the ground unit 420.

[0033] <Figure 5 Planning Information 210> Figure 5 is an explanatory diagram showing an example of planning information 210. The planning information 210 has the following fields: number 500, distance traveled 501, command 502, command argument 503, X position 504, Y position 505, elevation 506, pitch angle 507, roll angle 508, yaw angle 509, reference 510, (preferably) section type 511, and radius of curvature 512.

[0034] The number 500 holds the number k as an ascending integer starting from k=0. Number k=0 is the index indicating the starting point of mobile object 102. As the number k increases, the distance traveled L increases.

[0035] The distance traveled (501) holds the distance traveled (L) from the starting point (k=0) of the mobile object 102 corresponding to number k.

[0036] Command 502 holds, as a value, a command ("Start imaging" or "Stop imaging") that the control unit 401 instructs the imaging device 220 to perform at the position of the moving body 102 corresponding to number k.

[0037] Command argument 503 holds the argument of command 502 as its value. "ALL" means the right-side imaging device 407, the left-side imaging device 407, and the front imaging device 408. "Camera #0" means the front imaging device 408, "Camera #1" means the right-side imaging device 407, and "Camera #2" means the left-side imaging device 407.

[0038] The X position 504 holds, as a value, the position corresponding to the longitude direction of world coordinate system 100, which corresponds to the number k.

[0039] The Y position 505 holds, as a value, the position corresponding to the latitude direction of world coordinate system 100, which corresponds to the number k.

[0040] The elevation of 506 holds the height in the Z-axis direction of world coordinate system 100, corresponding to the number k, as a value.

[0041] The pitch angle 507 holds, as a value, the pitch angle β of the moving object 102 at the position of the moving object 102 in world coordinate system 100 corresponding to number k (X position 504, Y position 505, elevation 506).

[0042] The roll angle 508 holds, as a value, the roll angle γ of the moving object 102 at the position of the moving object 102 in world coordinate system 100 corresponding to number k (X position 504, Y position 505, elevation 506).

[0043] The yaw angle 509 holds the value of the yaw angle α of the moving object 102 at its position in world coordinate system 100 corresponding to number k (X position 504, Y position 505, elevation 506).

[0044] Reference 510 indicates the presence or absence of a reference reflector at the location in world coordinate system 100 corresponding to number k (X position 504, Y position 505, elevation 506).

[0045] Section type 511 is information that distinguishes whether the section with the travel distance of 501 and number k is a straight line or an arc.

[0046] A radius of curvature of 512 is, as a value, the reciprocal of the curvature, which is an indicator of the degree of curvature when the section of travel distance 501 at number k is a circular arc.

[0047] <Figure 6: Example of the self-position estimation process for the mobile object 102> Figure 6 is a flowchart showing an example of the self-position estimation process for the mobile object 102. Initially, the planning information 210 is assumed to have an entry with number 500 where k=0, a column for number 500, and a column for distance traveled 501. The x-axis, which is the direction of travel of the mobile object 102, is also set, and the transformation matrix from the local coordinate system 400 to the world coordinate system 100 is also assumed to be set.

[0048] (Step S601) The mobile unit 102 starts operation. As the wheels 410 rotate when operation starts, the rotation angle meter 411 outputs a pulse to the counter 405 each time the wheels 410 complete one rotation. The counting result of the counter 405 is N wheel This is how it is written.

[0049] (Step S602) The control unit 401 determines the counting result of the counter 405 as N wheel The control unit 401 obtains the counting result N and calculates the distance traveled L. During operation, except when stopped at intermediate stations, the pulse output from the rotation angle meter 411 continues, so the control unit 401 calculates the counting result N wheel Each time it is updated, the mileage L is calculated. The mileage L is calculated using the following formula (1). In the following formula (1), D is the diameter of the wheel 410. Also, N ppris the number of pulses (ppr: pulse per revolution) output by the rotation angle meter 411 per wheel rotation.

[0050]

Number

[0051] (Step S603) The control unit 401 determines whether the travel distance L satisfies L < L k+1 . L k+1 is the value of the travel distance 501. If the travel distance L does not satisfy L < L k+1 (Step S603: No), the process proceeds to step S604. If the travel distance L satisfies L < L k+1 (Step S603: Yes), the process proceeds to step S606.

[0052] (Step S604) The control unit 401 increments k and proceeds to step S605.

[0053] (Step S605) The control unit 401 determines whether k = K. K is the value of the number 500 corresponding to the arrival point of the moving body 102. If k ≠ K (Step S605: No), the process returns to step S603. If k = K (Step S605: Yes), the self-position estimation process of the moving body 102 ends.

[0054] (Step S606) The position and orientation calculation unit 402 calculates the travel distance l within the section as l = L k+1 - L. The section is between the point of the travel distance L and the point of L k+1 .

[0055] (Step S607) The position and attitude calculation unit 402 determines whether the section type 511 of the travel distance l within the section is a straight line or an arc. If it is determined to be a straight line (step S607: straight line), the process proceeds to step S608. If it is determined to be an arc (step S607: arc), the process proceeds to step S609. Alternatively, the position and attitude calculation unit 402 may refer to the radius of curvature 512 and determine that it is an arc if a value exists.

[0056] (Step S608) The position and orientation calculation unit 402 performs linear interpolation processing and proceeds to step S605. That is, the control unit 401 estimates its own position P in the current linear section. The own position P is calculated by the following equation (2). The own position P is a vector consisting of the X position and the Y position.

[0057]

number

[0058] (Step S609) The position and attitude calculation unit 402 calculates the amount of change in the azimuth angle θ of the tangent to the orbit 103.

[0059] [Figure 7 Relationship between World Coordinate System 100 and each variable] Figure 7 is an explanatory diagram showing the relationship between the world coordinate system 100 and each variable. In Figure 7, the center point of the arc in the k-th arc interval is C. k , C k and the kth self-position P k The distance to R k C k From the arc, the vector pointing to the self-position P is r φ,k In that case, r φ,k This is expressed by the following equation (3). Here, R k This is equal to the value stored in the k-th radius of curvature, 512.

[0060]

number

[0061] In equation (3) above, φ on the right-hand side is the polar angle, which is the angle with respect to the X-axis (eastward direction is positive), with counterclockwise being the positive direction. The relationship with the azimuth angle θ is expressed by equation (4) below.

[0062] φ = 90° - θ···(4)

[0063] Therefore, the change in the azimuth angle θ in the circular arc section, Δθ, is expressed by the following equation (5).

[0064]

number

[0065] (Step S610) The position and orientation calculation unit 402 performs arc interpolation processing and proceeds to step S605. That is, the control unit 401 estimates its own position P in the current arc section. The self-position P is calculated by the following equation (6).

[0066]

number

[0067] The r on the right side of equation (6) above k,φk+Δφ In the above equation (3), φ is replaced with φ k This is the vector when +Δφ is substituted.

[0068] This sets the values ​​for the X position 504 and Y position 505 in the planning information 210. The position and attitude calculation unit 402 calculates the travel distance L at k. k Alternatively, the altitude of 506 can be calculated using an altimeter. This sets the altitude value to 506.

[0069] Furthermore, the position and attitude calculation unit 402 calculates the pitch angle β by the derivative of the elevation 506, the yaw angle α from the azimuth angle θ of the tangent to the track 103, and the roll angle γ from the relationship between the curvature of the track 103 and the rail inclination angle determined at the time of track design.

[0070] [Figure 8: Vehicle body tilt angle] Figure 8 is an explanatory diagram showing the vehicle body tilt angle. When the moving body 102 has a roll angle compensation mechanism 406, the vehicle body tilt angle is added to the roll angle γ. The vehicle body tilt angle is the angle obtained by subtracting the known rail tilt angle from the roll angle γ.

[0071] Approximating the interval with a circular arc saves the table size of the planning information 210 and improves the accuracy of the self-localization process. For example, the horizon distance from a viewpoint height of 4m is approximately 7.5km. To achieve a resolution equivalent to satellite imagery at this distance (let's say 15m as an example), a side-viewing device 407 with a field of view of approximately 0.11 degrees per pixel is required. At the same time, the estimation accuracy of the azimuth angle θ must also be achieved on the same order of magnitude. With polygonal approximation (linear approximation), it would be necessary to divide the 90-degree curve interval into 810 rows and record them in the planning information 210. In contrast, when treating a curved interval as a circular arc, only two rows are needed for the start and end points of the arc, compressing the data volume to approximately 1 / 400th. This effect is particularly pronounced in intervals where curves are continuous.

[0072] The mobile unit 102 transmits the generated plan information 210 to the management device 201. The management device 201 sets the command 502 and command argument 503 through user operation. In this way, the plan information 210 shown in Figure 5 is generated. The management device 201 transmits the generated plan information 210 to the generation device 202.

[0073] <Figure 9: Example of Functional Configuration of Generation Device 202> Figure 9 is a block diagram showing an example of the functional configuration of the generation device 202. The generation device 202 includes an acquisition unit 901, a recording unit 902, a correction unit 903, a determination unit 904, an acquisition unit 905, a generation unit 906, and an output unit 907. These are realized by having the processor 301 execute a program stored in the storage device 302.

[0074] The acquisition unit 901 inputs data. Specifically, for example, it inputs image data sequences 110 from the mobile device 102, planning information 210 from the management device 201, and weather information 230 from the distribution device 203. The acquisition unit 901 also inputs topographic data 111 from an external source.

[0075] The recording unit 902 records the data input by the acquisition unit 901 into the storage device 302. As a result, the storage device 302 stores terrain data 111, planning information 210, and weather information 230.

[0076] The correction unit 903 corrects the position of the moving body 102 (X position 504, Y position 505, elevation 506) in the planning information 210.

[0077] The determination unit 904 determines a location on the terrain data 111 to which information about pixels in the image data sequence 110 will be allocated. Specifically, for example, the determination unit 904 determines a location to which information about a specific pixel corresponding to the line of sight vector from the imaging point of the moving object 102 to the subject (location G) will be allocated.

[0078] The acquisition unit 905 acquires spectral information. Spectral information includes the light source spectrum, luminance spectrum, and reflectance spectrum. The acquisition unit 905 also acquires information about a specific pixel based on the light source spectrum.

[0079] The generation unit 906 generates map data based on spectral information. Specifically, for example, the generation unit 906 generates map data corresponding to the terrain by assigning the information of a specific pixel determined by the determination unit 904 to the assigned point G. For example, the generation unit 906 generates reflectance spectral map data or vegetation evaluation map data.

[0080] Reflectance spectrum map data is map data that uses spectral information of the reflectance of the Earth's surface at various locations to plot the intensity of reflectance and its geographical distribution for specific wavelength bands (or multiple bands) required for the analysis.

[0081] Vegetation evaluation map data is map data that plots the distribution of vegetation evaluation indices at various locations, calculated based on luminance spectral information or reflectance spectral information. Known indices and their calculation methods, such as NDVI (Normalized Difference Vegetation Index), chlorophyll concentration, and water stress indices, are used to calculate these vegetation evaluation indices.

[0082] The output unit 907 outputs the map data generated by the generation unit 906 in a displayable format. Specifically, for example, the output unit 907 may display the map data, or it may transmit the map data to another computer that displays the map data.

[0083] <Figure 10: Example of map data generation process procedure> Figure 10 is a flowchart showing an example of the map data generation process procedure by the generation device 202.

[0084] (Step S1001) The acquisition unit 901 acquires data. Specifically, for example, the acquisition unit 901 acquires terrain data 111 and planning information 210 from the management device 201, weather information from the distribution device 203, and image data sequence 110 from the mobile device 102. The recording unit 902 records the data acquired by the acquisition unit 901 in the storage device 302.

[0085] (Step S1002) The correction unit 903 corrects its own position P. Specifically, for example, position estimation errors occur due to fluctuations in the diameter D of the wheel 410 or skidding. To correct these errors, the correction unit 903 performs a correction of its own position P using known markers on the ground.

[0086] For example, the on-board unit 412 receives the number k and location information (X position, Y position, altitude) of the ground unit 420 while the mobile unit 102 is moving. The generation device 202 receives the number k and location information (X position, Y position, altitude) of the ground unit 420 from the mobile unit 102 via the acquisition unit 901. The generation device 202 then updates the planning information 210 with the location information (X position, Y position, altitude) of number k.

[0087] Furthermore, by comparing the trajectory 103 of the image data for number k captured by the forward imaging device 408 with the trajectory 103 of the image data for number k+1, the difference in roll angle between numbers k and k+1 is detected. The generation device 202 updates the roll angle 508 so that the difference in roll angle between numbers k and k+1 in the acquired planning information 210 matches the detected difference in roll angle between numbers k and k+1.

[0088] Note that step S1002 may be performed on the mobile unit 102 instead of the generation device 202. This allows the acquisition unit 901 to acquire the corrected plan information 210 in step S1001.

[0089] (Step S1003) The determination unit 904 determines the assignment of pixel information of the image data to point G on the terrain data 111 based on the monoplot method of photogrammetry. Specifically, for example, the determination unit 904 converts (translates and rotates) the coordinate values ​​of each vertex of the mesh constituting the terrain data 111 from the coordinate values ​​of the world coordinate system 100 to the coordinate values ​​of the local coordinate system 400. The determination unit 904 may also rectify (correct distortion) each image data of the image data sequence 110 acquired by the acquisition unit 901 using internally acquired calibration parameters.

[0090] [Figure 11 Example of decision processing] Figure 11 is an explanatory diagram showing an example of the decision processing by the decision unit 904. The y-axis of the local coordinate system 400 coincides with the optical axis 1000 of the lateral imaging device 407. For the sake of explanation, in Figure 11, the terrain data 111 also shows the ground surface 101, the moving object 102, the trajectory 103, and the lateral imaging device 407 in real space.

[0091] The viewing frustum 1101 indicates the imaging range of the lateral imaging device 407. The viewing frustum 1101 has a front surface 1102 that is closer to the lateral imaging device 407 and a back surface 1103 that is further away from the lateral imaging device 407 than the front surface 1102. In other words, the content displayed from the front surface 1102 to the back surface 1103 becomes the image data that constitutes the image data sequence 110.

[0092] Let f(x,z) be the pixel of the image data at the front 1102 of the viewing frustum 1101. Let the line of sight vector of of be the vector extended from the camera origin o in the local coordinate system 400 to the pixel f(x,z).

[0093] The determination unit 904 extends the line-of-sight vector op and identifies the mesh that intersects with the terrain data 111, which is located beyond the foreground 1102 and is closest to the camera origin o (mesh 1104 in Figure 11), and its intersection point. Let a to d be the vertices that make up mesh 1104. Let g be the intersection point on mesh 1104 between mesh 1104 and the line-of-sight vector of. Mesh 1104 and intersection point g may be points inside or outside the frustum 1101, as long as they are on the far side in the y-axis direction from the foreground 1102.

[0094] The determination unit 904 associates pixel f(x,z) with intersection point g in the terrain data 111. In this way, the determination unit 904 searches for intersection point g for each pixel f(x,z) in the image data corresponding to the foreground 1102 and associates pixel f(x,z) with intersection point g.

[0095] Furthermore, the determination unit 904 performs an inverse transformation (translation and rotation) of the coordinate values ​​of intersection point g from the coordinate values ​​of the local coordinate system 400 to the coordinate values ​​of the world coordinate system 100. The intersection point g after the inverse transformation is designated as point G. As a result, the coordinate values ​​of the world coordinate system 100 are set for point G in the terrain data 111. In addition, various other information is set for point G.

[0096] [Figure 12 Example of imaging information] Figure 12 is an explanatory diagram showing an example of imaging information. For each location G, the imaging information 1200 includes location ID 1201, number 500, timestamp 1202, shooting location 1203, imaging conditions 1204, weather conditions 1205, wavelength information 1206, and adoption flag 1207.

[0097] Location ID 1201 is identification information that uniquely identifies location G. Number 500 is identification information that identifies the X position 504, Y position 505, and elevation 506 assigned to location G.

[0098] The timestamp 1202 is the date and time the image data containing the pixel to which pixel information is assigned at location G was captured, and is obtained from the metadata contained in the image data.

[0099] The imaging conditions 1204 are conditions set for imaging by the lateral imaging device 407, which generates image data having pixels to which pixel information has been assigned to point G, and are obtained from the metadata contained in the image data. Specifically, the imaging conditions 1204 are, for example, shutter speed, aperture, gain, and ISO sensitivity.

[0100] Weather conditions 1205 are weather information at the time of imaging specified by the timestamp 1202, and are acquired from the distribution device 203 by the acquisition unit 901. Specifically, weather conditions 1205 include, for example, weather, solar altitude, solar radiation, cloud cover, and visibility.

[0101] Wavelength information 1206 is the wavelength contained in the pixel of the image data assigned to location G, and is obtained from the metadata contained in the image data.

[0102] The adoption flag 1207 is a flag indicating whether or not to adopt the assignment of pixel information to location G. Specifically, for example, the determination unit 904 may make a decision based on the luminance spectrum included in the pixel information. For example, if the luminance spectrum is greater than or equal to a first threshold and less than or equal to a second threshold, the unit may decide to adopt (Y), and otherwise (for example, if it is less than the first threshold, there is insufficient light; if it is greater than the second threshold, it is saturated), the unit may decide not to adopt (N) the assignment of pixel information to location G.

[0103] Furthermore, the determination unit 904 determines whether or not location G is covered by the shadow of a building or tree, based on the solar altitude and the location of location G in the weather conditions. If location G is covered by the shadow, the determination unit 904 may decide not to assign pixel information to location G (N). The determination unit 904 may delete image data in which the number of pixels corresponding to location G that has been rejected exceeds a predetermined number. This reduces the capacity of the image data sequence 110 stored in the storage device 302. In addition, deleting such image data can suppress the deterioration of the quality of the image data sequence 110.

[0104] Furthermore, the determination unit 904 may accept or reject the assignment of pixel information to point G via user operation.

[0105] (Step S1004) The acquisition unit 905 acquires the light spectrum for each location G. The light spectrum includes the light source spectrum I(λ), the luminance spectrum A(λ), and the reflectance spectrum R(λ).

[0106] The light source spectrum I(λ) can be obtained using the following three methods. The method to be used is set in advance.

[0107] 1. Method of direct measurement A pyranometer with a bandwidth and wavelength resolution comparable to that of the lateral imaging device 407 is installed at the imaging point on the mobile body 102 or the orbit 103. The pyranometer measures and records the light source spectrum I(λ) in real time. The light source spectrum I(λ) measured by the pyranometer is transmitted to the generation device 202 via the network 204, along with a timestamp indicating the date and time of measurement.

[0108] 2. Method for imaging a reference reflector with known reflective properties. A reference reflector, such as a white reflector or checkerboard, with a known reflectance spectrum Rref(λ)[%] and shape pattern, is installed on track 103 (reference value 510 "present"). The reference reflector may be a white plate with a uniform known reflectance Rref(λ)[%] across its surface, or it may be a sign or facility wall surface with a standardized paint material. The side imaging device 407 images the reference reflector while the moving body 102 is moving.

[0109] The control unit 401 of the mobile unit 102 detects the shape pattern of the reference reflector from the image data captured of the reference reflector by pattern matching, and uses the luminance vector Aref(λ) of the pixels of the detected shape pattern and the known reflectance Rref(λ)[%] to calculate the light source spectrum I(λ) for each pixel of the image data using the following equation (7).

[0110] I(λ)=Aref(λ) / Rref(λ)...(7)

[0111] In this example, the light source spectrum I(λ) for each pixel is embedded in the metadata of the image data. Therefore, the acquisition unit 905 acquires the light source spectrum I(λ) for each point G to which the pixel information is assigned, from the metadata of each image data in the image data sequence 110.

[0112] 3. Methods for estimation from weather conditions, location, and time. Outdoor sunlight occurs when sunlight reaches the Earth's surface 101 after undergoing scattering and absorption processes by the atmosphere and clouds. Because different attenuations occur for each band during this process, the light source spectrum is determined by the distance traveled through the atmosphere (calculated from the solar altitude determined by latitude, longitude, date, and time) and the presence or absence of clouds. The path of sunlight roughly consists of a direct and a diffuse component, and by assigning weights to each component based on cloud cover, the light source spectrum I(λ) at the photographed point P can be obtained.

[0113] In this example, the light source spectrum I(λ) for each pixel is embedded in the metadata of the image data. Therefore, the acquisition unit 905 acquires the light source spectrum I(λ) for each point G to which the pixel information is assigned, from the metadata of each image data in the image data sequence 110.

[0114] Once the light source spectrum I(λ) is obtained by the above steps 1-3, the acquisition unit 905 determines the reflectance spectrum R(λ) by the following method. Specifically, for example, the luminance spectrum A(λ) of the light rays received by the image sensor of the side imaging device 407 is expressed by the following equation (8).

[0115] A(λ)=I(λ)·R(λ)·Stc(λ,L)···(8)

[0116] R(λ) is the reflectance spectrum [%] at each point on the ground surface 101 (i.e., each vertex of the mesh in the terrain data 111). Stc(λ,L) is the transmittance considering scattering and absorption along the path between the subject (point G) and the side imaging device 407.

[0117] Rearranging equation (8) for R(λ), we obtain equation (9) below.

[0118]

number

[0119] In equation (9) above, the unknown reflectance spectrum R(λ) can be determined from A(λ) observed by the image sensor of the side imaging device 407, the known I(λ), and Stc(λ,L).

[0120] Here, we will explain the transmittance Stc(λ,L). The transmittance Stc(λ,L) is a coefficient that indicates the amount of transmitted light after light rays have undergone scattering and absorption as they pass through the atmosphere. The scattering and absorption processes of light rays are typically known to be Rayleigh scattering and Mie scattering. The effects of these two processes differ depending on the relative sizes of the scattering material and the wavelength of the light ray. For example, in situations where there are few (or can be ignored) scattering materials such as raindrops or dust particles in the subject, the Rayleigh scattering coefficient σ for each wavelength λ is... R This is calculated using the following formula (10).

[0121]

number

[0122] Assuming the atmospheric density ρ is constant, the transmittance Stc(λ,L) for distance L is expressed by Lambert's law as shown in equation (11) below.

[0123]

number

[0124] In this case, N0 is the number density of scattered particles, and corresponds to the number of gas molecules present on a unit length of optical path. N0 is expressed by the following equation (12) using atmospheric density ρ, average molecular weight M (28.964 mol / g for the Earth's atmosphere), and Avogadro's number NA.

[0125]

number

[0126] From the above, it can be seen that the transmittance Stc(λ,L) decreases inversely proportional to the fourth power of the wavelength λ on the shorter wavelength side, and that the entire wavelength band is exponentially attenuated with respect to the distance L from the subject (point G).

[0127] In equation (10) above, n(λ) is the refractive index of light with wavelength λ, and depends on density and wavelength, but a typical value at Earth's surface 10¹ is n-1 = 280 × 10¹ -6 It is approximately ρ = 1.22 [kg / m 3 For λ=400[nm] and L=7.5[km], the transmittance Stc≈0.74 is not negligible. This effect changes depending on the distance L from the lateral imaging device 407 (i.e., the vertical direction within the camera's field of view), and is therefore appropriately corrected for each point G.

[0128] Since the distance L and positional relationship between the lateral imaging device 407 and point G are known from the planning information 210 and topographic data 111, the transmittance Stc(λ,L) is calculated. In addition, the light source spectrum I(λ) is obtained by the methods 1 to 3 above. Therefore, it is possible to solve for each point G in equation (8) for the reflectance spectrum R(λ). Note that if the transmittance Stc(λ,L) is ignored to simplify the calculation, Stc(λ,L) is excluded from equation (8).

[0129] (Step S1005) The generation unit 906 generates map data based on the information of the pixels assigned to each point G. Specifically, for example, the generation unit 906 generates monochrome map data, color map data, reflectance spectrum map data, and vegetation evaluation map data as map data based on the light source spectrum I(λ).

[0130] The information for the pixel assigned to location G may be the luminance spectrum A(λ) of the pixel from which location G was assigned, or the reflectance spectrum R(λ) obtained by equation (9) above. For example, when using the luminance spectrum A(λ) as the pixel information, the generation unit 906 converts the luminance components A(λa), A(λb), and A(λc) for wavelengths λa, λb, and λc of the pixel from which location G was assigned into luminance values ​​for the R, G, and B colors on the screen.

[0131] Alternatively, if the generation unit 906 uses the reflectance spectrum R(λ) as pixel information, it determines the luminance values ​​of R, G, and B based on the reflectance components R(λa), R(λb), and R(λc) for wavelengths λa, λb, and λc. For example, the generation unit 906 adjusts visibility by multiplying each color by an appropriate weight so that the larger the reflectance component R, the brighter it becomes.

[0132] Furthermore, the conversion from the wavelength λ component in the luminance or reflectance spectrum to luminance can be performed, for example, by a known wavelength component-RGB converter. The wavelength component-RGB converter may be a function that takes a spectrum and wavelength as input and outputs the luminance values ​​for each of the R, G, and B colors, or it may be a table that associates wavelength components with the luminance values ​​for each of the R, G, and B colors.

[0133] (Step S1006) The output unit 907 outputs the map data generated by the generation unit 906 in a displayable format. This completes the map data generation process by the generation device 202.

[0134] <Figure 13 Input / Output Screen> Figure 13 is an explanatory diagram showing an example of an input / output screen. The input / output screen 1300 is displayed on the management device 201 or the generation device 202. The input / output screen 1300 may also be displayed on a computer that can communicate with the management device 201 or the generation device 202.

[0135] The input / output screen 1300 displays terrain data 111, planning information 210, a cursor 1310, a first setting button 1301, a second setting button 1302, and an input completion button 1303.

[0136] When cursor 1310 selects a point corresponding to number 500 on the terrain data 111, the entry in the planning information 210 indicating the selected point is highlighted. In the example in Figure 13, the entry with number 500 and k=126 is highlighted. Alternatively, when cursor 1310 selects an entry in the planning information 210, the corresponding point on the terrain data 111 may be highlighted instead.

[0137] The first setting button 1301, the second setting button 1302, and the input completion button 1303 are user interfaces that can be switched ON or OFF by the user. In Figure 13, the white text indicates ON.

[0138] The first setting button 1301 is a user interface for inputting "Camera #1" (right-side imaging device 407) as the command argument 503 of the entry in the specified planning information 210. In Figure 13, the first setting button 1301 is in the ON state. If the first setting button 1301 is pressed again while in the ON state, it will turn OFF.

[0139] The second setting button 1302 is a user interface for inputting "Camera #2" (left-side imaging device 407) as the command argument 503 of the specified plan information 210 entry. In Figure 13, the second setting button 1302 is in the ON state. If the second setting button 1302 is pressed again while in the ON state, it will turn OFF.

[0140] If both the first setting button 1301 and the second setting button 1302 are ON, "ALL" will be entered as the command argument 503 for the specified plan information 210 entry.

[0141] The input completion button 1303 is a user interface for confirming and completing the input of command 502 and command argument 503. Command 502 can be entered by selecting from a pull-down menu or by text input. This sets command 502 and command argument 503 in the planning information 210.

[0142] <Figure 14 Display Screen> Figure 14 is an explanatory diagram showing an example of a display screen displayed by the output unit 907. The display screen 1400 has a wide-area map display area 1401, a detailed map display area 1402, and a spectral display area 1403.

[0143] The wide-area map display area 1401 displays map data generated by the generation unit 906. The wide-area map display area 1401 also displays a range selection cursor 1410 that can be moved by the user within the wide-area map display area 1401.

[0144] The detailed map display area 1402 displays the map data within the range selection cursor 1410 in an enlarged view. The detailed map display area 1402 includes a calendar 1421, an RGB selection button 1422, an NDVI selection button 1423, and a wavelength input area 1424.

[0145] Calendar 1421 is a user interface for specifying the period for displaying map data. It is a user interface for obtaining wavelength information 1206 of timestamp 1202 included in the period specified by Calendar 1421 from imaging information 1200.

[0146] The RGB selection button 1422 is a user interface for selecting the color display of the map data. When the RGB selection button 1422 is pressed, the location ID 1201 and wavelength of entries with values ​​within the visible light wavelength band of 380-780 [nm] in the wavelength information 1206 are obtained from the imaging information 1200. The location indicated by the obtained location ID 1201 is identified from the X position 504 and Y position 505 of the planning information 210 by the corresponding number 500 in the imaging information 1200.

[0147] The NDVI selection button 1423 is a user interface for selecting the NDVI (Normalized Difference Vegetation Index) display for map data. NDVI is a vegetation evaluation index that uses reflectance values ​​R at two wavelengths, as shown in equation (13) below, to represent the presence or absence and activity of vegetation based on the ratio of the difference and sum of the two values.

[0148]

number

[0149] Alternatively, a simpler calculation formula can be used, where the luminance value A is used instead of R. When the NDVI selection button 1423 is pressed, the wavelength band λ corresponding to the red color of vegetation is selected in the wavelength information 1206. RED (620~690 [nm]), near-infrared region λ that reflects light IR The location ID 1201 and wavelength of entries with values ​​within the range (720~1200 [nm]) are obtained from the imaging information 1200. The location indicated by the obtained location ID 1201 is identified from the X position 504 and Y position 505 of the planning information 210 by the corresponding number 500 in the imaging information 1200.

[0150] The wavelength input region 1424 is a region that accepts input of any wavelength. When a wavelength value is input to the wavelength input region 1424, the location ID 1201 of the entry with the input wavelength value is obtained from the imaging information 1200 in the wavelength information 1206. The location indicated by the obtained location ID 1201 is identified from the X position 504 and Y position 505 of the planning information 210 by the corresponding number 500 in the imaging information 1200.

[0151] The acquisition unit 905 uses the wavelength λ selected in this manner to acquire the light source spectrum I(λ), luminance spectrum A(λ), and transmittance Stc(λ,L) for each location G, and calculates the reflectance spectrum R(λ) using the above equation (8).

[0152] The spectral display area 1403 displays graph 1430 of the reflectance spectral waveforms 1431-1433 of the detailed map display area 1402 within the period specified in calendar 1421. The horizontal axis of the graph is wavelength, and the vertical axis is reflectance spectrum R(λ). Reflectance spectral waveforms 1431-1433 show the maximum, minimum, and average values ​​of the reflectance spectrum R(λ) based on the wavelength change at point G of cursor 1425 within the period specified in calendar 1421. Which of the reflectance spectral waveforms 1431-1433 to use can be selected by pre-configuration or by the user.

[0153] Bars 1434B, 1434G, and 1434R are displayed when the RGB selection button 1422 is pressed. Bar 1434B represents the blue wavelength, bar 1434G represents the green wavelength, and bar 1434R represents the red wavelength. Bars 1434B, 1434G, and 1434R can be moved horizontally by user input.

[0154] The generation unit 906 converts the values ​​at wavelength λ selected by bars 1434B, 1434G, and 1434R in the reflection spectrum waveform 1433 into R (red), G (green), and B (blue) luminance values ​​using an RGB converter. Alternatively, the generation unit 906 multiplies each of the reflectance spectra R(λ) selected by bars 1434B, 1434G, and 1434R in the reflection spectrum waveform 1433 by the spectral components of a pre-set light source, and then converts these values ​​into the respective luminance values ​​of the converted R, G, and B luminance values. Map data is generated by setting the converted R, G, and B luminance values ​​as pixel information for each assigned point G.

[0155] The generation unit 906 cannot set pixel information for locations where pixel information has not been assigned (hereinafter referred to as unassigned locations). In this case, the generation unit 906 sets a predetermined color (for example, black) for unassigned locations. Alternatively, the generation unit 906 may set separately acquired topographic maps or road maps for unassigned locations.

[0156] Thus, according to this embodiment, map data can be generated and displayed in which the colors of an image of the terrain are assigned to terrain data 111. Furthermore, by selecting the wavelength of the pixels, map data corresponding to the selected wavelength can be generated and displayed.

[0157] It should be noted that the present invention is not limited to the embodiments described above, but includes various modifications and equivalent configurations within the spirit of the attached claims. For example, the embodiments described above are described in detail to make the present invention easier to understand, and the present invention is not necessarily limited to having all of the described configurations. Furthermore, some of the configurations of one embodiment may be replaced with those of another embodiment. Furthermore, some of the configurations of one embodiment may be added to those of another embodiment. Furthermore, some of the configurations of each embodiment may be added, deleted, or replaced with other configurations.

[0158] Furthermore, each of the aforementioned configurations, functions, processing units, and processing means may be implemented in hardware, for example, by designing them as integrated circuits, or they may be implemented in software by having a processor interpret and execute programs that realize each function.

[0159] Information such as programs, tables, and files that implement each function can be stored in memory, hard disks, SSDs (Solid State Drives), or on recording media such as IC (Integrated Circuit) cards, SD cards, and DVDs (Digital Versatile Discs).

[0160] Furthermore, the control lines and information lines shown are those deemed necessary for explanation purposes and do not necessarily represent all control lines and information lines required for implementation. In reality, it can be assumed that almost all components are interconnected. [Explanation of symbols]

[0161] 100 World Coordinate Systems 101 Ground surface 102 Mobile Unit 103 orbit 110 Image Data Sequences 111 Topographic data 201 Management device 202 Generator 203 Distribution device 204 Network 210 Planning Information 220 Imaging device 230 Weather Information 301 Processor 302 Storage Devices 400 Local Coordinate System 401 Control Unit 402 Position and orientation calculation unit 403 Communications Department 404 Clock 405 counter 406 Roll Angle Compensation Mechanism 407 Lateral imaging device 408 Forward imaging device 410 wheels 411 Rotation Angle Gauge 412 Onboard unit 420 Ground Coil 500 number 504 X position 505 Y position 506 Elevation 507 pitch angle 508 Roll angle 509 Yaw angle 510 Reference 511 Section Type 512 radius of curvature 901 Input section 902 Records Department 903 Correction Unit 904 Decision Section 905 Acquisition Department 906 Generation part 907 Output section 1200 imaging information 1202 Timestamp 1203 Shooting location 1204 Imaging conditions 1205 Weather conditions 1206 Wavelength information 1207 Recruitment Flag

Claims

1. A generating apparatus having a processor for executing a program and a storage device for storing the program, The storage device stores terrain data that models the terrain and has location information for each point in the terrain, first image data obtained by capturing a subject that is part of the terrain from a first imaging point on the terrain, and location information of the first imaging point. The aforementioned processor, A determination process that determines the destination location to which information of a specific pixel corresponding to the line of sight vector from the first imaging point to the subject is assigned, based on the direction of the line of sight vector from the first imaging point to the subject, A generation process that generates map data corresponding to the terrain by assigning the information of the specific pixel determined by the determination process to the assigned location, A generating device characterized by performing the following actions.

2. A generating apparatus according to claim 1, The first image data is image data obtained by an imaging device of a moving body that moves on the ground surface constituting the terrain, capturing the subject from the side of the moving body. A generating apparatus characterized by the following features.

3. A generating apparatus according to claim 1, In the aforementioned decision process, the processor decides whether or not to accept the assigned location based on the information of the specific pixel. A generating apparatus characterized by the following features.

4. The generating apparatus according to claim 3, In the aforementioned decision process, the processor decides whether or not to accept the assigned location based on weather conditions. A generating apparatus characterized by the following features.

5. The generating apparatus according to claim 4, In the aforementioned decision process, the processor deletes from the storage device any first image data in which the number of pixels not selected as the assigned location is greater than or equal to a predetermined number. A generating apparatus characterized by the following features.

6. A generating apparatus according to claim 1, The aforementioned processor, Based on the light source spectrum, an acquisition process is performed to acquire information about the specific pixel. In the generation process, the processor generates the map data by assigning the information of the specific pixel acquired by the acquisition process to the assigned location. A generating apparatus characterized by the following features.

7. The generating apparatus according to claim 6, The information of the specific pixel is the first luminance spectrum of light rays from the assigned point received by an imaging device of a moving object moving on the ground surface constituting the terrain, or the reflectance spectrum of the subject calculated based on the light source spectrum and the first luminance spectrum. A generating apparatus characterized by the following features.

8. The generating apparatus according to claim 7, The first image data is image data generated by spectrally analyzing the light of the subject according to its wavelength. In the generation process, the processor generates the map data by converting the wavelength components of the first luminance spectrum or the reflectance spectrum into luminance values ​​as information for the specific pixel. A generating apparatus characterized by the following features.

9. The generating apparatus according to claim 8, In the acquisition process, the processor calculates the reflectance spectrum of the subject based on the light source spectrum, the first luminance spectrum, and the transmittance between the imaging device and the subject. A generating apparatus characterized by the following features.

10. The generating apparatus according to claim 6, The storage device stores second image data obtained by capturing a reflector having a known reflectivity from a second imaging point on the terrain. In the acquisition process, the processor acquires the light source spectrum based on the known reflectance and the second luminance spectrum of the light rays from the reflector received by the imaging device. A generating apparatus characterized by the following features.

11. The generating apparatus according to claim 6, In the acquisition process, the processor acquires the light source spectrum based on the weather conditions on the ground surface that constitute the terrain. A generating apparatus characterized by the following features.

12. The generating apparatus according to claim 8, The storage device stores a plurality of first image data with different capture dates and times. In the generation process, the processor calculates the reflectance spectrum for each of the multiple first image data with different capture dates and times, calculates a representative reflectance spectrum based on the multiple reflectance spectra, and generates information for the specific pixel based on the brightness value and the representative reflectance spectrum. A generating apparatus characterized by the following features.

13. The generating apparatus according to claim 8, In the generation process, the processor generates vegetation evaluation map data by selecting wavelengths related to vegetation evaluation for the assigned location. A generating apparatus characterized by the following features.

14. A generation method performed by a generation apparatus having a processor for executing a program and a storage device for storing the program, The storage device stores terrain data that models the terrain and has location information for each point in the terrain, first image data obtained by capturing a subject that is part of the terrain from a first imaging point on the terrain, and location information of the first imaging point. The aforementioned processor, A determination process that determines the destination location to which information of a specific pixel corresponding to the line of sight vector from the first imaging point to the subject is assigned, based on the direction of the line of sight vector from the first imaging point to the subject, A generation process that generates map data corresponding to the terrain by assigning the information of the specific pixel determined by the determination process to the assigned location, A generation method characterized by performing the following.

15. A processor that can access a storage device that stores terrain data which has location information for each point in the terrain and models the terrain, first image data which is a subject that is part of the terrain and is captured from a first imaging point on the terrain, and location information of the first imaging point, A determination process that determines the destination location to which information of a specific pixel corresponding to the line of sight vector from the first imaging point to the subject is assigned, based on the direction of the line of sight vector from the first imaging point to the subject, A generation process that generates map data corresponding to the terrain by assigning the information of the specific pixel determined by the determination process to the assigned location, A generation program characterized by causing the execution of a specific action.

Images