Display device, display method, and program

Abstract

To suppress discomfort of users.SOLUTION: A data generation device 12 includes: a data acquisition unit 50 for acquiring image data in which a ground is captured, and position data indicating a position where the image data is captured; an environment change data acquisition unit 52 for acquiring environment change data indicating a change level of earth environments per unit time in the position where the image data is captured; and a data generation unit 54 which associates the image data, the position data and the environment change data one another, for generating map data indicating a state of the ground.SELECTED DRAWING: Figure 4

Description

[Technical Field]

[0001] The present invention relates to a data generating device, a display device, a data generating method, a display method, and a program. [Background technology]

[0002] For example, Patent Document 1 describes the generation of polygons for drawing intermediate maps between rough spherical map data of an Earth object for car navigation and detailed planar map data, as map data showing the state of the ground at each location. Such map data may be constructed using images of the ground that have actually been captured. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 5921753 Summary of the Invention [Problem to be solved by the invention]

[0004] However, since it is difficult to simultaneously capture images of all points on the ground, map data is constructed by combining image data of each point captured at different times. Therefore, when a user views such ground data, the images for each point may not be seamless, which may cause the user to feel uncomfortable.

[0005] In view of the above-mentioned problems, an object of the present invention is to provide map data that can reduce the sense of discomfort felt by the user. [Means for solving the problem]

[0006] A data generation device according to one aspect of the present invention includes a data acquisition unit that acquires image data of the ground and location data indicating the location where the image data was acquired; an environmental change data acquisition unit that acquires environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was acquired; and a data generation unit that associates the image data, the location data, and the environmental change data to generate map data indicating the condition of the ground.

[0007] A display device according to one aspect of the present invention includes a map data acquisition unit that acquires map data including image data of the ground, location data indicating the location where the image data was acquired, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was acquired; a display condition acquisition unit that acquires information on the location on the ground to be displayed and date and time; a global environment setting unit that sets the global environment at the location and date and time to be displayed based on the environmental change data; an image data correction unit that corrects the image data so that it matches the set global environment; and a display control unit that displays the corrected image data.

[0008] A data generation method according to one aspect of the present invention includes the steps of acquiring image data of the ground and location data indicating the location where the image data was acquired, acquiring environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was acquired, and generating map data indicating the condition of the ground by associating the image data, the location data, and the environmental change data.

[0009] A display method according to one aspect of the present invention includes the steps of acquiring map data including image data of the ground, location data indicating the location where the image data was acquired, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was acquired; acquiring information on the location on the ground to be displayed and date and time; setting the global environment at the location and date and time to be displayed based on the environmental change data; correcting the image data so that it matches the set global environment; and displaying the corrected image data.

[0010] A program according to one aspect of the present invention causes a computer to execute the steps of acquiring image data of the ground and location data indicating the location where the image data was acquired, acquiring environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was acquired, and generating map data indicating the state of the ground by associating the image data, the location data, and the environmental change data.

[0011] A program according to one aspect of the present invention causes a computer to execute the following steps: acquiring map data including image data of the ground, location data indicating the location where the image data was captured, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured; acquiring information on the location on the ground to be displayed and the date and time; setting the global environment at the location and date and time to be displayed based on the environmental change data; correcting the image data so that it matches the set global environment; and displaying the corrected image data. [Effects of the Invention]

[0012] According to the present invention, it is possible to suppress the sense of discomfort felt by the user. [Brief explanation of the drawings]

[0013] [Figure 1] FIG. 1 is a schematic block diagram of a data generation system according to this embodiment. [Figure 2] FIG. 2 is a schematic block diagram of the image acquisition device according to this embodiment. [Figure 3] FIG. 3 is a schematic diagram showing an example of format data. [Figure 4] FIG. 4 is a schematic block diagram of a data generating device according to this embodiment. [Figure 5] FIG. 5 is a schematic diagram for explaining a method for calculating the incident angle of sunlight. [Figure 6] FIG. 6 is a schematic diagram showing an example of map data. [Figure 7] FIG. 7 is a flowchart illustrating the processing flow of the data generating device according to this embodiment. [Figure 8] FIG. 8 is a schematic block diagram of a display device according to this embodiment. [Figure 9] FIG. 9 is a schematic diagram showing an example of corrected image data. [Figure 10] FIG. 10 is a flowchart illustrating the processing flow of the display device according to this embodiment. [Figure 11] FIG. 11 is a block diagram showing a specific configuration of the data generating unit. [Figure 12] FIG. 12 is an explanatory diagram showing the positional relationship between two images to which the photogrammetry principle is applied. [Figure 13] FIG. 13 is an explanatory diagram showing the positional relationship between the two images. DETAILED DESCRIPTION OF THE INVENTION

[0014] DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS The present invention will be described in detail below with reference to the accompanying drawings. However, the present invention is not limited to the following embodiments.

[0015] (Data Generation System) Fig. 1 is a schematic block diagram of a data generation system according to this embodiment. As shown in Fig. 1, the data generation system 1 according to this embodiment includes an image acquisition device 10, a data generation device 12, and a display device 14. The data generation system 1 captures an image of the ground using the image acquisition device 10 to acquire image data of the captured image of the ground, generates map data including the image data using the data generation device 12, and displays the map data using the display device 14.

[0016] (Image acquisition device) FIG. 2 is a schematic block diagram of an image acquisition device according to this embodiment. The image acquisition device 10 is a device that captures images of the ground, and in this embodiment, is an air vehicle such as a drone. The image acquisition device 10 captures images of the ground while moving, thereby acquiring image data for each position on the Earth (image data captured at each point on the Earth). As shown in FIG. 2, the image acquisition device 10 includes a camera (image acquisition device) 20, a GPS receiver (GNSS signal receiver) 22, a direction sensor 24, a date and time data acquisition unit 26, a position data acquisition unit 28, a direction data acquisition unit 30, a formatting unit 32, and a storage unit 34.

[0017] Camera 20 captures images of the ground. The image data captured by camera 20 may be compressed into a format such as JPEG or PNG to the extent that degradation is not noticeable. Camera 20 sends the captured image data to formatting unit 32 and also sends it as trigger information to date and time data acquisition unit 26, position data acquisition unit 28, and orientation data acquisition unit 30.

[0018] The GPS receiver 22 receives radio waves from satellites and sequentially acquires the current date, time, and position detection results. The GPS receiver 22 sends the date and time detection results to the date and time data acquisition unit 26, and sends the position detection results to the position data acquisition unit 28.

[0019] The orientation sensor 24 uses a gyro (angular velocity sensor) to measure the tilt (elevation angle) of the camera 20. The orientation sensor 24 sends the measurement result to the orientation data acquisition unit 30.

[0020] The date and time data acquisition unit 26 acquires date and time data when the camera 20 captured the image data, based on the date and time detection result by the GPS receiving unit 22. Here, the date and time data is information indicating the timing when the image data was captured (the timing when the ground was captured by the camera 20), and in this embodiment includes information on the year, month, day, and time. The date and time data acquisition unit 26 sends the date and time data to the formatting unit 32.

[0021] The position data acquisition unit 28 acquires position data at the time when the camera 20 captured the image data, based on the position detection result by the GPS receiving unit 22. The position data is information indicating the position at which the image data was acquired (the position at which the ground was imaged by the camera 20), and indicates the position on the ground of the point at which the image data was captured. In this embodiment, the position data is the position of the image capture device 10 at the time the image data was captured, but this is not limiting. The position of the point at which the image data was captured may be derived based on the position of the image capture device 10 at the time the image data was captured, and the derived position may be used as the position data. The position data acquisition unit 28 sends the position data to the formatting unit.

[0022] The orientation data acquisition unit 30 acquires orientation data when the camera 20 captured the image data, based on the detection result of the tilt of the camera 20 by the orientation sensor 24. The orientation data is information that indicates the direction of the camera 20 at the time the image data was acquired (the time the ground was imaged by the camera 20). The orientation data acquisition unit 30 sends the orientation data to the format unit.

[0023] The format unit 32 associates image data, date and time data, position data, and orientation data to generate format data PD. That is, the format data PD is data in which image data, date and time data, position data, and orientation data are associated with each other. In this embodiment, the format unit 32 converts the image data, date and time data, position data, and orientation data into a predetermined format to generate the format data PD. The format unit 32 sets the number of bytes per parameter, converts various input information into a predetermined format, and generates the format data PD. The format unit 32 sends the format data PD to the storage unit 34. Note that the date and time data and orientation data are not essential, and the format data PD may be data in which image data and position data are associated with each other.

[0024] The date and time data acquisition unit 26, the position data acquisition unit 28, the orientation data acquisition unit 30, and the format unit 32 may be realized by a computing device included in the image acquisition device 10. The computing device here is a device including an arithmetic circuit such as a CPU (Central Processing Unit), and may realize the date and time data acquisition unit 26, the position data acquisition unit 28, the orientation data acquisition unit 30, and the format unit 32 and perform their processing by reading and executing a program (software) from the storage unit 34.

[0025] The storage unit 34 temporarily stores the format data PD. The storage unit 34 is a memory that stores various information such as the calculation contents of the calculation device and programs, and includes at least one of a main storage device such as a RAM (Random Access Memory), a ROM (Read Only Memory), and an external storage device such as an HDD (Hard Disk Drive). The storage unit 34 transmits the format data PD to an external database via a communication unit (not shown). The format data PD temporarily stored in the storage unit 34 is saved on a large scale in a database on the Internet, for example.

[0026] FIG. 3 is a schematic diagram showing an example of format data. As shown in FIG. 3, the format data PD is composed of a first half, a fixed-length portion, and a second half, a variable-length portion. The fixed-length portion has date and time data, position data, and direction data. The date and time data is year, month, and date information, and is composed, for example, of two bytes of year data, one byte of month data, one byte of day data, one byte of hour data, one byte of minute data, and one byte of second data. The position data is composed, for example, of eight bytes of latitude data and eight bytes of longitude data. The position information may be expressed as areas divided into a map of a predetermined size, and may be described as relative coordinates on the horizontal and vertical axes within the area. The direction data is composed, for example, of eight bytes of horizontal direction and eight bytes of elevation angle data. The orientation data acquired by camera 20 is expressed, for example, using an analog clock, with east being 3 o'clock (90°), south being 6 o'clock (180°), west being 9 o'clock (270°), and north being 12 o'clock (360° (0°)) in a clockwise direction. The elevation angle orientation is recorded as the degree of elevation angle when looking up, with horizontal being 0 degrees and directly above being 90 degrees. The variable-length portion is image data, and the image data is a compressed image file in JPG or PNG format.

[0027] As described above, the image acquisition device 10 acquires image data for each position on the Earth (image data captured at each point on the Earth) by capturing images of the Earth while moving. The formatting unit 32 generates format data PD for each position on the Earth, i.e., for each captured image data. In this embodiment, the image data included in one piece of format data PD is one piece of image data captured at one point, but this is not limited to this and may include multiple pieces of image data. In this case, the multiple pieces of image data may be image data captured at multiple points, or may be image data captured at the same point from different directions.

[0028] (Data generation device) Fig. 4 is a schematic block diagram of a data generating device according to this embodiment. The data generating device 12 according to this embodiment is, for example, a computer. As shown in Fig. 4, the data generating device 12 has a storage unit 40 and a control unit 42. The storage unit 40 is a memory that stores various information such as the calculation contents and programs of the control unit 42, and includes, for example, at least one of a RAM, a main storage device such as a ROM, and an external storage device such as an HDD.

[0029] The control unit 42 is a computing device and includes a computing circuit such as a CPU. The control unit 42 includes a data acquisition unit 50, an environmental change data acquisition unit 52, and a data generation unit 54. The control unit 42 realizes the data acquisition unit 50, the environmental change data acquisition unit 52, and the data generation unit 54 by reading and executing a program (software) from the storage unit 40. The control unit 42 may execute these processes using a single CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. At least a portion of the processes performed by the data acquisition unit 50, the environmental change data acquisition unit 52, and the data generation unit 54 may be implemented using hardware circuits.

[0030] (Data acquisition section) The data acquisition unit 50 acquires image data of the ground captured by the image acquisition device 10, location data indicating the location where the image data was captured, and date and time data indicating the date and time when the image data was captured. In this embodiment, the data acquisition unit 50 acquires format data PD generated by the format unit 32 of the image acquisition device 10. The data acquisition unit 50 may acquire the format data PD from an external database via a communication unit (not shown) included in the data generation device 12, or may acquire the format data PD directly from the image acquisition device 10.

[0031] In this way, the data acquisition unit 50 acquires the format data PM generated by the format unit 32 of the image acquisition device 10. However, this is not limiting, and the data acquisition unit 50 may have the functions of the format unit 32. In this case, the data acquisition unit 50 may acquire the image data, date and time data, position data, and orientation data acquired by the image acquisition device 10 via a communication unit (not shown), and associate these to generate the format data PM.

[0032] (Environmental change data acquisition section) The environmental change data acquisition unit 52 acquires environmental change data at the location where the image data was captured. The environmental change data is information indicating the degree of change in the global environment per unit time at that location. The global environment here refers to physical phenomena (natural phenomena) that change on a global scale, and can be said to be a natural phenomenon that is determined depending on the location and date and time. In other words, the global environment here refers to something that changes on an annual, seasonal, and hourly basis, and changes over time. Furthermore, the global environment here differs depending on the location on the ground (earth coordinates); for example, the global environment at different locations may differ even if the date and time are the same.

[0033] In this embodiment, the solar light incidence angle is defined as the Earth's environment. The solar light incidence angle is determined by the positional relationship between the sun and Earth coordinates, and therefore can be considered a natural phenomenon determined by location and date and time. The solar light incidence angle can be calculated, for example, as follows. FIG. 5 is a schematic diagram illustrating a method for calculating the solar light incidence angle. That is, the Earth's axis is tilted at approximately 23.5° with respect to the perpendicular to the orbital plane, and the solar light incidence angle changes depending on where the Earth is located in its orbit. Furthermore, since the Earth is a sphere and rotates, the solar light incidence angle varies depending on the latitude and time of day (i.e., longitude). Therefore, the solar light incidence angle can be considered to be determined by the orbital position (season), latitude, and time (longitude). For example, the solar light incidence angle at noon at a point with a longitude of 0° (the Greenwich Observatory position) can be calculated using the following formula (1).

[0034] α=90−Φ+δ (1)

[0035] As shown in Figure 5, α in equation (1) is the angle of incidence of sunlight at noon, which can be said to be the noon altitude angle (the angle between the sun and the Earth's surface at noon). Also, Φ in equation (1) is latitude, and δ is declination (the angle between the Earth's equator and the sun's rays).

[0036] The environmental change data acquisition unit 52 acquires environmental change data indicating the degree of change in the global environment per unit time at the location indicated by the position data included in the format data PD. As described above, the global environment changes depending on the location and the date and time. Therefore, once the location is specified, the degree of change in the global environment per unit time at that location can be calculated. Based on the location data, the environmental change data acquisition unit 52 acquires, as environmental change data, the degree of change in the global environment per unit time at the location indicated by the position data, i.e., the degree of change in the global environment per unit time at the location where the image data was captured. The environmental change data acquisition unit 52 may acquire the environmental change data using any method based on the position data. For example, a table in which locations (global coordinates) are associated with the degree of change in the global environment per unit time may be stored in the storage unit 40, and the environmental change data acquisition unit 52 may acquire, as environmental change data, the degree of change in the global environment per unit time associated with the location indicated by the position data in the table.

[0037] The environmental change data acquisition unit 52 acquires environmental change data for each piece of format data PD, that is, for each piece of image data captured at a different position.

[0038] (Data Generation Section) The data generation unit 54 associates image data, position data, date and time data, and environmental change data to generate map data MD that indicates the state of the ground.More specifically, the data generation unit 54 associates image data, position data, date and time data, and environmental change data to generate unit map data PM0 that indicates the state of the ground at one point (position), and combines the unit map data PM0 for each position on the ground to generate map data PM.In other words, the map data PM refers to a collection of unit map data PM0, and can be said to be a data group that indicates the state of the ground at each position.

[0039] FIG. 6 is a schematic diagram showing an example of map data. As shown in FIG. 6, the data generation unit 54 associates the format data PD with environmental change data for the position indicated by the position data included in the format data PD, and generates unit map data PM0 for that position. That is, the unit map data PM0 in this embodiment can be said to be data in which date and time data, position data, orientation data, image data, and environmental change data are associated with each other. The data generation unit 54 associates the environmental change data with each piece of format data PD, thereby generating unit map data PM0 for each position on the ground, and combines the unit map data PM0 to form map data PM. Note that the date and time data and orientation data are not essential, and the unit map data PM0 may be data in which image data, position data, and environmental change data are associated with each other.

[0040] The data generating device 12 generates map data PM in the above manner. The process flow for generating the map data PM described above will now be described. FIG. 7 is a flowchart illustrating the process flow of the data generating device according to this embodiment. As shown in FIG. 7, the data generating device 12 acquires format data PD, i.e., image data, position data, date and time data, and orientation data, using the data acquiring unit 50 (step S10). The data generating device 12 acquires environmental change data for the position indicated by the position data using the environmental change data acquiring unit 52 (step S12). The data generating device 12 then associates the image data, position data, date and time data, and orientation data (format data PD) with the environmental change data, using the data generating unit 54, to generate unit map data MD0 (step S14). If there is no other image data (format data PD) (step S16; No), the device combines the unit map data MD0 generated so far to generate map data MD, and ends this process. On the other hand, if there is other image data (format data PD) (step S16; Yes), the process returns to step S10 and repeats the creation of unit map data MD0.

[0041] When generating map data by capturing images of various locations on the ground, it is difficult to capture all locations simultaneously. Therefore, the map data is constructed by combining image data of each location captured at different times. Therefore, when the map data is displayed, images of the global environment captured at different times are displayed side by side, resulting in a lack of seamlessness for each location, which may cause discomfort to the user. In contrast, in this embodiment, image data of a particular location is stored in association with environmental change data indicating the degree of change in the global environment at that location. Therefore, for example, by using the environmental change data to correct the image data of each location so that the global environment is uniform, it is possible to suppress discrete changes in the global environment for each location and reduce discomfort to the user. Furthermore, for example, using the environmental change data to correct the image data to reflect the global environment at the time the user wants to view the image can also reduce discomfort to the user.

[0042] (display device) 8 is a schematic block diagram of a display device according to this embodiment. As shown in FIG. 8, the display device 14 includes an input unit 60, a display unit 62, a storage unit 64, and a control unit 66. The input unit 60 is a mechanism for accepting user operations, and may be, for example, a mouse, a keyboard, or a touch panel. The display unit 62 is a display for displaying images. The storage unit 64 is a memory for storing various information such as the contents of calculations performed by the control unit 66 and programs, and includes, for example, at least one of a RAM, a main storage device such as a ROM, and an external storage device such as an HDD.

[0043] The control unit 66 is a computing device and includes a computing circuit such as a CPU. The control unit 66 includes a map data acquisition unit 70, a display condition acquisition unit 72, a global environment setting unit 74, an image data correction unit 76, and a display control unit 78. The control unit 66 implements the display condition acquisition unit 72, the global environment setting unit 74, the image data correction unit 76, and the display control unit 78 by reading and executing a program (software) from the storage unit 64. The control unit 66 may implement these processes using a single CPU, or may be provided with multiple CPUs and execute the processes using the multiple CPUs. At least a portion of the processes performed by the display condition acquisition unit 72, the global environment setting unit 74, the image data correction unit 76, and the display control unit 78 may be implemented using hardware circuits.

[0044] (Map data acquisition section) The map data acquisition unit 70 acquires map data MD including image data, position data, and environmental change data. The map data acquisition unit 70 acquires the map data MD by receiving the map data MD generated by the data generation device 12 via a communication unit (not shown). Note that the map data MD generated by the data generation device 12 may be stored in the storage unit 64, and the map data acquisition unit 70 may read the map data MD from the storage unit 64.

[0045] (Display condition acquisition part) The display condition acquisition unit 72 acquires display conditions for displaying the map data MD as an image. The display conditions are information indicating the state in which the map data MD is to be displayed as an image. In this embodiment, the display conditions are a position and a date and time, or in other words, information on the position on the ground to be displayed and the date and time (information specifying the position and the date and time at which an image is to be displayed). The display condition acquisition unit 72 may acquire the display conditions by any method, but in this embodiment, it acquires display conditions input by the user via the input unit 60. That is, in this embodiment, the display condition acquisition unit 72 acquires display conditions set by the user. However, the display conditions are not limited to this, and may be set in advance as defaults.

[0046] (Global Environment Settings Department) The global environment setting unit 74 sets the global environment at the location and date and time indicated by the display conditions based on the environmental change data included in the map data MD. Specifically, the global environment setting unit 74 extracts, from the map data MD, unit map data MD0 corresponding to the location specified by the display conditions (unit map data MD whose position data matches the location specified by the display conditions). Then, the global environment setting unit 74 calculates the global environment at the location specified by the display conditions at the date and time indicated by the display conditions based on the date and time indicated by the display conditions and the environmental change data included in the extracted unit map data MD0. In other words, since the global environment at the reference date and time is known, the global environment at the location and date and time indicated by the display conditions can be calculated from the difference between the reference date and time and the date and time indicated by the display conditions and the environmental change data (the degree of change in the global environment per unit time). Note that the global environment at the reference date and time may be included in the unit map data MD0 in advance, or may be calculated by the global environment setting unit 74 based on the unit map data MD0. The reference date and time may be set arbitrarily, and may be, for example, the date and time when the image data was captured (the date and time indicated by the date and time data).

[0047] (Image data correction section) The image data correcting unit 76 corrects the image data to conform to the global environment set by the global environment setting unit 74. That is, the image data correcting unit 76 corrects the image data to conform to the global environment set by the global environment setting unit 74, in other words, to conform to the global environment at the date and time specified by the display conditions. The image data correcting unit 76 may correct the image data in any manner. For example, the image data may be corrected by calculating the luminance value of each pixel assuming that the image was captured in the global environment at the date and time specified by the display conditions, and correcting the image data to conform to the calculated luminance value. Alternatively, the image data may be corrected by calculating at least one of the chromaticity of each pixel and the length of a shadow of a light assuming that the image was captured in the global environment at the date and time specified by the display conditions, and correcting the image data to conform to at least one of the calculated chromaticity of the pixel and the length of the shadow of the light. Note that if a shadow is represented by the magnitude of the luminance of a pixel, the luminance of the pixel may be corrected to conform to the calculated length of the shadow.

[0048] FIG. 9 is a schematic diagram showing an example of corrected image data. In this embodiment, the global environment is the angle of incidence of sunlight, so the image data correction unit 76 corrects the image data so that the image is one in which sunlight is incident at the angle of incidence at the date and time specified in the display conditions. Image P1 in FIG. 9 is an example of an image based on the image data before correction, and image P2 is an example of an image based on the image data after correction. As shown in FIG. 9, since the brightness distribution changes when the angle of incidence of sunlight changes, the image data correction unit 76 may, for example, calculate the luminance value of each pixel at the angle of incidence of sunlight at the date and time specified in the display conditions, and correct the image data to the calculated luminance value.

[0049] (Display control unit) The display control unit 78 causes the display unit 62 to display the image data corrected by the image data correction unit 76. That is, the display control unit 78 causes the display unit 62 to display a captured image (map data) of the ground at the position and date and time specified by the user.

[0050] The user may switch the location to be displayed by zooming in and out of the map data, moving the location, etc. In this case, the image data correction unit 76 corrects the image data at each location to reflect the global environment at the specified date and time, and the display control unit 78 displays the corrected image at the switched location when the location to be displayed is switched. This unifies the global environment of the image at each location on the ground to the specified date and time, preventing discrete changes in the global environment at each location and reducing any sense of discomfort felt by the user.

[0051] The display device 14 generates the corrected image data as described above and displays an image of the ground using the corrected image data. The processing flow of the display device 14 will be described below. FIG. 10 is a flowchart illustrating the processing flow of the display device according to this embodiment. As shown in FIG. 10, the display device 14 according to this embodiment acquires map data MD using the map data acquisition unit 70 (step S20), acquires display conditions indicating the location and date and time to be displayed using the display condition acquisition unit 72 (step S22), and sets the global environment for the location and date and time to be displayed based on the environmental change data using the global environment setting unit 74 (step S24). The display device 14 corrects the image data for the location to be displayed based on the set global environment using the image data correction unit 76 (step S26), and causes the display control unit 78 to display the corrected image data on the display unit 62 (step S28).

[0052] In this way, the display device 14 sets the global environment for the location and date and time to be displayed based on the environmental change data, and corrects the image data of the captured ground to reflect that global environment. Therefore, for example, by correcting the image data for each location so that the global environment is uniform, it is possible to suppress discrete changes in the global environment for each location and reduce the sense of discomfort felt by the user. Furthermore, for example, the image data is corrected using the environmental change data to reflect the global environment at the date and time the user wants to view, thereby reducing the sense of discomfort felt by the user.

[0053] In the above description, the data generating device 12 that generates the map data and the display device 14 that displays the map data are separate devices, but the generation and display of map data may be performed by a single device. That is, for example, the data generating device 12 may have at least some of the functions of the display device 14 and generate and display the map data, or the display device 14 may have at least some of the functions of the data generating device 12 and generate and display the map data.

[0054] Furthermore, in this embodiment, the global environment is defined as the angle of incidence of sunlight, but is not limited to this. For example, the global environment may be snowfall, ice size, ocean currents, westerly winds, etc. Furthermore, the global environment may include changes in coastlines and lake water levels due to the positional relationship between the position of the Earth's surface and the position of the moon (daily tides), changes in the water levels and widths of oceans, lakes, and rivers due to the seasons (dry season, rainy season, etc.), and seasonal changes in the colors of mountains, farmland, trees, etc. (buds, new greenery, green, flowers, autumn leaves, fallen leaves, snowfall, snowmelt, etc.).

[0055] (Method for generating three-dimensional image data) Note that the image data in this embodiment may be two-dimensional image data or three-dimensional image data, as long as it is image data of the ground. In the case of three-dimensional image data, the three-dimensional image data of a point may be generated based on multiple image data obtained by capturing images of the same point from different directions. An example of a method for generating three-dimensional image data will be described below.

[0056] FIG. 11 is a block diagram showing the specific configuration of the data generation unit, FIG. 12 is an explanatory diagram showing the positional relationship between two images to which the photogrammetry principle is applied, and FIG. 13 is an explanatory diagram showing the positional relationship between the two images.

[0057] As shown in FIG. 11, the data generating unit 54 of the data generating device 12 has an epipolar line direction calculator 54A, an epipolar line orthogonal direction calculator 54B, a search range determiner 54C, a corresponding point detector 54D, and a distance calculator 54E.

[0058] The epipolar line direction calculator 54A calculates the direction of an epipolar line connecting corresponding pixel points of the image data of the object based on the image data acquired by the data acquisition unit 50. The epipolar line direction calculator 54A sends the calculated direction of the epipolar line to the epipolar line orthogonal direction calculator 54B.

[0059] The epipolar line orthogonal direction calculator 54B calculates an orthogonal direction that is orthogonal to the epipolar line, and outputs the calculated orthogonal direction to the epipolar line to the search range determiner 54C.

[0060] The search range determiner 54C determines a two-dimensional search range on the screen so as to include a plurality of pixel points that correspond to each other in the direction of the epipolar line and in the direction orthogonal to the epipolar line, and outputs the determined two-dimensional search range to the corresponding point detector 54D.

[0061] The corresponding point detector 54D performs a corresponding point search based on the multiple image data acquired by the data acquisition unit 50 and the determined two-dimensional search range, and obtains a disparity vector. The corresponding point detector 54D sends the obtained disparity vector to the distance calculator 54E.

[0062] The distance calculator 54E maps the parallax vector onto an epipolar line to find the epipolar line direction component of the parallax vector, and calculates the distance to the object based on the found epipolar line direction component.

[0063] The generation of three-dimensional image data will be specifically described below, but here, the case where three-dimensional image data is generated from two pieces of image data will be described.

[0064] The data generating unit 54 extracts, for example, two pieces of image data whose positions indicated by the position data are the same. Note that the positions being the same do not necessarily have to be exactly the same, and pieces whose positions are shifted by a predetermined amount may also be considered to be the same.

[0065] First, two sets of image data of the object are obtained by camera 20A for a field of view image and camera 20B for a field of view image (see FIG. 12 for both). Next, corresponding point detector 54D searches for corresponding points of feature points based on the two sets of image data. For example, corresponding point detector 54D performs pixel-by-pixel correspondence and searches for the position where the difference is smallest. Here, as shown in FIGS. 11 and 12, cameras 20A and 20B, which are assumed to exist simultaneously at two viewpoints, are positioned such that Yl = Yr so that the optical axes Ol and Or are included on the same XZ coordinate plane. Using the corresponding points searched by corresponding point detector 54D, a disparity vector corresponding to the angular difference for each pixel is calculated.

[0066] Since the obtained disparity vector corresponds to the distance from the cameras 20A and 20B in the depth direction, the distance calculator 54E calculates the distance in proportion to the magnitude of the disparity using perspective. Assuming that the photographer's cameras 20A and 20B only move approximately horizontally, by positioning the cameras 20A and 20B so that their optical axes Ol and Or are included on the same XZ coordinate plane, it is possible to search for corresponding points only on the scanning lines that are the epipolar lines Epl and Epr. The distance calculator 54E generates three-dimensional image data of the object using the two image data of the object and the respective distances from the cameras 20A and 20B to the object.

[0067] On the other hand, when point Ql(Xl,Yl) on the left image corresponds to point Qr(Xr,Yr) on the right image, the disparity vector at point Ql(Xl,Yl) is Vp(Xl-Xr,Yl-Yr). Here, since the two points Ql and Qr are on the same scanning line (epipolar line), Yl=Yr, and the disparity vector is expressed as Vp(Xl-Xr,0). The epipolar line direction calculator 54A calculates such disparity vector Vp for all pixel points on the image and creates a group of disparity vectors to obtain information on the depth direction of the image. Incidentally, for a set in which the epipolar line is not horizontal, the height of one of the camera positions may be different (though this is a low probability). In this case, the epipolar line orthogonal direction calculator 54B searches within a rectangle in the epipolar line direction and in a direction orthogonal to the epipolar line, which is approximately the deviation from the horizontal, compared to searching for corresponding points in a large two-dimensional area without considering corresponding point matching. The search range determiner 54C determines the search range for the minimum rectangle, as shown in Figure 13, when the epipolar line direction search range for the minimum rectangle is a-b = c-d and the orthogonal direction search range is b-c = d-a. In this case, the search width in the epipolar line direction is ΔE, and the search width in the direction T orthogonal to the epipolar line is ΔT. The desired area is the smallest non-tilted rectangle ABCD that contains the minimum slanted rectangle abcd.

[0068] In this way, the data generation unit 54 calculates the parallax vector from the corresponding feature points of the multiple cameras 20A and 20B under the epipolar constraint condition, obtains the depth direction information of each point, maps the texture on the surface of the three-dimensional shape, and generates three-dimensional image data. As a result, the model of the part in the image data used for calculation can reproduce the space as seen from the front hemisphere. Furthermore, if there is a part not captured in the image data of the three-dimensional image data, and if the part is connected by extending the lines or surfaces of the surrounding texture, the part in between is interpolated using the same texture.

[0069] The method for generating the three-dimensional image data is not limited to the above, and other methods may be used.

[0070] (effect) As described above, the data generation device 12 according to this embodiment includes a data acquisition unit 50, an environmental change data acquisition unit 52, and a data generation unit 54. The data acquisition unit 50 acquires image data of an image of the ground and location data indicating the location where the image data was captured. The environmental change data acquisition unit 52 acquires environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured. The data generation unit 54 associates the image data, location data, and environmental change data to generate map data MD indicating the state of the ground.

[0071] The data generating device 12 stores image data of a certain location and environmental change data indicating the degree of change in the global environment at that location in association with each other as map data. Therefore, by using such map data, it is possible to reduce the sense of discomfort felt by the user. For example, by using the environmental change data to correct the image data of each location so that the global environment is uniform, it is possible to reduce the sense of discomfort felt by the user. Furthermore, for example, it is also possible to reduce the sense of discomfort felt by the user by using the environmental change data to correct the image data so that the global environment at the time the user wants to view the image can be reflected.

[0072] Furthermore, the data generation unit 54 generates unit map data MD0 for each location on the ground, which associates image data, position data, and environmental change data, and combines the unit map data MD0 for each location on the ground to generate map data MD. By generating unit map data MD0 for each location, it becomes possible to correct the global environment for each location, and it is possible to more appropriately suppress any sense of discomfort felt by the user.

[0073] The environmental change data acquisition unit 52 also acquires, as environmental change data, the degree of change in the angle of incidence of sunlight per unit time at the position where the image data was captured. By correcting the angle of incidence of sunlight as a function of the global environment, it is possible to suppress discrete changes in the degree of sunlight incidence for each position, thereby reducing any discomfort felt by the user.

[0074] The display device 14 according to this embodiment includes a map data acquisition unit 70, a display condition acquisition unit 72, a global environment setting unit 74, an image data correction unit 76, and a display control unit 78. The map data acquisition unit 70 acquires map data MD including image data of the ground, location data indicating the location where the image data was captured, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured. The display condition acquisition unit 72 acquires information on the location on the ground to be displayed and the date and time. The global environment setting unit 74 sets the global environment at the location and date and time to be displayed based on the environmental change data. The image data correction unit 76 corrects the image data to achieve the set global environment. The display control unit 78 displays the corrected image data.

[0075] In this way, the display device 14 sets the global environment for the location and date and time to be displayed based on the environmental change data, and corrects the image data of the captured ground to reflect that global environment. Therefore, for example, by correcting the image data for each location so that the global environment is uniform, it is possible to suppress discrete changes in the global environment for each location and reduce the sense of discomfort felt by the user. Furthermore, for example, the image data is corrected using the environmental change data to reflect the global environment at the date and time the user wants to view, thereby reducing the sense of discomfort felt by the user.

[0076] In this way, the display device 14 sets the global environment for the location and date and time to be displayed based on the environmental change data, and corrects the image data of the captured ground to reflect that global environment. Therefore, for example, by correcting the image data for each location so that the global environment is uniform, it is possible to suppress discrete changes in the global environment for each location and reduce the sense of discomfort felt by the user. Furthermore, for example, the image data is corrected using the environmental change data to reflect the global environment at the date and time the user wants to view, thereby reducing the sense of discomfort felt by the user.

[0077] The environmental change data indicates the degree of change in the angle of incidence of sunlight per unit time at the position where the image data was captured, and the global environment setting unit 74 calculates the angle of incidence of sunlight at the position and date and time to be displayed, and the image data correction unit 76 corrects the image data to an image in which sunlight is incident at the calculated angle of incidence of sunlight. By correcting the angle of incidence of sunlight as the global environment, it is possible to suppress discrete changes in the degree of sunlight hitting the image for each position, thereby suppressing any sense of discomfort felt by the user.

[0078] Although the embodiments of the present invention have been described above, the embodiments are not limited to the contents of these embodiments. Furthermore, the above-described components include those that can be easily imagined by a person skilled in the art, those that are substantially the same, and those that are within the so-called equivalent range. Furthermore, the above-described components can be appropriately combined, and the configurations of each embodiment can also be combined. Furthermore, various omissions, substitutions, or modifications of the components can be made without departing from the spirit of the above-described embodiments. [Explanation of symbols]

[0079] 10 Image acquisition device 12 Data generation device 14 Display device 50 Data Acquisition Section 52 Environmental change data acquisition unit 54 Data Generation Unit 70 Map data acquisition unit 72 Display condition acquisition section 74 Global Environment Settings Department 76 Image data correction section 78 Display control unit MD map data MD0 unit map data

Claims

1. a map data acquisition unit that acquires map data including image data of the ground, the date and time when the image data was captured, location data indicating the location where the image data was captured, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured; a display condition acquisition unit that acquires information on the location on the ground and the date and time to be displayed; a global environment setting unit that sets the global environment at the location and date and time to be displayed based on the difference between the date and time when the image data was captured and the date and time on the ground to be displayed, and the environmental change data; an image data correcting unit that corrects the image data so that the set global environment is achieved; a display control unit that displays the corrected image data; A display device comprising:

2. the environmental change data indicates a degree of change in the angle of incidence of sunlight per unit time at the position where the image data was captured; the global environment setting unit calculates an incident angle of sunlight at a position and date and time to be displayed; The display device according to claim 1 , wherein the image data correcting section corrects the image data to an image in which sunlight is incident at the calculated incident angle of sunlight.

3. acquiring map data including image data of the ground, the date and time when the image data was captured, location data indicating the location where the image data was captured, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured; obtaining information on a terrestrial position and date and time to be displayed; setting the global environment at the location and date and time to be displayed based on the difference between the date and time when the image data was captured and the date and time on the ground to be displayed, and the environmental change data; correcting the image data so that the image data conforms to the set global environment; displaying the corrected image data; including, how it is displayed.

4. acquiring map data including image data of the ground, the date and time when the image data was captured, location data indicating the location where the image data was captured, and environmental change data indicating the degree of change in the global environment per unit time at the location where the image data was captured; obtaining information on a terrestrial position and date and time to be displayed; setting the global environment at the location and date and time to be displayed based on the difference between the date and time when the image data was captured and the date and time on the ground to be displayed, and the environmental change data; correcting the image data so that the image data conforms to the set global environment; displaying the corrected image data; The computer executes the program.

Images