Image correction device, image correction method, and program

Abstract

An image correction device according to the present invention comprises a geometric correction unit that: variably sets, according to a correction intensity parameter, the radius of a virtual cylindrical surface which is always in contact with a virtual spherical surface at one point or the radius of the virtual spherical surface; maps input image data onto the virtual spherical surface; projects, onto the virtual cylindrical surface, an image which has been mapped onto the virtual spherical surface; and uses the image projected onto the virtual cylindrical surface as output image data corresponding to pixels of a display device.

Description

Image correction device, image correction method, and program

[0001] The present technology relates to an image correction device, an image correction method, and a program, and relates to correction processing of a wide-field image such as a spherical image, for example.

[0002] For example, there are techniques for displaying image data with a wide angle range, such as a spherical image captured by a 360-degree camera or a fisheye image captured through a fisheye lens. Patent Document 1 below discloses a technique for correcting fisheye images.

[0003] JP 2012-99899 A

[0004] For example, it is conceivable that a wide angle range of a spherical image captured by a 360-degree camera or the like is displayed on a display, and the image can be scrolled in the desired direction as needed to view or use the wide-field image.

[0005] When a spherical image with a much wider angle range (e.g., 220 degrees horizontally) is displayed than the angle of view of the display as seen from the viewing position (e.g., 75 degrees horizontally), the subject is significantly distorted and directionality is lost, causing an unnatural feeling in the image. In particular, even if a correction process such as that described in Patent Document 1 is performed, the unnatural feeling cannot be sufficiently reduced when the viewing conditions are different. Furthermore, scrolling the image horizontally to display the desired direction results in an image that combines vertical axis rotation and horizontal movement, which can easily induce visually induced motion sickness.

[0006] Therefore, the present disclosure proposes a correction process that can reduce the sense of incongruity depending on the viewing conditions when displaying image data with a wide angle range.

[0007] An image correction device according to the present technology includes a geometric correction unit that variably sets the radius of a virtual cylindrical surface that always touches a virtual spherical surface or the radius of the virtual spherical surface according to a correction intensity parameter, maps input image data onto the virtual spherical surface, projects the image mapped onto the virtual spherical surface onto the virtual cylindrical surface, and generates output image data corresponding to pixels of a display device. For the virtual spherical surface and the virtual cylindrical surface in the correction calculation, the correction intensity varies depending on the relationship between the radius of the virtual spherical surface and the radius of the virtual cylindrical surface (the radius of curvature of the cylindrical circumferential surface). Therefore, the correction intensity is changed by variably setting the radius of the virtual cylindrical surface or the radius of the virtual spherical surface according to the correction intensity parameter.

[0008] 1 is a block diagram of a system configuration including an image correction device according to an embodiment of the present technology; FIG. 1 is an explanatory diagram of an input spherical image; FIG. 2 is an explanatory diagram of an image to which general cylindrical screen correction has been applied; FIG. 3 is an explanatory diagram of an image to which geometric correction according to an embodiment has been applied; FIG. 4 is an explanatory diagram of a virtual spherical surface and a virtual cylindrical surface in image correction according to an embodiment; FIG. 5 is an explanatory diagram of a virtual spherical surface and a virtual cylindrical surface in image correction according to an embodiment; FIG. 6 is an explanatory diagram of point contact between a virtual spherical surface and a virtual cylindrical surface; FIG. 7 is an explanatory diagram of a correspondence between a virtual cylindrical surface and pixels of a display according to an embodiment; FIG. 8 is an explanatory diagram of a correspondence between a virtual cylindrical surface and pixels of a display according to an embodiment; FIG. 9 is an explanatory diagram of a correspondence between a virtual cylindrical surface and pixels of a display according to an embodiment; FIG. 10 is an explanatory diagram of a change in correction intensity parameters according to an embodiment; FIG. 11 is an explanatory diagram of a change in correction intensity parameters according to an embodiment; FIG. 12 is a flowchart of geometric correction according to an embodiment; FIG. 13 is a flowchart of a correction intensity change according to an operation according to an embodiment; FIG. 14 is a flowchart of a correction intensity change according to an angle range of an image according to an embodiment; FIG. 15 is a flowchart of a correction intensity change according to a viewing distance according to an embodiment; FIG. 16 is a flowchart of a correction intensity change according to a curvature of a display according to an embodiment; FIG. 17 is a flowchart of a correction intensity change during scrolling according to an embodiment; and FIG. 18 is a block diagram of an information processing device according to an embodiment.

[0009] Hereinafter, the embodiments will be described in the following order: <1. System configuration including image correction device> <2. Configuration of image correction device> <3. Geometric correction> <4. Processing example> <5. Configuration of information processing device> <6. Summary and modified examples>

[0010] In this disclosure, "image" refers to both moving images and still images. Furthermore, "image" is not limited to images that are displayed on a display and are visually recognized by a user, but may also refer to image data in a transmission process, a signal processing process, or stored in a storage medium.

[0011] 1. System Configuration Including Image Correction Device In the embodiment, an image display system will be described that enables the display of a wide-field image with reduced distortion and less discomfort even when the viewing conditions change.

[0012] This image display system assumes that an image is displayed on a flat display or a curved display. The correction strength of the geometric correction is changed in response to changes in viewing conditions, such as the angle range of the displayed image specified by the user, the distance between the viewer and the display, and the curvature of the display, i.e., changes in external factors seen from the perspective of image display processing. Specifically, the system has a function for dynamically changing the geometric relationship between a virtual spherical surface 50 and a virtual cylindrical surface 51 (described later) from a predetermined design value to a more appropriate value.

[0013] Furthermore, when an image captured using a fisheye lens or a 360-degree camera is used as an input image, the device has the function of dynamically updating the geometric relationship between the virtual spherical surface 50 and the virtual cylindrical surface 51 in order to suppress motion sickness that occurs when the image is scrolled in the direction the viewer wants to see.

[0014] 1 shows an example of the configuration of an image display system including an image correction device 1 according to an embodiment. For example, the image display system includes an image correction device 1, an image generation device 2, an image display device 3, a controller 4, and a visual distance detection device 5. Note that not all of these components are required for the image display system, and devices not shown may also be added. Furthermore, each of these devices may be separate devices, or some or all of them may be integrated into one device.

[0015] The image generating device 2 is a device that generates image data and supplies it to the image correction device 1, and is, for example, a 360-degree camera or a 360-degree image player. The image output from the image generating device 2 is a celestial sphere image, that is, an image obtained by capturing or rendering the entire sphere including 360 degrees horizontally and 180 degrees vertically, and is, for example, an equirectangular format (equirectangular projection) or cube map format image.

[0016] In the embodiment, the image output from the image generating device 2 is described as a celestial sphere image of 360 degrees horizontally and 180 degrees vertically, but is not limited to this. The image may be an image of a partial range of the celestial sphere image, for example, an image of a range of 180 degrees horizontally and 100 degrees vertically, or a fisheye image. A fisheye image is an image captured using a fisheye lens and has a field angle of approximately 180 degrees in both the horizontal and vertical directions. Such an image, such as a celestial sphere image, output from the image generating device 2 becomes the input image Vin for the image correction device 1.

[0017] The image correction device 1 takes in an input image Vin such as a spherical image, corrects the image, and outputs the corrected image as an output image Vout. The configuration of the image correction device 1 will be described in detail later.

[0018] The image display device 3 has a display 31 and displays the corrected image (output image Vout) from the image correction device 1. The display 31 can be, for example, a curved display whose display surface curvature can be changed. For example, the display 31 can adjust the curvature in stages from a flat surface to a maximum curvature of 800R, and can adjust the curvature to an optimal value according to the viewing distance, installation location, and content, such as increasing the curvature when multiple people are watching and decreasing the curvature when watching alone for a more immersive experience.

[0019] The display 31 may be a flat display or a curved display with a fixed curvature. However, to display a wide angle range, such as 220 degrees horizontally, a curved display with a wide aspect ratio (e.g., 21:9 or 32:9) is more suitable than a flat display with a general aspect ratio (e.g., 16:9).

[0020] 1, the image display device 3 is assumed to be a curved display 31 with a variable curvature, and includes a curvature detection unit 32. The curvature detection unit 32 is provided as an option. The curvature detection unit 32 has a function of detecting the curvature of the display 31 and transmitting the detected curvature value as curvature information ri to the image correction device 1. For example, an internal signal linked to a curvature variable mechanism in the image display device 3 is used to detect the curvature.

[0021] The controller 4 is an input device for changing display settings and the like of an image output from the image correction device 1. A user or the like observing an image displayed on the image display device 3 can instruct display settings by operating a button, a touch panel, or the like provided on the controller 4. The controller 4 updates display setting information di based on a user operation and sends the updated information to the image correction device 1. Items of the display settings include an angle range, a display direction, a correction strength, and the like. The angle range is the range of angles displayed on the display 31 out of 360 degrees in the horizontal direction of the omnidirectional image. The display direction is the direction (latitude and longitude) of the omnidirectional image to be assigned to the center of the display 31. The correction strength is the correction strength of geometric correction, and details will be described later.

[0022] The visual distance detection device 5 is an optional device that has the function of detecting the distance between the viewer and the display 31 and providing the detected distance value as visual distance information ds to the image correction device 1. A distance sensor such as a TOF (Time of Flight) sensor is used to detect the distance.

[0023] 2. Configuration of Image Correction Device As shown in FIG. 1, the image correction device 1 includes an image input unit 21, an image storage unit 22, a geometric correction unit 23, an image output unit 24, and a parameter determination unit 25.

[0024] The image input unit 21 reads the input image Vin supplied from the image generating device 2 and writes it to the image storage unit 22. The pixel value of each pixel of the read input image Vin is a digitized value that takes one of 256 integer values ​​from "0" to "255" for each of the three reference colors, for example, R, G, and B.

[0025] The image input unit 21 may read the input image Vin from the image generation device 2 using an interface corresponding to the configuration of the image generation device 2. For example, if the input image Vin is supplied as a digital signal from the image generation device 2 via an HDMI (High-Definition Multimedia Interface (registered trademark)) interface or the like, the image input unit 21 may be configured to include an interface corresponding to that interface.

[0026] The image storage unit 22 includes a memory (RAM: Random Access Memory) large enough to store at least two screens of the input image Vin in order to store the entire input image Vin supplied from the image generation device 2. The image storage unit 22 uses a storage area for one screen (one frame) for writing the input image Vin from the image input unit 21, and uses the remaining storage area for one screen for reading from the geometric correction unit 23. The image storage unit 22 continuously performs writing and reading while alternately switching the roles of these storage areas for each screen.

[0027] The geometric correction unit 23 reads out the input image Vin stored in the image storage unit 22, performs geometric correction on the image data, and outputs the corrected image resulting from the geometric correction to the image output unit 24. This corrected image becomes the output image Vout. For example, when a predetermined angle range, such as 220 degrees horizontally, is displayed on the display 31, the geometric correction unit 23 performs geometric correction using a partial image range of the input image Vin, which is a spherical image, so as to obtain a corrected image of the predetermined angle range. As a result, a corrected image of the predetermined angle range is cut out from one frame of the spherical image and used as the output image Vout.

[0028] The geometric correction unit 23 performs geometric correction processing based on the correction strength parameter R, image rotation parameter P1, and angle range parameter P2 provided by the parameter determination unit 25. The correction strength parameter R is a parameter that specifies the strength of the geometric correction. The image rotation parameter P1 is a parameter that specifies the rotation angle about the X-axis, Y-axis, or Z-axis of the virtual spherical surface 50 onto which the omnidirectional image is mapped, and the image is scrolled by continuously changing this value. The angle range parameter P2 is a parameter that specifies the angle range in which correction processing is performed within 360 degrees in the horizontal direction of the omnidirectional image. For example, an angle range of 220 degrees, 180 degrees, 120 degrees, etc. is specified.

[0029] The geometric correction may be performed by calculating coordinates on the input image Vin corresponding to pixel coordinates on the output image Vout each time, or by using a correction table stored in a correction table storage unit (not shown). The correction table defines which pixel on the input image Vin stored in the image storage unit 22 should be referenced to determine the pixel value of each pixel in the output image Vout (corrected image) output by the geometric correction unit 23. The geometric correction unit 23 may be configured, for example, with a field programmable gate array (FPGA), an application specific integrated circuit (ASIC), a central processing unit (CPU), or a graphics processing unit (GPU).

[0030] The image output unit 24 converts the output image Vout (corrected image) output from the geometric correction unit 23 into an appropriate image signal and outputs it to the image display device 3. For example, if the image display device 3 is configured to display a digital image signal such as HDMI (registered trademark), the image output unit 24 may be configured to include an interface corresponding to that.

[0031] The parameter determination unit 25 determines the values ​​of the correction intensity parameter R, image rotation parameter P1, and angle range parameter P2 to be given to the geometric correction unit 23 based on the display setting information di transmitted from the controller 4, the viewing distance information ds transmitted from the viewing distance detection device 5, the curvature information ri transmitted from the curvature detection unit 32 of the image display device 3, etc.

[0032] The parameter determination unit 25 holds the initial values ​​of the correction strength parameter R, image rotation parameter P1, and angle range parameter P2, but updates the parameter values ​​when factors such as a change in display settings or observation conditions occur. The relationship between factors that cause parameters to be updated and the parameters that are reflected is as follows. Below, this is expressed as "factor that causes parameters to be updated / transmitted information / reflected parameters."

[0033] - Changing the angle range setting / display setting information di / angle range parameter - Changing the display direction setting / display setting information di / image rotation parameter - Changing the correction strength setting / display setting information di / correction strength parameter - Fluctuation in the distance between the viewer and the display / viewing distance information ds / correction strength parameter - Changing the display curvature / curvature information ri / correction strength parameter - Changing the display direction setting for horizontal scrolling / display setting information di / correction strength parameter

[0034] The parameter determination unit 25 may be configured, for example, by an MCU (Micro Controller Unit) or an MPU (Micro Processor Unit). Note that the parameter determination unit 25 may be a parameter determination device separate from the image correction device 1.

[0035] <3. Geometric Correction> The geometric correction performed by the geometric correction unit 23 of the image correction device 1 will be described. A schematic example of an image will be shown. Fig. 2 shows a celestial sphere image that is the input image Vin. This is a schematic illustration of an equirectangular celestial sphere image. The range of 360 degrees horizontally is expanded onto a plane.

[0036] Figures 3 and 4 are both examples of images obtained by applying geometric correction to the image in Figure 2, displayed on a 21:9 curved display, covering a horizontal range of 220 degrees. Figure 3 shows the image after applying general cylindrical screen correction, and it can be seen that road lines, buildings, etc. are distorted in an arc-like shape. For example, if an observer were to sit in the center of a large cylindrical screen and view the image in Figure 3, they would perceive it as a natural image. However, if the observer were to view the display in front of them from a distance, as in normal television viewing, the left and right sides would be distorted, creating an unnatural feeling.

[0037] FIG. 4 shows an example in which geometric correction according to this embodiment has been applied, and roads, buildings, and other features are corrected to be approximately linear. As a result, even when a viewer views the display in front of them from a distance, as in normal television viewing, there is no distortion on the left or right, and the image appears less strange. In this case, the degree to which the left and right curvatures are corrected to be linear depends on the correction strength. The appropriate correction strength also depends on the type of image and the external environment. For example, while FIG. 4 shows an image within an angular range of 220 degrees, the appropriate correction strength may change depending on factors such as the angle range, the distance between the viewer and the display, and the curvature of the display 31 in FIG. 1.

[0038] Next, the correction process will be described. In this embodiment, a method using virtual spherical and cylindrical surfaces is adopted.

[0039] 5 and 6 show models of the geometric correction process according to the embodiment. Fig. 5 is a perspective view of a virtual spherical surface 50 and a virtual cylindrical surface 51, and Fig. 6 is a plan view of the virtual spherical surface 50 and the virtual cylindrical surface 51 in Fig. 5 as viewed from above.

[0040] The virtual spherical surface 50 is a spherical surface on which a subject is projected in association with information of the input image Vin (spherical image), and is a spherical surface with a radius Ro and an origin at a point Oo in the XYZ coordinate system.

[0041] The imaginary cylindrical surface 51 is a cylindrical surface onto which the subject image projected onto the imaginary spherical surface 50 is projected, and is a cylindrical surface of radius Rc whose central axis passes through point Oc on the Z axis and is parallel to the Y axis. Point Oc is the intersection of the X' axis and the Y' axis. The X' axis is parallel to the X axis, and the Y' axis is parallel to the Y axis. The imaginary spherical surface 50 and the imaginary cylindrical surface 51 are always in contact at point A on the Z axis (XYZ coordinates (0,0,Ro)).

[0042] Here, the relative size of the radius Rc of the virtual cylindrical surface 51 to the radius Ro of the virtual spherical surface 50, i.e., Rc / Ro, is defined as a correction intensity parameter R, where R≧1. The value of the correction intensity parameter R is provided by a parameter determination unit 25 external to the geometric correction unit 23.

[0043] When the correction intensity parameter R=1, the radii Rc and Ro are the same size, which is the state when general cylindrical screen correction is applied. Figure 6 shows the state when the correction intensity parameter R=2. That is, the imaginary cylindrical surface 51 has its center at point Oc, and its radius Rc is twice the radius Ro of the imaginary spherical surface 50. Note that decreasing the value of the correction intensity parameter R will be expressed as "weakening the correction intensity," and increasing the value of the correction intensity parameter R will be expressed as "increasing the correction intensity."

[0044] The reason why the assumption that the imaginary spherical surface 50 and the imaginary cylindrical surface 51 always contact each other at point A is made is to reduce the number of elements that specify the geometric relationship between the imaginary spherical surface 50 and the imaginary cylindrical surface 51. This will be explained below. First, in order to specify the geometric relationship between the imaginary spherical surface 50 and the imaginary cylindrical surface 51, it is necessary to specify two elements: the relative relationship between the sizes of the radii of the two surfaces, and the relative relationship between the center positions of the two surfaces.

[0045] FIG. 7A shows an example in which the virtual spherical surface 50 and the virtual cylindrical surface 51 do not touch at point A. However, as shown in FIG. 7B , by increasing the radius Ro of the virtual spherical surface 50 without changing the position of the center Oo, the virtual spherical surface 50 can be modified to touch at point A. The results of the geometric correction are the same between FIG. 7A and FIG. 7B . For example, the pixels on the virtual spherical surface 50 projected onto the virtual cylindrical surface 51 are the same in FIG. 7A and FIG. 7B . Therefore, the two are equivalent. In other words, by setting the condition that the virtual spherical surface 50 and the virtual cylindrical surface 51 always touch at point A, the factors related to the results of the geometric correction can be narrowed down to the relative relationship between the radii of the virtual spherical surface 50 and the virtual cylindrical surface 51. Therefore, in the geometric correction of this embodiment, the "relative magnitude of the radius Rc of the virtual cylindrical surface 51 to the radius Ro of the virtual spherical surface 50," i.e., Rc / Ro, is used as the correction intensity parameter.

[0046] 6 is the range in the horizontal direction (longitude) of the celestial sphere image that is geometrically corrected by the image correction device 1, and is determined based on the value of the angle range parameter P2 given by the parameter determination unit 25. Fig. 6 shows a state when 2Θ = 220 degrees. The range of the celestial sphere image that is assigned to the range of 2Θ in the horizontal direction is determined based on the value of the image rotation parameter P1 given by the parameter determination unit 25. For example, when scrolling a displayed image in the horizontal direction, the virtual spherical surface 50 is rotated around the Y axis as the rotation axis.

[0047] The geometric correction process according to the embodiment is conceptually performed in the following steps (H1) to (H3). (H1) An input celestial sphere image is mapped onto a virtual spherical surface 50. (H2) The mapped image on the virtual spherical surface is projected (remapped) onto a virtual cylindrical surface 51 with the center Oo of the sphere as the viewpoint. (H3) The projected image on the virtual cylindrical surface is expanded onto a plane and output.

[0048] Fig. 8 schematically illustrates such geometric correction. First, as (H1) above, an image of the input image Vin that includes the angle range specified by the angle range parameter P2 is mapped onto the virtual sphere 50. This is the range indicated by the solid line on the virtual sphere 50 in Fig. 8. Note that the mapping itself may be performed on the entire omnidirectional image (an image of 360 degrees horizontally and 360 degrees vertically) onto the virtual sphere 50. In that case, the range of the output image Vout is the range indicated by the solid line on the virtual sphere 50 in Fig. 8.

[0049] In the above (H2), the image on the virtual spherical surface 50 is mapped onto the virtual cylindrical surface 51. This means that a vector pointing from the center Oo of the virtual spherical surface 50 onto the surface of the virtual spherical surface 50 is extended as it is, and virtual pixels on the surface of the virtual spherical surface 50 are mapped at the position where the vector reaches the virtual cylindrical surface 51. As a result, the image of the range indicated by the solid line on the virtual spherical surface 50, such as a range of 220 degrees horizontally, is mapped onto the virtual cylindrical surface 51.

[0050] In the above (H3), the virtual cylindrical surface 51 is developed, and the values ​​of the virtual pixels on the virtual cylindrical surface 51 are used as pixel data for the display 31. This is a process in which the image projected onto the virtual cylindrical surface 51 is directly transferred to a two-dimensional image with the circumferential direction as the horizontal direction. In FIG. 8, grids are shown on the virtual cylindrical surface 51, but for the sake of explanation, each grid corresponds to a pixel. The same grids are shown on the display 31, but the number of grids corresponds to the number of pixels in one horizontal line of the image. The number of grids on the virtual cylindrical surface 51 in the figure is the same as the number of grids on the display 31.

[0051] Although the size and curvature of the actual display 31 vary, the value of the virtual pixel on the virtual cylindrical surface 51 corresponding to each pixel of the display 31 is set to the value of each pixel in the geometrically corrected image.

[0052] However, in implementing the geometric correction process, it is necessary to calculate the position on the input image Vin that corresponds to the pixel position of the output image Vout (corrected image), so the concept is the reverse of the above steps (H1) to (H3).

[0053] This will be explained using Figures 9 and 10. As shown in Figure 9, assume that there are a predetermined number of pixels (number of squares) in the horizontal direction of the actual display 31. On the other hand, the imaginary cylindrical surface 51 does not actually exist, but is a virtual object calculated. Moreover, as will be described later, the radius Rc of the imaginary cylindrical surface 51 is variable. Figures 9 and 10 show cases where the radius Rc of the imaginary cylindrical surface 51 is different.

[0054] Geometric correction can also be considered a process of picking up pixel data in the output image Vout that corresponds to pixels of the display 31 from the virtual spherical surface 50. Therefore, the same number of pixels as the number of pixels in the horizontal and vertical directions of the display 31 are evenly allocated to a predetermined angular range of the virtual cylindrical surface 51. Then, on the virtual cylindrical surface 51, the pixels of the display 31, i.e., the pixel data of the output image Vout, are associated with the predetermined angular ranges in the horizontal and vertical directions displayed on the display 31.

[0055] Here, when the radius Rc of the imaginary cylindrical surface 51 is changed, the length L2 of the predetermined angle range of the imaginary cylindrical surface 51 also changes. The length L2 of the imaginary cylindrical surface 51 in Figures 9 and 10 is different. However, in either case, correspondence is performed according to the pixels of the display 31.

[0056] Then, pixel values ​​on the surface of the virtual spherical surface 50 are calculated on a vector line pointing from each pixel on the virtual cylindrical surface 51 to the center Oo of the virtual spherical surface 50. These become the pixel data values ​​of the output image Vout. If no corresponding pixel exists on the virtual spherical surface 50, that is, if the vector points between pixels, then pixel values ​​interpolated using surrounding pixels can be calculated. The pixel data obtained by calculating this for all pixels on the display 31 becomes the image data for one frame of the output image Vout.

[0057] The above-described geometric correction is performed, and in this embodiment, the radius of the imaginary cylindrical surface 51 is varied by the correction intensity parameter R. A method for determining the correction intensity parameter R by the parameter determination unit 25 will be described.

[0058] First, in the initial state of the system, the parameter determination unit 25 outputs the value of the correction intensity parameter R that is set in advance by design. The value of the correction intensity parameter R at this time is set to "R0".

[0059] Next, when the user operates the controller 4 to instruct a change in the correction intensity, the parameter determination unit 25 changes the value of the correction intensity parameter R based on the input display setting information di, that is, in this case, the correction intensity setting signal. The change in the value of the correction intensity parameter R is transmitted to the geometric correction unit 23, which changes the correction intensity of the geometric correction in the geometric correction unit 23. Specifically, the radius of the imaginary cylindrical surface 51 is changed. This also changes the degree of correction reflected in the image displayed on the display 31. By changing the correction intensity by operating the controller 4, the user can adjust the image on the display 31 so that it does not look strange to them.

[0060] The value of the correction intensity parameter R when adjustment of the correction intensity by user operation is completed (steady state) is set to "R1." When the user operates the controller 4 to change the image display settings, for example, and thereby change the observation conditions, the parameter determination unit 25 dynamically changes the value of the correction intensity parameter R from "R1" to "R2" based on at least one of the input display setting information di, viewing distance information ds, and curvature information ri. Four cases in which the correction intensity parameter R is dynamically changed are described below.

[0061] Case 1: When the angle range of the image is changed When display setting information di that changes the angle range of the image is provided from the controller 4, the parameter determination unit 25 dynamically changes the value of the correction intensity parameter R from “R1” to “R2”.

[0062] For example, assume that the correction intensity parameter R is in a steady state of R=R1 in FIG. 11A . Assume that R1=Rc / Ro=2.0. If the display settings are changed from this state to widen the angle range of the image, the value of the correction intensity parameter R is increased (R2>R1) as shown in FIG. 11B . For example, R2=2.5. Conversely, if the display settings are changed to narrow the angle range of the image, the value of the correction intensity parameter R is decreased (R2<R1) from the state of R1=2.0 shown in FIG. 12A to R1=1.5 shown in FIG. 12B.

[0063] The reason for this change is that as the angle range of the image becomes wider, the difference between the angle of view of the display 31 as seen from the observation position and the angle range of the image becomes larger, and therefore, increasing the correction strength can suppress image distortion. The difference becomes larger because the angle of view of the display as seen from the observation position remains constant but the angle range of the image becomes larger.

[0064] Case 2: When the distance between the observer and the display changes When the viewing distance information ds provided by the viewing distance detection device 5 changes due to a change in the distance between the observer and the display 31, the parameter determination unit 25 dynamically changes the value of the correction intensity parameter R from "R1" to "R2."

[0065] Specifically, when the distance between the viewer and the display 31 increases, the value of the correction intensity parameter R is increased (R2>R1) as shown in Figures 11A and 11B. Conversely, when the distance between the viewer and the display decreases, the value of the correction intensity parameter R is decreased (R2<R1) as shown in Figures 12A and 12B.

[0066] The reason for this change is that as the distance between the observer and the display increases, the difference between the angle of view of the display as seen from the observation position and the angular range of the image increases, and therefore increasing the correction strength can suppress image distortion. The difference increases because the angle of view of the display 31 as seen from the observation position decreases while the angular range of the image remains constant.

[0067] Case 3: When the curvature of the display is changed When the curvature information ri provided by the curvature detection unit 32 of the image display device 3 changes due to a change in the curvature of the display 31, the parameter determination unit 25 dynamically changes the value of the correction intensity parameter R from "R1" to "R2".

[0068] Specifically, when the radius of curvature of the display 31 increases, the value of the correction intensity parameter R is increased (R2>R1) as shown in Figures 11A and 11B. Conversely, when the radius of curvature of the display decreases, the value of the correction intensity parameter R is decreased (R2<R1) as shown in Figures 12A and 12B.

[0069] The reason for this change is that as the radius of curvature of the display 31 increases, the difference between the angle of view of the display as seen from the observation position and the angular range of the image increases, and therefore increasing the correction strength can suppress image distortion. The difference increases because the angular range of the image remains constant, but the angle of view of the display as seen from the observation position becomes smaller.

[0070] Case 4: When the displayed image is scrolled horizontally When display setting information for scrolling the displayed image horizontally, i.e., for moving the display direction of the image horizontally, is given from the controller 4, the parameter determination unit 25 dynamically changes the value of the correction intensity parameter R from "R1" to "R2."

[0071] Specifically, while the image is being scrolled horizontally, the value of the correction intensity parameter R is increased (R2>R1) as shown in Figures 11A and 11B. Meanwhile, while the image is not being scrolled horizontally (when the scrolling operation is completed), the value of the correction intensity parameter R is returned to the original value "R1."

[0072] The reason for this change is as follows: particularly when the correction strength is weak, i.e., when "R1" is relatively small, scrolling the image horizontally creates an image that combines vertical axis rotation and horizontal movement, which is likely to induce visually induced motion sickness. However, by increasing the correction strength, the horizontal movement component in the image weakens, resulting in an image similar to an image with only a vertical axis rotation component, which makes it possible to suppress the induction of visually induced motion sickness in the observer.

[0073] 13 shows an example of processing by the geometric correction unit 23. The geometric correction unit 23 performs the processing of FIG. 13 for each frame of the input image Vin input from the image storage unit 22.

[0074] In practice, the geometric correction unit 23 holds in a register the correction intensity parameter R, image rotation parameter P1, and angle range parameter P2 supplied from the parameter determination unit 25 for each frame of the input image Vin, and executes arithmetic processing accordingly. As a result, geometric correction is executed for each frame of the input image Vin, and a geometrically corrected image is output. In this embodiment, the geometric correction processing ultimately follows the concept shown in the following flowchart.

[0075] In the geometric correction unit 23, the process branches in step S10 depending on whether or not there has been a change in the correction intensity parameter R. If the value of the correction intensity parameter R from the parameter determination unit 25 has not been changed from the value at the time of processing the previous frame, the geometric correction unit 23 proceeds from step S10 to step S20.

[0076] If the value of the correction strength parameter R from the parameter determination unit 25 has changed from the value at the time of processing the previous frame, the geometric correction unit 23 changes the setting of the correction strength used in the calculation in step S11 and then proceeds to step S20. This change in the setting of the correction strength means that the radius of the virtual cylindrical surface 51 is changed in the geometric correction calculation according to the changed value of the correction strength parameter R.

[0077] In the geometric correction unit 23, the process branches in step S20 depending on whether or not the angle range parameter P2 has been changed. If the value of the angle range parameter P2 from the parameter determination unit 25 has not been changed from the value at the time of processing the previous frame, the geometric correction unit 23 proceeds from step S20 to step S30.

[0078] When the value of the angle range parameter P2 from the parameter determination unit 25 has been changed from the value at the time of processing the previous frame, the geometric correction unit 23 changes the setting of the angle range for performing the geometric correction process in step S21, and then proceeds to step S30. Changing the setting of the angle range means changing the range of the omnidirectional image that is the input image Vin, which is output as the corrected image Vout, in accordance with the newly specified angle range parameter P2.

[0079] In the geometric correction unit 23, the process branches in step S30 depending on whether the image rotation parameter P1 has been changed. If the value of the image rotation parameter P1 from the parameter determination unit 25 has not been changed from the value at the time of processing the previous frame, the geometric correction unit 23 proceeds from step S30 to step S40.

[0080] When the value of the image rotation parameter P1 from the parameter determination unit 25 has changed from the value at the time of processing the previous frame, the geometric correction unit 23 changes the image rotation setting in step S31 and then proceeds to step S40. Changing the image rotation setting means rotating the virtual spherical surface 51 onto which the omnidirectional image, which is the input image Vin, is mapped around the X-axis, Y-axis, or Z-axis in accordance with the image rotation parameter P1.

[0081] In step S40, the geometric correction unit 23 performs geometric correction. That is, the input image Vin is mapped onto a virtual spherical surface 50, the virtual spherical surface 50 is rotated according to the image rotation parameter P1, and the range according to the angle range parameter 51 is mapped onto a virtual cylindrical surface 51 according to the correction strength parameter R, and then the image is expanded onto a plane. Through the above processing, the geometric correction of one frame of image is performed, and one frame of output image Vout is obtained.

[0082] Next, the correction intensity setting process by the parameter determination unit 25 will be described with reference to Fig. 14 to Fig. 18. Fig. 14 shows the process of the parameter determination unit 25 in response to a user operation. The parameter determination unit 25 performs the process of Fig. 14 until it is determined in step S106 that the display has ended.

[0083] The parameter determination unit 25 initially outputs a correction intensity parameter R=R0 that is set in advance by design.

[0084] In step S101, the parameter determination unit 25 determines whether a signal to change the correction intensity has been received as the display setting information di from the controller 4. In other words, this is a signal that is received when the user performs an operation to change the correction intensity. If this signal has not been received, the parameter determination unit 25 proceeds to step S103 and does not change the correction intensity parameter R=R0.

[0085] If the display setting information di includes a signal for changing the correction intensity, the parameter determination unit 25 proceeds from step S101 to step S102 and checks whether the signal is for increasing the correction intensity.

[0086] If the signal is to increase the correction intensity, the parameter determination unit 25 proceeds to step S105, where it increases the correction intensity parameter R to R1 (where R1 > R0) to increase the correction intensity. That is, it increases the value of the correction intensity parameter R to be transmitted to the geometric correction unit 23. If the signal is to decrease the correction intensity, the parameter determination unit 25 proceeds to step S104, where it decreases the correction intensity parameter R to R1 (where R1 < R0) to decrease the correction intensity. That is, it decreases the value of the correction intensity parameter R to be transmitted to the geometric correction unit 23.

[0087] If no user operation is performed and no information for changing the correction strength is input, the parameter determination unit 25 proceeds to step S103, where the current correction strength is maintained. Therefore, the value of the correction strength parameter R is maintained as "R0" or "R1."

[0088] 15 shows an example of processing by the parameter determination unit 25 when the angle range of the image in Case 1 above is changed. It is assumed that the parameter determination unit 25 outputs a correction intensity parameter R=R1 to the geometric correction unit 23. Note that although the correction intensity parameter R may be set to R0 without any adjustment by a user operation, the processing in FIGS. 15 to 18 will be described below as processing after manual adjustment is performed in the processing in FIG. 14 and the correction intensity parameter R=R1. The parameter determination unit 25 also executes the processing in FIG. 15 until it is determined in step S207 that image display has ended.

[0089] 15, the parameter determination unit 25 determines whether or not a signal for changing the angle range of the image has been received based on the display setting information di from the controller 4. If the signal has not been received, the parameter determination unit 25 proceeds to step S203 and does not change the correction intensity parameter R=R1.

[0090] If the display setting information di includes a signal for changing the angle range of the image, the parameter determination unit 25 proceeds from step S201 to step S202 to check whether the signal is for widening the angle range. If the signal is for widening the angle range, the parameter determination unit 25 proceeds to step S205 to increase the correction intensity parameter R=R2 (where R2>R1) to strengthen the correction intensity. If the signal is for narrowing the angle range, the parameter determination unit 25 proceeds to step S204 to decrease the correction intensity parameter R=R2 (where R2<R1) to weaken the correction intensity.

[0091] In this case, it is desirable to set the value "R2" of the correction intensity parameter R according to the change width of the angle range. The larger the difference between the angle range before and after the change, the larger the difference between "R2" and "R1" as the value of the correction intensity parameter R. In this way, an appropriate correction intensity is set according to the amount of change in the angle range of the image.

[0092] 16 shows an example of processing by the parameter determination unit 25 when there is a change in the distance between the viewer and the display in the above-mentioned Case 2. It is assumed that the parameter determination unit 25 outputs a correction intensity parameter R=R1. The parameter determination unit 25 also executes the processing in FIG. 16 until it is determined in step S307 that the image display has ended.

[0093] In step S301, the parameter determination unit 25 determines whether there is a change in the viewing distance information ds from the viewing distance detection device 5. If there is no change in the viewing distance information ds, the parameter determination unit 25 proceeds to step S303 and does not change the correction intensity parameter R=R1.

[0094] If there is a change in the viewing distance information ds, the parameter determination unit 25 proceeds to step S302 to check whether the distance between the viewer and the display 31 has increased. If the distance has increased, the parameter determination unit 25 proceeds to step S305 to increase the correction intensity parameter R=R2 (where R2>R1) to strengthen the correction intensity. If the distance has decreased, the parameter determination unit 25 proceeds to step S304 to decrease the correction intensity parameter R=R2 (where R2<R1) to weaken the correction intensity.

[0095] In this case, it is desirable to set the value "R2" of the correction intensity parameter R according to the amount of change in distance. The greater the difference between the distance between the observer and the display 31 before the change and the distance after the change, the greater the difference between "R2" and "R1" as the value of the correction intensity parameter R. In this way, an appropriate correction intensity is set according to the amount of change in the distance between the observer and the display 31.

[0096] 17 shows an example of processing by the parameter determination unit 25 when the curvature of the display 31 is changed in the above-mentioned Case 3. It is assumed that the parameter determination unit 25 outputs a correction intensity parameter R=R1. The parameter determination unit 25 also executes the processing in FIG. 17 until it is determined in step S407 that the image display has ended.

[0097] In step S401, the parameter determination unit 25 determines whether or not there is a change in the curvature information ri from the curvature detection unit 32. If there is no change in the curvature information ri, the parameter determination unit 25 proceeds to step S403 and does not change the correction strength parameter R=R1.

[0098] If the curvature information ri has changed, the parameter determination unit 25 proceeds to step S402 to check whether the radius of curvature of the display 31 has increased. If the radius of curvature has increased, the parameter determination unit 25 proceeds to step S405 to increase the correction strength parameter R=R2 (where R2>R1) to strengthen the correction strength. If the radius of curvature has decreased, the parameter determination unit 25 proceeds to step S404 to decrease the correction strength parameter R=R2 (where R2<R1) to weaken the correction strength.

[0099] In this case, it is desirable to set the value "R2" of the correction intensity parameter R according to the amount of change in curvature. The greater the difference between the curvature of the display 31 before the change and the curvature after the change, the greater the difference between "R2" and "R1" as the value of the correction intensity parameter R. In this way, an appropriate correction intensity is set according to the amount of change in the curvature of the display 31.

[0100] 18 shows an example of the processing performed by the parameter determination unit 25 when the image displayed in Case 4 is scrolled horizontally. The parameter determination unit 25 outputs a correction intensity parameter R=R1. The parameter determination unit 25 continues the processing in FIG. 18 until it is determined in step S507 that the image display has ended.

[0101] In step S501, the parameter determination unit 25 determines whether or not a signal for scrolling an image has been provided in the display setting information di from the controller 4. If the signal has not been provided, the parameter determination unit 25 proceeds to step S503 and does not change the correction intensity parameter R=R1.

[0102] If the display setting information di includes a signal instructing image scrolling, the parameter determination unit 25 proceeds to step S502 to check whether the signal instructs horizontal scrolling. If the signal instructs vertical scrolling rather than horizontal scrolling, the parameter determination unit 25 proceeds to step S503 and does not change the correction intensity parameter R=R1.

[0103] If the signal indicates horizontal scrolling, the parameter determination unit 25 proceeds to step S505, where it increases the correction strength parameter R to R2 (where R2>R1) to strengthen the correction strength. Then, it returns to step S501. Therefore, if the scrolling operation continues, it proceeds again to steps S501, S502, and S505, where the correction strength parameter R continues to be set to R2.

[0104] In this case, the value "R2" of the correction strength parameter R is set according to the scroll speed. When the scroll speed is fast, the correction strength is increased significantly, and when the scroll speed is slow, the correction strength is increased slightly. When the scroll speed is constant, the value of "R2" may be set according to that constant speed. The speed of scrolling may also change depending on the user's operation state. Therefore, when the speed changes while scrolling is ongoing, the value of "R2" may be changed according to the speed each time in step S505.

[0105] In this way, by increasing the correction strength during scrolling and adjusting the degree of correction strength depending on the scrolling speed, it is possible to display scroll images that are less likely to cause motion sickness and that provide a less sense of discomfort.

[0106] When the scrolling operation is completed, the parameter determination unit 25 proceeds from step S501 to step S503, and returns the correction strength parameter R to R1, so that after the scrolling is completed, the image is displayed with the correction strength in the steady state.

[0107] 19 will now be used to describe an example of the configuration of an information processing device 70 that can be used as the image correction device 1. The information processing device 70 can be configured as, for example, a dedicated workstation, a general-purpose personal computer, a mobile terminal device, or the like.

[0108] 19 executes various processes in accordance with programs stored in a nonvolatile memory unit 74, such as a read-only memory (ROM) 72 or an electrically erasable programmable read-only memory (EEP-ROM), or programs loaded from a storage unit 79 to a random access memory (RAM) 73. The RAM 73 also stores data necessary for the CPU 71 to execute various processes. The CPU 71 executes various control and calculation functions, which will be described later, using programs.

[0109] In addition, a processor other than the CPU 71 may also be provided, such as a GPU (Graphics Processing Unit), a GPGPU (General-purpose computing on graphics processing units), or an AI (Artificial Intelligence) processor.

[0110] The CPU 71, ROM 72, RAM 73, and nonvolatile memory unit 74 are interconnected via a bus 83. The bus 83 is also connected to an input / output interface 75.

[0111] An input unit 76 consisting of operators and operation devices is connected to the input / output interface 75. For example, the input unit 76 may be various operators and operation devices such as a keyboard, a mouse, keys, a dial, a touch panel, a touch pad, a remote controller, etc. A user operation is detected by the input unit 76, and a signal corresponding to the input operation is interpreted by the CPU 71.

[0112] The input / output interface 75 is also connected, either integrally or separately, to a display unit 77 such as an LCD (Liquid Crystal Display) or an organic EL (Electro-Luminescence) panel, and an audio output unit 78 such as a speaker.

[0113] The display unit 77 performs various displays as a user interface. The display unit 77 is configured, for example, by a display device provided in the housing of the information processing device 70 or a separate display device connected to the information processing device 70. The display unit 77 displays various images on the display screen based on instructions from the CPU 71. The display unit 77 also displays various operation menus, icons, messages, etc., i.e., a GUI (Graphical User Interface), based on instructions from the CPU 71. The display unit 77 may function as the image display device 3 in FIG. 1.

[0114] The input / output interface 75 may be connected to a storage unit 79 configured with a solid state drive (SSD) or a hard disk drive (HDD), or a communication unit 80 configured with a modem, etc. The storage unit 79 can be used to store various data. A database can also be constructed in the storage unit 79. The communication unit 80 performs communication between devices and communication processing via a network. For example, when the information processing device 70 is used as the image correction device 1, the communication unit 80 performs communication between devices, and can receive an input image Vin from the image generation device 2 shown in FIG. 1 and transmit an output image Vout to the image display device 3.

[0115] A drive 82 is also connected to the input / output interface 75 as needed, and a removable recording medium 81 such as a flash memory, a memory card, a magnetic disk, an optical disk, or a magneto-optical disk is appropriately attached. The drive 82 allows data files such as image files and various computer programs to be read from the removable recording medium 81. The read data files are stored in the storage unit 79, and images and sounds contained in the data files are output on the display unit 77 and the audio output unit 78. In addition, the computer programs and the like read from the removable recording medium 81 are installed in the storage unit 79 as needed.

[0116] In this information processing device 70, software can be installed via network communication by the communication unit 80 or via a removable recording medium 81. Alternatively, the software may be stored in advance in the ROM 72, the storage unit 79, or the like.

[0117] The image correction device 1 according to the embodiment can be realized by such an information processing device 70. For example, it is possible to realize an image correction device 1 including a geometric correction unit 23, or an image correction device 1 including a geometric correction unit 23 and a parameter determination unit 25. Alternatively, an independent device serving as the parameter determination unit 25 can also be realized by the information processing device 70.

[0118] 6. Summary and Modifications The image correction device 1 according to the embodiment provides the following effects.

[0119] The image correction device 1 according to the embodiment includes a geometric correction unit 23 that variably sets the radius Rc of a virtual cylindrical surface 51 that always touches a virtual spherical surface 50 at one point (point A) according to a correction intensity parameter R, maps an input image Vin onto the virtual spherical surface 50, projects the image mapped onto the virtual spherical surface 50 onto the virtual cylindrical surface 51, and generates an output image Vout that corresponds to the pixels of the display 31. This image correction device 1 sets the correction intensity parameter R to the relative size of the radius Rc of the virtual cylindrical surface 51 to the radius Ro of the virtual spherical surface 50 when the virtual spherical surface 50 and the virtual cylindrical surface 51 are arranged so that they always touch at one point. This makes it possible to realize a system that can adjust the correction intensity with a correction intensity parameter that uses the ratio of the radii Ro and Rc. In the embodiment, an example has been described in which the radius Rc of the imaginary cylindrical surface 51 is variably set in accordance with the correction intensity parameter R, but the radius Rc of the imaginary cylindrical surface 51 may be fixed and the radius Ro of the imaginary spherical surface 50 may be variably set in accordance with the correction intensity parameter R. Furthermore, both the radius Rc of the imaginary cylindrical surface 51 and the radius Ro of the imaginary spherical surface 50 may be variably set in accordance with the correction intensity parameter R. In these cases as well, the system can adjust the correction intensity with a correction intensity parameter that uses the ratio of the radii Ro and Rc.

[0120] The image correction device 1 of the embodiment is exemplified as including a parameter determination unit 25 that variably sets the correction intensity parameter R in response to external factors and instructs the geometric correction unit 23 about the set correction intensity parameter R. The parameter determination unit 25 automatically variably sets the correction intensity parameter R in response to external factors such as a change in display settings or a change in the user's viewing conditions. This allows an image to be displayed with the correction intensity automatically adjusted even if the external viewing conditions change.

[0121] In the embodiment, an example has been given in which the parameter determination unit 25 increases or decreases the set value of the correction intensity parameter R in response to a change in the setting of the angular range of the image to be displayed. The user can change the angular range of the image to be displayed on the display 31 using the controller 4. The parameter determination unit 25 sets the correction intensity parameter R so that the correction intensity is increased when the angular range to be displayed on the display 31 is widened, and decreased when the angular range is narrowed. This automatically suppresses image distortion in response to the angular range of the image to be displayed, making it possible to display a wide-field-of-view image with less discomfort.

[0122] In the embodiment, an example has been given in which the parameter determination unit 25 increases or decreases the set value of the correction intensity parameter R in accordance with a change in the distance between the display 31 of the image display device 3 and the observer observing the screen. A user looking at the screen of the display 31 does not always remain still, and the distance between the observer and the screen may change. When the distance between the observer and the display 31 increases, the parameter determination unit 25 increases the value of the correction intensity parameter R to strengthen the correction intensity, and conversely, when the distance between the observer and the display 31 decreases, the parameter determination unit 25 decreases the value of the correction intensity parameter R to weaken the correction intensity. In this way, when the distance between the observer and the display 31 changes, image distortion is automatically suppressed, making it possible to display a wide-field-of-view image with less discomfort.

[0123] In the embodiment, an example has been given in which the parameter determination unit 25 increases or decreases the set value of the correction intensity parameter R in accordance with a change in the curvature of the screen of the image display device 3. When a display 31 having a screen with a variable curvature is used, the parameter determination unit 25 increases the value of the correction intensity parameter R to strengthen the correction intensity when the radius of curvature of the display 31 increases, and conversely, decreases the value of the correction intensity parameter R to weaken the correction intensity when the radius of curvature of the display 31 decreases. In this way, when the curvature of the display 31 changes, image distortion is automatically suppressed, making it possible to display a wide-field-of-view image with less discomfort.

[0124] In the embodiment, an example has been given in which the parameter determination unit 25 increases the set value of the correction intensity parameter R when the displayed image on the image display device 3 is scrolled horizontally. The user can scroll the image vertically or horizontally using the controller 4. The parameter determination unit 25 increases the value of the correction intensity parameter R to strengthen the correction intensity while the image is being scrolled horizontally. This weakens the horizontal movement component in the image during horizontal scrolling, making the image similar to an image with only a vertical axis rotation component, thereby making it possible to suppress the induction of visually-induced motion sickness in the observer.

[0125] In the embodiment, it has been described that the parameter determination unit 25 changes the increase amount of the set value of the correction strength parameter R according to the horizontal scroll speed. The parameter determination unit 25 changes the degree to which the correction strength is increased according to the image scroll speed. That is, when the scroll speed is fast, the correction strength is increased significantly, and when the scroll speed is slow, the correction strength is increased slightly. This also realizes optimization of correction according to the scroll speed.

[0126] In the embodiment, an example has been given in which the parameter determination unit 25 returns the set value of the correction intensity parameter R to the state before the start of scrolling, i.e., the steady-state value "R1", in response to the end of horizontal scrolling. By returning the correction intensity to the value before scrolling when the scrolling of the image ends, the parameter determination unit 25 can resume image display at the original correction intensity.

[0127] In the embodiment, an example has been given in which the parameter determination unit 25 increases or decreases the set value of the correction intensity parameter R in response to a user operation. For example, the correction intensity can be adjusted by a user, who is an observer of the display 31, operating the controller 4. This allows the user, for example, to adjust the image to a state where it looks most natural while viewing the screen of the display 31. The state adjusted in this way is set as the steady-state correction intensity parameter R=R1, and the correction intensity is dynamically varied in response to external factors based on this correction intensity. This allows the correction intensity to be dynamically varied in response to external factors based on the optimal state for the user, thereby achieving more appropriate image display.

[0128] In the embodiment, the correction intensity parameter R is the ratio of the radius Rc of the imaginary cylindrical surface 51 to the radius Ro of the imaginary spherical surface 50. By setting the condition that the imaginary spherical surface 50 and the imaginary cylindrical surface 51 always contact each other at one point (point A), it is possible to narrow down the elements related to the geometric correction result to the relative relationship between the radii of the imaginary spherical surface 50 and the imaginary cylindrical surface 51. Therefore, the relative magnitude (Rc / Ro) of the radius Rc of the imaginary cylindrical surface 51 to the radius Ro of the imaginary spherical surface 50 is set as the correction intensity parameter R, thereby making it possible to improve the efficiency of calculations.

[0129] In the embodiment, an example of geometric correction has been described in which the geometric correction unit 23 calculates pixel values ​​of the virtual spherical surface 50 corresponding to a line extending from a position on the virtual cylindrical surface 51 corresponding to each pixel of the display 31 to the center Oo of the virtual spherical surface 50 to generate the output image Vout. As a result, in the geometric correction using the virtual cylindrical surface 51, it is possible to calculate positions on the input image Vin corresponding to the pixel positions of the output image Vout (corrected image).

[0130] In the embodiment, the input image Vin that is mapped onto the virtual spherical surface 50 by the geometric correction unit 23 is a celestial sphere image, but it may be a fisheye image. When a celestial sphere image or a fisheye image is displayed on a flat or curved screen, the correction by the geometric correction unit 23 is useful, and by adjusting the correction strength in particular, it is possible to create an image that is less unnatural in response to various external factors.

[0131] 14 and 15 to 18 are executed by, for example, a CPU, a DSP (digital signal processor), an AI processor, or an information processing device 70 including any of these. That is, the program of the embodiment is a program that causes the information processing device 70 to execute the following processes: variably setting the radius Rc of a virtual cylindrical surface 51 that always touches the virtual spherical surface 50 at one point or the radius Ro of the virtual spherical surface according to the correction intensity parameter R; mapping the input image Vin onto the virtual spherical surface 50; projecting the image mapped onto the virtual spherical surface 50 onto the virtual cylindrical surface 51; and generating the image of the projected virtual cylindrical surface 51 as an output image Vout corresponding to the pixels of the display 31.

[0132] The program according to the embodiment may also be configured to variably set the correction intensity parameter R in response to an external factor and to apply the set correction intensity parameter R to the geometric correction process.

[0133] With such a program, the image correction device 1 according to the embodiment can be realized in an information processing device 70, for example, specifically, a computer device, a mobile terminal device, or other device capable of executing information processing.

[0134] Such a program can be pre-recorded on a hard disk drive (HDD) as a recording medium built into a computer or other device, or on a ROM within a microcomputer having a CPU. Alternatively, the program can be temporarily or permanently stored (recorded) on a removable recording medium such as a flexible disk, a CD-ROM (Compact Disc Read Only Memory), an MO (Magneto Optical) disc, a DVD (Digital Versatile Disc), a Blu-ray Disc (registered trademark), a magnetic disk, a semiconductor memory, or a memory card. Such removable recording media can be provided as so-called packaged software. Furthermore, such a program can be installed on a personal computer or the like from a removable recording medium, or can be downloaded from a download site via a network such as a LAN (Local Area Network) or the Internet.

[0135] Furthermore, such a program is suitable for widely providing information processing devices 70 that realize the image correction device 1 according to the embodiment. For example, by downloading the program to a mobile terminal device such as a smartphone or tablet, a mobile phone, a personal computer, a game device, a video device, a PDA (Personal Digital Assistant), or the like, these devices can be made into information processing devices 70 that function as the image correction device 1 according to the present disclosure.

[0136] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0137] The present technology can also be configured as follows. (1) An image correction device comprising: a geometric correction unit that variably sets the radius of a virtual cylindrical surface that always touches a virtual spherical surface at one point or the radius of the virtual spherical surface in accordance with a correction intensity parameter, maps input image data onto the virtual spherical surface, projects the image mapped onto the virtual spherical surface onto the virtual cylindrical surface, and sets the projected image of the virtual cylindrical surface as output image data corresponding to pixels of a display device. (2) The image correction device described in (1) above, comprising: a parameter determination unit that variably sets the correction intensity parameter in accordance with an external factor and instructs the geometric correction unit about the set correction intensity parameter. (3) The image correction device described in (2) above, in which the parameter determination unit increases or decreases the set value of the correction intensity parameter in accordance with a change in the setting of an angle range of an image to be displayed. (4) The image correction device described in (2) or (3) above, in which the parameter determination unit increases or decreases the set value of the correction intensity parameter in accordance with a change in the distance between a screen of an image display device and a viewer observing the screen. (5) The image correction device according to any one of (2) to (4) above, wherein the parameter determination unit increases or decreases the set value of the correction intensity parameter in accordance with a change in the curvature of the screen of the image display device. (6) The image correction device according to any one of (2) to (5) above, wherein the parameter determination unit increases the set value of the correction intensity parameter when the image displayed on the image display device is scrolled horizontally. (7) The image correction device according to (6) above, wherein the parameter determination unit changes the increase amount of the set value of the correction intensity parameter in accordance with the horizontal scrolling speed. (8) The image correction device according to (6) or (7) above, wherein the parameter determination unit returns the set value of the correction intensity parameter to the state before the scrolling started in accordance with the end of the horizontal scrolling. (9) The image correction device according to any one of (2) to (8) above, wherein the parameter determination unit increases or decreases the set value of the correction intensity parameter in accordance with a user operation. (10) The image correction device according to any one of (1) to (9) above, wherein the correction intensity parameter is the ratio of the radius of the virtual cylindrical surface to the radius of the virtual spherical surface.(11) The image correction device according to any of (1) to (10), wherein the geometric correction unit calculates pixel values ​​of the virtual spherical surface corresponding to a line extending from a position on the virtual cylindrical surface corresponding to each pixel of the image display device to the center of the virtual sphere, thereby generating the output image data. (12) The image correction device according to any of (1) to (11), wherein the input image data mapped onto the virtual spherical surface by the geometric correction unit is a celestial sphere image or a fisheye image. (13) An image correction method, wherein the image correction device variably sets a radius of a virtual cylindrical surface that always touches the virtual spherical surface at one point or a radius of the virtual sphere according to a correction intensity parameter, maps input image data onto the virtual sphere, projects the image mapped onto the virtual sphere onto the virtual cylindrical surface, and sets the projected image of the virtual cylindrical surface as output image data corresponding to pixels of a display device. (14) A program that causes an information processing device to execute the following processes: variably setting the radius of a virtual cylindrical surface that always touches a virtual spherical surface at one point or the radius of the virtual spherical surface according to a correction intensity parameter; mapping input image data onto the virtual spherical surface; projecting the image mapped onto the virtual spherical surface onto the virtual cylindrical surface; and converting the projected image of the virtual cylindrical surface into output image data corresponding to the pixels of a display device.

[0138] REFERENCE SIGNS LIST 1 Image correction device 2 Image generation device 3 Image display device 4 Controller 5 Viewing distance detection device 21 Image input unit 22 Image storage unit 23 Geometric correction unit 24 Image output unit 25 Parameter determination unit 31 Display 32 Curvature detection unit 50 Virtual sphere 51 Virtual surface Vin Input image Vout Output image ri Curvature information ds Viewing distance information di Display setting information

Claims

1. An image correction device comprising a geometric correction unit that variably sets the radius of a virtual cylindrical surface that always touches a virtual spherical surface at one point or the radius of the virtual spherical surface in accordance with a correction intensity parameter, maps input image data onto the virtual spherical surface, projects the image mapped onto the virtual spherical surface onto the virtual cylindrical surface, and converts the projected image of the virtual cylindrical surface into output image data corresponding to the pixels of a display device.

2. The image correction device according to claim 1, further comprising a parameter determination section that variably sets the correction intensity parameter in response to an external factor and indicates the set correction intensity parameter to the geometric correction section.

3. The image correction device according to claim 2, wherein the parameter determination unit increases or decreases the set value of the correction strength parameter in response to a change in the setting of the angle range of the image to be displayed.

4. The image correction device according to claim 2, wherein the parameter determination section increases or decreases the set value of the correction intensity parameter in accordance with a change in the distance between the screen of the image display device and the viewer viewing the screen.

5. The image correction device according to claim 2, wherein the parameter determination section increases or decreases the set value of the correction strength parameter in accordance with a change in the curvature of the screen of the image display device.

6. The image correction device according to claim 2, wherein the parameter determination section increases the set value of the correction intensity parameter when the image displayed on the image display device is scrolled horizontally.

7. The image correction device according to claim 6, wherein the parameter determination section changes the increment of the set value of the correction strength parameter in accordance with the horizontal scroll speed.

8. The image correction device according to claim 6, wherein the parameter determination unit returns the set value of the correction intensity parameter to the state before scrolling started in response to the end of horizontal scrolling.

9. The image correction device according to claim 2, wherein the parameter determination section increases or decreases the set value of the correction intensity parameter in response to a user operation.

10. The image correction device according to claim 1, wherein the correction strength parameter is a ratio of the radius of the virtual cylindrical surface to the radius of the virtual spherical surface.

11. The image correction device according to claim 1, wherein the geometric correction unit calculates pixel values ​​of the virtual sphere corresponding to a line extending from a position on the virtual cylindrical surface corresponding to each pixel of the image display device to the center of the virtual sphere, and generates the output image data.

12. The image correction device according to claim 1, wherein the input image data that the geometric correction unit maps onto the virtual sphere is a celestial sphere image or a fisheye image.

13. An image correction method, in which an image correction device variably sets the radius of a virtual cylindrical surface that always touches a virtual spherical surface at one point or the radius of the virtual spherical surface in accordance with a correction intensity parameter, maps input image data onto the virtual spherical surface, projects the image mapped onto the virtual spherical surface onto the virtual cylindrical surface, and sets the projected image of the virtual cylindrical surface as output image data corresponding to the pixels of a display device.

14. A program that causes an information processing device to execute the following process: variably setting the radius of a virtual cylindrical surface that always touches a virtual spherical surface at one point or the radius of the virtual spherical surface according to a correction intensity parameter; mapping input image data onto the virtual spherical surface; projecting the image mapped onto the virtual spherical surface onto the virtual cylindrical surface; and converting the projected image of the virtual cylindrical surface into output image data corresponding to the pixels of a display device.

Images