Information processing device, information processing method, and program

Abstract

An information processing device (1) comprises a viewpoint position detection unit (15) and a correction processing unit (11). The viewpoint position detection unit (15) detects a viewpoint position (PS) of an observer. The correction processing unit (11) detects a gaze point (GP) on a viewpoint image (VPI) corresponding to the viewpoint position (PS). The correction processing unit (11) blurs an outer edge portion of the viewpoint image (VPI) that is apart from the gaze point (GP) in a parallax direction.

Description

[Technical Field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] Crosstalk can be a problem for display devices that display stereoscopic or multi-viewpoint images. To address this issue, a technique has been proposed to suppress crosstalk by using an inverse correction process that performs a transformation opposite to that caused by crosstalk. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2013-183426 [Patent Document 2] Japanese Patent Application Laid-Open No. 2013-214052 Summary of the Invention [Problem to be solved by the invention]

[0004] Inverse correction involves image processing that reduces the signal values ​​of pixels whose brightness increases due to crosstalk and increases the signal values ​​of pixels whose brightness decreases due to crosstalk. However, the tonal range of an image is limited to 0 to 255. For pixels whose signal values ​​decrease or increase beyond this saturation constraint, the signal values ​​are clipped to 0 or 255. In areas where the signal value exceeds the saturation constraint, image correction is insufficient, making it difficult to effectively suppress crosstalk.

[0005] Therefore, the present disclosure proposes an information processing device, an information processing method, and a program that can effectively suppress crosstalk. [Means for solving the problem]

[0006] According to the present disclosure, there is provided an information processing device including a viewpoint position detection unit that detects the viewpoint position of an observer, and a correction processing unit that detects a fixation point on a viewpoint image that corresponds to the viewpoint position and blurs an outer edge of the viewpoint image that is away from the fixation point in a parallax direction. Also, according to the present disclosure, there is provided an information processing method in which information processing of the information processing device is executed by a computer, and a program that causes a computer to realize the information processing of the information processing device. [Brief explanation of the drawings]

[0007] [Figure 1] FIG. 1 illustrates an example of an information processing apparatus. [Figure 2] FIG. 10 is a diagram illustrating an example of crosstalk correction processing. [Figure 3] FIG. 10 is a diagram illustrating an example of crosstalk correction processing. [Figure 4] FIG. 10 is a diagram illustrating an example of crosstalk correction processing. [Figure 5] 10A and 10B are diagrams illustrating another example of crosstalk correction processing. [Figure 6] 10A and 10B are diagrams illustrating another example of crosstalk correction processing. [Figure 7] 10A and 10B are diagrams illustrating other application examples of crosstalk correction processing. [Figure 8] 10A and 10B are diagrams illustrating other application examples of crosstalk correction processing. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same components are designated by the same reference numerals, and redundant description will be omitted.

[0009] The explanation will be given in the following order. [1. Overview] [2. Configuration of information processing device] [3. Crosstalk correction processing] [4. Effects] [5. Modifications]

[0010] [1. Overview] The present disclosure proposes a method for suppressing crosstalk using blurring. The blurring is selectively performed on an image that is far from the observer's gaze point. The blurring obscures a portion of the image that is contaminated by crosstalk, making the crosstalk less noticeable. Blurring an image that is far from the gaze point, which is less noticeable to the observer, does not cause degradation in image quality. Therefore, crosstalk can be suppressed while minimizing degradation in image quality. The crosstalk correction process of the present disclosure will be specifically described below.

[0011] [2. Configuration of information processing device] FIG. 1 is a diagram illustrating an example of an information processing device 1 for performing the crosstalk correction processing of the present disclosure.

[0012] The information processing device 1 includes a processing device 10 and a storage device 20. The crosstalk correction process of the present disclosure can be applied to crosstalk between two viewpoints and crosstalk between multiple viewpoints (three or more viewpoints). A 3D (dimensional) display is known as a display that targets two viewpoints. Below, an example will be described in which the crosstalk correction process of the present disclosure is applied to a naked-eye 3D display.

[0013] The processing device 10 includes a correction processing unit 11, a viewpoint position detection unit 15, and a blur adjustment unit 16.

[0014] The correction processing unit 11 corrects the input image IM IN Blur processing is applied to the output image IM out Generates the input image IM IN is the original image or a corrected image obtained by applying some correction processing to the original image. IN includes a plurality of viewpoint images VPI. In this embodiment, since 3D display is performed, the input image IM IN are the multiple viewpoint images VPI, and the left eye input image LI IN and right eye input image RI IN (See Figure 3).

[0015] The viewpoint position detection unit 15 detects the viewpoint position PS (see FIG. 2) of the viewer based on head tracking. For example, the viewpoint position PS (left eye viewpoint position PSL) at which the viewpoint image VPI of the left eye is observed is the center of the pupil of the left eye. The viewpoint position PS (right eye viewpoint position PSR) at which the viewpoint image VPI of the right eye is observed is the center of the pupil of the right eye. The viewpoint position detection unit 15 outputs viewpoint position information including coordinate information of the viewpoint position PS to the correction processing unit 11 and the blur adjustment unit 16.

[0016] The correction processing unit 11 detects a gaze point GP (see FIG. 2) on the viewpoint image VPI corresponding to the viewpoint position PS. The gaze point GP is detected by tracing the path of light incident on the viewpoint position PS. The left eye gaze point GPL is located at the center of the viewpoint image VPI for the left eye. The right eye gaze point GPR is located at the center of the viewpoint image VPI for the right eye. The correction processing unit 11 blurs the outer edge of the viewpoint image VPI that is away from the gaze point GP in the parallax direction.

[0017] The blur adjustment unit 16 detects the movement speed of the viewpoint position PS based on the viewpoint position information. The blur adjustment unit 16 adjusts the distribution of the blur amount in the viewpoint image VPI based on the movement speed of the viewpoint position PS. For example, the blur adjustment unit 16 reduces the blur amount of all pixels in the viewpoint image VPI more significantly as the movement speed of the viewpoint position increases.

[0018] The storage device 20 stores, for example, a program 29 executed by the processing device 10 and parameter information 21. The program 29 is a program that causes a computer to execute information processing according to the present disclosure. The processing device 10 performs various processes in accordance with the program 29 stored in the storage device 20. The storage device 20 may be used as a work area for temporarily storing processing results of the processing device 10. The storage device 20 includes, for example, any non-transitory storage medium such as a semiconductor storage medium or a magnetic storage medium. The storage device 20 includes, for example, an optical disk, a magneto-optical disk, or a flash memory. The program 29 is stored, for example, in a non-transitory storage medium readable by a computer.

[0019] The processing device 10 is, for example, a computer configured with a processor and a memory. The memory of the processing device 10 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). The processing device 10 executes a program 29 to function as a correction processing unit 11, a viewpoint position detection unit 15, and a blur adjustment unit 16.

[0020] [3. Crosstalk correction processing] An example of the crosstalk correction process of the present disclosure will be described below with reference to FIGS.

[0021] Blurring is a crosstalk correction process that blurs the outer edges of the viewpoint image VPI that are farther away from the center of the viewpoint image VPI (the observer's gaze point GP) in the parallax direction. "Blurring" means making it less visible. Blurring processes include smoothing processes using Gaussian filters, mosaic processing, and level correction processes that lower the signal value to darken the outer edges.

[0022] As shown in FIG. 2, a plurality of viewpoint images VPI are arranged in the parallax direction on the screen SCR. The parallax direction is the direction in which parallax occurs. For example, the direction in which a line connecting the right and left eyes of the viewer projected onto the screen SCR extends is the parallax direction. One viewpoint image VPI includes a plurality of line images LP adjacent to each other in the parallax direction. One viewpoint image VPI has a width equivalent to the width of the plurality of line images.

[0023] In the example shown on the left side of Figure 2, the original image IM org Multiple line images contained in LP org is assigned to one viewpoint image VPI. A line image LP that has not undergone crosstalk correction processing org is displayed as is, so the left eye perception image LI view and right eye perception image RI view In the left eye perceived image LI, crosstalk is strongly perceived. view is the image perceived by the observer's left eye, and the image perceived by the right eye RI viewis the perceived image perceived by the observer's right eye. The perceived image is an image in which multiple viewpoint images VPI are mixed together due to crosstalk.

[0024] In the example on the right side of Figure 2, the corrected image IM c Multiple line images contained in LP c is assigned to one viewpoint image VPI. c The blurred image is mixed with the left eye perceived image LI due to crosstalk. view and right eye perception image RI view In this case, crosstalk becomes less noticeable.

[0025] The correction processing unit 11 selectively blurs one or more line images LP among multiple adjacent line images LP where the gaze point GP is not located. Light emitted from the outer edge of the viewpoint image VPI is incident at a position shifted from the center of the pupil. Therefore, the line images LP at the outer edge of the viewpoint image VPI are hardly recognized by the observer. However, light may spread due to the influence of aberrations of the lens LE, and the line images LP at the outer edge may be recognized as crosstalk components. In this disclosure, the center of the viewpoint image VPI, which is recognized as the left eye image and the right eye image, is referred to as the normal viewpoint, and positions other than the normal viewpoint are referred to as intermediate viewpoints. The line images LP at the intermediate viewpoints are unlikely to be recognized as the left eye image and the right eye image. Therefore, the correction processing unit 11 selectively blurs the line images LP at the intermediate viewpoints.

[0026] 3, the correction processing unit 11 varies the magnitude of blur in the line image LP depending on the distance from the gaze point GP. IN The five line images LP included in the image RI are assigned viewpoint indices L1 to L5. The viewpoint index L3 is the left eye's normal viewpoint, and the viewpoint indices L1, L2, L4, and L5 are the left eye's intermediate viewpoints. INThe five line images LP included in the image LP are assigned viewpoint indices R1 to R5, respectively. The viewpoint index R3 is the right eye's normal viewpoint, and the viewpoint indices R1, R2, R4, and R5 are the right eye's intermediate viewpoints.

[0027] For example, Gaussian blurring is used as the blurring process. The magnitude of the blur is represented by a standard deviation σ. The correction processing unit 11 blurs the line image LP more as it is farther from the gaze point GP. The magnitude of the blur (standard deviation σ) is represented by a monotonic function that monotonically increases from the center of the viewpoint image VPI toward the end of the viewpoint image VPI. In the example of FIG. 3, the monotonic function is represented by a cosine curve or a cosine squared curve. However, the monotonic function is not limited to this. Information regarding the correspondence between the viewpoint index and the magnitude of the blur is included in the parameter information 21.

[0028] 4, the correction processing unit 11 determines the correspondence between the pixel groups of the screen SCR and each viewpoint image VPI, the allocation of viewpoint indexes within the viewpoint image VPI, and the distribution of the magnitude of blur (amount of blur) within the viewpoint image VPI based on the viewpoint position information of the observer. Based on the determined information, the correction processing unit 11 generates a multi-level blur image in which the amount of blur is controlled in multiple levels according to the distance from the gaze point GP, and generates a corrected image IM c Output as

[0029] In FIG. 3, Gaussian blurring is used as the blurring process, but the blurring process is not limited to this. For example, as shown in FIG. 5, by reducing the brightness of the line image LP of the intermediate viewpoint, it is possible to make the outer edge of the viewpoint image VPI less noticeable than the central portion. For example, the magnitude of the blur is defined as a brightness control value of the line image LP. The brightness control value is expressed as a monotonic function that monotonically increases from the center of the viewpoint image VPI toward the edge of the viewpoint image VPI. The correction processing unit 11 reduces the signal values ​​of all pixels included in the line image LP by the brightness control value. Information regarding the brightness control value is included in the parameter information 21.

[0030] The correction processing unit 11 generates the corrected image IM in accordance with the timing when the observer's viewpoint position PS (observer's head) moves so that an image without blur is displayed at the gaze point GP. c However, if the moving speed of the viewpoint position PS is high, the corrected image IM c If the change of viewpoint position PS is not made in time, the viewer may notice a line image LP (blurred image) at the intermediate viewpoint. Therefore, as shown in Fig. 6, the correction processing unit 11 can vary the distribution of the blur amount in the viewpoint image VPN depending on the moving speed of the viewpoint position PS.

[0031] For example, a threshold value of 1 or more is set for the movement speed of the viewpoint position PS. A blur adjustment value is set for each threshold value. The blur adjustment value is expressed as a monotonic function that monotonically increases as the movement speed of the viewpoint position PS increases. When the movement speed of the viewpoint position PS is greater than the threshold value, the correction processing unit 11 reduces the amount of blur of all intermediate viewpoint line images LP by the blur adjustment value corresponding to the threshold value. The greater the movement speed of the viewpoint position PS, the smaller the correction amount (blur amount) of the blur processing. Information regarding the threshold value and the blur adjustment value is included in the parameter information 21.

[0032] A similar problem may occur when the measurement accuracy of the viewpoint position PS by head tracking is low. Therefore, the correction processing unit 11 may set a blur adjustment value for each measurement accuracy of the viewpoint position PS and vary the distribution of the blur amount in the viewpoint image VPN depending on the measurement accuracy.

[0033] [4. Effects] The information processing device 1 has a viewpoint position detection unit 15 and a correction processing unit 11. The viewpoint position detection unit 15 detects the viewpoint position PS of the observer. The correction processing unit 11 detects a fixation point GP on the viewpoint image VPI corresponding to the viewpoint position PS. The correction processing unit 11 blurs the outer edge of the viewpoint image VPI that is away from the fixation point GP in the parallax direction. In the information processing method of this embodiment, the processing of the information processing device 1 described above is executed by a computer. A program 29 of this embodiment causes the computer to realize the processing of the information processing device 1 described above.

[0034] According to this configuration, the viewpoint image VPI that has been subjected to the blurring process is mixed with crosstalk. Therefore, crosstalk is less likely to be perceived. Images at the outer edge that are far from the gaze point GP are less likely to be perceived by the observer. Therefore, even if the image at the outer edge is blurred, the image quality is less likely to be impaired. Therefore, crosstalk can be reduced while suppressing deterioration of image quality due to blurring.

[0035] The viewpoint image VPI includes a plurality of line images LP adjacent in the parallax direction. The correction processing unit 11 selectively blurs one or more line images LP where the gaze point GP is not located, among the plurality of line images LP.

[0036] According to this configuration, blurring is not performed on the line image LP near the fixation point GP, which is easily recognized by the viewer, and therefore, deterioration of image quality due to blurring is unlikely to occur.

[0037] The correction processing unit 11 blurs the line image LP more as it is farther from the fixation point GP.

[0038] This configuration makes it possible to effectively reduce crosstalk while suppressing deterioration of image quality due to blurring.

[0039] The correction processing unit 11 varies the distribution of the blur amount in the viewpoint image VPI according to the moving speed of the viewpoint position PS.

[0040] According to this configuration, it is possible to control the ease with which a blurred image is perceived when the viewpoint position PS moves.

[0041] The faster the movement speed of the viewpoint position PS, the smaller the amount of blur.

[0042] With this configuration, blurring is less noticeable when the viewpoint position PS moves quickly.

[0043] The effects described in this specification are merely examples and are not limiting, and other effects may also be present.

[0044] [5. Modifications] 7 and 8 are diagrams showing other application examples of the crosstalk correction process.

[0045] In the above-described embodiment, the crosstalk correction process of the present disclosure is applied to a naked-eye 3D display. However, the information processing of the present disclosure may also be applied to an eyeglass-type 3D display.

[0046] The crosstalk correction process of the present disclosure can also be applied to crosstalk between multiple viewpoints of three or more viewpoints. In the example of FIG. 7, the information processing of the present disclosure is applied to a display that displays four viewpoint images VPI, but the number of viewpoints may be three, five, or more. In the example of FIG. 7, the four viewpoint images VPI are distributed to viewpoint positions PS1, PS2, PS3, and PS4 by a lenticular lens LE, but the means for spatially separating the viewpoint images VPI is not limited to the lenticular lens LE. A parallax barrier can also be used as the spatial separation means. Furthermore, a projector array method can also be applied as a method for displaying multi-viewpoint images.

[0047] In the example of FIG. 7, parallax occurs only in the horizontal direction. Therefore, four viewpoint images VPI corresponding to viewpoint positions PS1 to PS4 are displayed alternately in the horizontal direction on the screen SCR. However, as shown in FIG. 8, parallax may occur in both the horizontal and vertical directions. In this case, the four viewpoint images VPI are arranged in a matrix in the horizontal and vertical directions on the screen SCR. The line image LP is a closed linear image such as a circle or a square. In the example of FIG. 8, the viewpoint image VPI is, for example, a square image. The line image LP is a rectangular linear image centered on the gaze point GP. One viewpoint image VPI includes multiple pixels PX. The correction processing unit 11 determines the correspondence between the pixel groups on the screen SCR and each viewpoint image VPI so that the center of the viewpoint image VPI is the gaze point GP. For example, the distribution of the blur amount is set as a concentric circular distribution centered on the gaze point GP.

[0048] [Note] The present technology can also be configured as follows. (1) a viewpoint position detection unit that detects the viewpoint position of an observer; a correction processing unit that detects a fixation point on a viewpoint image corresponding to the viewpoint position and blurs an outer edge portion of the viewpoint image that is distant from the fixation point in a parallax direction; An information processing device having the above. (2) the viewpoint image includes a plurality of line images adjacent in a parallax direction, The correction processing unit selectively blurs one or more line images in which the gaze point is not located, among the plurality of line images. The information processing device according to (1) above. (3) The correction processing unit blurs the line image more as it is farther from the gaze point. The information processing device according to (2) above. (4) The correction processing unit varies the distribution of the blur amount in the viewpoint image depending on the moving speed of the viewpoint position. The information processing device according to any one of (1) to (3) above. (5) The greater the moving speed of the viewpoint position, the smaller the blur amount. The information processing device according to (4) above. (6) Detecting the observer's viewpoint position, detecting a gaze point on a viewpoint image corresponding to the viewpoint position; blurring an outer edge of the viewpoint image that is distant from the gaze point in a parallax direction; 10. A computer-implemented information processing method comprising: (7) Detecting the observer's viewpoint position, detecting a gaze point on a viewpoint image corresponding to the viewpoint position; blurring an outer edge of the viewpoint image that is distant from the gaze point in a parallax direction; A program that makes a computer do something. [Explanation of symbols]

[0049] 1. Information processing equipment 11 Correction processing section 15 Viewpoint position detection unit GP gaze point LP Line Image PS viewpoint position VPI viewpoint image

Claims

1. a viewpoint position detection unit that detects the viewpoint position of an observer; a correction processing unit that detects a fixation point on a viewpoint image corresponding to the viewpoint position and blurs an outer edge portion of the viewpoint image that is distant from the fixation point in a parallax direction; and the correction processing unit varies a distribution of blur amounts in the viewpoint image in accordance with a moving speed of the viewpoint position; The greater the moving speed of the viewpoint position, the smaller the blur amount. Information processing device.

2. the viewpoint image includes a plurality of line images adjacent in a parallax direction, The correction processing unit selectively blurs one or more line images in which the gaze point is not located, among the plurality of line images. The information processing device according to claim 1 .

3. The correction processing unit blurs the line image more as it is farther from the gaze point. The information processing device according to claim 2 .

4. Detecting the observer's viewpoint position, detecting a gaze point on a viewpoint image corresponding to the viewpoint position; blurring an outer edge of the viewpoint image that is distant from the gaze point in a parallax direction; a distribution of blur amounts in the viewpoint images is varied according to a moving speed of the viewpoint position; The greater the moving speed of the viewpoint position, the smaller the blur amount.

10. A computer-implemented information processing method comprising:

5. Detecting the observer's viewpoint position, detecting a gaze point on a viewpoint image corresponding to the viewpoint position; blurring an outer edge of the viewpoint image that is distant from the gaze point in a parallax direction; a distribution of blur amounts in the viewpoint images is varied according to a moving speed of the viewpoint position; The greater the moving speed of the viewpoint position, the smaller the blur amount. A program that makes a computer do something.

Images