Information Processing Apparatus, Information Processing Method, and Program

Abstract

An information processing device (1) includes a cost evaluating unit (12) and a correction processing unit (11). The cost evaluating unit (12) evaluates a correction residual after a crosstalk correction process, as a cost (CS). The correction processing unit (11) implements a plurality of different crosstalk correction processes while determining a correction range and a correction amount on the basis of the cost (CS).

Description

Technical Field

[0001] The present invention relates to an information processing apparatus, an information processing method, and a program.

Background Art

[0002] In a display device that displays a stereoscopic image or a multi-viewpoint image, image degradation due to crosstalk becomes a problem. Therefore, a technique has been proposed in which crosstalk is suppressed by inverse correction processing that performs conversion opposite to the conversion caused by crosstalk.

Prior Art Document

Patent Document

[0003]

Patent Document 1

Non-Patent Document

[0004]

Non-Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In inverse correction processing, image processing is performed to reduce the signal value of pixels whose luminance increases due to crosstalk and increase the signal value of pixels whose luminance decreases. However, the gradation range of an image is limited to 0 to 255. For pixels whose signal value is reduced or increased beyond this saturation constraint, the signal value is clipped to 0 or 255. In a region where the signal value exceeds the saturation constraint, image correction becomes insufficient, and it is difficult to suppress crosstalk well.

[0006] Therefore, the present disclosure proposes an information processing apparatus, an information processing method, and a program that can satisfactorily suppress crosstalk.

Means for Solving the Problems

[0007] According to the present disclosure, there is provided an information processing apparatus including: a cost evaluation unit that evaluates, as a cost, a correction residual after crosstalk correction processing; and a correction processing unit that performs a plurality of different crosstalk correction processes while determining a correction range and a correction amount based on the cost. Further, according to the present disclosure, there are provided an information processing method in which the information processing of the information processing apparatus is executed by a computer, and a program that causes a computer to realize the information processing of the information processing apparatus.

Brief Description of the Drawings

[0008]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Figure 18

Figure 19

Figure 20

Figure 21

Figure 22

Figure 23

Figure 24

Figure 25

Embodiments for Carrying Out the Invention

[0009] Hereinafter, embodiments of the present disclosure will be described in detail with reference to the drawings. In the following embodiments, the same parts are denoted by the same reference numerals, and redundant descriptions are omitted.

[0010] Note that the description will be made in the following order. [1. Outline of Multi-Stage Crosstalk Correction Process] [2. Configuration of Information Processing Apparatus] [3. Crosstalk Correction Process] [3-1. Inverse Correction Process] [3-2. Compression Process] [3-3. Blurring Process] [4. Specific Example of Multi-Stage Crosstalk Correction Process] [5. Effects] [6. Modification Example] [6-1. Modification Example 1] [6-2. Modification Example 2] [6-3. Modification Example 3] [6-4. Modification Example 4] [6-5. Modification Example 5]

[0011] [1. Overview of Multi-Stage Crosstalk Correction Processing] FIG. 1 is a diagram for explaining the overview of multi-stage crosstalk correction processing.

[0012] This disclosure proposes a method of performing multiple crosstalk correction processes in a layered manner by repeating the simulation and cost evaluation of the perceived image IM view . In the simulation, the perceived image IM view of the observer after the crosstalk correction process is simulated. The perceived image IM view is an image in which a plurality of viewpoint images VPI after the crosstalk correction process are mixed by crosstalk. In the cost evaluation, based on the simulation results, the correction residual after the crosstalk correction process is evaluated as the cost CS.

[0013] The correction residual is caused by, for example, the saturation of the signal value due to the crosstalk correction process and the side effects on the image quality of the crosstalk correction process. The side effects mean, for example, the blurring of the image caused by the blurring process and the reduction of the contrast caused by the compression of the luminance range. The correction residual is calculated, for example, as the difference between the image that should be originally displayed (the original image IM org ) and the perceived image IM view after the crosstalk correction process.

[0014] Based on the cost CS obtained in the most recent cost evaluation, the correction range and correction amount of the subsequent crosstalk correction process are determined. In the example of FIG. 1, the simulation and the cost evaluation are each performed N times (N is an integer of 2 or more). In the i-th (i is an integer from 1 to N) simulation, the perceived image IM obtained by performing the i crosstalk correction processes from the 1st to the i-th on the original image IM org ​view,i is simulated. In the i-th cost evaluation, the cost CS view,i for the perceptual image IM i is evaluated.

[0015] Based on the cost CS i , the correction range and correction amount of the (i + 1)-th crosstalk correction process are determined. The correction range and correction amount are determined such that the cost CS i after the crosstalk correction process is smaller than the cost CS i+1 obtained in the most recent cost evaluation. Based on the determined correction range and correction amount, the (i + 1)-th crosstalk correction process is performed. In this way, multiple crosstalk correction processes are performed while determining the correction range and correction amount so as to reduce the cost CS.

[0016] [2. Configuration of Information Processing Apparatus] FIG. 2 is a diagram showing an example of an information processing apparatus 1 for performing the multi-stage crosstalk correction process described above.

[0017] The information processing apparatus 1 includes a processing apparatus 10 and a storage apparatus 20. The crosstalk correction process of the present disclosure can be applied to crosstalk between two viewpoints and crosstalk between multiple viewpoints of three or more viewpoints. As a display for two viewpoints, a 3D (Dimension) display is known. Hereinafter, an example in which the multi-stage crosstalk correction process of the present disclosure is applied to a 3D display will be described.

[0018] The processing apparatus 10 includes a correction processing unit 11 and a cost evaluation unit 12.

[0019] The correction processing unit 11 performs a plurality of crosstalk correction processes on the original image IM org to generate an output image IM out . The original image IM org includes a plurality of viewpoint images VPI. In the present embodiment, since 3D display is performed, the original image IM org includes, as a plurality of viewpoint images VPI, a left-eye original image LI organd the right-eye original image RI org are included.

[0020] The correction processing unit 11 includes a correction amount determination unit 13 and a corrected image generation unit 14.

[0021] The correction amount determination unit 13 determines the correction range and correction amount of subsequent crosstalk correction processing based on the cost CS obtained in the most recent cost evaluation. The correction amount determination unit 13 determines that there is a correction residual when the correction residual is greater than a preset residual threshold value. The correction amount determination unit 13 determines the image area where a correction residual is determined as the correction range, and determines the correction amount according to the magnitude of the correction residual.

[0022] For example, the correction amount determination unit 13 determines the correction range and correction amount of the (i + 1)-th crosstalk correction processing based on the cost CS obtained in the i-th cost evaluation. i The correction amount determination unit 13 acquires the parameters required for the crosstalk correction processing from the storage device 20. The parameters required for the crosstalk correction processing are stored in the storage device 20 as parameter information 21.

[0023] The corrected image generation unit 14 performs subsequent crosstalk correction processing to generate a corrected image IM c The corrected image generation unit 14 holds the corrected image IM generated in the most recent crosstalk correction processing until the subsequent crosstalk correction processing is performed. The corrected image generation unit 14 performs subsequent crosstalk correction processing on the held most recent corrected image IM c to generate a new corrected image IM c c is generated.

[0024] For example, the corrected image generation unit 14 stores the corrected image IM generated in the i-th crosstalk correction processing until the (i + 1)-th crosstalk correction processing is performed. The corrected image IM c,i is the corrected image IM obtained by subjecting the original image IM c,i to i crosstalk correction processes from the first to the i-th. org c ​​is the corrected image IM c,i By performing the (i + 1)-th crosstalk correction process on, the original image IM org is subjected to (i + 1) crosstalk correction processes from the 1st to the (i + 1)-th, and the corrected image IM c,i+1 is generated.

[0025] The cost evaluation unit 12 simulates the observer's perceived image IM c,i after the i-th crosstalk correction process based on the corrected image IM view,i obtained by the i-th crosstalk correction process. The cost evaluation unit 12 evaluates the correction residual after the i-th crosstalk correction process as the cost CS i based on the simulation result. In this way, the correction processing unit 11 performs a plurality of different crosstalk correction processes while determining the correction range and correction amount based on the cost CS.

[0026] The number (N + 1) of crosstalk correction processes is, for example, 3 or more. The correction processing unit 11 performs a plurality of crosstalk correction processes within a range where the number of crosstalk correction processes does not exceed a preset number in order to balance the image quality of the perceived image IM view and the calculation load.

[0027] [3. Crosstalk correction process] Hereinafter, an example of the crosstalk correction process used in the information processing of the present disclosure will be described. The correction processing unit 11 performs, for example, an inverse correction process, a compression process, and a blurring process as the crosstalk correction process.

[0028] [3-1. Inverse correction process] FIG. 3 and FIG. 4 are diagrams showing an example of the inverse correction process.

[0029] The inverse correction process is a crosstalk correction process that performs a conversion opposite to the conversion caused by crosstalk on the input image IM IN . The input image IM IN is the original image IM org or the corrected image IM cis. In the example of FIG. 3, as a crosstalk model, the left-eye input image LI IN and the right-eye input image RI IN are mixed at a mixing rate α using a linear mixing model. However, the crosstalk model is not limited to the one shown in FIG. 3. A crosstalk model may be constructed based on a non-linear mixing model with an expanded range where crosstalk occurs.

[0030] In the crosstalk model of FIG. 3, for the input signals of the left-eye input image LI IN and the right-eye input image RI IN , gamma processing, mixing processing, and degamma processing are performed in this order. The mixing processing is represented by a 4×4 determinant. Therefore, in the inverse correction processing, an inverse matrix operation processing using the inverse matrix of this determinant is performed. The correction processing unit 11 performs gamma processing, inverse matrix operation processing, and degamma processing on the left-eye input image LI IN and the right-eye input image RI IN in this order, and generates a left-eye corrected image LI c and a right-eye corrected image RI c . The correction processing unit 11 integrates the left-eye corrected image LI c and the right-eye corrected image RI c and outputs them as a corrected image IM c .

[0031] Information on the gamma value γ used in the gamma processing and degamma processing, and the mixing rate α used in the inverse matrix operation processing is stored in the storage device 20 as parameter information 21. When performing the inverse correction processing, the correction processing unit 11 acquires information on the mixing rate α and the gamma value γ included in the parameter information 21 from the storage device 20.

[0032] The left side of FIG. 4 shows an example where no inverse correction processing is performed. In the example of FIG. 4, the input signal of the left-eye input image LI IN is larger than the input signal of the right-eye input image RI IN . Therefore, when crosstalk occurs, the flow of the input signal from the left-eye side to the right-eye side increases. As a result, the luminance of the left-eye perceived image LI view is lower than that of the left-eye original image LI orgbecomes smaller, and the luminance of the right-eye perception image RI view is greater than that of the right-eye original image RI org .

[0033] The right side of FIG. 4 shows an example of performing inverse correction processing. In crosstalk, the input signal on the left-eye side decreases and the input signal on the right-eye side increases. Therefore, in the inverse correction processing, signal processing is performed such that the input signal on the left-eye side increases and the input signal on the right-eye side decreases. When crosstalk occurs between the left-eye corrected image LI c and the right-eye corrected image RI c , the change in the signal value due to the inverse correction processing and the change in the perceived luminance due to crosstalk cancel each other out. As a result, the luminance of the left-eye perception image LI view becomes equal to that of the left-eye input image LI IN , and the luminance of the right-eye perception image RI view becomes equal to that of the right-eye input image RI IN . Thereby, crosstalk is eliminated.

[0034] [3-2. Compression Processing] FIGS. 5 to 9 are diagrams showing an example of compression processing.

[0035] The compression processing is a crosstalk correction processing that compresses the luminance range of the correction range. In the inverse correction processing shown in FIG. 4, since the contrast between the viewpoint images VPI is not large, the signal value after the inverse correction processing does not exceed the limit value of the signal (for example, the gradation range of 0 to 255). However, as shown in FIG. 6, when the contrast between the viewpoint images VPI becomes large, the signal value after the inverse correction processing may exceed the limit value.

[0036] When the signal value after the inverse correction processing exceeds the upper limit value (255) or the lower limit value (0) of the signal, the signal value after the inverse correction processing is clipped to the upper limit value or the lower limit value of the signal. In the example of FIG. 6, due to the inverse correction processing, the signal value of the left-eye corrected image LI c′ exceeds the upper limit value, and the signal value of the right-eye corrected image RI c′ falls below the lower limit value. Therefore, due to clipping, the left-eye corrected image LI cand a right-eye corrected image RI c are generated. As a result, the luminance of the left-eye perception image LI view becomes smaller than that of the left-eye input image LI IN , and the luminance of the right-eye perception image RI view becomes larger than that of the right-eye input image RI IN .

[0037] When the signal value changes due to clipping, the change in the signal value due to inverse correction processing and the change in the perceived luminance due to crosstalk no longer cancel each other out. Therefore, as shown in FIG. 7, crosstalk remains, and the right-eye input image RI view appears as a ghost in the left-eye perception image LI IN , and the left-eye input image LI view appears as a ghost in the right-eye perception image RI IN . Therefore, the correction processing unit 11 compresses the luminance range of the input signal so that clipping does not occur.

[0038] As shown in FIG. 8, the correction processing unit 11 divides the input image IM IN into a plurality of local regions LBA. The local region LBA is set as, for example, a rectangular region of 64 pixels × 64 pixels, but the size of the local region LBA is not limited to this.

[0039] The correction processing unit 11 calculates the signal value after inverse correction processing for all the pixels in the local region LBA. For the pixels whose signal value after inverse correction processing exceeds the limit value, the correction processing unit 11 calculates the signal value of the portion exceeding the limit value as the saturation value. For example, the correction processing unit 11 calculates the saturation value of the portion exceeding the upper limit value of the signal as a positive value, and calculates the saturation value of the portion below the lower limit value of the signal as a negative value.

[0040] As shown in FIG. 9, for each local region LBA, the correction processing unit 11 determines the maximum value L c′ and the minimum value L max of the saturation values of the left-eye corrected image LI min , and the maximum value R c′ and the minimum value R max of the saturation values of the right-eye corrected image RI minCalculate them as range parameters. The correction processing unit 11 selects one or more local regions LBA including pixels whose signal values are saturated. For each selected local region LBA, the correction processing unit 11 determines the amount of compression of the luminance range such that the signal values of all pixels within the local region LBA do not exceed the limit value by the inverse correction processing as the correction amount of the compression processing. For example, L max and R max The larger of the two is LR max Let it be, and L min and R min The smaller of the two is LR min Let it be. The correction processing unit 11 calculates the compression amount C as C = (255 + LR max − LR min ) / 255. The correction processing unit 11 compresses the luminance range of each selected local region LBA based on the determined compression amount. The correction processing unit 11 does not perform compression processing on local regions LBA that do not include pixels whose signal values are saturated.

[0041] As shown in FIG. 5, the correction processing unit 11 generates a local luminance compressed image BCI in which one or more local regions LBA are compressed for the input image IM IN . In the local luminance compressed image BCI, a luminance step occurs at the boundary of local regions LBA with different luminance ranges. Therefore, the correction processing unit 11 generates a plurality of local luminance compressed images BCI in which the luminance range is adjusted to a range that is not saturated by the inverse correction processing for each local region LBA while shifting the division position of the input image IM IN . The correction processing unit 11 overlaps and averages the plurality of local luminance compressed images BCI. Thereby, a corrected image IM c is generated by synthesizing a plurality of local luminance compressed images BCI. The corrected image IM c is an image in which the luminance step between local regions LBA is reduced. Information regarding the size of the local region LBA and the set position of the local region LBA (the division position of the input image IM IN ) is included in the parameter information 21.

[0042] [3-3. Blurring Processing] FIGS. 10 to 14 are diagrams showing an example of blurring processing.

[0043] The blurring process is a crosstalk correction process that blurs the outer edge of the viewpoint image VPI away from the center of the viewpoint image VPI (the observer's fixation point) in the parallax direction. "Blurring" means making it in a state where it cannot be clearly seen. The blurring process includes, in addition to the smoothing process using a Gaussian filter or the like, a mosaic process, a level correction process of lowering the signal value to darken the outer edge, and the like.

[0044] As shown in FIG. 10, a plurality of viewpoint images VPI are arranged in the parallax direction on the screen SCR. The parallax direction is the direction in which parallax occurs. For example, the extending direction of the line obtained by projecting the line connecting the observer's right eye and left eye onto the screen SCR is the parallax direction. One viewpoint image VPI includes a plurality of line images LP adjacent to each other in the parallax direction. One viewpoint image VPI has a width corresponding to a plurality of line images.

[0045] In the example shown on the left side of FIG. 10, the plurality of line images LP org included in the original image IM org are assigned to one viewpoint image VPI. Since the line image LP without crosstalk correction is displayed as it is, strong crosstalk is recognized in the left-eye perception image LI org and the right-eye perception image RI. view view

[0046] In the example on the right side of FIG. 10, the plurality of line images LP c included in the corrected image IM c are assigned to one viewpoint image VPI. The corrected image IM c is subjected to a blurring process. Since the image subjected to the blurring process is mixed in by crosstalk, crosstalk is less likely to be recognized in the left-eye perception image LI view and the right-eye perception image RI. view

[0047] For example, the correction processing unit 11 acquires the viewpoint position information detected by head tracking. The viewpoint position information includes the coordinate information of the viewpoint position PS from which the viewpoint image VPI is observed. For example, the viewpoint position PS (left-eye viewpoint position PSL) from which the viewpoint image VPI (left-eye correction image LI c ) of the left eye is observed is the center of the pupil of the left eye. The viewpoint position PS (right-eye viewpoint position PSR) from which the viewpoint image VPI (right-eye correction image RI c ) of the right eye is observed is the center of the pupil of the right eye.

[0048] The correction processing unit 11 detects the fixation point GP on the viewpoint image VPI corresponding to the viewpoint position PS. The fixation point GP is detected by tracing the path of the light incident on the viewpoint position PS. The left-eye fixation point GPL is located at the center of the viewpoint image VPI of the left eye. The right-eye fixation point GPR is located at the center of the viewpoint image VPI of the right eye. The correction processing unit 11 selectively blurs one or more line images LP among a plurality of adjacent line images LP where the fixation point GP is not located.

[0049] The light radiated from the outer edge of the viewpoint image VPI enters at a position deviated from the center of the pupil. Therefore, the line image LP at the outer edge of the viewpoint image VPI is hardly recognized by the observer. However, due to the influence of the aberration of the lens LE or the like, the light spreads, and the line image LP at the outer edge may be recognized as a crosstalk component. In the present disclosure, the central part of the viewpoint image VPI recognized as the left-eye image and the right-eye image is referred to as the orthoscopic point, and the positions other than the orthoscopic point are referred to as the intermediate viewpoints. The line image LP of the intermediate viewpoint is difficult to be recognized as the left-eye image and the right-eye image. Therefore, the correction processing unit 11 selectively blurs the line image LP of the intermediate viewpoint.

[0050] As shown in FIG. 11, the correction processing unit 11, for example, varies the blurring degree of the line image LP according to the distance from the fixation point GP. In FIG. 11, five line images LP included in the left-eye input image LI IN are respectively assigned viewpoint indexes L1 to L5. The viewpoint index L3 is the orthoscopic point of the left eye, and the viewpoint indexes L1, L2, L4, and L5 are the intermediate viewpoints of the left eye. The right-eye input image RI INThe five line images LP included in are each assigned viewpoint indices R1 to R5. The viewpoint index R3 is the direct viewpoint of the right eye, and the viewpoint indices R1, R2, R4, and R5 are the intermediate viewpoints of the right eye.

[0051] As the blurring process, for example, Gaussian blurring is adopted. The degree of blur is represented by the standard deviation σ. The correction processing unit 11 blurs the line image LP farther from the fixation point GP more significantly. The degree of blur (standard deviation σ) is represented by a monotonic function that monotonically increases from the center of the viewpoint image VPI toward the edge of the viewpoint image VPI. In the example of FIG. 11, the monotonic function is represented by a cosine curve or a cosine-squared curve. However, the monotonic function is not limited to this. Information regarding the correspondence relationship between the viewpoint index and the degree of blur is included in the parameter information 21.

[0052] As shown in FIG. 12, the correction processing unit 11 determines the correspondence relationship between the pixel group of the screen SCR and each viewpoint image VPI, the assignment of the viewpoint index within the viewpoint image VPI, and the distribution of the degree of blur (amount of blur) within the viewpoint image VPI based on the observer's viewpoint position information. The correction processing unit 11 generates a multi-stage blurred image in which the amount of blur is controlled in multiple stages according to the distance from the fixation point GP based on the determined information, and outputs it as the corrected image IM c to output.

[0053] In FIG. 11, Gaussian blurring is used as the blurring process, but the blurring process is not limited to this. For example, as shown in FIG. 13, by reducing the luminance of the line image LP of the intermediate viewpoint, the outer edge portion of the viewpoint image VPI can be made less conspicuous than the central portion. For example, the degree of blur is defined as the luminance control value of the line image LP. The luminance control value is represented by a monotonic function that monotonically increases from the center of the viewpoint image VPI toward the edge of the viewpoint image VPI. The correction processing unit 11 reduces the signal value of all pixels included in the line image LP by the luminance control value. Information regarding the luminance control value is included in the parameter information 21.

[0054] The correction processing unit 11 corrects the correction image IM in accordance with the timing when the viewpoint position PS (the head of the observer) of the observer moves so that an image without blurring is displayed at the fixation point GP. c However, if the moving speed of the viewpoint position PS is high, the correction of the correction image IM c may not be in time, and there is a possibility that the observer strongly recognizes the line image LP (blurred image) of the intermediate viewpoint. Therefore, as shown in FIG. 14, the correction processing unit 11 can vary the distribution of the blur amount in the viewpoint image VPN according to the moving speed of the viewpoint position PS.

[0055] For example, a threshold of 1 or more is set for the moving speed of the viewpoint position PS. For each threshold, a blur adjustment value is set. The blur adjustment value is represented by a monotonic function that monotonically increases as the moving speed of the viewpoint position PS increases. When the moving speed of the viewpoint position PS is greater than the threshold, the correction processing unit 11 reduces the blur amount of all the line images LP of the intermediate viewpoints by the blur adjustment value corresponding to the threshold. The greater the moving speed of the viewpoint position PS, the smaller the correction amount (blur amount) of the blurring process. Information regarding the threshold and the blur adjustment value is included in the parameter information 21.

[0056] A similar problem may occur even when the measurement accuracy of the viewpoint position PS by head tracking is low. Therefore, the correction processing unit 11 may set a blur adjustment value for each measurement accuracy of the viewpoint position PS and vary the distribution of the blur amount in the viewpoint image VPN according to the measurement accuracy.

[0057] [4. Specific Example of Multistage Crosstalk Correction Processing] FIGS. 15 to 19 are diagrams showing an example of multistage crosstalk correction processing. Hereinafter, the processing flow shown in FIGS. 15 and 19 will be described with reference to FIGS. 16 to 18.

[0058] In step S1, the correction processing unit 11 performs inverse correction processing on the original image IM org to generate a correction image IM c The inverse correction processing in step S1 is the perceptual image IM after the inverse correction processing viewis performed to simulate, and the output image IM out is not performed to generate.

[0059] In step S2, the cost evaluation unit 12 simulates the perceptual image IM after the inverse correction process view . Based on the simulation result, the cost evaluation unit 12 evaluates the correction residual after the inverse correction process as the cost CS.

[0060] In step S3, the correction processing unit 11 determines the correction range and correction amount of the blurring process based on the cost CS obtained by the cost evaluation after the inverse correction process. The correction processing unit 11 performs the blurring process on the original image IM org based on the determined correction range and correction amount.

[0061] The simulation of the correction residual of the inverse correction process is performed based on, for example, the processing flow shown in FIG. 16. The cost evaluation unit 12 applies the left-eye corrected image LI c and the right-eye corrected image RI c to the crosstalk model to perform the crosstalk simulation. By the crosstalk simulation, the left-eye perceptual image LI c and the right-eye perceptual image RI c when the left-eye corrected image LI view and the right-eye corrected image RI view are displayed on the screen SCR are simulated.

[0062] The cost evaluation unit 12 calculates the left-eye correction residual (LI org - LI view ) based on the difference between the left-eye original image LI org and the left-eye perceptual image LI view . The cost evaluation unit 12 calculates the right-eye correction residual (RI org - RI view ) based on the difference between the right-eye original image RI org and the right-eye perceptual image RI view ). The correction processing unit 11, based on the left-eye correction residual, for the left-eye original image LI orgCalculate the distribution of the blurring amount used for the blurring process. The correction processing unit 11 calculates, based on the right-eye correction residual, the original right-eye image RI org Calculate the distribution of the blurring amount used for the blurring process.

[0063] The blurring amount is calculated, for example, using the correction curve CV shown in FIG. 17. The correction curve CV is a curve that defines the correspondence between the correction residual and the blurring amount (e.g., standard deviation σ). In FIG. 17, the correction curve CV is shown as a curve in which the blurring amount increases in proportion to the increase amount of the correction residual within a certain residual range, and the blurring amount is constant regardless of the value of the correction residual in other residual ranges. However, the correction curve CV is not limited to this. The shape of the correction curve CV is determined, for example, by the subjective evaluation of the designer in step S7. Step S7 is carried out in an offline inspection.

[0064] In step S4, the cost evaluation unit 12 simulates the perceived image IM view after the blurring process. For example, the cost evaluation unit 12 performs inverse correction processing and blurring processing on the original image IM org to obtain the left-eye corrected image LI c and the right-eye corrected image RI c and blurs them using the point spread function PSF shown in FIG. 18. The point spread function PSF has a peak at the center of the viewpoint image VPI (the center of the lens LE). The cost evaluation unit 12 applies the left-eye corrected image LI c and the right-eye corrected image RI c blurred by the point spread function PSF to the crosstalk model and performs crosstalk simulation. By the crosstalk simulation, the left-eye perceived image LI c and the right-eye perceived image RI c when the left-eye corrected image LI view and the right-eye corrected image RI view are displayed on the screen SCR are simulated.

[0065] The cost evaluation unit 12 evaluates the correction residual after the blurring process as the cost CS based on the simulation result. For example, the cost evaluation unit 12 calculates the left-eye original image LI org and the left-eye perception image LI view to calculate the left-eye correction residual (LI org -LI view ) based on the difference therebetween. The cost evaluation unit 12 calculates the right-eye correction residual (RI org and the right-eye perception image RI view based on the difference between org -RI view ).

[0066] In step S5, the correction processing unit 11 determines the correction range and the correction amount of the compression process based on the cost CS obtained by the cost evaluation after the blurring process. For example, the correction processing unit 11 determines an image area with the left-eye correction residual and the right-eye correction residual as the correction range of the compression process. The correction processing unit 11 selects one or more local areas LBA in which pixels included in the correction range exist. For each selected local area LBA, the correction processing unit 11 calculates a range parameter and compresses the luminance range of the local area LBA based on the range parameter. The correction processing unit 11 does not perform the compression process on the local area LBA in which no pixels included in the correction range exist.

[0067] The correction processing unit 11 generates a local luminance compression image BCI obtained by compressing one or more local areas LBA for the corrected image IM c on which the blurring process has been performed. The correction processing unit 11 generates a plurality of local luminance compression images BCI in which the set positions of the local areas LBA are shifted. The correction processing unit 11 overlaps and averages the plurality of local luminance compression images BCI. Thereby, the corrected image IM c on which the blurring process and the compression process have been performed is generated.

[0068] In step S6, the correction processing unit 11 performs an inverse correction process on the corrected image IM c obtained in step S5 to generate an output image IM out . Thereby, a display image with less deterioration in image quality due to crosstalk is obtained.​

[0069] The storage device 20 stores, for example, the program 29 executed by the processing device 10 and the parameter information 21. The program 29 is a program for causing a computer to execute the information processing according to the present disclosure. The processing device 10 performs various processes according to the program 29 stored in the storage device 20. The storage device 20 may be used as a work area for temporarily storing the processing results of the processing device 10. The storage device 20 includes, for example, any non-transitory storage medium such as a semiconductor storage medium and a magnetic storage medium. The storage device 20 includes, for example, an optical disk, a magneto-optical disk, or a flash memory. The program 29 is stored, for example, in a non-transitory storage medium readable by a computer.

[0070] The processing device 10 is, for example, a computer including a processor and a memory. The memory of the processing device 10 includes a RAM (Random Access Memory) and a ROM (Read Only Memory). By executing the program 29, the processing device 10 functions as a correction processing unit 11, a cost evaluation unit 12, a correction amount determination unit 13, and a corrected image generation unit 14.

[0071] [5. Effects] The information processing apparatus 1 includes a cost evaluation unit 12 and a correction processing unit 11. The cost evaluation unit 12 evaluates the correction residual after the crosstalk correction processing as a cost CS. The correction processing unit 11 performs a plurality of different crosstalk correction processes while determining a correction range and a correction amount based on the cost CS. In the information processing method of the present embodiment, the processing of the information processing apparatus 1 described above is executed by a computer. The program 29 of the present embodiment causes a computer to realize the processing of the information processing apparatus 1 described above.

[0072] According to this configuration, the correction residual can be reduced by a plurality of crosstalk correction processes performed in a layered manner. Therefore, crosstalk is suppressed well.

[0073] The correction processing unit 11 performs a plurality of crosstalk correction processes within a range where the number of crosstalk correction processes does not exceed a preset number.

[0074] According to this configuration, it is possible to improve the image quality while suppressing the computational load of the crosstalk correction process.

[0075] As the crosstalk correction process, the correction processing unit 11 performs an inverse correction process, a compression process, and a blurring process. The inverse correction process is a crosstalk correction process that performs a conversion opposite to the conversion caused by crosstalk. The compression process is a crosstalk correction process that compresses the luminance range of the correction range. The blurring process is a crosstalk correction process that blurs the outer edge portion of the viewpoint image.

[0076] According to this configuration, the side effects caused by the inverse correction process can be reduced by the blurring process and the compression process. The blurring process and the compression process have significantly different correction viewpoints. Therefore, by adjusting the contributions of the blurring process and the compression process, crosstalk can be suppressed better.

[0077] The correction processing unit 11 determines the correction range and the correction amount of the blurring process based on the cost obtained by the cost evaluation after the inverse correction process. The correction processing unit 11 determines the correction range and the correction amount of the compression process based on the cost obtained by the cost evaluation after the blurring process.

[0078] According to this configuration, the blurring process, which has a greater side effect on the image quality than the compression process, is performed as the second crosstalk correction process. Since the blurring process has a large side effect on the image quality, the contribution of the blurring process to the correction is set small. The final correction is performed by the third compression process, but since the contribution of the blurring process is small, the contribution of the compression process becomes relatively large. Since the contribution of the compression process with a small side effect becomes large, the degradation of the image quality caused by the side effects of the blurring process and the compression process is minimized.

[0079] The correction processing unit 11 reduces the correction amount of the blurring process as the moving speed of the observer's viewpoint position PS increases.

[0080] According to this configuration, it becomes difficult for the observer to recognize the blurring of the image due to the blurring process.

[0081] [6. Modification Example] [6-1. Modification Example 1] FIG. 20 is a diagram showing a first modification example of the multi-stage crosstalk correction process.

[0082] In the example of FIG. 15, the inverse correction process was performed by performing numerical operations of gamma processing, inverse matrix operation processing, and de-gamma processing. However, as shown in FIG. 20, the operation result of the inverse correction process can be stored in advance as a look-up table LUT, and the numerical operation can be omitted by referring to the look-up table LUT.

[0083] In this modification example, for all combinations of input signal values regarding the left-eye original image LI org and the right-eye original image RI org the operation of the inverse correction process of FIG. 3 is performed. The gamma value γ and the mixing ratio α are measured in advance based on the display characteristics and the crosstalk characteristics. The look-up table LUT stores the combination of the input signal value and the output signal value obtained by the above-described operation. The look-up table LUT is stored in the storage device 20.

[0084] [6-2. Modification Example 2] FIG. 21 is a diagram showing a second modification example of the multi-stage crosstalk correction process.

[0085] In the example of FIG. 15, the processes were performed in the order of blurring process, compression process, and inverse correction process. However, the order of the crosstalk correction process is not limited to this. As shown in FIG. 21, the processes may be performed in the order of compression process, blurring process, and inverse correction process.

[0086] In the example of FIG. 21, the correction processing unit 11 determines the correction range and correction amount of the compression processing based on the cost CS obtained by the cost evaluation after the inverse correction processing. The correction processing unit 11 determines the correction range and correction amount of the blurring processing based on the cost CS obtained by the cost evaluation after the compression processing. Also in this case, the correction residual can be reduced by a plurality of crosstalk correction processes performed in a layered manner. Therefore, crosstalk is well suppressed.

[0087] [6-3. Modification Example 3] FIG. 22 is a diagram showing a third modification example of the multi-stage crosstalk correction process.

[0088] In the example of FIG. 15, the correction amount of the blurring processing was determined using the correction curve CV determined by the subjective evaluation of the designer (step S7). However, the method for determining the correction amount of the blurring processing is not limited to this. For example, it is also possible to quantitatively evaluate the blurring feeling of the corrected image after the blurring processing and determine the correction amount of the blurring processing while feeding back the evaluation result of the blurring feeling.

[0089] In the example of FIG. 22, the correction processing unit 11 c performs FFT (Fast Fourier Transform) on the corrected image IM after the blurring processing. The correction processing unit 11 quantitatively evaluates the blurring feeling of the corrected image IM c based on the result of the FFT. When the blurring of the corrected image is large, the value of the high-frequency component obtained by the FFT processing becomes small. Therefore, by setting a threshold value for the value of the high-frequency component, the blurring feeling is quantitatively evaluated. The correction processing unit 11 determines the correction amount of the blurring processing based on the evaluation result of the blurring feeling. According to this configuration, the blurring amount of the corrected image can be set by an objective evaluation.

[0090] [6-4. Modification Example 4] FIG. 23 is a diagram showing a fourth modification example of the multi-stage crosstalk correction process.

[0091] In the example of Fig. 15, cost evaluation was performed for all image regions in which correction residuals occurred due to inverse correction processing. However, crosstalk is likely to be recognized mainly at the edge portions of an image. At the edge portions, the luminance difference between adjacent pixels is large through the boundary of the edge. Therefore, crosstalk is likely to occur. Even if crosstalk correction processing is performed, since the signal value is likely to saturate, it is difficult to obtain the effect of correction. Therefore, it is also possible to selectively perform crosstalk correction processing on the edge portions.

[0092] In the example of Fig. 23, the cost evaluation unit 12 detects an edge portion from the original image IM org The edge portion is detected using a known edge extraction filter such as a Sobel filter, a Laplacian filter, and a Canny filter. The cost evaluation unit 12 evaluates only the correction residual of the edge portion of the image as the cost CS. According to this configuration, the image quality of the edge portion where crosstalk is likely to be recognized is improved. In addition, since the correction range is limited to the edge portion, the calculation load due to crosstalk correction processing is reduced.

[0093] [6-5. Modification Example 5] Figs. 24 and 25 are diagrams showing other application examples of multi-stage crosstalk correction processing.

[0094] In the above-described embodiment, the multi-stage crosstalk correction processing of the present disclosure was applied to the autostereoscopic 3D display. However, the information processing of the present disclosure may be applied to a stereoscopic 3D display.

[0095] The multi-stage crosstalk correction process of the present disclosure can also be applied to crosstalk between three or more viewpoints. In the example of FIG. 24, the information processing of the present disclosure is applied to a display that displays four viewpoint images VPI, but the number of viewpoints may be three, or five or more. In the example of FIG. 24, four viewpoint images VPI are distributed to the viewpoint positions PS1, PS2, PS3, and PS4 by the lenticular lens LE, but the means for spatially separating the viewpoint images VPI is not limited to the lenticular lens LE. A parallax barrier can also be used as the spatial separation means. Also, as a method for displaying multi-viewpoint images, a projector array method can be applied.

[0096] In the example of FIG. 24, parallax occurs only in the horizontal direction. Therefore, four viewpoint images VPI corresponding to the viewpoint positions PS1 to PS4 are alternately displayed in the horizontal direction of the screen SCR. However, as shown in FIG. 25, parallax may occur in two directions, the horizontal direction and the vertical direction. In this case, the four viewpoint images VPI are arranged in a matrix in the horizontal and vertical directions of the screen SCR. The line image LP is a closed linear image such as a circle or a square. In the example of FIG. 25, the viewpoint image VPI is, for example, a square image. The line image LP is a square linear image centered on the fixation point GP. One viewpoint image VPI includes a plurality of pixels PX. The correction processing unit 11 determines the correspondence relationship between the pixel group of the screen SCR and each viewpoint image VPI so that the center of the viewpoint image VPI becomes the fixation point GP. For example, the distribution of the blur amount is set as a concentric circular distribution centered on the fixation point GP.

[0097] Note that the effects described in this specification are merely examples and are not limiting, and there may be other effects.

[0098] [Appendix] Note that the present technology can also adopt the following configuration. (1) A cost evaluation unit that evaluates the correction residual after crosstalk correction as a cost, A correction processing unit that performs a plurality of different crosstalk correction processes while determining a correction range and a correction amount based on the cost, An information processing apparatus having the same. (2) The cost evaluation unit evaluates only the correction residual of the edge portion of the image as the cost. The information processing apparatus according to (1) above. (3) The correction processing unit performs a plurality of crosstalk correction processes within a range where the number of the crosstalk correction processes does not exceed a preset number. The information processing apparatus according to (1) or (2) above. (4) The correction processing unit performs, as the crosstalk correction process, an inverse correction process that performs a conversion opposite to the conversion caused by crosstalk, a compression process that compresses the luminance range of the correction range, and a blurring process that blurs the outer edge portion of the viewpoint image. The information processing apparatus according to any one of (1) to (3) above. (5) The correction processing unit determines the correction range and the correction amount of the blurring process based on the cost obtained by the cost evaluation after the inverse correction process, and determines the correction range and the correction amount of the compression process based on the cost obtained by the cost evaluation after the blurring process. The information processing apparatus according to (4) above. (6) The correction processing unit performs FFT (Fast Fourier Transform) on the corrected image after the blurring process, quantitatively evaluates the blurring feeling of the corrected image based on the result of the FFT, and determines the correction amount of the blurring process based on the evaluation result of the blurring feeling. The information processing apparatus according to (4) or (5) above. (7) The greater the moving speed of the observer's viewpoint position, the smaller the correction amount of the blurring process. The information processing apparatus according to any one of (4) to (6) above. (8) Evaluate the correction residual after crosstalk correction processing as a cost, and perform a plurality of different crosstalk correction processes while determining the correction range and the correction amount based on the cost, An information processing method executed by a computer, comprising the above. (9) Evaluate the correction residual after crosstalk correction processing as a cost, and perform a plurality of different crosstalk correction processes while determining the correction range and the correction amount based on the cost, A program that causes a computer to realize the above.

Explanation of symbols

[0099] 1 Information processing apparatus 11 Correction processing unit 12 Cost evaluation unit CS Cost

Claims

[

1. ] A correction processing unit that performs crosstalk correction processing for suppressing degradation of a viewer's perceptual image due to mixing of perspective images recognized as crosstalk by correcting signal values of each perspective image, A cost evaluation unit that evaluates, as a cost, a correction residual that is a difference between each perspective image that is an original image and the perceptual image after the crosstalk correction processing, comprising: The correction processing unit: determines an image region having the correction residual as a correction range, determines a correction amount of the signal value by the crosstalk correction processing according to the magnitude of the correction residual, sequentially performs a plurality of different crosstalk correction processes while determining the correction range and the correction amount based on the cost so that the cost is sequentially reduced, An information processing apparatus. [

2. ] The cost evaluation unit evaluates only the correction residual of an image region indicating an edge portion as the cost The information processing apparatus according to claim 1. [

3. ] The correction processing unit performs a plurality of crosstalk correction processes within a range where the number of the crosstalk correction processes does not exceed a preset number The information processing apparatus according to claim 1. [

4. ] As the crosstalk correction processing, the correction processing unit performs an inverse correction process that performs an inverse conversion of the conversion of the signal value of the perceptual image generated by the crosstalk, a compression process that compresses a luminance range of an image region corresponding to the correction range, and a blurring process that blurs an outer edge portion of the perspective image. The information processing apparatus according to claim 1. [

5. ] The correction processing unit determines the correction range and the correction amount of the blurring process based on the cost obtained by cost evaluation after the inverse correction process, and determines the correction range and the correction amount of the compression process based on the cost obtained by cost evaluation after the blurring process. The information processing apparatus according to claim 4. [

6. ] The correction processing unit performs FFT (Fast Fourier Transform) on a corrected image after the blurring process, quantitatively evaluates a blurred feeling of the corrected image based on the result of the FFT, and determines a correction amount of the blurring process based on the evaluation result of the blurred feeling. The information processing apparatus according to claim 4. [

7. ] The correction processing unit acquires information on a viewpoint position at which the perspective image is observed by the viewer, and makes the correction amount of the blurring process smaller as the moving speed of the viewpoint position is larger. The information processing apparatus according to claim 4.

8. Implement cross-talk correction processing that suppresses degradation of the observer's perceptual image due to mixing of perspective images recognized as cross-talk by correcting the signal values of each perspective image. Evaluate, as a cost, the correction residual that is the difference between each perspective image, which is the original image, and the perceptual image after the cross-talk correction processing. Determine, as a correction range, an image area where the correction residual exists. Determine the correction amount of the signal value by the cross-talk correction processing according to the magnitude of the correction residual. Sequentially perform a plurality of different cross-talk correction processes while determining the correction range and the correction amount based on the cost so that the cost is sequentially reduced. An information processing method executed by a computer, comprising the above.

9. Implement cross-talk correction processing that suppresses degradation of the observer's perceptual image due to mixing of perspective images recognized as cross-talk by correcting the signal values of each perspective image. Evaluate, as a cost, the correction residual that is the difference between each perspective image, which is the original image, and the perceptual image after the cross-talk correction processing. Determine, as a correction range, an image area where the correction residual exists. Determine the correction amount of the signal value by the cross-talk correction processing according to the magnitude of the correction residual. Sequentially perform a plurality of different cross-talk correction processes while determining the correction range and the correction amount based on the cost so that the cost is sequentially reduced. A program that causes a computer to implement the above.

Images