Image processing device, image processing method, and computer program

Abstract

The purpose of the present invention is to suppress deterioration in display quality of an image. An image processing device (1OO) configured to execute a frame rate control process for expressing a desired gradation that is larger than the n-th gradation and smaller than the N-th gradation on the basis of a plurality of gradations including at least an n-th gradation (nMO) and an N-th gradation (NMn+2). The image processing device comprises an image data acquisition unit (10), and an image processing unit (20) that executes the frame rate control process. The image data acquisition unit acquires a plurality of input frames, and the image processing unit: selects, on the basis of the magnitude of a random number and a random number threshold, a gradation from the plurality of gradations that is to be allocated to the pixel of interest in the plurality of input frames; and generates a plurality of output frames for expressing the desired gradation on the basis of the selected gradation. The random number is given for each pixel of interest, and the random number threshold is associated with the desired gradation, the n-th gradation, and the N-th gradation, the random number threshold being set such that the ratio at which the N-th gradation is selected exceeds the ratio at which the n-th gradation is selected commensurately with an increase in the desired gradation.

Description

Image processing device, image processing method, and computer program

[0001] The present invention relates to an image processing device, an image processing method, and a computer program.

[0002] Frame rate control processing is known for gradation display in image display devices (see, for example, Patent Document 1). The frame rate control processing utilizes the eye's integral effect, which occurs when a human perceives multiple frames with different gradations, to allow a user to artificially perceive an intermediate gradation (target gradation) that is different from the gradations of the multiple frames. For example, by performing frame rate control processing using adjacent gradations (e.g., gradation values ​​of 4 and 5) that can be displayed by a panel of an image processing device, it is possible to artificially generate an intermediate gradation (e.g., 4.5 gradation) that cannot be displayed by the panel, thereby expanding the gradation representation of the image display device.

[0003] Japanese Unexamined Patent Publication No. 7-160224

[0004] Depending on the type of panel that displays the output frame generated by the image processing device, the display quality of certain gradations may be degraded. For example, unlike liquid crystal displays, organic electroluminescence (EL) displays do not have backlights, making it easier to display low gradations (black) and capable of displaying clear images with a high contrast ratio. However, when displaying images with lower gradations on an organic EL display, there are gradations among the lower gradations (excluding those with a gradation value of 0) where the display quality of the image is degraded due to, for example, circuit noise. It is also expected that there are gradations on liquid crystal displays where the display quality of the image is degraded due to, for example, the backlight.

[0005] In addition, in the frame rate control process, for example, it is possible to express an intermediate gradation (target gradation) by switching between relatively high and low gradations, but if there is a bias in the gradations that are displayed sequentially in the time direction, the image will be perceived as flickering, and the image display quality is likely to deteriorate. Furthermore, if the difference between the relatively high and low gradations is large, the effect of this flickering becomes more noticeable.

[0006] The present invention has been made in view of the above circumstances, and aims to provide an image processing device, an image processing method, and a computer program that can prevent a decrease in image display quality.

[0007] According to the present invention, there is provided an image processing device according to [1]. [1] An image processing device configured to execute frame rate control processing for expressing a target gradation greater than the nth gradation and less than the Nth gradation based on a plurality of gradations including at least an nth gradation (n≧0) and an Nth gradation (N≧n+2), the image processing device comprising: an image data acquisition unit; and an image processing unit that executes the frame rate control processing, wherein the image data acquisition unit acquires a plurality of input frames, and the image processing unit selects a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generates a plurality of output frames for expressing the target gradation based on the selected gradation, wherein the random number is provided for each pixel of interest, and the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation, and the random number threshold is set so that as the target gradation increases, the Nth gradation is selected more frequently than the nth gradation.

[0008] In the present invention, a frame rate control process can be performed to express a target gray scale that is greater than the nth gray scale and less than the Nth gray scale. Here, the relationship N≧n+2 is satisfied, and there is a gap between the nth gray scale and the Nth gray scale. As described above, when displaying an output frame on an organic EL display, for example, there are gray scales that degrade the image display quality due to circuit noise, etc. However, in the present invention, by using the nth gray scale and the Nth gray scale, which are sufficiently separated from each other, it is possible to intentionally generate a pseudo gray scale that degrades the image display quality, and this gray scale can be compensated for by the frame rate control process. Furthermore, in the present invention, the image processing unit selects a gray scale to be assigned to a pixel of interest in multiple input frames from multiple gray scales based on the magnitude of a random number and a random number threshold. Therefore, relatively high and low gray scales are probabilistically assigned from the multiple gray scales. This prevents flickering, such as a bias in the gray scales displayed in the output frame, from being introduced into the output frame. In this way, the present invention can artificially generate gradations that would reduce the display quality of an image, and also prevents flickering factors such as bias in the gradations displayed in the output frame from being added to the output frame, thereby preventing a reduction in the display quality of the image.

[0009] Various embodiments of the present invention are exemplified below. The embodiments shown below can be combined with each other. [2] The image processing device according to [1], wherein the random numbers of adjacent pixels of interest have different forms. [3] The image processing device according to [1] or [2], wherein, when the gradations of the pixels of interest in consecutive output frames within a predetermined period are within a range from an nth gradation to an Nth gradation, n and N satisfy the following formula (1), S is 0.008 or more and 0.042 seconds or less, S' is 0.042 or more and 1.000 seconds or less, a function f is a function related to the target gradation and returns a range of a visual gradation discrimination threshold, and T is the value of the target gradation.

[0010]

[0011] [4] The image processing device according to any one of [1] to [3], wherein the n-th gradation is gradation 0. [5] The image processing device according to any one of [1] to [4], wherein the image processing unit uses the random number threshold only when expressing the target gradation that is equal to or less than a predetermined upper limit value. [6] The image processing device according to [1], wherein the random numbers for the adjacent pixels of interest are generated by process (a), process (b), process (c), or process (d), wherein in process (a), the random numbers for the adjacent pixels of interest are generated sequentially in the time direction using a different algorithm for each of the adjacent pixels of interest, in process (b), the random numbers for the adjacent pixels of interest are generated sequentially in the time direction using the same algorithm for each of the adjacent pixels of interest under conditions where at least one of the initial value of the random number and the period of the random number is different for each of the adjacent pixels of interest, in process (c), the random numbers for multiple pixels of interest included in multiple adjacent regions are generated using a different algorithm for each of the adjacent regions, and in process (d), the random numbers for multiple pixels of interest in multiple adjacent regions are generated using the same algorithm for each of the adjacent regions. [7] The image processing device according to any one of [1] to [6], comprising an image display unit configured to display the output frame generated by the image processing unit or a frame processed based on the output frame.[8] An image processing method executed by a computer, which executes frame rate control processing to express a target gradation greater than the nth gradation and less than the Nth gradation based on a plurality of gradations including at least an nth gradation (n≧0) and an Nth gradation (N≧n+2), the image processing method comprising: an image data acquisition step; and an image processing step of executing the frame rate control processing, wherein the image data acquisition step acquires a plurality of input frames; and the image processing step selects a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generates a plurality of output frames for expressing the target gradation based on the selected gradation, wherein the random number is assigned to each pixel of interest, and the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation, and the random number threshold is set so that the larger the target gradation, the greater the rate at which the Nth gradation is selected compared to the rate at which the nth gradation is selected. [9] A computer program for executing the image processing method described in [8].

[0012] FIG. 1 is a schematic diagram showing an image processing device 100 according to an embodiment and an output device 200 that outputs a video signal. FIG. 2 is a functional block diagram of the image processing device 100 shown in FIG. 1. FIG. 3 is a schematic diagram showing a first random number assigned to each pixel and a second random number for a frame at each timing generated using the first random number as an initial value. FIG. 4 is a schematic diagram of a table included in the setting processing unit 2 shown in FIG. 2. FIG. 5 shows an example of an input frame gradation transition A1 as a target gradation, an output frame gradation transition A2, and a moving average A3 of the gradation of the output frame. Note that the output frame gradation transition A2 in FIG. 5 does not represent a transition when the frame rate control process according to the embodiment is executed, but is merely an example. FIG. 6 shows an example of a transition of the output frame gradation transition A2 and the moving average A3 when the frame rate control process according to the embodiment is executed for the input frame gradation transition A1 similar to that in FIG. 5. FIG. 6 illustrates a situation in which fluctuations in the moving average A3 are suppressed (a constant value in FIG. 6 ), thereby suppressing flickering. FIG. 7 illustrates an example of a situation in which the difference between the maximum and minimum values ​​of the moving average exceeds the visual tone discrimination threshold and is easily perceived as flickering. In FIG. 7 , the input frame tone transition A1 is the same as in FIG. 5 , but the output frame tone and its moving average differ from those in FIG. 5 , resulting in a transition that is more likely to cause flickering. FIG. 8 is a graph illustrating how flickering occurs due to changes in the spatial tone of a frame at an arbitrary timing. FIG. 8 also illustrates a connecting line A12 connecting the tone values ​​of pixels in the output frame aligned in an arbitrary direction at an arbitrary timing, and a moving average A13 of the connecting line A12. Like FIG. 5 , FIG. 8 does not illustrate a situation in which the frame rate control process according to this embodiment is executed, but is merely an example.

[0013] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other. Furthermore, each feature can be an invention independently.

[0014] 1. Description of Overall Configuration The overall configuration of an image processing device 100 according to an embodiment will be described. In the embodiment, the image processing device 100 is communicably connected to an output device 200, as shown in Fig. 1. The output device 200 is an information processing device (e.g., a personal computer) configured to be able to output input frames that are video signals.

[0015] As shown in FIG. 2 , the image processing device 100 includes an image data acquisition unit 10, an image processing unit 20, an image display unit 30, and a storage unit 40 configured to store various data. The storage unit 40 stores, for example, data used in various processes performed by the image processing unit 20. The image processing device 100 also includes a random number generation unit 1, a setting processing unit 2, and an output frame generation unit 3. Each component of the image processing device 100 may be implemented by software or hardware. When implemented by software, various functions can be realized by a CPU executing a computer program. The program may be stored in an internal storage unit or in a computer-readable non-transitory recording medium. Alternatively, the program may be read from an external storage unit and implemented using so-called cloud computing. When implemented by hardware, the function may be realized by various circuits such as an ASIC, an FPGA, or a DRP (Dynamically Reconfigurable Processor). In this embodiment, various information and concepts that encompass this information are handled, which are represented by high or low signal values ​​as a collection of binary bits consisting of 0 or 1, and communication and calculations can be performed using the above-mentioned software, hardware, or a combination of software and hardware.

[0016] The image data acquisition unit 10 is configured to acquire a plurality of input frames. The input frames acquired by the image data acquisition unit 10 are sent to the image processing unit 20 and the like, where they are subjected to frame rate control processing and the like, which will be described later. In the embodiment, the image data acquisition unit 10 is described as acquiring the input frames from the output device 200, but this is not limited thereto, and the image data may also be acquired from the Internet, for example.

[0017] The image processing unit 20 is configured to be able to perform frame rate control processing. For example, when the gradation (gradation value) of a pixel is switched rapidly, the human eye has the tendency to perceive the pixel as displaying a gradation intermediate between the gradations before and after the switch (integral effect). The frame rate control processing can express an intermediate gradation (target gradation) by rapidly switching between relatively high and low gradations. In other words, the frame rate control processing can utilize the integral effect of human vision to pseudo-generate intermediate gradations (target gradations), thereby enabling an expansion of the range of gradations that can be expressed. Here, when generating intermediate gradations (target gradations), the frame rate control processing may cause humans to perceive image flickering, for example, depending on the gradation values ​​of pixels in the output frame. Note that flickering can be understood, for example, as a bias in the data contained in the output frame (e.g., the gradation of any pixel in the output frame remains the same value for a long period of time). The image processing device 100 according to this embodiment has the function of suppressing the above-mentioned data imbalance and suppressing flickering in the image displayed on the image display unit 30 when executing frame rate control processing.

[0018] The image display unit 30 is configured to display the output frame output from the image processing unit 20. For example, the image processing device 100 may include an image processing circuit (not shown) downstream of the output frame generation unit 3, and the image display unit 30 may display a frame image-processed by the image processing circuit based on the output frame. In the embodiment, the display of the image display unit 30 may be configured as an organic electroluminescence (EL) display (OELD) but is not limited thereto. The display of the image display unit 30 may also be other displays such as a liquid crystal display (LCD) or electronic paper. The embodiment is suitable for a case where the image display unit 30 employs an OLED display. Unlike an LCD display, an OLED display does not have a backlight, and therefore is capable of expressing low gradations (e.g., black) and clear images with a high contrast ratio. However, when displaying images with low gradations on an OLED display, some of the low gradations (excluding those with a gradation value of 0) result in reduced display quality. Therefore, by using frame rate control processing to express gradations that would otherwise degrade display quality, it is expected that the disadvantages of organic EL displays can be eliminated. The degradation in display quality is thought to be caused by, for example, variations in organic EL elements, driver IC characteristics, analog noise, etc. However, as described above, when frame rate control processing according to conventional technology is performed, flickering is likely to occur. Therefore, in this embodiment, it is possible to suppress the occurrence of flickering even when frame rate control processing is performed.

[0019] 2. Image Processing Unit 20 The image processing unit 20 has a random number generation unit 1, a setting processing unit 2, and an output frame generation unit 3. The image processing unit 20 is configured to select a gradation to be assigned to a pixel of interest in a plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold. Then, the image processing unit 20 generates a plurality of output frames for expressing a target gradation based on the selected gradation. The functions of the image processing unit 20 will be described in detail below.

[0020] In the embodiment, the image processing unit 20 stochastically allocates relatively high and low gradations to the multiple output frames to generate target gradations corresponding to the pseudo-grayscale. Since gradations are specified for each pixel (each pixel of interest), in the embodiment, relatively high and low gradations are stochastically allocated to each pixel of interest in the multiple output frames to be generated. In other words, in the embodiment, a stochastic element is incorporated into the determination of the gradations of the pixels in the output frames, thereby preventing bias in the data included in the output frames (e.g., the gradation of any pixel in the output frame remaining the same value for a long period of time). The above-mentioned pixel of interest corresponds to a pixel on which the frame rate control process described in the embodiment is executed. In the embodiment, to incorporate a stochastic element into the determination of the content of the output frames, the image processing unit 20 uses a random number and a random number threshold, which will be described later.

[0021] 2-1 Random Number Generation Unit 1 The random number generation unit 1 is configured to generate random numbers used when generating an output frame (random numbers for comparison with a random number threshold, described later). The random number generation unit 1 then assigns the generated random numbers to each pixel. In the embodiment, the random numbers take values ​​from 0 to 255, but are not limited to this range and can be changed depending on the bit depth of the gradation. In the embodiment, the random numbers used when generating an output frame are generated through multiple random number generation processes. That is, the random number generation unit 1 includes a first random number generation unit 1A and a second random number generation unit 1B. The first random number generation unit 1A is configured to generate first random numbers by executing a first random number generation process, and the second random number generation unit 1B is configured to generate second random numbers by executing a second random number generation process. In the embodiment, the second random numbers are the random numbers used when generating an output frame (random numbers for comparison with a random number threshold, described later). The first random number generation process and the second random number generation process can employ various algorithms, but an example in the embodiment will be described below.

[0022] The first random number generation process may employ, for example, the XORSHIFT algorithm. The XORSHIFT algorithm is an algorithm capable of generating uniformly distributed random numbers (pseudo-random numbers). The second random number generation process may be the same as or different from the first random number generation process. In the embodiment, the second random number generation process is the same as the first random number generation process and employs the XORSHIFT algorithm. The second random number generation process generates a second random number based on the first random number generated in the first random number generation process. In other words, the second random number generation process uses the first random number generated in the first random number generation process as an initial value and generates a second random number (a random number to be compared with a random number threshold described below) using the XORSHIFT algorithm. In the embodiment, the random number generation unit 1 recursively executes the XORSHIFT algorithm as described above, thereby improving the quality of the random number to be ultimately used (a random number to be compared with a random number threshold described below). In the embodiment, the above-mentioned XORSHIFT is used as the algorithm for the first and second random number generation processes, but instead of XORSHIFT, for example, Mersenne Twister or WELL may be used.

[0023] A random number is assigned to each pixel. The process for calculating the random number for a frame is explained below. The coordinates of the pixels in the frame shown in FIG. 3 are defined as P1, P2, P3, and P4. Although there are other pixels in the frame, for the sake of convenience, only the coordinates P1 to P4 will be explained here. In FIG. 3, the first random number for the pixel at coordinate P1 is 16, the first random number for the pixel at coordinate P2 is 39, the first random number for the pixel at coordinate P3 is 2, and the first random number for the pixel at coordinate P4 is 41, as an example.

[0024] By executing the first random number generation process, the first random numbers generated sequentially are assigned individually to the coordinates of the frame. Note that any value can be used as the initial value used in the first random number generation process.

[0025] The second random number for the pixel at coordinate P1 at any timing is obtained using the first random number for the pixel at coordinate P1. Specifically, the second random number for the pixel at coordinate P1 is generated by applying a second random number generation process (in this embodiment, the XORSHIFT algorithm) to the first random number (16) as an initial value. In other words, by executing the second random number generation process using the first random number as the initial value, the random numbers generated sequentially become the second random number for each timing of the pixel at coordinate P1. In FIG. 3 , the second random number for the pixel at coordinate P1 at timing t is 169, and the second random number for the pixel at coordinate P1 at timing t+1 is 238. Second random numbers for pixels at other coordinates at any timing can also be obtained in a similar manner.

[0026] In this embodiment, the random numbers for adjacent pixels (pixels of interest) in the spatial and temporal directions of a frame are generated in different ways. Specifically, the random numbers for adjacent pixels (pixels of interest) are generated by process (a), process (b), process (c), or process (d). In process (a), random numbers for adjacent pixels of interest are generated sequentially in the temporal direction using different algorithms for each adjacent pixel of interest. Process (a) corresponds, for example, to employing the above-described XORSHIFT for the pixel at coordinate P1 and the Mersenne Twister for the pixel at coordinate P2. In process (b), random numbers for adjacent pixels of interest are generated sequentially in the temporal direction using the same algorithm for each adjacent pixel of interest. However, in process (b), at least one of the initial value and the period of the random numbers differs for each adjacent pixel of interest. Process (b) corresponds, for example, to employing the above-described XORSHIFT for both the pixel at coordinate P1 and the pixel at coordinate P2, but using different initial values ​​and periods of the random numbers. For example, among random numbers having a certain period, the position or length of the period used may be different. Specifically, as an example of a different period position, if a random number sequence related to XORSHIFT has a period consisting of, for example, 10,000 to 300,000 terms, a random number from 10,000 to 100,000 terms may be used for the pixel corresponding to coordinate P1, and the remaining random number may be used for the pixel corresponding to coordinate P2. In process (c), random numbers for multiple target pixels contained in multiple adjacent regions are generated using different algorithms for each adjacent region. Note that the region in process (c) can be defined as multiple pixels contained within a predetermined range of the image display unit 30. Furthermore, the multiple adjacent regions are adjacent to each other but do not overlap. For example, one of two adjacent regions is a multiple of pixels arranged horizontally in the top row, and random numbers for that multiple pixels are generated using XORSHIFT. The other of the two adjacent regions is a multiple of pixels arranged horizontally in the row one row below the top row, and random numbers for that multiple pixels are generated using Mersenne Twister. In process (d), random numbers for a plurality of pixels of interest in a plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions.However, in process (d), at least one of the initial value of the random number and the period of the random number may be different or the same for each of the adjacent regions. Note that in processes (c) and (d), the region may be one pixel.

[0027] Supplementally, processes (a) and (b) correspond to processes for generating random numbers to be assigned to pixels (pixels of interest) at each coordinate that transition in the time direction. For example, the random number for the pixel at coordinate P1 shown in FIG. 3 transitions from 16 to 169 to 238 in the time direction. Processes (a) and (b) correspond to generating random numbers that transition in this way in the time direction. On the other hand, processes (c) and (d) correspond to processes for generating random numbers to be assigned to pixels at each coordinate that transition in the space direction. For example, as shown in FIG. 3, the random number for coordinate P1 transitions in the space direction as 16, the random number for coordinate P2 transitions as 39, the random number for coordinate P3 transitions as 2, and the random number for coordinate P4 transitions as 41. Process (c) corresponds to generating random numbers that transition in this way in the space direction. In other words, processes (a) and (b) are processes for assigning random numbers that transition in the time direction, and processes (c) and (d) are processes for assigning random numbers that transition in the space direction.

[0028] 2-2 Setting Processing Unit 2 The setting processing unit 2 is configured to be able to set a random number threshold for comparison with the random number according to the target gradation. The random number threshold set by the setting processing unit 2 is associated with the target gradation, a relatively high gradation (an example of an Nth gradation), and a relatively low gradation (an example of an nth gradation). In other words, the setting processing unit 2 is set with a gradation value of a relatively high gradation, a gradation value of a relatively low gradation, and a random number threshold for each target gradation.

[0029] In the following description, the target gradation, the gradation value of a relatively high gradation, the gradation value of a relatively low gradation, and the random number threshold are also referred to as setting data. In the following description, a relatively high gradation is also referred to as a high gradation, and a relatively low gradation is also referred to as a low gradation. In the embodiment, the setting data is described as being set as a table, but this is not limited thereto and may be set as a function. Various values ​​of the setting data (table) of the setting processing unit 2 may be set in advance by, for example, the manufacturer or user of the image processing device 100. Furthermore, multiple pieces of setting data (multiple tables) may be stored in the image processing device 100, and the setting data (table) to be used may be changed as appropriate depending on the situation.

[0030] <Target Gradation> The setting data (table) of the setting processing unit 2 will be described in detail with reference to FIG. 4 . The target gradation in the table is an intermediate gradation generated pseudo-wise by switching between high and low gradation frames. This target gradation is a value greater than a low gradation (an example of an nth gradation) and less than a high gradation (an example of an Nth gradation). The target gradation is a number greater than 0. In the example shown in FIG. 4 , the target gradation is an integer between 1 and 15. Note that the target gradation is not limited to an integer as long as it is greater than 0, and may be, for example, 0.5 gradation or 1.5 gradation. As described above, organic EL displays have gradations among their low gradations that degrade display quality due to, for example, analog noise (hereinafter also referred to as display-quality-degrading gradations). Such gradations depend on the panel characteristics of the organic EL display and can be predicted in advance. Therefore, the target gradations include display-quality-degrading gradations. In other words, although this is a gradation that can be displayed without using frame rate control processing, display quality will deteriorate if this gradation is displayed without using frame rate control processing, so this gradation is artificially generated using frame rate control processing.

[0031] Furthermore, a predetermined upper limit is set for the target gradation, and in the embodiment, the upper limit is 15. That is, in the embodiment, the frame rate control process according to the embodiment is not executed for all gradations (all pixels). This is because it is expected that there will be no gradations that will cause display quality degradation at higher gradations. Furthermore, if the frame rate control process according to the embodiment were executed for all gradations, there is a possibility that the data processing load on the image processing unit 20 would increase. Therefore, in the embodiment, a predetermined upper limit is set for the target gradation, thereby suppressing the data processing load on the image processing device 100.

[0032] <Low Gradation and Low Gradation> The high and low gradations in the table are set to gradation values ​​that are considered preferable for pseudo-generating a specific target gradation. When the number of low gradations is n and the number of high gradations is N, the relationship n≧0 and N≧n+2 is satisfied. For example, in the table in FIG. 4, in order to express a target gradation of 1, the high gradation is set to 6 and the low gradation is set to 0 (i.e., n=0), satisfying the above-mentioned relationship. The high and low gradations for other target thresholds also satisfy the same relationship.

[0033] Here, the fact that the low gradation is 0 is particularly effective when the image display unit 30 employs an organic electroluminescence (EL) display. Liquid crystal displays have the disadvantage that a gradation value of 0 impairs display quality due to the difficulty in expressing black caused by the influence of the backlight. On the other hand, organic EL displays eliminate this disadvantage and are characterized by the fact that display quality can be easily ensured even when the gradation value is 0. Therefore, from the perspective of display quality, it is preferable to set the low gradation of the target gradation to 0 in the table of the setting processing unit 2, but this is not limited thereto. For example, if only the low gradation in the table is set to 0, it is conceivable that it may be difficult or impossible to generate the target gradation. In such cases, the low gradation may be set to a value greater than 0 (in FIG. 4 , the low gradation is set to a value greater than 0 when the target gradation is 4 or greater). In this case, the low and high gradations may be selected, for example, from gradations other than those that degrade display quality, or a combination of gradations with relatively good quality among the gradations that degrade display quality may be selected.

[0034] <Random Number Threshold> The random number threshold is a value greater than 0 and equal to or less than 255, but is not limited to this range and can be changed depending on the bit depth of the gradation. The random number threshold is set so that the higher the target gradation, the greater the proportion of high gradations selected compared to low gradations. Specifically, as shown in the table in FIG. 4 , the higher the target gradation, the greater the random number threshold. For example, when the target gradation is 1, the random number threshold is 10, and when the target gradation is 2, the random number threshold is 20. In this way, by increasing the random number threshold as the target gradation increases, the proportion (probability) of high gradations being selected as gradations for the output frame increases, as will be described later in the description of the output frame generation unit 3. Note that, while it is preferable to use the median (e.g., 127 or 128) of the entire gradation range (256 in this embodiment), this is not limiting and may vary depending on the situation.

[0035] 2-3 Output Frame Generation Unit 3 The output frame generation unit 3 generates an output frame based on the input frame, the random number generated by the random number generation unit 1 (in this embodiment, the second random number generated in the second random number generation process), and the setting data of the setting processing unit 2. The output frame generation unit 3 has a determination processing unit 3A and a generation processing unit 3B.

[0036] <Determination processing unit 3A> The determination processing unit 3A has the function of determining whether or not to perform the frame rate control processing according to the embodiment for each pixel of the input frame based on the gradation value of each pixel of the input frame and the upper limit value of the target gradation of the setting processing unit 2.

[0037] In the embodiment, the generation processing unit 3B (described later) uses the random number threshold only when the determination processing unit 3A performs the above-described determination to express a target gradation equal to or less than a predetermined upper limit. In other words, the determination processing unit 3A designates pixels of the input frame with gradation values ​​equal to or less than the upper limit as target pixels, and designates pixels with gradation values ​​exceeding the upper limit as non-target pixels. The target pixel is a pixel that is subject to the frame rate control processing according to the embodiment, and the non-target pixel is a pixel that is not subject to the frame rate control processing according to the embodiment. For example, if the gradation value of a pixel at any coordinate in the input frame is greater than the upper limit of the target gradation (15 in the embodiment), the frame rate control processing according to the embodiment is not performed on the pixel at that coordinate, and the gradation value of the pixel at that coordinate in the input frame becomes the gradation value of that coordinate in the output frame. Conversely, if the gradation value of a pixel at any coordinate in the input frame is equal to or less than the upper limit of the target gradation (15 in the embodiment), the frame rate control processing according to the embodiment is performed on the pixel at that coordinate. The gradation value of the pixel at that coordinate is then processed by the generation processing unit 3B (described later). For example, if the gradation value of a pixel at any coordinate in the input frame is 16 and the gradation value of a pixel at a coordinate adjacent to that arbitrary coordinate is 14, the former is a non-target pixel and the frame rate control processing according to the embodiment is not performed, but the latter is a target pixel and the frame rate control processing according to the embodiment is performed.

[0038] <Generation Processing Unit 3B> The generation processing unit 3B is configured to select a gradation to be assigned to a pixel of interest in an input frame from a plurality of gradations (in the embodiment, high gradations with N gradations and low gradations with n gradations) of the setting data of the setting processing unit 2, based on the random number of the random number generation unit 1 and the magnitude of the random number threshold of the table in the setting processing unit 2. In other words, the generation processing unit 3B sets the selected high gradation and low gradation as the gradation of the pixel of interest in the output frame, as shown in the following formula (2).

[0039]

[0040] For example, if the random number of a given pixel is equal to or greater than the random number threshold, a low gradation value corresponding to the target gradation of the given pixel is selected from the table, and if the random number of the given pixel is otherwise less than the random number threshold, a high gradation value corresponding to the target gradation of the given pixel is selected from the table.

[0041] An example will be described in which the gradation value (target gradation) of a pixel at any coordinate in the input frame is 4, and the random number for the pixel at the arbitrary coordinate is 80. Because the gradation value (target gradation) of the pixel at the arbitrary coordinate is 4, which is smaller than the upper limit (15), the determination processing unit 3A subjects the pixel at the arbitrary coordinate to execution of the frame rate control process according to the embodiment. Furthermore, because the random number (80) is smaller than the random number threshold (96) corresponding to the target gradation (4), the generation processing unit 3B selects a high gradation value (6) as the gradation of the output frame. In this example, because the target gradation of the pixel at any coordinate in the input frame is 4, the random number threshold is significantly higher than when the target gradation is 1 to 3. Therefore, compared to when the target gradation is 1 to 3, it can be said that the random number is below the random number threshold, and the likelihood of a high gradation being selected is increased. In this example, the random number (80) is actually below the random number threshold (96). In this embodiment, by using the random number and the random number threshold in this way, it is determined probabilistically whether the gradation of the pixel of interest in the output frame will be a low gradation or a high gradation. In other words, if the target gradation is high, the possibility that a high gradation will be selected increases accordingly, and conversely, if the target gradation is low, the possibility that a low gradation will be selected increases accordingly.

[0042] The gradation (target gradation), low gradation, high gradation, and random number threshold of the output frame can be explained as generally satisfying the relationship expressed by the following formula (3). In the following formula (3), the random number threshold is normalized, and the value shown in the table divided by the bit depth (8 bits in this embodiment) is used.

[0043]

[0044] 3 Operational Description 3-1 Gradation Transition This section explains the transition of the gradation of pixels in an output frame when frame rate control processing is performed on the gradation of pixels in an input frame. For the sake of convenience, only a pixel of interest at an arbitrary coordinate in a frame will be explained here. Also, for the sake of convenience, the gradation (target gradation) of the pixel of interest at the arbitrary coordinate in the input frame is assumed to be 12.5 at all times (frames 1 to 13), and the gradation (target gradation) is constant.

[0045] Fig. 5 shows an example of the process by which pixels of an output frame transition during frame rate control. Fig. 5 is not intended to show whether the transition process corresponds to the frame rate control process according to the embodiment or the frame rate control process of a conventional method, but rather to specify the conditions under which flickering is likely to occur and those under which it is unlikely to occur by defining a graph of gradation transition using parameters. Therefore, as shown in Fig. 5, a transition in gradation of an output frame does not necessarily mean that flickering will be less likely to occur.

[0046] The graphs and parameters shown in FIG. 5 are as follows: A1 represents the gradation (target gradation) of the input frame, which is fixed at 12.5 in this example; A2 represents the transition of the output frame after frame rate control processing; A3 represents the moving average of multiple consecutive frames (four frames in this example) at any timing; it can be assumed that the moving average simulates the integral effect due to the characteristics of human vision. If the gradation difference of this moving average (for example, the difference between the maximum and minimum values ​​of the moving average) exceeds the visual gradation discrimination threshold, the transition of gradation will be perceived and recognized as flickering.

[0047] S is the visual integration time. That is, S corresponds to the time during which an image is displayed so as to produce an integration effect on the eye. S is, for example, greater than or equal to 0.008 seconds and less than or equal to 0.042 seconds. The notation of S is omitted in FIG. 5 .

[0048] S' corresponds to the length from the first timing of a period in which the gradation of an output frame does not change to the first timing of a change in the gradation in a given period (the period from frame 1 to frame 13 in the example of FIG. 5 ) in which the gradation of the output frame does not change. In other words, S' can be considered an index representing the bias of the data in the output frame. Note that FIG. 5 shows the longest S' among the periods from frame 1 to frame 13. If S' is long, the gradation of the output frame does not change to converge (approach) the target gradation, and flickering is likely to occur in the image. Here, in the frame rate control process, the target gradation is expressed by switching between high and low gradations. If it takes a long time to switch between high and low gradations, this can lead to perceiving the high or low gradation itself rather than perceiving an intermediate gradation between the high and low gradations. Furthermore, if the difference between the high and low gradations exceeds the visual gradation discrimination threshold, it will be perceived as flickering. For this reason, it can be said that the shorter the length of S', the easier it is to suppress image flicker. Specifically, S' is preferably equal to or greater than 0.042 seconds and equal to or less than 1.000 seconds. The length of S' varies depending on the values ​​of the random number and the random number threshold. For this reason, S' can be defined as the convergence time to the target gradation related to the random number and the random number threshold. For example, if the quality of the generated random number is poor, S' will be long, and flickering will be more likely to occur in the image.

[0049] n is the value of the low gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship n≧0. N is the value of the high gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship N≧n+2. T is the value of the target gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship n<target gradation<N. The function f is a function related to the target gradation, and returns the range of the visual gradation discrimination threshold. Note that this function f can be a function that returns a larger range of the gradation discrimination range the larger the gradation value, and a smaller range of the gradation discrimination range the smaller the gradation value. For example, the grayscale standard display function (GSDF) defined by DICOM can be used. In other words, the function f can be a function that returns one step (range of the gradation discrimination range) when a target gradation generated based on the DICOM grayscale standard function is input. The function f is not limited to the DICOM grayscale standard function, and for example, a Barten Ramp curve may be used.

[0050] Here, in the moving average, among the gradations recognized by the visual integral effect, the lower gradation is defined as Amin and the higher gradation is defined as Amax. When S' is sufficiently short with respect to S, the visual integral effect causes Amin and Amax to converge to the target gradation value T. In other words, the following formula (4) is satisfied.

[0051]

[0052] On the other hand, when S' is long relative to S, it can be approximated that n or N is output as the gradation value during the time when Amin and Amax have not converged to the target gradation value T. Therefore, when S' is long relative to S, Amin and Amax can be expressed as in the following equations (5) and (6). Note that in the following equations (5) and (6), when S = S', they are the same as the above equation (4). On the other hand, as S' approaches infinity, Amin converges to n, and Amax converges to N. If the difference between n and N exceeds the visual gradation discrimination range, it will be perceived as image flickering.

[0053]

[0054]

[0055] If the difference between Amin and Amax (tone difference) exceeds the human tone discrimination threshold, it will be perceived as flickering by humans, so with regard to the above parameters, it is preferable that the high tone (N) and low tone (n) satisfy the following formula (7): In other words, when the tone of a pixel of interest in successive output frames within a predetermined period is within the range from the nth tone to the Nth tone, it is preferable that n and N satisfy the following formula (7):

[0056]

[0057] As mentioned above, the length of S' varies depending on the values ​​of the random number and the random number threshold. For example, if S' becomes too large, the above formula can be satisfied by adjusting the values ​​of the high gradation (N) and low gradation (n). That is, by converting the above formula (7), it can be expressed as formula (8). In other words, it is sufficient to ensure that the difference between the high gradation value and the low gradation value used for each target gradation in the setting data (table) of the setting processing unit 2 is smaller than the value corresponding to the left side of formula (8). In other words, it is preferable to set an upper limit value for the difference between the high gradation value and the low gradation value used for each target gradation in the setting data (table) of the setting processing unit 2.

[0058]

[0059] 3-2 Gradation Transition When Flickering is Suppressed A situation in which flickering is suppressed will be described with reference to FIG. 6. FIG. 6 shows that an appropriate output frame is generated using the random number and random number threshold used in the frame rate control process according to the embodiment, and the moving average is constant. In the example of FIG. 6, although the difference between the low gradation n and the high gradation N is large, the time spent at the low gradation n or the time spent at the high gradation N is short. Therefore, FIG. 6 can be said to show a situation in which the convergence time S' is shorter than the integral time S. In other words, FIG. 6 shows an example of a situation in which Amin and Amax are equal to the target gradation value T due to the visual integral effect, satisfying the above formula (4).

[0060] 3-3 Gradation Transitions When Flickering Occurs Conversely, a situation in which flickering occurs will be described with reference to FIG. 7 . FIG. 7 can be considered to be a situation in which conventional frame rate control processing, different from the frame rate control processing according to the embodiment, is executed. In the output frame shown in FIG. 7 , the difference between Amin and Amax in the moving average is quite large. This is due not only to the large difference between the low gradation n and the high gradation N of the output frame, but also to the long time the gradation of the output frame remains at the low gradation n or the long time the gradation remains at the high gradation N. If this dwell time is long, the convergence time S' tends to be longer than the integration time S. Furthermore, if the difference between Amin and Amax (the gradation difference of the moving average) exceeds the visual gradation discrimination threshold, humans will perceive the gradation transition, resulting in flickering.

[0061] 3-4 Flickering Due to Changes in Tone in the Spatial Direction While FIGS. 5 to 7 have discussed the presence or absence of flickering due to changes in output frames over time, flickering can also occur due to changes in tone in the spatial direction, which will be explained with reference to FIG. 8 . When pixels of the same tone appear consecutively in the spatial direction of the output frame, the value of the moving average tends to fluctuate significantly. If the difference between the maximum and minimum values ​​of the moving average exceeds the visual tone discrimination threshold, image flickering occurs. Therefore, as described in the configuration of the random number generation unit 1, in this embodiment, the random numbers for the pixel of interest at each coordinate in the spatial direction are different (corresponding to process (a), process (b), process (c), or process (d)). Therefore, when frame rate control processing is performed, the tone levels of adjacent pixels are prevented from becoming biased toward the same tone, thereby suppressing image flickering.

[0062] 4 Other Embodiments In the embodiment, the table as the setting data specifies two gradations, high and low, but this is not limited thereto. Three or more gradations may be specified. That is, gradations other than high and low gradations may be specified in the table. In the embodiment, the target gradation of the setting processing unit 2 is described as having a predetermined upper limit, but this is not limited thereto. An upper limit may not be set. That is, the frame rate control process described in the embodiment may be performed for all gradations. In the embodiment, the target gradation of the setting data of the setting processing unit 2 is described as having a predetermined upper limit, but this is not limited thereto. All pixels in a frame may be the pixel of interest without setting an upper limit. The frequency of the input data output from the output device 200 may be lower than the frequency of the image display unit 30. In this case, to improve the quality of the displayed image, an additional frame may be added between each frame of the input data. In this case, this additional frame may be generated by the frame rate control process described in the embodiment. In the embodiment, the random number generation unit 1 and the setting processing unit 2 are described as being included in the image processing unit 20 of the image processing device 100, but this is not a limitation. For example, the random number generation unit 1 and the setting processing unit 2 may be provided in a device separate from the image processing device 100, and the image processing unit 20 may acquire random numbers and setting data from this separate device. In the embodiment, the output device 200 and the image processing device 100 are described as being separate, but this is not a limitation, and these configurations may be integrated. In the embodiment, the image processing device 100 is described as including an image display unit 30, but this is not a limitation, and the image processing device 100 may not include an image display unit 30. In the embodiment, one pixel is described as the smallest unit for executing the frame rate control process, but this is not a limitation. Each pixel has an R (red), G (green), and B (blue) subpixel. Therefore, these subpixels may be used as the smallest unit for executing the frame rate control process.In other words, when performing frame rate control processing, the pixel of interest may be processed pixel by pixel, as described in the embodiment, or sub-pixel by sub-pixel. For example, when frame rate control processing is performed pixel by pixel, assume that the RGB gradation values ​​transition from (0,0,0), (0,0,0), (5,5,5), (0,0,0), (0,0,0). When frame rate control processing is performed sub-pixel by sub-pixel, transitions such as (5,0,0), (0,0,0), (0,5,0), (0,0,0), (0,0,5) can achieve the same integral effect as the pixel-by-pixel transition described above. The image processing unit 20 selects a gradation to assign to the pixel of interest for multiple input frames from multiple gradations based on the magnitudes of the random number and the random number threshold. In the embodiment, as shown in FIG. 4, the random number threshold increases as the target gradation increases, and a low gradation is selected when the random number is equal to or greater than the random number threshold. However, this is not limited to this. The random number threshold may be a smaller value as the target gradation becomes larger, in which case a low gradation may be selected when the random number is below the random number threshold (a high gradation is selected when the random number is equal to or greater than the random number threshold). - In the embodiment, the random number generation unit 1 is described as generating (calculating) random numbers, but this is not limited to this, and random numbers stored in the storage unit 40 may be used, for example.

[0063] 1: Random number generation unit 1A: First random number generation unit 1B: Second random number generation unit 2: Setting processing unit 3: Output frame generation unit 3A: Determination processing unit 3B: Generation processing unit 10: Image data acquisition unit 20: Image processing unit 30: Image display unit 40: Storage unit 100: Image processing device 200: Output device

Claims

1. An image processing device configured to execute frame rate control processing for expressing a target gradation that is larger than the n-th gradation and smaller than the N-th gradation based on a plurality of gradations including at least an n-th gradation (n≧0) and an N-th gradation (N≧n+2), an image data acquisition unit and an image processing unit that executes the frame rate control process; the image data acquisition unit acquires a plurality of input frames; the image processing unit selects a gradation to be assigned to a pixel of interest of the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generates a plurality of output frames for expressing the target gradation based on the selected gradation; the random number is given for each pixel of interest, the random number threshold is associated with the target gradation, an nth gradation, and an Nth gradation, and the random number threshold is set so that as the target gradation increases, the rate at which the Nth gradation is selected becomes higher than the rate at which the nth gradation is selected; the random number of the adjacent pixel of interest is generated by a process (a), a process (b), a process (c), or a process (d); In the process (a), the random numbers for the adjacent pixels of interest are generated sequentially in a time direction using different algorithms for each of the adjacent pixels of interest; In the process (b), the random numbers for the adjacent pixels of interest are sequentially generated in the time direction using the same algorithm for each of the adjacent pixels of interest under a condition that at least one of the initial value of the random numbers and the period of the random numbers is different for each of the adjacent pixels of interest; In the process (c), the random numbers for the plurality of pixels of interest included in the plurality of adjacent regions are generated using different algorithms for each of the adjacent regions; In the process (d), the random numbers for the plurality of pixels of interest in the plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions.

2. 2. The image processing device according to claim 1, The random numbers of the adjacent pixels of interest have different forms.

3. 3. The image processing device according to claim 1, When the gradations of the pixel of interest in the successive output frames within a predetermined period are within a range from the n-th gradation to the N-th gradation, n and N satisfy the following formula: S is equal to or greater than 0.008 seconds and equal to or less than 0.042 seconds, S' is equal to or greater than 0.042 and equal to or less than 1.000 seconds, An image processing device, wherein a function f is a function relating to the target gradation and returns a range of a visual gradation discrimination threshold, and T is a value of the target gradation. [Equation 1]

4. 3. The image processing device according to claim 1, The image processing device, wherein the nth gradation is 0 gradation.

5. 3. The image processing device according to claim 1, The image processing device, wherein the image processing unit uses the random number threshold only when expressing the target gradation that is equal to or less than a predetermined upper limit value.

6. 3. The image processing device according to claim 1, An image processing device comprising: an image display unit configured to display the output frame generated by the image processing unit or a frame processed based on the output frame.

7. An image processing method performed by a computer, which executes frame rate control processing to express a target gradation that is larger than the n-th gradation and smaller than the N-th gradation based on a plurality of gradations including at least an n-th gradation (n≧0) and an N-th gradation (N≧n+2), an image data acquisition step and an image processing step of executing the frame rate control processing, The image data acquisition step acquires a plurality of input frames; In the image processing step, a gradation to be assigned to the pixel of interest of the plurality of input frames is selected from the plurality of gradations based on the magnitude of a random number and a random number threshold, and a plurality of output frames for expressing the target gradation are generated based on the selected gradation; the random number is given for each pixel of interest, the random number threshold is associated with the target gradation, an nth gradation, and an Nth gradation, and the random number threshold is set so that as the target gradation increases, the rate at which the Nth gradation is selected becomes higher than the rate at which the nth gradation is selected; the random number of the adjacent pixel of interest is generated by a process (a), a process (b), a process (c), or a process (d); In the process (a), the random numbers for the adjacent pixels of interest are generated sequentially in a time direction using different algorithms for each of the adjacent pixels of interest; In the process (b), the random numbers for the adjacent pixels of interest are sequentially generated in the time direction using the same algorithm for each of the adjacent pixels of interest under a condition that at least one of the initial value of the random numbers and the period of the random numbers is different for each of the adjacent pixels of interest; In the process (c), the random numbers for the plurality of pixels of interest included in the plurality of adjacent regions are generated using different algorithms for each of the adjacent regions; In the process (d), the random numbers for the plurality of pixels of interest in the plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions.

8. A computer program that executes the image processing method according to claim 7.

Images