Image processing device, image processing method, and computer program

Abstract

The purpose of the present invention is to suppress deterioration in display quality of an image. An image processing device (1OO) configured to execute a frame rate control process for expressing a desired gradation that is larger than the n-th gradation and smaller than the N-th gradation on the basis of a plurality of gradations including at least an n-th gradation (nMO) and an N-th gradation (NMn+2). The image processing device comprises an image data acquisition unit (10), and an image processing unit (20) that executes the frame rate control process. The image data acquisition unit acquires a plurality of input frames, and the image processing unit: selects, on the basis of the magnitude of a random number and a random number threshold, a gradation from the plurality of gradations that is to be allocated to the pixel of interest in the plurality of input frames; and generates a plurality of output frames for expressing the desired gradation on the basis of the selected gradation. The random number is given for each pixel of interest, and the random number threshold is associated with the desired gradation, the n-th gradation, and the N-th gradation, the random number threshold being set such that the ratio at which the N-th gradation is selected exceeds the ratio at which the n-th gradation is selected commensurately with an increase in the desired gradation.

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and a computer program. [Background technology]

[0002] Frame rate control processing is known for gradation display in image display devices (see, for example, Patent Document 1). Frame rate control processing utilizes the integral effect of the eye that occurs when a human perceives multiple frames with different gradations, allowing a pseudo perception of an intermediate gradation (target gradation) that is different from the gradations of the multiple frames. For example, by executing frame rate control processing using adjacent gradations that can be displayed by a panel of an image processing device (e.g., gradation value 4 and gradation value 5), it is possible to pseudo-generate an intermediate gradation that cannot be displayed by the panel (e.g., gradation 4.5), thereby expanding the gradation expression of the image display device. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 7-160224 Summary of the Invention [Problem to be solved by the invention]

[0004] Depending on the type of panel that displays the output frame generated by the image processing device, the display quality of a certain gradation may be degraded. For example, unlike LCD displays, OLED displays do not have a backlight, making it easier to display low gradations (black) and capable of displaying clear images with a high contrast ratio. However, when displaying images with low gradations on an OLED display, there are gradations among the low gradations (excluding those with a gradation value of 0) where the display quality of the image is degraded due to, for example, circuit noise. It is also expected that there will be gradations on LCD displays where the display quality of the image is degraded due to, for example, the backlight.

[0005] In addition, in the frame rate control process, for example, it is possible to express an intermediate gradation (target gradation) by switching between relatively high and low gradations, but if there is a bias in the gradations that are displayed sequentially in the time direction, the image will be perceived as flickering, and the image display quality is likely to deteriorate. Furthermore, if the difference between the relatively high and low gradations is large, the effect of this flickering becomes more noticeable.

[0006] The present invention has been made in view of the above circumstances, and aims to provide an image processing device, an image processing method, and a computer program that can prevent a decrease in image display quality. [Means for solving the problem]

[0007] According to the present invention, an image processing device according to [1] is provided. [1] An image processing device configured to execute frame rate control processing to express a target gradation greater than the nth gradation and less than the Nth gradation based on a plurality of gradations including at least an nth gradation (n≧0) and an Nth gradation (N≧n+2), the image processing device comprising: an image data acquisition unit; and an image processing unit that executes the frame rate control processing, wherein the image data acquisition unit acquires a plurality of input frames; the image processing unit selects a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold; and generates a plurality of output frames for expressing the target gradation based on the selected gradation, wherein the random number is given for each pixel of interest; the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation; and the random number threshold is set so that the larger the target gradation, the greater the rate at which the Nth gradation is selected compared to the rate at which the nth gradation is selected.

[0008] In the present invention, by executing frame rate control processing, it is possible to express a target gray scale that is greater than the nth gray scale and less than the Nth gray scale. Here, the relationship N≧n+2 is satisfied, and there is a gap between the nth gray scale and the Nth gray scale. As described above, when displaying an output frame on an organic EL display, for example, there are gray scales at which the image display quality deteriorates due to the influence of circuit noise, etc. However, in the present invention, by using the nth gray scale and the Nth gray scale, which have a sufficient gap between them, it is possible to intentionally generate a pseudo gray scale at which the image display quality deteriorates, and this gray scale can be handled by frame rate control processing. Furthermore, in the present invention, the image processing unit selects a gradation to be assigned to a pixel of interest in a plurality of input frames from a plurality of gradations based on the magnitude of a random number and a random number threshold, so that relatively high and low gradations are probabilistically assigned from the plurality of gradations, thereby suppressing flickering caused by bias in the gradations displayed in the output frame from being introduced into the output frame. In this way, the present invention can artificially generate gradations that would reduce the display quality of an image, and also prevents flickering factors such as bias in the gradations displayed in the output frame from being added to the output frame, thereby preventing a reduction in the display quality of the image.

[0009] Various embodiments of the present invention will be described below as examples, and the embodiments shown below can be combined with each other. [2] The image processing device according to [1], wherein the random numbers of the adjacent pixels of interest have different forms. [3] In the image processing device according to [1] or [2], when the gradations of the pixel of interest in the successive output frames within a predetermined period are within a range from the nth gradation to the Nth gradation, n and N satisfy the following formula (1), S is equal to or greater than 0.008 and equal to or less than 0.042 seconds, S' is equal to or greater than 0.042 and equal to or less than 1.000 seconds, and the function f is An image processing device, wherein the function related to the target gradation is a function returning a range of a visual gradation discrimination threshold, and T is a value of the target gradation.

[0010]

number

[0011] [4] The image processing device according to any one of [1] to [3], wherein the n-th gradation is 0-gradation. [5] An image processing device according to any one of [1] to [4], wherein the image processing unit uses the random number threshold only when expressing the target gradation that is equal to or less than a predetermined upper limit value. [6] An image processing device as described in [1], wherein the random numbers for adjacent pixels of interest are generated by process (a), process (b), process (c), or process (d), wherein in process (a), the random numbers for adjacent pixels of interest are generated sequentially in the time direction using a different algorithm for each adjacent pixel of interest; in process (b), the random numbers for adjacent pixels of interest are generated sequentially in the time direction using the same algorithm for each adjacent pixel of interest, under conditions where at least one of the initial value of the random number and the period of the random number is different for each adjacent pixel of interest; in process (c), the random numbers for multiple pixels of interest included in multiple adjacent regions are generated using a different algorithm for each adjacent region; and in process (d), the random numbers for multiple pixels of interest in multiple adjacent regions are generated using the same algorithm for each adjacent region. [7] An image processing device according to any one of [1] to [6], comprising an image display unit configured to display the output frame generated by the image processing unit or a frame processed based on the output frame. [8] An image processing method executed by a computer, which executes frame rate control processing to express a target gradation that is larger than the nth gradation and smaller than the Nth gradation based on a plurality of gradations including at least an nth gradation (n≧0) and an Nth gradation (N≧n+2), the image processing method comprising: an image data acquisition step; and an image processing step of executing the frame rate control processing, wherein the image data acquisition step acquires a plurality of input frames; and the image processing step selects a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generates a plurality of output frames to express the target gradation based on the selected gradation; the random number is provided for each pixel of interest; and the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation; and the random number threshold is set so that the larger the target gradation, the greater the rate at which the Nth gradation is selected compared to the rate at which the nth gradation is selected. [9] A computer program for executing the image processing method described in [8]. [Brief explanation of the drawings]

[0012] [Figure 1] FIG. 1 is a schematic diagram showing an image processing device 100 according to an embodiment and an output device 200 that outputs a video signal. [Figure 2] FIG. 2 is a functional block diagram of the image processing device 100 shown in FIG. [Figure 3] FIG. 3 schematically shows the first random numbers assigned to each pixel and the second random numbers of the frames at each timing that are generated using the first random numbers as initial values. [Figure 4] FIG. 4 is a diagram showing a table included in the setting processing unit 2 shown in FIG. [Figure 5] 5 shows an example of a grayscale transition A1 of an input frame as a target grayscale, a grayscale transition A2 of an output frame, and a moving average A3 of the grayscale of the output frame. Note that in FIG. 5, the grayscale transition A2 of the output frame does not represent a transition when the frame rate control process according to this embodiment is executed, but is merely an example. [Figure 6] Fig. 6 shows an example of a gradation transition A2 of an output frame and a transition of a moving average A3 when the frame rate control process according to the embodiment is performed on a gradation transition A1 of an input frame similar to that of Fig. 5. Fig. 6 shows a situation in which fluctuations in the moving average A3 are suppressed (a constant value in Fig. 6), and flickering is suppressed. [Figure 7] Figure 7 shows an example of a situation where the difference between the maximum and minimum values ​​of the moving average exceeds the visual gradation discrimination threshold and is easily perceived as flickering. In Figure 7, the gradation transition A1 of the input frame is the same as in Figure 5, but the gradation of the output frame and the contents of its moving average are different from those in Figure 5, resulting in a transition that is likely to cause flickering. [Figure 8]Fig. 8 is a graph illustrating the occurrence of flickering due to changes in the spatial gradation of a frame at any timing. Fig. 8 shows a connecting line A12 connecting the gradation values ​​of pixels of an output frame aligned in any direction at any timing, and a moving average A13 of the connecting line A12. Like Fig. 5, Fig. 8 does not show a situation in which the frame rate control process according to this embodiment is executed, but is merely an example. DETAILED DESCRIPTION OF THE INVENTION

[0013] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings. Various features shown in the following embodiments can be combined with each other. Furthermore, each feature can be an invention independently.

[0014] 1. Overall configuration The overall configuration of an image processing device 100 according to an embodiment will be described. In the embodiment, the image processing device 100 is communicably connected to an output device 200, as shown in Fig. 1. The output device 200 is an information processing device (for example, a personal computer) configured to be able to output input frames that are video signals.

[0015] 2, the image processing device 100 includes an image data acquisition unit 10, an image processing unit 20, an image display unit 30, and a storage unit 40 configured to be able to store various data. The storage unit 40 stores, for example, data used in various processes by the image processing unit 20. The image processing device 100 also includes a random number generation unit 1, a setting processing unit 2, and an output frame generation unit 3. Each component of the image processing device 100 may be implemented by software or hardware. When implemented by software, various functions can be realized by a CPU executing a computer program. The program may be stored in an internal storage unit or a computer-readable non-transitory recording medium. Alternatively, the program may be read from an external storage unit and implemented using so-called cloud computing. When implemented by hardware, the functions can be realized by various circuits such as an ASIC, an FPGA, or a DRP (Dynamically Reconfigurable Processor). This embodiment deals with various pieces of information and concepts that encompass them. These are represented by high and low signal values ​​as binary bit groups consisting of 0s and 1s, and communication and calculations can be performed using the above software, hardware, or a combination of software and hardware.

[0016] The image data acquisition unit 10 is configured to acquire a plurality of input frames. The input frames acquired by the image data acquisition unit 10 are sent to the image processing unit 20 and the like, where they are subjected to frame rate control processing and the like, which will be described later. In the embodiment, the image data acquisition unit 10 is described as acquiring the input frames from the output device 200, but this is not limited thereto, and the image data may be acquired from the Internet, for example.

[0017] The image processing unit 20 is configured to be able to execute frame rate control processing. For example, when the gradation (gradation value) of a certain pixel is switched at high speed, the human eye has the property of seeing the pixel as displaying a gradation intermediate between the gradations before and after the switch (integral effect). The frame rate control processing can express an intermediate gradation (target gradation) by switching between relatively high and low gradations at high speed. In other words, the frame rate control processing can utilize the integral effect of human vision to pseudo-generate an intermediate gradation (target gradation), thereby enabling the expansion of the gradation that can be expressed. Here, in the frame rate control process, when generating intermediate gradations (target gradations), humans may perceive flickering in the image depending on, for example, the gradation values ​​of pixels in the output frame. Note that flickering can be understood, for example, as a bias in the data included in the output frame (for example, the gradation of any pixel in the output frame remaining the same value for a long period of time). The image processing device 100 according to this embodiment has a function of suppressing the above-mentioned data bias and suppressing flickering in the image displayed on the image display unit 30 while executing the frame rate control process.

[0018] The image display unit 30 is configured to display the output frame output from the image processing unit 20. Note that, for example, the image processing device 100 may be provided with an image processing circuit (not shown) downstream of the circuit of the output frame generation unit 3, and the image display unit 30 may be configured to display a frame that has been image-processed by the image processing circuit based on the output frame. In the embodiment, the display of the image display unit 30 may be configured as an organic light-emitting diode (OLED) display, but is not limited thereto. The display of the image display unit 30 may also be other displays such as a liquid crystal display or electronic paper. Note that the embodiment is suitable for the case where the image display unit 30 employs an OLED display. Unlike LCDs, OLED displays do not have backlights, making it easier to display low gradations (e.g., black) and capable of displaying clear images with a high contrast ratio. However, when displaying images with low gradations on an OLED display, some low gradations (excluding those with a gradation value of 0) result in a decrease in display quality. Therefore, by using frame rate control processing to display gradations that result in a decrease in display quality, it is expected that the disadvantages of OLED displays can be eliminated. The decrease in display quality is thought to be due to, for example, variations in OLED elements, driver IC characteristics, and analog noise. However, as described above, when the frame rate control process according to the conventional technology is executed, flickering is likely to occur. Therefore, in this embodiment, it is possible to suppress the occurrence of flickering even when the frame rate control process is executed.

[0019] 2. Image processing unit 20 The image processing unit 20 includes a random number generation unit 1, a setting processing unit 2, and an output frame generation unit 3. The image processing unit 20 is configured to select a gradation to be assigned to a pixel of interest in a plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold. Then, the image processing unit 20 generates a plurality of output frames for expressing a target gradation based on the selected gradation. The functions of the image processing unit 20 will be described in detail below.

[0020] In the embodiment, the image processing unit 20 stochastically allocates relatively high and low gradations to the multiple output frames to generate target gradations corresponding to the pseudo-grayscale. Since the gradation is specified for each pixel (each pixel of interest), in the embodiment, relatively high and low gradations are stochastically allocated to each pixel of interest in the multiple output frames to be generated. In other words, in the embodiment, a stochastic element is incorporated into the determination of the gradations of the pixels in the output frames, thereby preventing bias in the data included in the output frames (for example, the gradation of any pixel in the output frame remaining the same value for a long period of time). The above-mentioned pixel of interest corresponds to the pixel on which the frame rate control process described in the embodiment is executed. In this embodiment, in order to incorporate a probabilistic element into the determination of the content of the output frame, the image processing unit 20 uses a random number and a random number threshold, which will be described later.

[0021] 2-1 Random number generator 1 The random number generation unit 1 is configured to generate random numbers to be used when generating an output frame (random numbers to be compared with a random number threshold, which will be described later). The random number generation unit 1 then assigns the generated random numbers to each pixel. In the embodiment, the random numbers take values ​​from 0 to 255, but are not limited to this range and can be changed according to the bit depth of the gradation. In the embodiment, the random number used when generating an output frame is generated through multiple random number generation processes. That is, the random number generation unit 1 has a first random number generation unit 1A and a second random number generation unit 1B. The first random number generation unit 1A is configured to execute the first random number generation process to generate a first random number, and the second random number generation unit 1B is configured to execute the second random number generation process to generate a second random number. In the embodiment, the second random number is a random number used when generating an output frame (a random number to be compared with a random number threshold, described later). The first random number generation process and the second random number generation process can employ various algorithms, and an example in the embodiment will be described below.

[0022] The first random number generation process may employ, for example, the XORSHIFT algorithm, which is an algorithm capable of generating uniformly distributed random numbers (pseudo-random numbers). The second random number generation process may be the same as or different from the first random number generation process. In this embodiment, the second random number generation process is the same as the first random number generation process and employs the XORSHIFT algorithm. The second random number generation process generates a second random number based on the first random number generated in the first random number generation process. In other words, the second random number generation process uses the first random number generated in the first random number generation process as an initial value and generates a second random number (a random number to be compared with a random number threshold, described below) using the XORSHIFT algorithm. In the embodiment, the random number generation unit 1 recursively executes the XORSHIFT algorithm as described above, thereby improving the quality of the random numbers that are ultimately used (random numbers to be compared with the random number threshold described below). In the embodiment, the above-mentioned XORSHIFT is used as the algorithm for the first and second random number generation processes, but instead of XORSHIFT, for example, Mersenne Twister or WELL may be used.

[0023] A random number is assigned to each pixel. The process by which the random number for a frame is calculated is explained below. The coordinates of pixels in the frame shown in Fig. 3 are defined as P1, P2, P3, and P4. Although other pixels exist in the frame, for the sake of convenience, only the coordinates P1 to P4 will be described here. Fig. 3 shows an example in which the first random number of the pixel at coordinate P1 is 16, the first random number of the pixel at coordinate P2 is 39, the first random number of the pixel at coordinate P3 is 2, and the first random number of the pixel at coordinate P4 is 41.

[0024] By executing the first random number generation process, the first random numbers generated sequentially are assigned individually to each coordinate of the frame. Note that any value can be used as the initial value used in the first random number generation process.

[0025] The second random number for the pixel at coordinate P1 at any timing is obtained using the first random number for the pixel at coordinate P1. Specifically, the second random number for the pixel at coordinate P1 is generated by applying a second random number generation process (the XORSHIFT algorithm in this embodiment) to the first random number (16) as an initial value. In other words, by executing the second random number generation process using the first random number as the initial value, the random numbers generated sequentially become the second random number for each timing of the pixel at coordinate P1. In FIG. 3, the second random number for the pixel at coordinate P1 at timing t is 169, and the second random number for the pixel at coordinate P1 at timing t+1 is 238. Second random numbers for pixels at other coordinates at any timing can also be obtained in a similar manner.

[0026] In the embodiment, the random numbers of adjacent pixels (pixels of interest) in the spatial and temporal directions of a frame are generated in different ways. Specifically, the random numbers of adjacent pixels (pixels of interest) are generated by process (a), process (b), process (c), or process (d). In process (a), random numbers for adjacent pixels of interest are generated sequentially in the time direction using different algorithms for each adjacent pixel of interest. Process (a) corresponds to, for example, using the above-mentioned XORSHIFT for the pixel at coordinate P1 and using Mersenne Twister for the pixel at coordinate P2. In process (b), random numbers for adjacent pixels of interest are generated sequentially in the time direction using the same algorithm for each pixel of interest. However, in process (b), at least one of the initial value and the period of the random numbers differs for each adjacent pixel of interest. Process (b) corresponds to, for example, the pixel at coordinate P1 and the pixel at coordinate P2 employing the above-mentioned XORSHIFT, but with at least one of the initial value and the period of the random numbers being different. For example, among random numbers having a certain period, the position or length of the period used may be different. Specifically, as an example of a different period position, if a random number sequence related to XORSHIFT has a period consisting of, for example, 10,000 to 300,000 terms, the pixel corresponding to coordinate P1 may use, for example, random numbers from 10,000 to 100,000 terms, and the pixel corresponding to coordinate P2 may use, for example, the remaining random numbers. In process (c), random numbers for multiple pixels of interest contained in multiple adjacent regions are generated using different algorithms for each adjacent region. Note that the regions in process (c) can be defined as multiple pixels contained within a predetermined range of the image display unit 30. The multiple adjacent regions are adjacent to each other but do not overlap. For example, one of the two adjacent regions is a group of pixels arranged horizontally in the top row, and random numbers for the multiple pixels are generated using XORSHIFT. The other of the two adjacent regions is a group of pixels arranged horizontally in the row below the top row, and random numbers for the multiple pixels are generated using Mersenne Twister. In process (d), random numbers for a plurality of pixels of interest in a plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions, but in process (d), at least one of the initial value and the period of the random numbers may be different or the same for each of the adjacent regions. In the processes (c) and (d), the area may be one pixel.

[0027] Supplementally, processes (a) and (b) correspond to processes for generating random numbers that transition in the time direction and are assigned to pixels at each coordinate (pixel of interest). For example, the random number for the pixel at coordinate P1 shown in Figure 3 transitions from 16 to 169 to 238 in the time direction, and processes (a) and (b) correspond to generating random numbers that transition in this way in the time direction. On the other hand, processes (c) and (d) correspond to processes for generating random numbers that transition in the spatial direction and are assigned to pixels at each coordinate (pixel of interest). For example, as shown in Figure 3, the random number at coordinate P1 transitions in the spatial direction as follows: 16, the random number at coordinate P2 as 39, the random number at coordinate P3 as 2, and the random number at coordinate P4 as 41. Process (c) corresponds to the generation of random numbers that transition in this spatial direction. In other words, processes (a) and (b) are processes for assigning random numbers that transition in the time direction, and processes (c) and (d) are processes for assigning random numbers that transition in the spatial direction.

[0028] 2-2 Setting processing section 2 The setting processing unit 2 is configured to be able to set a random number threshold for comparison with the random number according to the target gradation. The random number threshold set by the setting processing unit 2 is associated with the target gradation, a relatively high gradation (an example of an Nth gradation), and a relatively low gradation (an example of an nth gradation). In other words, the setting processing unit 2 is set with a gradation value of a relatively high gradation, a gradation value of a relatively low gradation, and a random number threshold for each target gradation.

[0029] In the following description, the target gradation, the gradation value of a relatively high gradation, the gradation value of a relatively low gradation, and the random number threshold are also referred to as setting data. In the following description, the relatively high gradation is also referred to as a high gradation, and the relatively low gradation is also referred to as a low gradation. Furthermore, in the embodiment, the setting data is described as being set as a table, but this is not limiting and the setting data may be set as a function. Moreover, various values ​​of the setting data (table) of the setting processing unit 2 can be set in advance by, for example, the manufacturer or user of the image processing device 100. Furthermore, a plurality of setting data (a plurality of tables) may be stored in the image processing device 100, and the setting data (table) to be used may be changed as appropriate depending on the situation.

[0030] <Target tone> The setting data (table) of the setting processing unit 2 will be specifically described with reference to FIG. The target gradation of the table is an intermediate gradation that is generated artificially by switching between high and low gradation frames. This target gradation is greater than the low gradation (an example of an nth gradation) and less than the high gradation (an example of an Nth gradation). The target gradation is a number greater than 0, and in the example shown in Fig. 4, the target gradation is an integer between 1 and 15. Note that the target gradation is not limited to being an integer as long as it is a number greater than 0, and may be, for example, a value such as 0.5 gradation or 1.5 gradation. As described above, organic EL displays have, among the low gradations, gradations at which the display quality deteriorates due to, for example, analog noise (hereinafter also referred to as "display quality deteriorating gradations"). Such gradations depend on the panel characteristics of the organic EL display and can be predicted in advance. Therefore, the target gradations include display quality deteriorating gradations. In other words, although these gradations can be displayed without using frame rate control processing, display quality deteriorates if they are displayed without using frame rate control processing. Therefore, the frame rate control processing is used to artificially generate these gradations.

[0031] Furthermore, a predetermined upper limit is set for the target gradation, and in the embodiment, the upper limit is 15. That is, in the embodiment, the frame rate control process according to the embodiment is not executed for all gradations (all pixels). This is because it is assumed that there are no gradations that degrade display quality at higher gradations. Furthermore, if the frame rate control process according to the embodiment is executed for all gradations, there is a possibility that the data processing load on the image processing unit 20 will increase. Therefore, in the embodiment, a predetermined upper limit is set for the target gradation, thereby suppressing the data processing load on the image processing device 100.

[0032] <Low gradation and low gradation> The high and low gradations in the table are set to gradation values ​​that are considered preferable for artificially generating a specific target gradation. When the number of low gradations is n and the number of high gradations is N, the relationship n≧0 and N≧n+2 is satisfied. For example, in the table in FIG. 4, in order to express target gradation=1, the high gradation is set to 6 and the low gradation is set to 0 (i.e., n=0), which satisfies the above relationship. Other Target Tone Similarly, the high and low gradations in the above example also satisfy this relationship.

[0033] Here, the fact that the low gradation is 0 is particularly effective when an organic EL display is used for the image display unit 30. This is because, while a liquid crystal display has the disadvantage that it is difficult to express black due to the influence of the backlight when the gradation value is 0, and the display quality is impaired, an organic EL display does not have this disadvantage and has the characteristic that it is easy to ensure the display quality when the gradation value is 0. For this reason, from the viewpoint of display quality, it is preferable that 0 be set for the low gradations of the target gradations in the table of the setting processing unit 2, but this is not limitative. For example, if only 0 is set for the low gradations in the table, it is expected that there may be cases where it is difficult or impossible to generate the target gradation. In such cases, the low gradations may be set to a value greater than 0 (in FIG. 4, when the target gradation is 4 or greater, the low gradations are set to a value greater than 0). In this case, for example, gradations other than the display quality degrading gradations may be selected for the low and high gradations, or a combination of gradations with relatively good quality may be selected from among the display quality degrading gradations.

[0034] <Random number threshold> The random number threshold is a value greater than 0 and equal to or less than 255, but is not limited to this range and can be changed depending on the bit depth of the gradation. The random number threshold is set so that the higher the target gradation, the greater the rate at which high gradations are selected compared to low gradations. Specifically, as shown in the table in FIG. 4, the higher the target gradation, the greater the random number threshold. For example, when the target gradation is 1, the random number threshold is 10, but when the target gradation is 2, the random number threshold is 20. In this way, by increasing the random number threshold as the target gradation increases, the rate (probability) at which high gradations are selected as gradations for the output frame increases, as will be described later in connection with the output frame generation unit 3. Note that, while it is preferable to use the median (e.g., 127 or 128) of the entire gradation range (256 in this embodiment), the random number threshold may vary depending on the situation, and is not limited to this.

[0035] 2-3 Output frame generator 3 The output frame generation unit 3 generates an output frame based on the input frame, the random number generated by the random number generation unit 1 (in this embodiment, the second random number generated in the second random number generation process), and the setting data of the setting processing unit 2. The output frame generation unit 3 has a determination processing unit 3A and a generation processing unit 3B.

[0036] <Determination processing unit 3A> The judgment processing unit 3A has the function of determining whether or not to perform the frame rate control processing according to the embodiment for each pixel of the input frame based on the gradation value of each pixel of the input frame and the upper limit value of the target gradation of the setting processing unit 2.

[0037] In this embodiment, only when the determination processing unit 3A executes the above-described determination to express a target gradation equal to or less than a predetermined upper limit value, the generation processing unit 3B, which will be described later, uses a random number threshold. That is, the determination processing unit 3A sets, among the pixels of the input frame, pixels with gradation values ​​equal to or less than this upper limit as target pixels, and pixels with gradation values ​​exceeding the upper limit as non-target pixels. The target pixels are pixels that are subject to the frame rate control processing according to the embodiment, and the non-target pixels are pixels that are not subject to the frame rate control processing according to the embodiment. For example, if the gradation value of a pixel at any coordinate in the input frame is greater than the upper limit of the target gradation (15 in the example embodiment), the frame rate control processing according to the embodiment is not performed on the pixel at that coordinate, and the gradation value of the pixel at that coordinate in the input frame becomes the gradation value of that coordinate in the output frame. Conversely, if the gradation value of a pixel at any coordinate in the input frame is equal to or less than the upper limit of the target gradation (15 in this embodiment), the pixel at that coordinate is subjected to the frame rate control process according to this embodiment. The gradation value of the pixel at that coordinate is then processed by the generation processing unit 3B, which will be described later. For example, if the gradation value of a pixel at any coordinate in the input frame is 16 and the gradation value of a pixel at a coordinate adjacent to that arbitrary coordinate is 14, the former is a non-target pixel and the frame rate control processing according to the embodiment is not performed, but the latter is a target pixel and the frame rate control processing according to the embodiment is performed.

[0038] <Generation processing unit 3B> The generation processing unit 3B is configured to select a gradation to be assigned to a pixel of interest in the input frame from a plurality of gradations (in this embodiment, high gradations with N gradations and low gradations with n gradations) of the setting data of the setting processing unit 2, based on the random number of the random number generation unit 1 and the magnitude of the random number threshold in the table of the setting processing unit 2. In other words, the generation processing unit 3B selects the selected high gradation and low gradation as the gradation of the pixel of interest in the output frame, as shown in the following equation (2).

[0039]

number

[0040] For example, if the random number of a given pixel is equal to or greater than the random number threshold, a low gradation value corresponding to the target gradation of the given pixel is selected from the table. If the random number of a given pixel is less than the random number threshold (otherwise), a high gradation value corresponding to the target gradation of the given pixel is selected from the table. Select the tone value from the table.

[0041] An example will be described in which the gradation value (target gradation) of a pixel at any coordinate in the input frame is 4, and the random number of the pixel at the arbitrary coordinate is 80. Because the gradation value (target gradation) of the pixel at the arbitrary coordinates is 4, which is smaller than the upper limit (15), the judgment processing unit 3A selects the pixel at the arbitrary coordinates as a target for execution of the frame rate control process according to the embodiment. Then, because the random number (80) is smaller than the random number threshold (96) corresponding to the target gradation (4), the generation processing unit 3B selects the high gradation value (6) as the gradation of the output frame. In this example, the target gradation of a pixel at any coordinate in the input frame is 4, so the random number threshold is significantly higher than when the target gradation is 1 to 3. Therefore, compared to when the target gradation is 1 to 3, it can be said that the random number falls below the random number threshold, increasing the likelihood that a high gradation will be selected. And in this example, the random number (80) is actually below the random number threshold (96). In this embodiment, by using the random number and the random number threshold in this way, it is determined probabilistically whether the gradation of the pixel of interest in the output frame will be a low gradation or a high gradation. In other words, if the target gradation is high, the possibility that a high gradation will be selected increases accordingly, and conversely, if the target gradation is low, the possibility that a low gradation will be selected increases accordingly.

[0042] The gradation (target gradation), low gradation, high gradation, and random number threshold of the output frame can be explained as generally satisfying the relationship expressed by the following formula (3). In the following formula (3), the random number threshold is normalized, and the value shown in the table divided by the bit depth (8 bits in this embodiment) is used.

[0043]

number

[0044] 3 Operational explanation 3-1 Gradation transition The transition of the gradation of pixels in an output frame when frame rate control processing is performed on the gradation of pixels in an input frame will be described. For the sake of convenience, the following description will be given only for a pixel of interest at an arbitrary coordinate in a frame. Also, for the sake of convenience, the gradation (target gradation) of the pixel of interest at the arbitrary coordinate in the input frame is assumed to be 12.5 at any timing (frame 1 to frame 13), and the gradation (target gradation) is constant.

[0045] Fig. 5 shows an example of the process by which pixels of an output frame transition during frame rate control. Fig. 5 is not intended to show whether the transition process corresponds to the frame rate control process according to the embodiment or the frame rate control process of a conventional method, but rather to specify the conditions under which flickering is likely to occur and those under which it is unlikely to occur by defining a graph of gradation transition using parameters. Therefore, as shown in Fig. 5, a transition in gradation of an output frame does not necessarily mean that flickering will be less likely to occur.

[0046] The graphs and parameters shown in Figure 5 are as follows: A1 represents the grayscale of the input frame (target grayscale), which is fixed at 12.5 in this example. A2 represents the transition of the output frame subjected to frame rate control processing. A3 represents the moving average of multiple consecutive frames (four frames in this example) at any given time. It can be assumed that the moving average simulates the integral effect of human visual characteristics. If the gradation difference of this moving average (for example, the difference between the maximum and minimum values ​​of the moving average) exceeds the gradation discrimination threshold of the human eye, the transition in gradation will be perceived, and this will be recognized as flickering.

[0047] S is the visual integration time. That is, S corresponds to the time during which an image is displayed so as to produce an integration effect on the eye. S is, for example, greater than or equal to 0.008 seconds and less than or equal to 0.042 seconds. The notation of S is omitted in FIG. 5.

[0048] S' is the time period in a given period (frames 1 to 13 in the example of Figure 5). In other words, S' corresponds to the length from the first timing of the period in which the gray level of the output frame does not change to the first timing of the change in the gray level. It can be said that this is an index that indicates the bias of data. Note that in FIG. 5, the longest S' is shown among the periods from frame 1 to frame 13. As S' gets longer, the tone of the output frame changes so that it converges (approaches) to the target tone. It will not be converted, and the image is likely to flicker. Here, in the frame rate control process, the target gradation is expressed by switching between high gradation and low gradation. If it takes time for the high gradation and the low gradation to switch, instead of recognizing the intermediate gradation between the high gradation and the low gradation, it will lead to recognizing the high gradation itself or the low gradation itself. And if the difference between the high gradation and the low gradation exceeds the visual gradation discrimination threshold, it will be recognized as flicker. Therefore, it can be said that when the length of S' is short it is easier to suppress the flicker of the image. Specifically, S' is preferably 0.0 or more and 1.000 seconds or less. The length of S' changes according to the values of the random number and the random number threshold. Therefore, S' can be defined as the convergence time to the target gradation related to the random number and the random number threshold. For example, when the quality of the generated random number is poor, S' becomes long and the image is likely to flicker.

[0049] n is the value of the low gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship n≧0. N is the value of the high gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship N≧n + 2. T is the value of the target gradation in the setting data (table) of the setting processing unit 2, and satisfies the relationship n < target gradation < N. The function f is a function related to the target gradation and is a function that returns the range of the visual gradation discrimination threshold. As this function f, a function can be adopted in which the range of the gradation discrimination region returned is larger as the gradation value is larger, and the range of the gradation discrimination region returned is smaller as the gradation value is smaller. For example, the grayscale standard function (GSDF: Grayscale Standard Display Function) defined in DICOM can be adopted. In other words, the function f is the grayscale of DICOM When a target grayscale level generated based on a standard function is input, a function that returns one step (range of the grayscale discrimination range) can be used. Note that the function f is not limited to the DICOM grayscale standard function, and for example, a Barten Ramp curve may be used.

[0050] Here, in the moving average, among the gradations recognized by the integral effect of vision, the low gradation is defined as Amin and the high gradation is defined as Amax. When S' is sufficiently short compared to S, Due to the integration effect, Amin and Amax converge to the target gradation value T. In other words, the following formula (4) is satisfied.

[0051]

number

[0052] On the other hand, if S' is longer than S, Amin and Amax do not converge to the target gradation value T. In the unbundled time, it can be approximated that n and N are output as gradation values. Therefore, when S' is longer than S, Amin and Amax are expressed by the following formula (5): and can be expressed as equation (6). In the following formulas (5) and (6), when S=S', the above formula (4) is the same as: On the other hand, as S' approaches infinity, Amin converges to n, and Amax converges to N. If the difference between n and N exceeds the range of visual gradation discrimination, it will be perceived as flickering in the image.

[0053]

number

[0054]

number

[0055] If the difference between Amin and Amax (tone difference) exceeds the human tone discrimination threshold, it will be perceived as flickering by humans, so with regard to the above parameters, it is preferable that the high tone (N) and low tone (n) satisfy the following formula (7): In other words, when the tone of a pixel of interest in consecutive output frames within a predetermined period is within the range from the nth tone to the Nth tone, it is preferable that n and N satisfy the following formula (7):

[0056]

number

[0057] As mentioned above, the length of S' varies depending on the random number and the random number threshold value. For example, if S' becomes too large, adjust the values ​​of the high gradation (N) and low gradation (n). By adjusting the above, the above formula can be satisfied. In other words, by converting the above formula (7), it can be expressed as formula (8). In other words, it is sufficient to make the difference between the high gradation value and the low gradation value used for each target gradation in the setting data (table) of the setting processing unit 2 smaller than the value corresponding to the left side of formula (8). In other words, it is preferable to set an upper limit value for the difference between the high gradation value and the low gradation value used for each target gradation in the setting data (table) of the setting processing unit 2.

[0058]

number

[0059] 3-2 Grayscale transition when flickering is suppressed The circumstances under which flickering is suppressed will be described with reference to FIG. FIG. 6 shows that an appropriate output frame is generated by the random number and random number threshold used in the frame rate control process according to the embodiment, and the moving average is constant. In the example of FIG. 6, although the difference between the low gradation n and the high gradation N is large, the time spent at the low gradation n or the time spent at the high gradation N is short. For this reason, it can be said that FIG. 6 shows a situation in which the convergence time S' is shorter than the integration time S. In other words, Fig. 6 shows an example of a situation in which Amin and Amax are equal to the target gradation value T due to the integral effect of vision, and the above formula (4) is satisfied.

[0060] 3-3 Grayscale transitions when flickering occurs Conversely, a situation in which flickering occurs will be described with reference to FIG. 7 can be considered to be a situation in which a conventional frame rate control process, different from the frame rate control process according to the embodiment, is executed. In the output frame shown in FIG. 7, the difference between Amin and Amax in the moving average is quite large. This is due not only to the large difference between the low gradation n and the high gradation N of the output frame, but also to the fact that the time during which the gradation of the output frame remains at the low gradation n or the time during which it remains at the high gradation N is long. When this stay time is long, the convergence time S' becomes longer than the integration time S. If the difference between Amin and Amax (the difference in the moving average gradation) exceeds the visual gradation discrimination threshold, humans will perceive the gradation transition, causing flickering.

[0061] 3-4 Flickering caused by spatial gradation changes 5 to 7, the presence or absence of flicker based on the time change of the output frame has been described, but flicker can also occur due to changes in grayscale in the spatial direction, which will be described with reference to FIG. When pixels of the same gradation are consecutive in the spatial direction of the output frame, the value of the moving average is likely to fluctuate significantly. If the difference between the maximum and minimum values ​​of the moving average exceeds the visual gradation discrimination threshold, image flickering occurs. Therefore, as explained in the configuration of the random number generation unit 1, in this embodiment, the random numbers for the pixel of interest at each coordinate in the spatial direction are different (corresponding to process (a), process (b), process (c), or process (d)). Therefore, when the frame rate control process is executed, the gradations of adjacent pixels are prevented from becoming biased toward the same gradation, and as a result, image flickering is suppressed.

[0062] 4. Other embodiments In the embodiment, two gradations, high and low, are specified in the table as setting data, but this is not limiting and three or more gradations may be specified. In other words, gradations other than high and low gradations may be specified in the table. In the embodiment, a predetermined upper limit is set for the target gradation of the setting processing unit 2, but this is not limiting and an upper limit may not be set. In other words, the frame rate control process described in the embodiment may be performed for all gradations. In the embodiment, a predetermined upper limit is set for the target gradation of the setting data of the setting processing unit 2, but this is not limited to this. All pixels in a frame may be the pixel of interest without setting an upper limit. The frequency of the input data output from the output device 200 may be lower than the frequency of the image display unit 30. In this case, to improve the quality of the displayed image, additional frames may be added between each frame of the input data. In this case, these additional frames may be generated by the frame rate control process according to the embodiment. In the embodiment, the random number generation unit 1 and the setting processing unit 2 are described as being included in the image processing unit 20 of the image processing device 100, but this is not limiting. For example, the random number generation unit 1 and the setting processing unit 2 may be provided in a device separate from the image processing device 100, and the image processing unit 20 may obtain the random numbers and setting data from this separate device. In the embodiment, the output device 200 and the image processing device 100 are described as being separate entities, but this is not limiting, and these may be integrated into one structure. In the embodiment, the image processing device 100 is described as including the image display unit 30, but this is not limited to this, and the image processing device 100 does not necessarily have to include the image display unit 30. In the embodiment, the frame rate control process is performed on a pixel-by-pixel basis. However, this is not limiting. Each pixel has an R (red), G (green), and B (blue) subpixel. Therefore, these subpixels may be used as the smallest unit for the frame rate control process. In other words, the frame rate control process may be performed on a pixel-by-pixel basis, as described in the embodiment, or on a subpixel-by-subpixel basis. For example, when the frame rate control process is performed on a pixel-by-pixel basis, assume that the RGB gradation values ​​transition as follows: (0,0,0), (0,0,0), (5,5,5), (0,0,0), (0,0,0). When the frame rate control process is performed on a subpixel-by-subpixel basis, the transitions as follows: (5,0,0), (0,0,0), (0,5,0), (0,0,0), (0,0,5) can achieve the same integral effect as the pixel-by-pixel transition described above. The image processing unit 20 selects a gradation to be assigned to a pixel of interest in a plurality of input frames from a plurality of gradations based on the magnitude of the random number and the random number threshold. In the embodiment, as shown in FIG. 4, the random number threshold assumes a larger value as the target gradation increases, and a low gradation is selected when the random number is equal to or greater than the random number threshold. However, this is not limited to this. The random number threshold may assume a smaller value as the target gradation increases, and in this case, a low gradation may be selected when the random number is below the random number threshold (a high gradation is selected when the random number is equal to or greater than the random number threshold). In the embodiment, the random number generation unit 1 is described as generating (calculating) random numbers, but this is not limited to this. For example, random numbers stored in the storage unit 40 may be used. [Explanation of symbols]

[0063] 1: Random number generator 1A: First random number generator 1B: Second random number generator 2: Setting processing section 3: Output frame generation section 3A: Judgment processing section 3B: Generation processing section 10: Image data acquisition unit 20: Image processing section 30: Image display section 40: Storage section 100: Image processing device 200: Output device

Claims

1. 1. An image processing device configured to execute a frame rate control process for expressing a target gradation corresponding to a pseudo-halftone gradation that is larger than an n-th gradation (n≧0) and smaller than an N-th gradation (N≧n+2), based on a plurality of gradations including at least the n-th gradation (n≧0) and the N-th gradation (N≧n+2), an image data acquisition unit and an image processing unit that executes the frame rate control process; the image data acquisition unit acquires a plurality of input frames; The image processing unit Among the pixels of the input frame, a pixel to be expressed with a gradation equal to or less than a predetermined upper limit is selected as a pixel of interest; selecting a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generating a plurality of output frames for expressing the target gradation based on the selected gradation, wherein the target gradation is the gradation of the pixel of interest in the input frame; the random number is given for each pixel of interest, the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation, and the random number threshold is set so that as the target gradation increases, the rate at which the Nth gradation is selected becomes greater than the rate at which the nth gradation is selected; the random number of the adjacent pixel of interest is generated by a process (a), a process (b), a process (c), or a process (d); In the process (a), the random numbers for the adjacent pixels of interest are generated sequentially in a time direction using different algorithms for each of the adjacent pixels of interest; In the process (b), the random numbers for the adjacent pixels of interest are sequentially generated in the time direction using the same algorithm for each of the adjacent pixels of interest under a condition that at least one of the initial value of the random numbers and the period of the random numbers is different for each of the adjacent pixels of interest; In the process (c), the random numbers for the plurality of pixels of interest included in the plurality of adjacent regions are generated using different algorithms for each of the adjacent regions; In the process (d), the random numbers for the plurality of pixels of interest in the plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions.

2. 2. The image processing device according to claim 1, The image processing device, wherein the nth gradation is 0th gradation.

3. 2. The image processing device according to claim 1, The image processing unit executes the frame rate control process using the random number threshold when a pixel of the input frame is the pixel of interest, and executes a process of using the gradation of the pixel as the gradation of the output frame when the pixel is not the pixel of interest.

4. 2. The image processing device according to claim 1, An image processing device comprising: an image display unit configured to display the output frame generated by the image processing unit or a frame processed based on the output frame.

5. 1. An image processing method executed by a computer, which executes frame rate control processing to express a target gradation corresponding to a pseudo halftone gradation that is larger than the nth gradation (n≧0) and smaller than the Nth gradation (N≧n+2) based on a plurality of gradations including at least the nth gradation (n≧0) and the Nth gradation (N≧n+2), an image data acquisition step and an image processing step of executing the frame rate control processing, The image data acquisition step acquires a plurality of input frames; In the image processing step, Among the pixels of the input frame, a pixel to be expressed with a gradation equal to or less than a predetermined upper limit is selected as a pixel of interest; selecting a gradation to be assigned to a pixel of interest in the plurality of input frames from the plurality of gradations based on the magnitude of a random number and a random number threshold, and generating a plurality of output frames for expressing the target gradation based on the selected gradation, wherein the target gradation is the gradation of the pixel of interest in the input frame; the random number is given for each pixel of interest, the random number threshold is associated with the target gradation, the nth gradation, and the Nth gradation, and the random number threshold is set so that as the target gradation increases, the rate at which the Nth gradation is selected becomes greater than the rate at which the nth gradation is selected; the random number of the adjacent pixel of interest is generated by a process (a), a process (b), a process (c), or a process (d); In the process (a), the random numbers for the adjacent pixels of interest are generated sequentially in a time direction using different algorithms for each of the adjacent pixels of interest; In the process (b), the random numbers for the adjacent pixels of interest are sequentially generated in the time direction using the same algorithm for each of the adjacent pixels of interest under a condition that at least one of the initial value of the random numbers and the period of the random numbers is different for each of the adjacent pixels of interest; In the process (c), the random numbers for the plurality of pixels of interest included in the plurality of adjacent regions are generated using different algorithms for each of the adjacent regions; In the process (d), the random numbers for the plurality of pixels of interest in the plurality of adjacent regions are generated using the same algorithm for each of the adjacent regions.

6. A computer program that executes the image processing method according to claim 5.

Images