Image processing device, image processing method, image processing system, program, and storage medium

Abstract

To provide an image processing device, an image processing system, a program, and a storage medium for easily correcting a piecewise linear function that performs gradation compression of images.SOLUTION: In an image processing device 100, a function acquisition unit 103 acquires a piecewise linear function for converting gradation of an input image data image acquired by an image acquisition unit 101. A first correction unit 104, when a magnitude of slope of a piecewise linear function in a segment to be processed is greater than a prescribed upper limit value, corrects such that the magnitude of the slope of the piecewise linear function in the segment to be processed becomes the upper limit value by changing a value for an end point of the segment to be processed. The image processing device sequentially performs the correction by the first correction unit to segments adjacent to the segment to be processed.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to image tone conversion technology.

Background Art

[0002] As a method of image conversion such as reduction of the data amount of image data or correction of an image, there is a method of performing tone conversion of image data by tone mapping of the image data using a piecewise linear function. Here, the piecewise linear function is a group of functions in which the range of an explanatory variable is divided into a plurality of sections (hereinafter also referred to as “classes”), and a function is defined for each section, and the function of each section is defined as a linear function. When applying a piecewise linear function to image tone conversion, the range of signal values (hereinafter also referred to as “pixel values”) of an image (hereinafter referred to as “input image”) data to be converted is divided into several classes, and a linear function indicating the input-output relationship is defined for each class to define a piecewise linear function. Tone mapping of image data using a piecewise linear function is to convert each pixel value in the input image data using the piecewise linear function defined in this way to obtain image (hereinafter referred to as “output image”) data after conversion. In particular, tone conversion that reduces the total number of tones of the output image data compared to the total number of tones of the input image data is also called tone compression.

[0003] When the pixel value of the output image data obtained by tone compression is an integer value, depending on the slope of the linear function corresponding to each class in the piecewise linear function, tone crushing may occur. However, in tone compression for reducing the data amount of input image data, shortening the bit length of each pixel value and tone crushing are in a trade-off relationship. Therefore, in order to realize suitable tone compression, it is preferable to dynamically determine a piecewise linear function according to the distribution of pixel values of the image.

[0004] Patent Document 1 discloses a technique in the field of image correction technology in which a piecewise linear function obtained based on a histogram of input image data is corrected based on the maximum grayscale value in the histogram, and the corrected piecewise linear function is used to perform image transformation. Specifically, in the technique disclosed in Patent Document 1, first, the grayscale value (hereinafter also referred to as "frequency") of each class in the histogram is divided by the total number of pixels to normalize the frequency of each class. Next, the slope of the linear function corresponding to each class in the piecewise linear function is calculated based on this normalization to obtain a piecewise linear function based on the histogram. Furthermore, the piecewise linear function is corrected by setting an upper or lower limit of the slope of the linear function corresponding to each class in the piecewise linear function obtained based on the histogram, based on the magnitude of the maximum grayscale value in the histogram. [Prior art documents] [Patent Documents]

[0005] [Patent Document 1] Japanese Patent Application Publication No. 11-331598 [Overview of the Initiative] [Problems that the invention aims to solve]

[0006] However, in the technology disclosed in Patent Document 1, the value of the target variable at the final class of the corrected piecewise linear function may not match the value normalized by dividing the cumulative grayscale value (accumulated from the grayscale values ​​up to the final class in the histogram) by the total number of pixels. Therefore, in the technology disclosed in Patent Document 1, it was necessary to set an upper or lower limit on the slope of the linear function corresponding to each class in the piecewise linear function obtained based on the histogram, and then normalize the histogram again. The purpose of this disclosure is to provide an image processing apparatus that can easily modify the piecewise linear function for grayscale compression of an image without having to renormalize the histogram. [Means for solving the problem]

[0007] The image processing apparatus according to this disclosure is an image processing apparatus that performs grayscale conversion of an image, comprising: an acquisition means for acquiring a piecewise linear function for grayscale conversion; and a first modification means that, if the magnitude of the slope of the piecewise linear function in a section to be processed is greater than a predetermined upper limit, modifies the slope of the piecewise linear function in the section to be processed by changing the value of the endpoint of the section to be processed so that the magnitude of the slope of the piecewise linear function in the section to be processed becomes the upper limit, and sequentially performs the modification by the first modification means on sections adjacent to the section to be processed. i) The aforementioned upper limit is a positive value less than or equal to 1.0. . [Effects of the Invention]

[0008] According to this disclosure, the piecewise linear function used for image grayscale compression can be easily modified without renormalizing the histogram. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram showing an example of the configuration of a functional block in an image processing apparatus according to the first embodiment. [Figure 2] This is a block diagram showing an example of the hardware configuration of an image processing device according to the first embodiment. [Figure 3] This figure shows an example of a histogram and a cumulative histogram of an input image according to the first embodiment. [Figure 4] This figure shows the correspondence between the input values, calculated values, and output values ​​of a piecewise linear function in a certain category. [Figure 5] This figure shows an example of a histogram and cumulative histogram of an input image in which pixels belonging to a specific category are concentrated. [Figure 6] This figure shows an example of a piecewise linear function obtained by the function acquisition unit according to the first embodiment. [Figure 7] This figure illustrates an example of a method for modifying a piecewise linear function according to the first embodiment. [Figure 8] This figure shows an example of a piecewise linear function after the correction process performed by the first and second correction units according to the first embodiment. [Figure 9] This flowchart shows an example of the processing flow of the image processing apparatus according to the first embodiment. [Figure 10] This block diagram shows an example of the configuration of a functional block in an image processing apparatus according to the second embodiment. [Figure 11] This flowchart shows an example of the processing flow of the image processing device 100a according to the second embodiment. [Modes for carrying out the invention]

[0010] The embodiments of this disclosure will be described in detail below with reference to the attached drawings. Note that the configurations shown in the following embodiments are merely examples, and the scope of this disclosure is not limited to these configurations.

[0011] [First Embodiment] The image processing apparatus 100 according to the first embodiment will be described with reference to Figures 1 to 9. First, the configuration of the image processing apparatus 100 will be described with reference to Figures 1 and 2. Figure 1 is a block diagram showing an example of the configuration of the functional blocks of the image processing apparatus 100 according to the first embodiment. The image processing apparatus 100 includes an image acquisition unit 101, a histogram acquisition unit 102, a function acquisition unit 103, a first modification unit 104, a second modification unit 105, an image conversion unit 106, and an image output unit 109. The processing of each part of the image processing apparatus 100 is performed by hardware such as an ASIC (Application Specific Integrated Circuit) built into the image processing apparatus 100. The processing of each part of the image processing apparatus 100 may also be performed by hardware such as an FPGA (Field Programmable Gate Array) built into the image processing apparatus 100. Alternatively, the processing may be performed by software using a CPU (Central Processor Unit) or GPU (Graphic Processor Unit) and memory. The processing of each part of the image processing apparatus 100 will be described later.

[0012] Figure 2 is a block diagram showing an example of the hardware configuration of the image processing device 100 according to the first embodiment. Specifically, the block diagram shown in Figure 2 shows an example of the hardware configuration of the image processing device 100 when each part of the image processing device 100 operates as software. The image processing device 100 is composed of a computer, which has a CPU 201, ROM 202, RAM 203, auxiliary storage device 204, display unit 205, operation unit 206, communication unit 207, and bus 208, as shown as an example in Figure 2.

[0013] The CPU 201 controls the computer using programs or data stored in the ROM 202 or RAM 203, thereby making the computer function as the various components of the image processing device 100 shown in Figure 1. The image processing device 100 may have one or more dedicated hardware components separate from the CPU 201, and at least a portion of the processing performed by the CPU 201 may be executed by the dedicated hardware. Examples of dedicated hardware include ASICs, FPGAs, and DSPs (Digital Signal Processors). The ROM 202 stores programs and other data that do not require modification. The RAM 203 temporarily stores programs or data supplied from the auxiliary storage device 204, or data supplied from an external source via the communication unit 207. The auxiliary storage device 204 is, for example, a hard disk drive and stores various types of data such as image data or audio data.

[0014] The display unit 205 is composed of, for example, a liquid crystal display or an LED, etc., and displays a GUI (Graphical User Interface) etc. for the user to operate or view the image processing apparatus 100. The operation unit 206 is composed of, for example, a keyboard, a mouse, or a touch panel, etc., and receives operations by the user and inputs various instructions to the CPU 201. In addition to the above-described configuration, the operation unit 206 may be configured to receive operations by the user's gestures through known gesture recognition processing. In this case, the operation unit 206 has a configuration for acquiring image data by visible light or infrared light, etc., and a configuration for recognizing the user's actions from the acquired image data and converting them into commands. Also, the operation unit 206 may be configured to receive operations by the user's voice through known voice recognition processing. In this case, the operation unit 206 has a configuration for recognizing the user's speech from the voice data output by a sound collection device such as a microphone and converting it into a command.

[0015] The CPU 201 also operates as a display control unit for controlling the display unit 205 and an operation control unit for controlling the operation unit 206. The communication unit 207 is used for communication with devices external to the image processing apparatus 100. For example, when the image processing apparatus 100 is wired-connected to an external device, a communication cable is connected to the communication unit 207. When the image processing apparatus 100 has a function of wireless communication with an external device, the communication unit 207 includes an antenna. The bus 208 connects each part of the image processing apparatus 100 and transmits information. In the first embodiment, the display unit 205 and the operation unit 206 are described as existing inside the image processing apparatus 100, but at least one of the display unit 205 and the operation unit 206 may exist outside the image processing apparatus 100 as another device.

[0016] The processing of each part of the image processing apparatus 100 will be described. The image acquisition unit 101 acquires the data of the input image (hereinafter referred to as "input image data"). Specifically, for example, the image acquisition unit 101 acquires the input image data by reading out the input image data stored in the auxiliary storage device 204 from the auxiliary storage device 204. The image acquisition unit 101 may acquire the input image data output by an external device via the communication unit 207, or may acquire the signal output by an image sensor such as a CMOS image sensor as the input image data. When the image acquisition unit 101 acquires the signal output by the image sensor, the image acquisition unit 101 may be provided with an A / D converter such as an analog / digital conversion circuit, and may generate and acquire the input image data by converting the signal into a digital signal.

[0017] The histogram acquisition unit 102 generates and acquires a histogram of the input image based on the input image data acquired by the image acquisition unit 101. The function acquisition unit 103 generates and acquires a piecewise linear function for grayscale conversion based on the input image histogram acquired by the histogram acquisition unit 102. The method for generating the histogram and the method for generating the piecewise linear function will be explained with reference to Figure 3. Hereinafter, it will be explained assuming that the number of histogram divisions (hereinafter also referred to as "classes") generated by the histogram acquisition unit 102 is 8. However, the number of classes in the histogram is not limited to 8; it may be 2 or more, 7 or less, or 9 or more. Preferably, the number of classes in the histogram generated by the histogram acquisition unit 102 is a number that corresponds to a value less than or equal to the value obtained by dividing the size of the range of pixel values ​​that each pixel in the input image can take by the size of the range of pixel values ​​that each pixel in the output image can take by. By setting the number of classes in the histogram to a number corresponding to the values ​​mentioned above, the size of the range of pixel values ​​that each class of pixels can take can be made to be less than or equal to the size of the range of pixel values ​​that each pixel in the output image can take. As an example, we will explain by assuming that the value obtained by dividing the size of the range of pixel values ​​that each pixel in the input image can take by the size of the range of pixel values ​​that each pixel in the output image can take is 8. In other words, we will explain below assuming that the size of the range of pixel values ​​that each class of pixels in the histogram can take and the size of the range of pixel values ​​that each pixel in the output image can take are equal to each other.

[0018] Figure 3 shows an example of an input image histogram and a cumulative histogram according to the first embodiment. The histogram shown as an example in Figure 3(a) is divided into eight classes, bin0 to 7, and the frequencies 300 to 307 represent the respective frequencies in each class. For example, the pixel value of each pixel in the input image is classified into eight classes, and its frequency is recorded in each class. Simply put, for example, the pixel value of each pixel in the input image is divided by 8, and the frequency is recorded in the class number (bin0 to 7) that matches the quotient. Specifically, for example, if the input image is a grayscale image, the pixel value of each pixel in the input image can be simply divided by the number of classes, which is 8. Also, for example, if the input image is a 3-channel RGB color image, the value of a specific channel data, such as the G channel data, among the 3-channel RGB data corresponding to each pixel in the input image can be divided by the number of classes, which is 8. Furthermore, for example, in this case, the system may be configured to calculate brightness, etc., from the RGB 3-channel data, and then divide the calculated brightness value by 8, which is the number of classes, to calculate the frequency, or it may be configured to calculate the frequency regardless of the channel.

[0019] The same configuration can be used even if the signal from the image sensor is a known Bayer format signal. Specifically, in a Bayer format signal, each pixel value corresponding to a unit composed of 2x2 photodiodes has one R channel data and one B channel data, and two G channel data. For example, averaging the two G channel data will simply yield one data for each of the RGB channels, so the same processing as in the case of the three RGB channel data described above can be performed. Alternatively, for example, one R channel data, one B channel data, and two G channel data can be processed directly for each pixel. The histogram generation method described above is merely an example, and the histogram generation method in the histogram acquisition unit 102 is not limited to the above, as long as a histogram of the input image can be generated.

[0020] Figure 3(b) shows an example of a cumulative histogram (also called a "cumulative frequency diagram"), which is the cumulative histogram of frequencies 300 to 307 for each class in the histogram shown in Figure 3(a). Bin0' to 7' in Figure 3(b) are the classes corresponding to bin0 to 7 in Figure 3(a). Here, the frequency of binN' (where N is an integer greater than or equal to 0 and less than or equal to 7) in the cumulative histogram shown in Figure 3(b) is the sum of the frequencies from bin0 to binN shown in Figure 3(a). Therefore, the frequency of bin7' in Figure 3(b) is the sum of the frequencies from bin0 to bin7 shown in Figure 3(a), and thus represents the sum of the frequencies for each class shown in Figure 3(a), i.e., the total number of pixels in the input image (hereinafter also called "total frequency"). The cumulative histogram may be generated by the histogram acquisition unit 102 or by the function acquisition unit 103, but it will be described below assuming that it is generated by the histogram acquisition unit 102.

[0021] The function acquisition unit 103 generates and acquires a piecewise linear function based on the cumulative histogram generated by the histogram acquisition unit 102. Specifically, the function acquisition unit 103 first divides the frequency of each class in the cumulative histogram shown in Figure 3(b) by the total number of pixels in the input image (total frequency). At this time, the value obtained by dividing the frequency of bin7' by the total number of pixels in the input image is 1.0 because the frequency of bin7' corresponds to the total number of pixels in the input image (total frequency). Next, the function acquisition unit 103 multiplies the value obtained by dividing the frequency of each class in the cumulative histogram by the total number of pixels in the input image (total frequency) by the maximum output gradation level when equalizing the histogram. Specifically, for example, if the output grayscale level is an integer value represented by an 8-bit bit length, that is, an integer value from 0 to 255, the function acquisition unit 103 multiplies the value obtained by dividing the frequency of each class in the cumulative histogram by the total number of pixels in the input image (total frequency) by 255. As a result, the maximum value of the output grayscale level in bin7' becomes 255. Furthermore, the function acquisition unit 103 generates a piecewise linear function by linearly connecting the frequency values ​​of each class in the cumulative histogram, which has been transformed as described above, in which the frequency of each class takes values ​​from 0 to 255.

[0022] The group of line segments 311, composed of multiple line segments as shown in Figure 3(b), represents an example of a piecewise linear function generated by the function acquisition unit 103. The method for generating piecewise linear functions described above is merely one example; the method for generating piecewise linear functions in the function acquisition unit 103 is not limited to the above, as long as it can generate a piecewise linear function for equalizing the histogram of the input image.

[0023] Histogram equalization can be performed by applying the piecewise linear function generated by the function acquisition unit 103 as a grayscale conversion function to the input image. However, as described above, when grayscale compression of the input image is performed simply by using a piecewise linear function generated based on the histogram, grayscale loss may occur. The following describes the grayscale loss that occurs when grayscale compression of the input image is performed using a piecewise linear function.

[0024] Grayscale compression using a piecewise linear function can be rephrased as a process that assigns the number of output grayscale levels to each segment of the input grayscale when the number of output grayscale levels is less than the number of input grayscale levels defined by the bit length of the input pixel values. From this perspective, the known piecewise linear function for equalizing the histogram generated by the function acquisition unit 103 can be understood as distributing the number of grayscale levels according to the ratio of the number of pixels belonging to each segment of the input grayscale.

[0025] Referring to Figure 4, a specific example of how to assign tones to each section of the input tone will be explained. Figure 4 is a diagram showing the correspondence between the input value, calculated value, and output value of a piecewise linear function in a certain section. Specifically, Figure 4(a) shows the calculated value and output value of the piecewise linear function when the input value to the piecewise linear function is an integer value from 0 to 9 and the slope of the piecewise linear function is 0.7. Also, Figure 4(b) shows the calculated value and output value of the piecewise linear function when the input value to the piecewise linear function is an integer value from 0 to 9 and the slope of the piecewise linear function is 1.3. Referring to Figure 4(a), when the input value is 1 and 2, the output value is 1 in both cases. Similarly, when the input value is 5 and 6, the output value is 4 in both cases, and when the input value is 8 and 9, the output value is 6 in both cases. This means that when tone conversion of an image is applied, the original tone cannot be preserved and the original tone is lost, that is, tone crushing may occur due to tone conversion of the image. However, when using tone compression as the purpose of tone conversion, as mentioned above, it is necessary to accept to some extent that the original tonal range will be lost after the tone conversion.

[0026] In contrast, referring to Figure 4(b), we can see that the output values ​​2, 6, and 11 do not appear. In the case of Figure 4(b), there is no loss of gradation as in Figure 4(a), but conversely, there are output values ​​that do not appear. In other words, for input values ​​in the range from 0 to 9, 13 gradations from 0 to 12 are assigned as output values, but 3 of these gradations are not used. This can be said to be a redundant assignment for a gradation conversion aimed at gradation compression. For example, when creating a known piecewise linear function for equalizing histograms, when pixels belonging to a certain range are concentrated, the number of gradations is distributed according to the ratio of the number of pixels belonging to that range, as described above. Therefore, in this case, redundant gradations may be assigned to the range where pixels are concentrated. In general, if the slope of the piecewise linear function is less than 1.0, there is a possibility that gradation loss will occur due to the gradation conversion of the image, and if it is greater than 1.0, redundant gradation assignments will be made due to the gradation conversion of the image.

[0027] Figure 5 shows an example of a histogram and cumulative histogram of an input image in which pixels belonging to a specific category are concentrated. Specifically, Figure 5(a) shows an example of a histogram of an input image in which pixels belonging to a specific category are concentrated, and Figure 5(b) shows an example of a cumulative histogram obtained by accumulating the frequencies of each bin0 to 7 in the histogram shown in Figure 5(a). In the histogram shown in Figure 5(a), pixels belonging to bin0 and bin1 are concentrated. As a result, as shown in Figure 5(b), it can be seen that in the piecewise linear function, which is the conversion function for grayscale conversion, the majority of the number of grayscale levels of the output value are assigned to bin0' and bin1'.

[0028] Referring to Figures 6 to 8, a method for modifying the piecewise linear function according to the first embodiment will be described. Figure 6 is a diagram showing an example of a piecewise linear function obtained by the function acquisition unit 103 according to the first embodiment. In the piecewise linear function shown in Figure 6, for eight divisions from division 0 to division 7, an output value corresponding to the minimum input value in each division (hereinafter referred to as the "start point") is determined, and in Figure 6, the output values ​​of each start point are represented by the dotted points 601 to 608 (hereinafter also referred to as "control points"). Also in Figure 6, the output value corresponding to the maximum input value in division 7 (hereinafter referred to as the "end point") is represented by the dotted point 609. As shown in Figure 6, the output value of the start point in one adjacent division and the output value of the end point in the other division are the same value and correspond to the values ​​of points 602 to 608.

[0029] In Figure 6, the line shown by the dashed line is a line that starts at point 601 and where the input value and output value are the same, that is, a line where y=x. In other words, this line passes through point 601 and has a slope of 1 . This is a straight line with a value of 0. This line is displayed as an auxiliary line to make it easier to understand the shape of the piecewise linear function shown by the solid line passing through each point from point 601 to point 609, and will be referred to as the auxiliary line below. In Figure 6, the slope of the line segment connecting points 602 and 603, which is the gradation characteristic of section 1, is greater than the slope of the auxiliary line (1.0). Here, as described above, in this embodiment, the size of the range of pixel values ​​that each pixel belonging to each class in the histogram can take is equal to the size of the range of pixel values ​​that each pixel in the output image can take. Therefore, the fact that the slope of the line segment connecting points 602 and 603, which is the gradation characteristic of section 1, is greater than the slope of the auxiliary line (1.0) means that redundant gradation assignments are occurring in section 1. Furthermore, the gradient characteristics of sections 0 and 2 through 7, specifically the slope of the line segment connecting point 601 and point 602, and the slope of each line segment connecting two adjacent points from point 603 to point 609, are smaller than the slope of the auxiliary lines. Therefore, this means that in sections 0 and 2 through 7, there is a possibility of tonal degradation due to the gradient conversion.

[0030] Referring to Figure 7, a method for improving this redundant gradation assignment, that is, a method for modifying the piecewise linear function obtained by the function acquisition unit 103, will be described. Figure 7 is a diagram illustrating an example of a method for modifying the piecewise linear function according to the first embodiment. In Figure 7, components similar to those in Figure 6 are denoted by the same reference numerals, and their explanations are omitted. The white dots 712 to 718 in Figure 7 correspond to values ​​calculated during the processing described later. Also, for explanatory purposes, Figure 7 shows three auxiliary lines, two more than in Figure 6.

[0031] The piecewise linear function obtained by the function acquisition unit 103 is modified by the first modification unit 104 and the second modification unit 105. Specifically, for example, the modification process by the first modification unit 104 and the second modification unit 105 is performed sequentially, starting with section 0 as the first section to be processed, and moving from section 0 toward section 7, with each adjacent section being the next section to be processed. Using section 0 as the first section to be processed is merely an example and is not limited to this. For example, the modification process by the first modification unit 104 and the second modification unit 105 may be performed sequentially, starting with section 7 as the first section to be processed, and moving from section 7 toward section 0, with each adjacent section being the next section to be processed.

[0032] First, the first modification unit 104 takes section 0 as the first section to be processed and compares the slope of the piecewise linear function in section 0, that is, the slope of the line segment connecting points 601 and 602, with the slope of the auxiliary line, which is the upper limit of the slope. If the slope of the piecewise linear function in the section to be processed is less than or equal to the slope of the auxiliary line, the first modification unit 104 terminates the process without modifying the piecewise linear function in the section to be processed. That is, in the piecewise linear function shown as an example in Figure 7, the slope of the piecewise linear function in section 0 is less than or equal to the slope of the auxiliary line, so the first modification unit 104 terminates the process without modifying the piecewise linear function in section 0. Note that if the slope of the piecewise linear function in the section to be processed is less than or equal to the slope of the auxiliary line, the second modification unit 105 does not perform any processing, and the modification process by the second modification unit 105 is omitted.

[0033] The above-described process in the first modification unit 104 can also be described as follows. First, the first modification unit 104 draws a straight line with a slope of 1 from point 601 and finds the value of point 712, which corresponds to the endpoint of section 0 (the starting point of section 1). Next, the first modification unit 104 compares the value of point 712 found with the value of point 602, which corresponds to the starting point of section 1 of the piecewise linear function shown by the solid line, and selects the point 602 with the smaller value, i.e., the point 602 with the smaller y-coordinate. In this case, the linear function corresponding to the line segment connecting point 601 and the selected point 602 becomes the piecewise linear function in section 0 after the modification process by the first modification unit 104. Therefore, as a result, the piecewise linear function in section 0 is not modified, and the first modification unit 104 terminates its process.

[0034] Next, the first modification unit 104 takes section 1 as the section to be processed and compares the slope of the piecewise linear function in section 1, that is, the slope of the line segment connecting points 602 and 603, with the slope of the auxiliary line, which is the upper limit of the slope. If the slope of the piecewise linear function in the section to be processed is greater than the slope of the auxiliary line, the first modification unit 104 modifies the piecewise linear function in the section to be processed so that it passes through the starting point in the section to be processed and its slope is equal to the slope of the auxiliary line, i.e., the upper limit of the slope. That is, in Figure 7, the slope of the piecewise linear function in section 1 is greater than the slope of the auxiliary line, so the first modification unit 104 modifies the piecewise linear function in section 1 so that it passes through point 602, which is the starting point in section 1, and its slope is equal to the slope of the auxiliary line. Through the modification process by the first modification unit 104, the piecewise linear function in section 1 is modified to be a linear function corresponding to the line segment connecting points 602 and 713. Furthermore, if the slope of the piecewise linear function in the section to be processed is greater than the slope of the auxiliary line, the correction process by the second correction unit 105 is performed after the correction process by the first correction unit 104.

[0035] The second modification unit 105 first calculates an output value corresponding to point 713, which is the endpoint of the piecewise linear function in section 1 after it has been modified by the first modification unit 104. Next, the second modification unit 105 modifies the piecewise linear function in section 2, which is the next section to be processed after section 1, that is, the linear function corresponding to the line segment connecting point 603 and point 604, to the linear function corresponding to the unillustrated line segment connecting point 713 and point 604. Through the modification process by the second modification unit 105, a modified piecewise linear function is obtained that has been processed up to section 1.

[0036] The above-described processing in the first modification unit 104 and the second modification unit 105 can also be described as follows. First, the first modification unit 104 draws a straight line with a slope of 1 from point 602 and finds the value of point 713, which corresponds to the endpoint of section 1 (the starting point of section 2). Next, the first modification unit 104 compares the value of point 713 found with the value of point 603, which corresponds to the starting point of section 2 of the piecewise linear function shown by the solid line, and selects the point 713 with the smaller value, i.e., the point 713 with the smaller y-coordinate. In this case, the linear function corresponding to the line segment connecting point 602 and the selected point 713 becomes the piecewise linear function in section 1 after the modification processing by the first modification unit 104. Also, the linear function corresponding to the line segment connecting the selected point 713 and point 604 becomes the piecewise linear function in section 2 after the modification processing by the second modification unit 105.

[0037] Next, the first modification unit 104, treating section 2 as the section to be processed, compares the slope of the modified piecewise linear function in section 2, that is, the slope of the unillustrated line segment connecting points 713 and 604, with the slope of the auxiliary line, which is the upper limit of the slope. In Figure 7, since the slope of the modified piecewise linear function in section 2 is greater than the slope of the auxiliary line, the first modification unit 104 modifies the piecewise linear function in section 2 so that it passes through point 713, which is the modified starting point in section 2, and its slope is equal to the slope of the auxiliary line. In other words, the piecewise linear function in section 2 is modified by the modification process of the first modification unit 104 to become a linear function corresponding to the line segment connecting points 713 and 714. After the modification process by the first modification unit 104, the second modification unit 105 first calculates the output value corresponding to point 714, which is the endpoint of the piecewise linear function in section 2 after it has been modified by the first modification unit 104. Next, the second modification unit 105 modifies the piecewise linear function in section 3, which is the next section to be processed after section 2, that is, the linear function corresponding to the line segment connecting points 604 and 605, to the linear function corresponding to the line segment (not shown) connecting points 714 and 605. Through the modification process by the second modification unit 105, a modified piecewise linear function is obtained that has been processed up to section 2.

[0038] The above-described processing in the first modification unit 104 and the second modification unit 105 can also be described as follows. First, the first modification unit 104 draws a straight line with a slope of 1 from point 713 and finds the value of point 714, which corresponds to the endpoint of section 2 (the starting point of section 3). Next, the first modification unit 104 compares the value of point 714 found with the value of point 604, which corresponds to the starting point of section 2 of the piecewise linear function shown by the solid line, and selects the point 714 with the smaller value, i.e., the point 714 with the smaller y-coordinate. In this case, the linear function corresponding to the line segment connecting point 713 and the selected point 714 becomes the piecewise linear function in section 2 after the modification processing by the first modification unit 104. Also, the linear function corresponding to the line segment connecting the selected point 714 and point 605 becomes the piecewise linear function in section 3 after the modification processing by the second modification unit 105.

[0039] Thereafter, the image processing device 100 sequentially processes sections 3 to 7, performing the same correction process as described above. Note that for section 7, since there is no subsequent section to be processed, the processing of the second correction unit 105 may be omitted when section 7 is selected as the processing section, or section 7 may not be selected as the processing section after section 6 has been selected. The above processing, explained with reference to Figures 6 to 8, can be generalized using mathematical formulas as follows. In the following explanation, M is the total number of control points of the piecewise linear function obtained by the function acquisition unit 103. i is an integer greater than or equal to 0 and less than M, and is an index variable for explanation. F(i) is the control point of the piecewise linear function in section i before correction, and G(i) is the control point of the piecewise linear function in section i after correction. x(P) is the x-coordinate of point P, and y(P) is the y-coordinate of point P. a is a predetermined upper limit of the slope, which is the slope of the auxiliary line described above. D(i) is the width of segment i, and min() is a function that outputs the smallest explanatory variable among the input explanatory variables as the target variable.

[0040] If i is 0, the control point of the piecewise linear function in piece i before modification is used as the control point after modification. That is, if i is 0, the control point after modification is calculated by the following equation (1). G(i) = F(i) ... Equation (1) F(0) and G(0) correspond to point 601, which is shown as an example in Figure 7.

[0041] If i is 1 or greater, first, the coordinates of point Q are determined. Specifically, the x-coordinate of point Q corresponds to the x-coordinate of the control point of the piecewise linear function before modification in section i, and corresponds to the x-coordinate value corresponding to the boundary between section i being processed and section i-1, which was the section immediately preceding the section being processed (hereinafter also referred to as the "boundary value"). Also, the y-coordinate of point Q corresponds to the y-coordinate of the point where a half-line drawn with slope a from the control point of the modified piecewise linear function in section i-1 intersects with the endpoint of that section. That is, the x-coordinate and y-coordinate values ​​of point Q are calculated by the following equations (2) and (3). x(Q)=x(F(i))=x(F(i-1))+D(i-1) ··· Formula (2) y(Q)=y(F(i-1))+a×D(i-1) ··· Equation (3)

[0042] Next, we determine the coordinates of point G(i). Specifically, the x and y coordinates of point G(i) are calculated using equations (4) and (5) below. x(G(i))=x(F(i)) ··· Equation (4) y(G(i))=min(y(F(i)),y(Q)) Equation (5)

[0043] The operations from equation (2) to equation (5) are repeatedly performed, incrementing i from 1 to M-1.

[0044] Figure 8 shows an example of a piecewise linear function after correction processing by the first correction unit 104 and the second correction unit 105 according to the first embodiment. Specifically, Figure 8 shows an example of a piecewise linear function after correction processing of the piecewise linear function in each of the sections from section 0 to section 7. In Figure 8, the same reference numerals are used for components that are the same as those in Figure 6 or Figure 7. In the corrected piecewise linear function shown as an example in Figure 8, the slope of the piecewise linear function in each of the sections from section 1 to section 4 is all 1.0. That is, in sections 0 and sections 2 to 7, where there was a possibility of tonal collapse due to tone conversion, the condition has been improved so that tonal collapse due to tone conversion does not occur in sections 2 to 4. Furthermore, in section 5, the slope of the piecewise linear function in section 5 is greater than before the correction, improving the condition so that tonal collapse due to tone conversion is less likely to occur than before the correction. Furthermore, for Category 1, the image remains free from tonal degradation due to tone conversion even after the correction. For Categories 0, 6, and 7, the possibility of tonal degradation due to tone conversion remains the same even after the correction.

[0045] In other words, comparing Figure 6 and Figure 8, in the uncorrected piecewise linear function shown in Figure 6, the slope of section 1 is greater than 1.0, resulting in redundant gradation allocation in section 1, as exemplified in Figure 4(b). Also, in each section from section 2 to section 4, the slope of the uncorrected piecewise linear function is less than 1.0, which can cause gradation collapse as exemplified in Figure 4(a). In contrast, in the corrected piecewise linear function shown in Figure 8, the slope of section 1 becomes 1.0, eliminating redundant gradation allocation in section 1. Furthermore, by eliminating the redundant gradation allocation in section 1, the remaining gradations are allocated to sections 2 to 4 (and possibly section 5), thus resolving gradation collapse in each section from section 2 to 4.

[0046] Furthermore, comparing Figure 6 and Figure 8, as shown as an example in Figure 8, the range of the target variable that the modified piecewise linear function can take is the same as the range of the target variable that the unmodified piecewise linear function can take, as shown in Figure 6. Therefore, the image processing device 100 can easily obtain a modified piecewise linear function for performing grayscale compression of the input image without renormalizing the histogram. Finally, there are cases where the magnitude of the slope of the unmodified piecewise linear function in the last section to be processed, specifically, for example, section 7 in Figure 6, is greater than the upper limit of the slope. In this case, the range of the target variable that the modified piecewise linear function can take may be smaller than the range of the target variable that the unmodified piecewise linear function can take, as shown in Figure 6. However, even in such cases, only the redundant assignment of grayscale is corrected, and the desired grayscale compression is achieved. In such cases, the histogram may be renormalized and the modified piecewise linear function may be modified again so that the range of the target variable that the modified piecewise linear function can take and the range of the target variable that the unmodified piecewise linear function can take are similar.

[0047] Furthermore, although the above explanation assumes that the slope of the guideline is 1.0, the slope of the guideline is not limited to 1.0. Specifically, in this embodiment, from the viewpoint of correcting redundant assignment of gradations, a slope of 1.0 for the guideline is preferable, but the slope of the guideline may be less than 1.0 as long as it is a positive value of 1.0 or less. If the slope of the guideline is a positive value less than 1.0, the number of gradations assigned to each section is reduced, and furthermore, the number of gradations assigned to each section is distributed on an average basis.

[0048] The image conversion unit 106 performs grayscale conversion on the input image data acquired by the image acquisition unit 101 using the piecewise linear function that has been modified by the first modification unit 104 and the second modification unit 105. The image output unit 109 outputs the image data after grayscale conversion by the image conversion unit 106 as output image data. Specifically, the image output unit 109 outputs the output image data to the auxiliary storage device 204, and the output image data is stored in the auxiliary storage device 204. The image output unit 109 may also output the output image data to the display unit 205, and the output image may be displayed on a display device such as a liquid crystal display or LED. Alternatively, the image output unit 109 may output the output image data to an external device via the communication unit 207.

[0049] In this case, the external device may acquire the output image data, perform the inverse conversion of the grayscale conversion performed by the image processing device 100 on the acquired output image data, and acquire image data corresponding to the original input image data. When the external device performs the inverse conversion of the grayscale conversion, the image processing device 100 may output information necessary for the inverse conversion of the grayscale conversion, such as information indicating the modified piecewise linear function used for the grayscale conversion, in association with the output image data. In this case, for example, the communication unit 207 may be configured to transmit the output image data and the information necessary for the inverse conversion of the grayscale conversion on different channels. Alternatively, for example, the communication unit 207 may be configured to transmit the output image data and the information necessary for the inverse conversion of the grayscale conversion on the same channel using a protocol or format that can identify them. Furthermore, for example, the external device may be configured to request the information necessary for the inverse conversion of the grayscale conversion from the image processing device 100 via the communication unit 207 and read the information stored in RAM 203 or the like.

[0050] The operation of the image processing device 100 will be explained with reference to Figure 9. Figure 9 is a flowchart showing an example of the processing flow of the image processing device 100 according to the first embodiment. In the following explanation, the symbol "S" means step. First, at S901, the image acquisition unit 101 acquires input image data. Next, at S902, the histogram acquisition unit 102 acquires a histogram of the input image. Next, at S903, the function acquisition unit 103 acquires a piecewise linear function based on the cumulative histogram.

[0051] Next, in S904, the first modification unit 104 selects a category to be processed and determines whether the magnitude of the slope of the piecewise linear function in the category to be processed is greater than a predetermined upper limit of the slope, that is, the magnitude of the slope of the auxiliary line. Specifically, for example, the first modification unit 104 selects category 0 shown in Figure 6 as the first category to be processed and determines whether the magnitude of the slope of the piecewise linear function in category 0 is greater than the upper limit of the slope. If it is determined in S904 that the magnitude of the slope of the piecewise linear function in the category to be processed is greater than the upper limit of the slope, then in S905, the first modification unit 104 modifies the piecewise linear function in the category to be processed.

[0052] The processes in S904 and S905 can be explained using the above-mentioned equations (1) to (5) as follows. First, the first modification unit 104 initializes the index variable i to 0 only when it selects section 0, shown as an example in Figure 6, as the first section to be processed, and performs the operation of equation (1) to set point G(0) to F(0). Next, the first modification unit 104 increments the value of the index variable i and uses equations (1) to (5) to find the coordinates of point G(i). If the y-coordinate of point G(i) is the y-coordinate of point Q, it means that the magnitude of the slope of the piecewise linear function in the section to be processed is greater than the upper limit of the slope. Therefore, in this case, the first modification unit 104 modifies the piecewise linear function in the section to be processed to a linear function corresponding to the line segment connecting point G(i-1) and point G(i). The case where the y-coordinate of point G(i) is the y-coordinate of point F(i) will be described later.

[0053] After S905, in S906, the second modification unit 105 processes the current target of processing. classification Next, the piecewise linear function in the section to be processed is modified. To explain the process of S906 using the above equations (1) to (5), the second modification unit 105 modifies the current processing target. classification Next, the piecewise linear function in the section to be processed is modified to a line segment connecting point G(i) and point F(i+1). After S906, in S907, for example, the first modification unit 104 modifies the current processing target. classificationNext, it is determined whether there is a category to be processed. The process in S907 is equivalent to the process of determining whether the value of i is less than M-1. In S907, the current processing target classification If it is determined that there is another section to be processed next, the image processing device 100 returns to the process of S904 and executes the process of S904. That is, in this case, the image processing device 100 returns to the process of S904, selects the next section to be processed as the current section to be processed, and performs the processing from S904 onwards. In other words, in this case, the image processing device 100 returns to the process of S904, increments the value of the index variable i, and executes the processing after the increment described above.

[0054] If it is determined in S904 that the magnitude of the slope of the piecewise linear function in the section to be processed is not greater than the upper limit of the slope, that is, it is less than or equal to the upper limit, the image processing device 100 executes the process in S907. In other words, if such a section is the target of processing, the image processing device 100 executes the process in S907 without performing any modifications to the piecewise linear function by the first modification unit 104 and the second modification unit 105. Note that if the y-coordinate of point G(i) is the same as the y-coordinate of point F(i), this corresponds to the case where it is determined that the magnitude of the slope of the piecewise linear function in the section to be processed is not greater than the upper limit of the slope.

[0055] S907 is currently processing target classification If it is determined that there are no further sections to be processed, in S908 the image conversion unit 106 performs grayscale conversion of the input image data using the modified piecewise linear function. After S908, in S909 the image output unit 109 outputs the grayscale-converted image data as output image data. After S909 the image processing device 100 completes the processing shown in the flowchart in Figure 9.

[0056] As described above, the image processing device 100 allows for the easy modification of the piecewise linear function used for image tone compression without renormalizing the histogram. Furthermore, the tone compression process using the modified piecewise linear function optimizes the allocation of redundant tones. As a result of optimizing the allocation of redundant tones, the remaining tones can be sequentially allocated to other sections, improving the tone characteristics of those sections.

[0057] In this embodiment, the case where the input image data is still image data has been described as an example, but the input image data may be moving image data. When the input image data is moving image data, the image processing device 100 performs the above processing for each of the data of multiple frames included in the moving image data acquired by the image acquisition unit 101. However, when moving image data is transmitted continuously from an imaging device, etc., the generation of a histogram corresponding to a certain frame may not be completed in time for the start of the subsequent processing. In such a case, it is not possible to generate the uncorrected piecewise linear function corresponding to that frame from the histogram of that frame. Here, the main focus of this disclosure is a method for correcting the piecewise linear function used for grayscale conversion, so the method for acquiring the original uncorrected piecewise linear function may be various, and the uncorrected piecewise linear function may include approximations, etc. Therefore, in the above case, the system may be configured to generate the uncorrected piecewise linear function by using a histogram acquired from a past frame, preferably a frame from about one or two frames ago, as an approximate histogram.

[0058] Furthermore, although this embodiment has been described as having all of the parts shown in Figure 1 in a single image processing device 100, the image processing system may also be configured as having a first device having some of the parts shown in Figure 1 and a second device having the other parts.

[0059] Furthermore, in this embodiment, as an example, a configuration was described in which the piecewise linear function correction process is performed sequentially from a small class, i.e., a division to which pixels with small pixel values ​​belong, to a large class, i.e., a division to which pixels with large pixel values ​​belong. However, the image processing device 100 may also perform the piecewise linear function correction process sequentially as the division to be processed, from a large class to a small class. In this case, the right end of the division (the endpoint on the large class side) is set as the starting point, the endpoint (the endpoint of the small class) is tentatively determined according to the upper limit of the slope, and the process can be appropriately replaced by selecting the point with the larger y-coordinate between the tentatively determined endpoint and the corresponding control point before correction.

[0060] [Second Embodiment] The image processing apparatus 100a according to the second embodiment will be described with reference to Figures 10 and 11. The image processing apparatus 100 according to the first embodiment sequentially performs piecewise linear function correction processing from small classes to large classes or from large classes to small classes in the histogram, and the direction is predetermined. In contrast, the image processing apparatus 100a according to the second embodiment has a function that determines, based on the histogram, whether to perform sequential correction processing from small classes to large classes or from large classes to small classes.

[0061] Figure 10 is a block diagram showing an example of the configuration of a functional block in the image processing apparatus 100a according to the second embodiment. The image processing apparatus 100a includes an image acquisition unit 101, a histogram acquisition unit 102, a function acquisition unit 103, a determination unit 1001, a first modification unit 104, a second modification unit 105, an image conversion unit 106, and an image output unit 109. In other words, the image processing apparatus 100a is the image processing apparatus 100 according to the first embodiment with the determination unit 1001 added. Hereafter in Figure 10, components similar to those shown in Figure 1 are denoted by the same reference numerals and their descriptions are omitted. Specifically, the image acquisition unit 101, histogram acquisition unit 102, function acquisition unit 103, first modification unit 104, second modification unit 105, image conversion unit 106, and image output unit 109 according to the second embodiment are the same as those in the first embodiment and their descriptions are omitted. The processing of each part of the image processing device 100a is performed by hardware such as an ASIC or FPGA built into the image processing device 100a, similar to the first embodiment 1. This processing may also be performed by software executed by hardware as shown as an example in Figure 2.

[0062] The determination unit 1001 first determines the class (category) to be processed. Specifically, the determination unit 1001 obtains the magnitude of the slope of the piecewise linear function in the smallest class (hereinafter referred to as the "minimum class") among all possible categories to be processed, that is, the class to which the pixel with the smallest pixel value belongs. The determination unit 1001 also obtains the magnitude of the slope of the piecewise linear function in the largest class (hereinafter referred to as the "maximum class") among all possible categories to be processed, that is, the class to which the pixel with the largest pixel value belongs. Furthermore, if the magnitude of the slope of the piecewise linear function in the minimum class is greater than the upper limit of the slope, and the magnitude of the slope of the piecewise linear function in the maximum class is less than or equal to the upper limit of the slope, the minimum class is determined as the first class (category) to be processed. Also, if the magnitude of the slope of the piecewise linear function in the maximum class is greater than the upper limit of the slope, and the magnitude of the slope of the piecewise linear function in the minimum class is less than or equal to the upper limit of the slope, the maximum class is determined as the first class (category) to be processed.

[0063] By determining the initial processing target class in this way, it is possible to output output image data that makes full use of the range of pixel values ​​that the output image data can take. Furthermore, by determining the initial processing target class in this way, the allocation of redundant gradations can be optimized, and as a result of optimizing the allocation of redundant gradations, the remaining gradations can be sequentially allocated to other classes, improving the gradation characteristics of those classes.

[0064] Furthermore, for example, if the magnitude of the slope of the piecewise linear function in the smallest class and the largest class is greater than the upper limit of the slope, the determination unit 1001 determines the class with the greater slope as the first class (section) to be processed. By determining the first class to be processed in this way, it is possible to output output image data that uses a wider range of possible pixel values. Also, by determining the first class to be processed in this way, it is possible to optimize sections that have been allocated more redundant gradations. Moreover, as a result of optimizing the allocation of redundant gradations, the remaining gradations can be sequentially allocated to other sections, improving the gradation characteristics of those sections.

[0065] Furthermore, for example, when the magnitude of the slope of the piecewise linear function in the smallest class and the largest class is both less than or equal to the upper limit of the slope, the determination unit 1001 determines the first class (classification) to be processed as follows. For example, the determination unit 1001 first identifies the class with the largest magnitude of the slope of the piecewise linear function from among all classes that can be processed. Next, from among the smallest class and the largest class, it determines the class that is closer to the class identified above as the first class (classification) to be processed. When the magnitude of the slope of the piecewise linear function in the smallest class and the largest class is both less than or equal to the upper limit of the slope, the determination unit 1001 may determine the first class (classification) to be processed as follows. For example, the determination unit 1001 first identifies the class in the cumulative histogram whose total frequency value exceeds half of the total frequency value in the class with the largest total frequency value from among all classes that can be processed. Next, from among the smallest class and the largest class, it determines the class that is closer to the class identified above as the first class (classification) to be processed. By determining the class to be processed first in this way, output image data that represents the features of the input image can be output.

[0066] Figure 11 is a flowchart showing an example of the processing flow of the image processing apparatus 100a according to the second embodiment. Hereinafter, in Figure 11, steps similar to those shown in Figure 9 are denoted by the same reference numerals and their explanation is omitted. First, the image processing apparatus 100a executes the processes from S901 to S903. After S903, in S1101, the determination unit 1001 determines the class (category) of the first processing target. After S1101, the image processing apparatus 100a executes the processes from S904 to S909 as appropriate and completes the processing shown in the flowchart of Figure 11.

[0067] As described above, the image processing device 100a allows for the easy modification of the piecewise linear function used for image tone compression without renormalizing the histogram. Furthermore, by performing tone compression processing using the modified piecewise linear function, the allocation of redundant tones can be optimized. As a result of optimizing the allocation of redundant tones, the remaining tones can be sequentially allocated to other sections, improving the tone characteristics of those sections. In particular, the image processing device 100a allows for further improvement of the tone characteristics of each section by appropriately determining which section to process first when modifying the piecewise linear function.

[0068] In the above embodiment, an example of a configuration in which still images are temporarily stored in RAM 203 was shown. However, as the image processing device 100, it is also possible to use, for example, a video capture device that acquires multiple time-sequentially consecutive frames as a video image.

[0069] In this case, the frame acquired by the image acquisition unit 101 is passed to the histogram acquisition unit 102, and the function acquisition unit 103 obtains a piecewise linear function from the histogram based on the frame. After the piecewise linear function has been modified as necessary, the image conversion unit 106 performs grayscale conversion of the input image using the modified piecewise linear function.

[0070] However, in the above configuration, it is conceivable that all frames may be input to the image conversion unit 106 before the modified piecewise linear function is input to the image conversion unit 106. Therefore, in this case, the histogram obtained based on the current frame cannot be used for grayscale conversion of the current frame (hereinafter referred to as the "current frame"). To address this, the system is configured to use a histogram obtained based on a past frame (hereinafter referred to as a "past frame"), preferably a past frame from one or two frames prior, as an approximation. The function acquisition unit 103 obtains a piecewise linear function from the histogram obtained based on the past frame. After the necessary modifications are made to the piecewise linear function, the image conversion unit 106 performs grayscale conversion of the current frame using the modified piecewise linear function.

[0071] Furthermore, in the above embodiment, the function acquisition unit 103 acquires a piecewise linear function from a histogram based on the input image. However, it is also possible to configure the system to calculate and acquire the necessary piecewise linear function based on user instructions and input, without using an input image.

[0072] Furthermore, in the above embodiment, the first modification unit 104 and the second modification unit 105 are used when modifying the piecewise linear function. However, it is also possible to store only the boundary values ​​of the sections, rather than storing the start and end values ​​for each section, as the data format for storing the piecewise linear function. In this case, for example, changing the boundary value corresponding to the end value of the section to be processed will change the start value of the next section to be processed. Therefore, depending on the data format for storing the piecewise linear function, modification by the second modification unit 105 may be unnecessary, and modification by only the first modification unit 104 may be performed.

[0073] [Other embodiments] This disclosure can also be implemented by supplying a program that implements one or more of the functions of the embodiments described above to a system or device via a network or storage medium, and by having one or more processors in the computer of that system or device read and execute the program. It can also be implemented by a circuit such as an ASIC that implements one or more functions.

[0074] Within the scope of this disclosure, it is possible to freely combine the embodiments, modify any component of each embodiment, or omit any component in each embodiment. [Explanation of Symbols]

[0075] 100 Image Processing Devices 103 Function acquisition section 104 First Amendment Part 105 Second Amendment Part

Claims

1. An image processing device that performs grayscale conversion on an image, A means for obtaining a piecewise linear function for grayscale conversion, The magnitude of the slope of the piecewise linear function in the processing target category exceeds a predetermined upper limit. If the value is also large, the value of the endpoint of the processing target section can be changed. First modification: Modify the slope of the piecewise linear function in minutes so that it is equal to the upper limit. The system has means, and sequentially performs modifications by the first modification means on sections adjacent to the section to be processed. The image processing apparatus is characterized in that the aforementioned upper limit is a positive value of 1.0 or less.

2. The acquisition means, based on the histogram of the image, determines the dividing line for the grayscale conversion. The image processing apparatus according to claim 1, characterized by acquiring a shape function.

3. The acquisition means accumulates the frequencies from the smallest division in the histogram to each division. The frequency of each segment in the resulting cumulative histogram is divided by the total frequency of the histogram. Based on the value obtained by multiplying the value by the magnitude of the range that the piecewise linear function can take as the dependent variable, To obtain a piecewise linear function The image processing apparatus according to claim 2, characterized by the above.

4. The aforementioned upper limit is the magnitude of the range of the dependent variable that the piecewise linear function can take. This value is obtained by dividing the function by the range of explanatory variables it can take. An image processing apparatus according to any one of claims 1 to 3, characterized by the above.

5. The aforementioned upper limit is 1.

0. The image processing apparatus according to claim 1, characterized in that

6. The smallest division in the aforementioned piecewise linear function is defined as the first division to be processed, and the size of the division The adjacent sections are processed sequentially as the next section to be processed, in the direction that increases the value. The image processing apparatus according to any one of claims 1 to 5.

7. The image processing apparatus according to claim 6, further comprising: determination means for determining the smallest division as the first division to be processed when the slope of the piecewise linear function in the largest division of the piecewise linear function is smaller than the upper limit.

8. The image processing apparatus according to any one of claims 1 to 5, characterized in that the largest division in the piecewise linear function is set as the first division to be processed, and adjacent divisions in the direction of decreasing division size are processed sequentially as the next division to be processed.

9. The image processing apparatus according to claim 8, further comprising: determination means for determining the largest portion as the first portion to be processed when the slope of the piecewise linear function in the smallest portion of the piecewise linear function is smaller than the upper limit.

10. The system further comprises a conversion means that performs the grayscale conversion on the image using the piecewise linear function after the correction process has been applied to all sections. An image processing apparatus according to any one of claims 1 to 9, characterized by the above.

11. The image processing apparatus according to any one of claims 1 to 10, further comprising a second modification means for changing the starting point value of the next section to be processed if the magnitude of the slope of the piecewise linear function in the section to be processed is greater than the upper limit.

12. The aforementioned image is a moving image, The acquisition means, based on past frames in the video, performs the grayscale conversion The image processing apparatus according to any one of claims 1 to 11, which obtains the aforementioned piecewise linear function.

13. The acquisition means acquires the piecewise linear function for grayscale conversion based on user instructions. The image processing apparatus according to claim 1, characterized in that...

14. An image processing method that performs grayscale conversion on an image, A step to obtain a piecewise linear function for the grayscale conversion, The magnitude of the slope of the piecewise linear function in the processing target category exceeds a predetermined upper limit. If the value is also large, the value of the endpoint of the processing target section can be changed. First modification: Modify the slope of the piecewise linear function in minutes so that it is equal to the upper limit. The process, The process is carried out sequentially for the sections adjacent to the section to be processed, according to the first modification step. Make corrections, The image processing method is characterized in that the upper limit is a positive value of 1.0 or less.

15. An image processing system that performs grayscale conversion on an image, A means for obtaining a piecewise linear function for grayscale conversion, The magnitude of the slope of the piecewise linear function in the processing target category exceeds a predetermined upper limit. If the value is also large, the value of the endpoint of the processing target section can be changed. First modification: Modify the slope of the piecewise linear function in minutes so that it is equal to the upper limit. means and The first modification means sequentially applies to the sections adjacent to the section to be processed. Make corrections, The aforementioned upper limit is a positive value of 1.0 or less, characterized in that it is an image processing system.

16. The computer is operated as an image processing device according to any one of claims 1 to 13. A program to enable it.

17. The computer is operated as an image processing device according to any one of claims 1 to 13. A storage medium that stores a program for execution.

Images