Image processing device, method, and program

Abstract

To easily recognize whether or not the designation of an extraction range corresponding to a specific area is appropriate. [Solution] A captured image 110 of a chart including multiple patches is acquired, and multiple extraction ranges (chart position information 113) are designated on the captured image 110 displayed on a display unit 100 based on a user operation, and the presence or absence of pixel variation in the extraction range is determined from the pixel values ​​of the extraction range. A control unit 101 causes the display unit 100 to display the extraction range in a display mode based on the determination result.

Description

[Technical Field]

[0001] The present invention relates to an image processing device, method, and program. [Background technology]

[0002] In video production, multiple imaging devices are often used for filming. Even when shooting the same subject, the color and brightness characteristics obtained from each imaging device vary depending on the individual imaging device and lens used, as well as differences in environmental lighting. In order to match the characteristics of the captured image output from the imaging devices, there are color matching methods that use charts.

[0003] Patent Document 1 discloses an image processing device that extracts patch information from a photographed chart image acquired by photographing the chart. In Patent Document 1, a transformation matrix is ​​calculated from the four vertices of the chart and the coordinates of the four vertices of the chart in the photographed image, and the center coordinates of all patches in the photographed image are found from the center coordinates of each patch of the chart and the transformation matrix. Then, when all patches are non-overlapping and can be extracted, the pixel values ​​of the extraction range are obtained. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2015-226219 Summary of the Invention [Problem to be solved by the invention]

[0005] However, pixel variations can occur in the extraction range due to misalignment between the extraction range and the patch, compression noise, random noise, etc. When performing color matching, if the color matching is based on data that includes pixel variations, the accuracy of the color matching decreases, and it becomes necessary to reselect the chart or reacquire the image, which reduces workability.

[0006] An object of the present invention is to enable, when specifying a certain area in an image, a user to easily recognize whether or not the designation of a range corresponding to that area is appropriate. [Means for solving the problem]

[0007] In order to achieve the above object, the present invention is a program for controlling an image processing device, which causes a computer of the image processing device to execute the following steps: a first acquisition step of acquiring a captured image of a subject including a predetermined area; a control step of displaying the captured image acquired by the first acquisition step on a display unit; a designation step of designating a plurality of extraction ranges for the subject based on a user operation; a second acquisition step of acquiring pixel values ​​of each extraction range designated by the designation step in the captured image; and a determination step of determining whether a predetermined condition is satisfied for each extraction range from the pixel values ​​acquired by the second acquisition step, wherein the control step causes the display unit to display the plurality of extraction ranges in a display mode based on a determination result by the determination step. [Effects of the Invention]

[0008] According to the present invention, it is possible to easily recognize whether or not the designation of an extraction range corresponding to a specific area is appropriate. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram of the main parts of the image processing device. [Figure 2] FIG. 10 is a diagram showing an image displayed on a display unit. [Figure 3] FIG. 10 is a diagram showing an example of a display on the display unit during specification of an extraction range. [Figure 4] FIG. 10 is a diagram showing an example of a display on the display unit after the extraction range has been specified. [Figure 5] FIG. 10 is a diagram illustrating an example of drawing chart position information. [Figure 6] FIG. 1 is a diagram showing one rectangular range. [Figure 7]10 is a flowchart showing an extraction range drawing process. [Figure 8] 10 is a flowchart showing an extraction range drawing process. [Figure 9] FIG. 10 is a diagram showing a modified example of a chart image. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0011] (First embodiment) FIG. 1 is a block diagram of the main components of an image processing device according to a first embodiment of the present invention. This image processing device includes a display unit 100, a control unit 101, a pixel value extraction unit 102, and a pixel value determination unit 103. The image processing device also includes a CPU, ROM, RAM, etc., all of which are not shown. The ROM stores a control program executed by the CPU. The RAM provides a work area for the CPU to execute the control program. The functions of the control unit 101, pixel value extraction unit 102, and pixel value determination unit 103 are realized by cooperation between the CPU, ROM, RAM, etc.

[0012] 2 is a diagram showing an image displayed on the display unit 100. The display unit 100 displays a captured image 110 and multiple pieces of chart position information 113. The captured image 110 is image data acquired by capturing an image of a printout (not shown) on which an original chart including multiple patches serving as measurement images is printed. The captured image 110 is acquired by the control unit 101, pixel value extraction unit 102, and pixel value determination unit 103, which serve as first acquisition means.

[0013] The captured image 110 includes a chart image 111 corresponding to the above chart. The chart image 111 includes a patch image 112 and a frame image 114. The frame image 114 is an image of the area of ​​the chart image 111 excluding the patch image 112. The patch image 112 is a chart element corresponding to a patch included in the chart.

[0014] The display unit 100 displays the input photographed image 110 and chart position information 113 output from the control unit 101. The chart position information 113 is displayed superimposed on the chart image 111.

[0015] The chart position information 113 is an "extraction range" that is the target when extracting pixel values ​​for each patch image 112. The pixel values ​​of the extraction range become image information for determining whether or not there is pixel variation. A specific determination method will be described later. Determining whether or not there is pixel variation is an example of determining whether or not a predetermined condition is satisfied.

[0016] The control unit 101 generates chart position information 113 based on the number of extractions (h, v) and the extraction size (w), a user operation specifying the extraction range, and pixel value determination information 115 output from the pixel value determination unit 103, and outputs the chart position information 113 to the display unit 100. The control unit 101 also generates coordinate information 116 on the captured image 110 from the coordinates on the display unit 100 (explained in Equation 4 below), and outputs the coordinate information 116 to the pixel value extraction unit 102.

[0017] First, a method for generating the chart position information 113 will be described. The extraction number (h, v) indicates the number of patch images 112 in the horizontal direction (h) and the number of patch images 112 in the vertical direction (v) within the chart image 111. In the example shown in FIG. 2, the horizontal direction is 6 and the vertical direction is 4. The extraction size (w) is a numerical value indicating the width of the chart position information 113. The extraction number and extraction size may be input in advance by the user, or may be stored in advance in ROM as parameters for each chart.

[0018] Fig. 3 is a diagram showing an example of the display on the display unit 100 while an extraction range is being specified. Fig. 3 shows the central coordinates of the patch images 112 at the four corners of the multiple patch images 112. These central coordinates are the upper left patch coordinate 120, the upper right patch coordinate 121, the lower left patch coordinate 122, and the lower right patch coordinate 123.

[0019] The initial specification of the extraction range is performed by determining the four vertices of a rectangle 124. The four vertices of the rectangle 124 are the upper left designated coordinate Q1, the upper right designated coordinate Q2, the lower left designated coordinate Q3, and the lower right designated coordinate Q4.

[0020] The user can specify these coordinates using a mouse 119 or the like. There are no limitations on the method of specification, but as an example, the user first specifies one of patch coordinates 120, 121, 122, and 123 as the start point. At this point, the patch coordinates 120, 121, 122, and 123 do not necessarily coincide with the specified coordinates Q1, Q2, Q3, and Q4.

[0021] If the upper left patch coordinate 120 is set as the starting point, the user drags the mouse toward the lower right patch coordinate 123 located diagonally opposite the patch coordinate 120. This changes the range of the rectangle 124.

[0022] The matrix of chart position information 113 for the number of extractions (h, v) is defined as the extraction range by the rectangle 124. The center coordinates of each piece of chart position information 113 defined by the rectangle 124 while it is being specified are coordinates 126. The center coordinates of each piece of chart position information 113 defined by the rectangle 124 after the specification is complete are coordinates 130 (see FIG. 4).

[0023] The coordinate values ​​of coordinates 126 and 130 (x i, y j ) can be calculated by Equation 1, assuming that the origin of the coordinate axes shown in FIG.

[0024]

number

[0025] As mentioned above, h in Equation 1 is the number of horizontal extractions, and v is the number of vertical extractions. (x 1, y1) is the coordinate value of the upper left patch coordinate 120, (x h, y v ) indicates the coordinate value of the specified coordinate Q4 at the bottom right.

[0026] The starting point is not limited to the upper left patch coordinate 120, but may be another vertex such as the upper right patch coordinate 121. Also, it is not necessary to set one of the patch coordinates at the four corners as the starting point; for example, if the number of extractions is four horizontally and two vertically, the user may determine the patch coordinates that will be the starting point depending on the number of extractions.

[0027] When the specified coordinate Q4 reaches the lower right patch coordinate 123, the user releases the mouse button. At this point, the initial specification of the extraction range is completed. Next, the user modifies the extraction range by specifying it a second time or later. For example, the user fine-tunes the specified coordinate Q2 by operating the mouse so that it matches the upper right patch coordinate 121, and then fine-tunes the specified coordinate Q3 so that it matches the lower left patch coordinate 122. This changes the shape and range of the rectangle 124, as shown in the example of FIG. 4.

[0028] Fig. 4 is a diagram showing an example of the display on the display unit 100 after the extraction range has been specified at least once. As shown in Fig. 4, when the extraction range is ideally specified or corrected, patch coordinates 120, 121, 122, and 123 coincide with specified coordinates Q1, Q2, Q3, and Q4. Furthermore, coordinates 130, which are the center coordinates of each piece of chart position information 113, coincide with the center coordinates of the patch image 112.

[0029] The control unit 101 generates chart position information 113 based on the coordinates 130 and the extraction size. i, Y j ) can be calculated using Equation 2 and Equation 3.

[0030]

number

[0031] The coordinate value (x 1, y1) is the coordinate value of the specified coordinate Q1 at the top left of the rectangle 124, and the coordinate value (x h, y v) is the coordinate value of the designated coordinate Q4 at the bottom right of the rectangle 124. 1, Y1) is the coordinate value of the upper left coordinate 130, which is synonymous with the coordinate value of the designated upper left coordinate Q1 on the rectangle 124 in the example of FIG. 4, Y4) is the coordinate value of the bottom right coordinate 130, which is synonymous with the coordinate value of the bottom right designated coordinate Q4 in the example of FIG. 2, Y2) is the coordinate value of the upper right coordinate 130, which is synonymous with the coordinate value of the upper right designated coordinate Q2 in the example of FIG. 3, Y3) is the coordinate value of the bottom left coordinate 130, which is synonymous with the coordinate value of the bottom left designated coordinate Q3 in the example of Figure 4. The coordinate values ​​of both coordinates 130 are coordinate values ​​acquired when the user performs an operation.

[0032] The control unit 101 calculates the coordinate value (X i, Y j ) and the extraction size, the chart position information 113 can be generated. For example, if the extraction size is set to w=10, the coordinates (X i -5 , Y j -5), (X i +5 , Y j +5) is the chart position information 113.

[0033] The control unit 101 changes the drawing method of the extraction range (chart position information 113) based on pixel value determination information 115 output from the pixel value determination unit 103. The control unit 101 changes the display mode of the extraction range so that it is possible to distinguish between cases where there is pixel variation in the chart position information 113 and cases where there is no pixel variation.

[0034] Fig. 5 is a diagram showing an example of how chart position information 113 is displayed on the display unit 100. In the example shown in Fig. 5, some of the multiple pieces of chart position information 113 are misaligned with respect to the corresponding patch images 112. As a result, variations occur in the extracted pixel values ​​for some of the chart position information 113.

[0035] For example, suppose that it is determined that the chart position information 113-A has pixel value variations, and that the chart position information 113-B has pixel value variations. In this case, the control unit 101 marks the chart position information 113-A with a mark such as an "x" in a rectangle. No mark is added to the chart position information 113-B. This allows the presence or absence of pixel variation to be visually recognized.

[0036] However, the method of distinction is not limited to this. For example, at least one of the color, mark, or frame shape may be different between an extraction range (chart position information 113) determined to have no pixel variation and an extraction range determined to have pixel variation. Alternatively, the rectangle (frame shape) may not be displayed for an extraction range determined to have pixel variation.

[0037] Next, a method for generating coordinate information 116 (FIG. 1) on a captured image will be described.

[0038] 6(a) and (b) are diagrams showing one rectangular range 150. Each rectangular range 150 corresponds to a drawing range of each piece of chart position information 113. The center coordinates (CX i, CY j ) are the center coordinates of the rectangular area 150. The coordinate information 116 on the captured image is the center coordinates (CX i, CY j ) and width CX w , C.Y. w It is defined as:

[0039] The resolution of the captured image 110 does not necessarily match the display resolution of the display unit 100. Therefore, it is necessary to convert the coordinate system on the display unit 100 into the coordinate system on the captured image 110. in , vertical resolution is V in , the horizontal resolution of the captured image 110 is H out , vertical resolution is V out In this case, the coordinate system on the display unit 100 can be converted into the coordinate system on the captured image 110 using Equation 4.

[0040]

number

[0041] The pixel value extraction unit 102 (FIG. 1) acquires (extracts) pixel values ​​based on the input captured image 110 and coordinate information 116 on the captured image input from the control unit 101. Here, the pixel value extraction unit 102 acquires pixel values ​​for each extraction range corresponding to a patch image 112 of a chart image 111 included in the captured image 110.

[0042] In the example of Fig. 6(a), the pixel value extraction unit 102 extracts pixel values ​​for each RGB color in a rectangular area 150, which is the extraction range, and performs this process for all rectangular areas 150. The pixel value extraction unit 102 then outputs the pixel values ​​acquired for each extraction range and for each RGB color to the pixel value determination unit 103 as a pixel value group 117 (Fig. 1).

[0043] The pixel value determination unit 103 determines whether or not there is pixel variation based on the pixel value group 117 output from the pixel value extraction unit 102, and outputs the determination result to the control unit 101 as pixel value determination information 115. A method for generating the pixel value determination information 115 will be described.

[0044] Each pixel value of R, G, B in the extracted range is R (n,m) , G (n,m) , B (n,m) Here, n and m are coordinates within the extraction range. The pixel value determination unit 103 determines the central pixel value R c、 G c、 B c and threshold T, it is determined whether or not Equation 5, which includes the following three conditions, is satisfied for the coordinates within the extraction range. Threshold T<|R c -R (n,m) | Threshold T<|G c -G (n,m) |...Formula 5 Threshold T<|B c -B (n,m) |

[0045] If at least one of the conditions in Equation 5 is met, it is determined that pixel variation exists. If none of the conditions are met, it is determined that pixel variation does not exist. In other words, if the difference between the pixel value at the center of the extraction range and the pixel value outside the center for any of R, G, or B exceeds the threshold T, it is determined that pixel variation exists in the extraction range.

[0046] The pixel value determination information 115 is P which indicates whether or not there is pixel variation for each coordinate 130. (i,j) When the pixel value determination unit 103 determines that there is pixel variation, it sets P (i,j) = 1, and if it is determined that there is no pixel variation, P (i,j) Set =0.

[0047] The threshold value T may be set for each of R, G, and B, such as threshold values ​​Tr, Tg, and Tb. In this case, the presence or absence of pixel variation is determined for each of R, G, and B using the threshold values ​​Tr, Tg, and Tb, and the P in the pixel value determination information 115 is (i,j) , Pr (i,j) , Pg (i,j) , Pb (i,j) Alternatively, the chart position information 113 may be provided for each color. If the pixel value determination information 115 is distinguished by RGB, it is possible to determine which color's pixel values ​​have had variation. Furthermore, the drawing mode of the chart position information 113 may be changed depending on the color in which pixel variation has occurred. For example, if there is variation only in the RB pixel values, the chart position information 113 may be drawn in magenta, and if there is variation only in the GB pixel values, the chart position information 113 may be drawn in cyan.

[0048] Alternatively, the presence or absence of pixel variation in the extraction range may be determined for each of the multiple regions obtained by dividing the extraction range. For example, as shown in FIG. 6(b), a rectangular range 150 may be divided into four quadrants passing through the center coordinate, and the presence or absence of pixel variation may be determined for each of the four quadrants 151, 152, 153, and 154. This makes it possible to determine which quadrant has pixel variation. Therefore, the user can reduce pixel variation by shifting the extraction range in the direction opposite to the quadrant where pixel variation is occurring, or by narrowing the extraction range. Note that the divided regions are not limited to quadrants.

[0049] Alternatively, the average value of the pixel values ​​in the extraction range may be calculated and linked to the determination result. For example, the pixel value determination unit 103 averages the pixel value group 117 input from the pixel value extraction unit 102 and outputs the average pixel value. This average pixel value can be used as input data for performing color matching between imaging devices. This average pixel value may then be linked to pixel value determination information 115 and output. This is because, when performing color matching between imaging devices, it is possible to determine whether or not to use the average pixel value depending on whether or not there is pixel variation in the average pixel value data.

[0050] Next, the processing from the specification of the extraction range to the drawing of the extraction range, which has been described above, will be described with reference to FIG.

[0051] 7 is a flowchart showing the extraction range drawing process. This process is realized by the CPU of the image processing device expanding a program stored in the ROM into the RAM and executing it. This process is started, for example, when a user instructs the start of the extraction range drawing process.

[0052] In this explanation, it is assumed that the patch image 112 in the chart image 111 included in the photographed image 110 has uniform pixel values ​​(R=255, G=255, B=255). It is also assumed that the portion of the frame image 114 excluding the patch image 112 in the chart image 111 also has uniform pixel values ​​(R=128, G=128, B=128). The threshold value T used for determining pixel values ​​is assumed to be 30.

[0053] In step S100, the control unit 101 starts specifying an extraction range on the captured image 110 displayed on the display unit 100 in response to a user operation. First, the control unit 101 specifies, as a start point, a point that is first specified by the user's mouse operation. For example, when the user places the cursor on patch coordinate 120 in the upper left of the chart image 111 and presses the mouse 119, the control unit 101 specifies the patch coordinate 120 as a start point (first point) (for example, specified coordinate Q1 in FIG. 3).

[0054] In step S101, the control unit 101 specifies a point designated by a mouse drag operation by the user as the end point. Here, the user drags the mouse 119 toward patch coordinate 123 at the bottom right diagonally opposite the start point. The pressed position of the mouse 119 may change from moment to moment. The control unit 101 specifies the current pressed position of the mouse 119 as the end point (second point) (for example, designated coordinate Q4 in FIG. 3). This tentatively determines the rectangle 124.

[0055] In step S102, the control unit 101 calculates the coordinate value (X ) of the coordinate 130 (or the coordinate 126) which is the center coordinate of each piece of chart position information 113 defined by the rectangle 124 using the formulas 1 to 3. i, Y j The end point remains tentatively determined until the mouse 119 is released, and the control unit 101 continues to calculate the necessary coordinate values ​​while the mouse 119 is being moved.

[0056] In this way, in steps S100 to S102, the control unit 101 as a designation unit designates a plurality of extraction ranges to correspond to a plurality of patches on the captured image 110 displayed on the display unit 100 based on a user operation.

[0057] In step S103, pixel value extraction unit 102, which serves as second acquisition means, extracts (acquires) pixel values ​​for each extraction range using Equation 4, and outputs these pixel values ​​to pixel value determination unit 103 as pixel value group 117. In step S104, pixel value determination unit 103, which serves as determination means, performs pixel value determination using pixel value group 117 output from pixel value extraction unit 102. That is, pixel value determination unit 103 determines whether or not each extraction range satisfies a predetermined condition.

[0058] Consider the example of Figure 5. The chart image 111 is tilted, but the rectangle 124 is not tilted. When the pressed position of the mouse 119 is near the patch coordinate 123 at the bottom right, the area of ​​the chart position information 113-A at the top right straddles the corresponding patch image 112 and frame image 114. In this case, from Equation 5, the central pixel value R of the chart position information 113-A, which is the extraction range, is c、 G c、 B c The difference between the threshold value T and the chart position information 113-A is |255-128|=128, which exceeds the threshold value T=32. In such a case, the pixel value determination unit 103 determines that there is pixel variation, and sets the pixel value determination information 115 relating to the chart position information 113-A to P (6,1) = 1 and output to the control unit 101.

[0059] On the other hand, the area of ​​the chart position information 113-B is contained within the corresponding patch image 112. In this case, from Equation 5, the central pixel value R of the chart position information 113-B, which is the extraction range, c、 G c、 B c The difference between the threshold value T and the chart position information 113-B is |255-255|=0, which does not exceed the threshold value T=32. Therefore, the pixel value determination unit 103 determines that there is no pixel variation, and sets the pixel value determination information 115 relating to the chart position information 113-B as P (6,4) = 0 and output to the control unit 101.

[0060] The pixel value determination unit 103 performs similar determinations for the other extraction ranges, generates pixel value determination information 115 , and outputs it to the control unit 101 .

[0061] In step S105, pixel value determination unit 103 determines whether pixel value determination information 115 for all patch images has been output to control unit 101. If pixel value determination unit 103 has not output pixel value determination information 115 for all patch images, it returns to step S103, and if pixel value determination unit 103 has output pixel value determination information 115 for all patch images, it proceeds to step S106.

[0062] In step S106, the control unit 101 as a control means displays the extraction range (chart position information 113) on the display unit 100 based on the pixel value determination information 115 output from the pixel value determination unit 103, that is, in a mode based on the determination result. Note that if the chart position information 113 is already displayed, the display of the chart position information 113 is updated. The chart position information 113 displayed here corresponds to the extraction range in a provisionally determined state (in the process of being specified).

[0063] For example, the control unit 101 (i,j) Pixel value determination information 115 of P (i,j) = 0 and output to the display unit 100. (i,j) The pixel value determination information 115 of P = 1 is information for marking a rectangle with a mark such as "x", (i,j) The pixel value determination information 115 of =0 is information that does not put a mark such as "x" in the rectangle. (i,j) The pixel value determination information 115 for P = 1 is a red rectangle, (i,j) The pixel value determination information 115 of 0 is a green rectangle. Based on the pixel value determination information 115, the control unit 101 draws a plurality of chart position information 113 on the captured image 110 in an overlapping manner.

[0064] In step S107, the control unit 101 determines whether the mouse 119 has been released. If the mouse 119 has not been released, the end point has not been determined, so the control unit 101 returns to step S101. If the mouse 119 has been released, the end point has been determined, so the control unit 101 proceeds to step S107. At this point, the initial rectangle 124 is determined, and the specification of the initial extraction range is completed.

[0065] If the chart image 111 included in the captured image 110 is not tilted, the specification of the extraction range can usually be completed with one operation. However, as shown in the examples of Figures 3 and 5, the chart image 111 may be tilted, or the rectangle defined by the outline of the chart image 111 may be misaligned with the rectangle 124.

[0066] Therefore, in step S108 and thereafter, control unit 101 fine-tunes the vertices of rectangle 124 based on user operations. Note that adjusting the vertices may cause rectangle 124 to change into a quadrangle whose interior angles are not right angles, but for convenience, the term "rectangle" will still be used hereinafter. The processing up to step S107 corresponds to the initial specification of an extraction range, and the processing from step S108 onwards corresponds to the second or subsequent specification of an extraction range.

[0067] In step S108, the control unit 101 determines whether or not the user has started an operation to change the vertex positions of the rectangle 124. Here, the vertices that can be changed include the start point and the end point. If the mouse 119 is pressed at a position within a predetermined range from any vertex, it is determined that an operation to change the vertex position has started.

[0068] 7 when it is determined that an operation to change the vertex position has not been started, and proceeds to step S109 when it is determined that an operation to change the vertex position has been started. Note that the control unit 101 may finalize the determination result in step S108 when a predetermined time has elapsed since the start of the determination. In other words, the process shown in FIG. 7 may be ended when a predetermined time has elapsed since the completion of the first or second extraction range specification.

[0069] In step S109, the control unit 101 acquires the coordinates of the changed vertices based on a user operation. For example, as described above, the user may use a mouse operation to move the designated coordinate Q2 so that it matches the upper right patch coordinate 121, or move the designated coordinate Q3 so that it matches the lower left patch coordinate 122. Alternatively, the user may move the position of the designated coordinate Q1 or the designated coordinate Q4. To move the designated coordinates, the user may drag the mouse 119 from the selected vertex toward the target point.

[0070] In steps S110 to S114, the control unit 101 executes the same processes as in steps S102 to S106. Since the chart position information 113 has already been displayed, the display in step S114 corresponds to updating the display of the chart position information 113.

[0071] In step S115, the control unit 101 determines whether the mouse 119 has been released from the vertex to be changed. If the mouse 119 has not been released, the position of the vertex after the change has not been confirmed, and the control unit 101 returns to step S108. If the mouse 119 has been released, the position of the vertex after the change has been confirmed, and the control unit 101 ends the processing shown in FIG. 7. Note that, similar to step S108, the control unit 101 may confirm the determination result in step S115 when a predetermined time has elapsed since the start of the determination. In other words, the processing shown in FIG. 7 may end when a predetermined time has elapsed since the completion of the correction and specification of the extraction range. When the processing shown in FIG. 7 ends, the extraction range is finally confirmed.

[0072] According to this embodiment, multiple extraction ranges (chart position information 113) are specified on the captured image 110 displayed on the display unit 100 based on user operation, and the presence or absence of pixel variation in the extraction range is determined based on the pixel values ​​of the extraction range. Then, the control unit 101 causes the display unit 100 to display the extraction range in a manner based on the determination result. For example, the control unit 101 causes an extraction range without pixel variation to have a different display manner from an extraction range with pixel variation. This makes it possible to visually recognize the presence or absence of pixel variation in the extraction range. This allows the user to easily recognize whether the designation of an extraction range corresponding to a specific area (corresponding to a patch) is appropriate.

[0073] In other words, when extracting chart pixel values ​​to match the characteristics of an imaging device, by displaying the image in a way that makes it visually clear whether or not there is pixel variation in the extraction range during the operation of specifying the extraction range, it is possible to reduce the need to go back and forth, such as reselecting the chart or re-acquiring the image.

[0074] Furthermore, if the extraction range is changed after pixel values ​​of the extraction range have been acquired, pixel values ​​of the changed extraction range are acquired again, and the presence or absence of pixel variation in the changed extraction range is determined (S108 to S112). For example, as illustrated in Fig. 4, the chart position information 113 of the original image matches the position of the corresponding patch image 112. This allows the extraction range to be appropriately corrected.

[0075] Furthermore, after the initial extraction range specification has begun but before the specification is completed, the display of the extraction range begins (S106), so the user can recognize whether the extraction range specification is appropriate even while the extraction range specification is in progress.

[0076] Furthermore, by determining whether or not there is pixel variation in the extraction range for each of the R, G, and B colors, or for each of the multiple regions into which the extraction range is divided, it is possible to determine in more detail whether or not the extraction range has been specified appropriately.

[0077] Furthermore, by linking the average pixel values ​​in the extraction range to the determination result, it is possible to provide information for determining whether or not to use the average pixel value when performing color matching between imaging devices.

[0078] (Second embodiment) In the first embodiment, the display of the extraction range begins after the initial specification of the extraction range begins but before the specification is completed. However, as the end point changes due to mouse dragging, the display mode of the extraction range changes successively, which can cause the screen to flicker.

[0079] Therefore, in the second embodiment of the present invention, the control unit 101 does not perform drawing based on the pixel value judgment information 115 until the end point is tentatively determined, because there is a high possibility of pixel variation occurring, and instead performs drawing based on the pixel value judgment information 115 after the end point is tentatively determined.

[0080] 8 is a flowchart showing the extraction range drawing process. The execution body and starting conditions of this process are the same as those of the extraction range drawing process shown in FIG.

[0081] The processing of steps S100 to S105 is the same as that described in FIG. 7. In step S200, the control unit 101 causes the display unit 100 to display the extraction range (chart position information 113) in a manner that is not based on the determination result. For example, the control unit 101 draws the extraction range in the same manner as when there is no pixel variation in all extraction ranges. Therefore, all chart position information 113 is displayed in the same color and shape. This prevents visual flickering.

[0082] In step S201, the control unit 101 executes the same process as step S107 in Fig. 7. If the mouse 119 has not been released, the end point has not been determined, so the control unit 101 returns to step S101. If the mouse 119 has been released, the end point has been determined, so the control unit 101 proceeds to step S202. At this point, the initial rectangle 124 is determined, and the specification of the initial extraction range is completed.

[0083] In step S202, similarly to step S106, the control unit 101 causes the display unit 100 to display the extraction range (chart position information 113) based on the pixel value determination information 115 output from the pixel value determination unit 103, that is, in a mode based on the determination result. Although some illustrations are omitted, in step S203 and thereafter, the control unit 101 executes the same processes as steps S108 to S115.

[0084] According to this embodiment, the same effect as in the first embodiment can be achieved in that the user can easily recognize whether the designation of the extraction range corresponding to a specific area is appropriate.

[0085] Furthermore, the control unit 101 displays the extraction range in a manner not based on the determination result before the initial specification of the extraction range is completed, and displays the extraction range in a manner based on the determination result after the initial specification of the extraction range is completed, thereby suppressing visual flickering during the initial specification of the extraction range.

[0086] Furthermore, when the extraction range is changed by the second or subsequent specification, the control unit 101 displays the changed extraction range after the start of the second or subsequent specification and before the specification is completed, thereby allowing the user to recognize whether the specification of the extraction range is appropriate.

[0087] In each of the above embodiments, the shape of the rectangle 124 defined by the initial tentative determination of the end points may be a quadrangle of a predetermined shape whose interior angles are not right angles.

[0088] In each of the above embodiments, the multiple extraction ranges may be specified so as to include an arrangement corresponding to the arrangement of the multiple patches. For example, when the frame image 114 is wide as shown in Fig. 9(a), the number of extraction ranges may be greater than the number of patches as shown in Fig. 9(b).

[0089] That is, pixel values ​​may be extracted including a portion of the frame image 114. In addition to the patch images 112, the control unit 101 provides a virtual patch image 160 in the region of the frame image 114 surrounding the patch images 112. The control unit 101 may specify an extraction range so that the number of extractions is 8 horizontally and 6 vertically, including the virtual patch image 160, instead of 6 horizontally and 4 vertically, which corresponds to the patch image 112. By determining pixel variation by pixel value extraction also in the portion of the virtual patch image 160, it is possible to obtain an average pixel value including the frame data. Note that if a uniform region also extends around the periphery of the chart image 111, an extraction range may be specified using a similar method, making it possible to extract pixel values.

[0090] (Other Examples) In the above embodiment, an example using a chart including patches has been described, but the present invention is not necessarily limited to this. For example, the present invention can be applied to a specific subject that includes a certain predetermined area, such as a structure with a specific pattern.

[0091] In this embodiment, pixel variation is used. That is, whether or not a predetermined condition is satisfied is determined by determining whether or not pixel variation exists. However, this method is not necessarily limited to this. For example, the display of the extracted range may be controlled by determining the S / N ratio of the extracted range or determining whether or not an edge exists. In this case, too, a predetermined condition based on the noise ratio or edge strength may be set, and if, for example, the noise ratio or edge strength is greater than a predetermined value, the same control as when pixel variation is determined to exist may be performed.

[0092] The present invention can also be realized by supplying a program that realizes one or more of the functions of the above-described embodiments to a system or device via a network or a non-transitory storage medium, and having one or more processors in the computer of the system or device read and execute the program. The above program and the storage medium storing the program constitute the present invention. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more of the functions.

[0093] While the present invention has been described in detail above based on preferred embodiments thereof, the present invention is not limited to these specific embodiments, and various forms within the scope of the gist of the present invention are also included in the present invention. Parts of the above-described embodiments may be combined as appropriate.

[0094] The disclosure of this embodiment includes the following configurations (and methods and programs). (Configuration 1) a first acquisition means for acquiring a captured image of a chart including a plurality of patches; a control means for displaying the photographed image acquired by the first acquisition means on a display unit; a designation means for designating a plurality of extraction ranges corresponding to the plurality of patches based on a user operation; a second acquiring means for acquiring pixel values ​​of each extraction range designated by the designating means in the captured image; a determination means for determining whether or not a predetermined condition is satisfied for each of the extraction ranges from the pixel values ​​acquired by the second acquisition means, The image processing device is characterized in that the control means causes the display unit to display the plurality of extraction ranges in a display mode based on the determination result by the determination means. (Configuration 2) The image processing device according to configuration 1, wherein the control means, when causing the display unit to display the plurality of extraction ranges, differentiates the display manner between extraction ranges determined to satisfy the predetermined condition and extraction ranges determined not to satisfy the predetermined condition. (Configuration 3) 3. The image processing device according to configuration 2, wherein the control means changes the display mode by changing at least one of a color, a mark, and a frame shape. (Configuration 4) 3. The image processing device according to claim 1, wherein the determining means determines whether the predetermined condition is satisfied by determining whether there is pixel variation. (Configuration 5) The image processing device according to configuration 4, wherein the determination means determines that there is pixel variation in the extraction range when a difference between a central pixel value in each of the acquired extraction ranges and pixel values ​​other than the central pixel value exceeds a threshold value, and determines that there is no pixel variation in the extraction range when the difference does not exceed the threshold value. (Configuration 6) 4. The image processing device according to any one of configurations 1 to 3, wherein the second acquisition means acquires pixel values ​​of the respective extraction ranges, and then, when the plurality of extraction ranges are changed by designation by the designation means, re-acquires pixel values ​​of the respective extraction ranges after the change. (Configuration 7) 7. The image processing device according to any one of configurations 1 to 3 or 6, wherein the control means displays the plurality of extraction ranges after starting to specify the plurality of extraction ranges for the first time and before completing the specification. (Configuration 8) The image processing device according to any one of configurations 1 to 3, 6, or 7, wherein the control means displays the plurality of extraction ranges in a manner not based on the determination result before the initial specification of the plurality of extraction ranges is completed, and displays the plurality of extraction ranges in a manner based on the determination result after the initial specification of the plurality of extraction ranges is completed. (Configuration 9) 9. The image processing device according to configuration 8, wherein, when the plurality of extraction ranges are changed by the second or subsequent designation by the designation means, the plurality of extraction ranges after the change are displayed after the start of the second or subsequent designation and before the designation is completed. (Configuration 10) 4. The image processing device according to any one of configurations 1 to 3, wherein the initial designation of the plurality of extraction ranges by the designation means is designation of a quadrangle having vertices that are a first point on the displayed captured image and a second point diagonally opposite to the first point. (Configuration 11) The image processing device according to configuration 10, wherein the second and subsequent designations by the designation means include designation to correct the position of at least one of the four vertices of the quadrangle. (Configuration 12) 4. The image processing device according to any one of configurations 1 to 3, wherein the control means causes the display unit to display the plurality of extraction ranges superimposed on the captured image. (Configuration 13) 13. The image processing device according to any one of configurations 1 to 3 or 12, wherein the plurality of extraction ranges are specified so as to include an arrangement corresponding to an arrangement of the plurality of patches. (Configuration 14) 6. The image processing device according to configuration 4 or 5, wherein the determining means determines whether or not there is pixel variation in the extraction range for each of R, G, and B colors. (Configuration 15) 6. The image processing device according to configuration 4 or 5, wherein the determining means determines whether or not there is pixel variation in the extraction range for each of a plurality of regions obtained by dividing the extraction range into a plurality of regions. (Configuration 16) The image processing device according to any one of configurations 1 to 15, wherein the control means calculates an average value of pixel values ​​of the plurality of extraction ranges acquired by the second acquisition means, and associates the average value with the determination result by the determination means. (Method 1) a first acquisition step of acquiring a captured image of a chart including a plurality of patches; a control step of displaying the photographed image acquired in the first acquisition step on a display unit; a designation step of designating a plurality of extraction ranges corresponding to the plurality of patches based on a user operation; a second acquisition step of acquiring pixel values ​​of each extraction range designated by the designation step in the captured image; a determination step of determining whether or not a predetermined condition is satisfied for each of the extraction ranges from the pixel values ​​acquired in the second acquisition step, The image processing method is characterized in that, in the control step, the plurality of extraction ranges are displayed on the display unit in a display mode based on the determination result in the determination step. (Program 1) A program for causing a computer to function as each of the means of the image processing device according to any one of configurations 1 to 16. [Explanation of symbols]

[0095] 100 Display 101 Control section 110 captured images 102 Pixel value extraction unit 103 Pixel value determination unit 113 Chart Position Information

Claims

1. A program for controlling an image processing device, The computer of the image processing device a first acquisition step of acquiring a captured image of a subject including a predetermined area; a control step of displaying the photographed image acquired in the first acquisition step on a display unit; a designation step of designating a plurality of extraction ranges for the subject based on a user operation; a second acquisition step of acquiring pixel values ​​of each extraction range designated by the designation step in the captured image; a determination step of determining whether a predetermined condition is satisfied for each of the extraction ranges from the pixel values ​​acquired in the second acquisition step, The control step causes the display unit to display the plurality of extraction ranges in a display mode based on the determination result obtained in the determination step.

2. 2. The program according to claim 1, wherein the control step, when displaying the plurality of extraction ranges on the display unit, makes the display manner different between extraction ranges determined to satisfy the predetermined condition and extraction ranges determined not to satisfy the predetermined condition.

3. 3. The program according to claim 2, wherein the control step varies the display mode by varying at least one of a color, a mark, and a frame shape.

4. 4. The program according to claim 1, wherein the determining step determines whether the predetermined condition is satisfied by determining whether there is pixel variation.

5. 5. The program according to claim 4, wherein the determination step determines that there is pixel variation in the extraction range if a difference between a central pixel value in each of the acquired extraction ranges and pixel values ​​other than the central pixel value exceeds a threshold value, and determines that there is no pixel variation in the extraction range if the difference does not exceed the threshold value.

6. 4. The program according to claim 1, wherein the second acquisition step acquires pixel values ​​of each of the extraction ranges, and then, when the plurality of extraction ranges are changed as specified in the designation step, re-acquires pixel values ​​of each of the extraction ranges after the change.

7. 4. The program according to claim 1, wherein the control step displays the plurality of extraction ranges after the start of designating the plurality of extraction ranges for the first time and before the designation is completed.

8. 4. The program according to claim 1, wherein the control step displays the plurality of extraction ranges in a manner not based on the determination result before the initial specification of the plurality of extraction ranges is completed, and displays the plurality of extraction ranges in a manner based on the determination result after the initial specification of the plurality of extraction ranges is completed.

9. 9. The program according to claim 8, wherein, when the plurality of extraction ranges are changed by a second or subsequent designation in the designation step, the plurality of extraction ranges after the change are displayed after the start of the second or subsequent designation and before completion of the designation.

10. 4. The program according to claim 1, wherein the initial designation of the plurality of extraction ranges in the designation step is designation of a rectangle having vertices that are a first point on the displayed captured image and a second point diagonally opposite the first point.

11. 11. The program according to claim 10, wherein the second and subsequent designations in the designation step include designation to modify the position of at least one of the four vertices of the quadrangle.

12. 4. The program according to claim 1, wherein the control step displays the plurality of extraction ranges on the display unit so as to be superimposed on the captured image.

13. the subject including the predetermined area is a chart including a plurality of patches, 4. The program according to claim 1, wherein the plurality of extraction ranges are specified so as to include an arrangement corresponding to an arrangement of the plurality of patches.

14. 5. The program according to claim 4, wherein the determining step determines whether or not there is pixel variation in the extraction range for each of R, G, and B colors.

15. 5. The program according to claim 4, wherein the determining step determines whether or not there is pixel variation in the extraction range for each of a plurality of regions obtained by dividing the extraction range.

16. The program according to any one of claims 1 to 3, characterized in that the control step calculates an average value of pixel values ​​of the plurality of extraction ranges acquired by the second acquisition step, and links the average value to the judgment result by the judgment step.

17. 4. The program according to claim 1, wherein the specifying step specifies a range to be used for color matching between a plurality of image capturing devices.

18. a first acquisition step of acquiring a captured image of a subject including a predetermined area; a control step of displaying the photographed image acquired in the first acquisition step on a display unit; a designation step of designating a plurality of extraction ranges for the subject based on a user operation; a second acquisition step of acquiring pixel values ​​of each extraction range designated by the designation step in the captured image; a determination step of determining whether a predetermined condition is satisfied for each of the extraction ranges from the pixel values ​​acquired in the second acquisition step, The image processing method is characterized in that, in the control step, the plurality of extraction ranges are displayed on the display unit in a display mode based on the determination result in the determination step.

19. a first acquisition means for acquiring a captured image of a subject including a predetermined area; a control means for displaying the photographed image acquired by the first acquisition means on a display unit; a designation means for designating a plurality of extraction ranges for the subject based on a user operation; a second acquiring means for acquiring pixel values ​​of each extraction range designated by the designating means in the captured image; a determining means for determining whether a predetermined condition is satisfied for each of the extraction ranges from the pixel values ​​acquired by the second acquiring means, The image processing device is characterized in that the control means causes the display unit to display the plurality of extraction ranges in a display mode based on the determination result by the determination means.

Images