Image processing apparatus and method, program

Abstract

To easily recognize the propriety of designation of an extraction range corresponding to a specific area.SOLUTION: A photographed image 110 obtained by imaging a chart including a plurality of patches is acquired, a plurality of extraction ranges (chart position information 113) are designated on the photographed image 110 displayed on a display unit 100 based on a user operation, and presence or absence of pixel variation in the extraction ranges is determined from pixel values of the extraction ranges. The control unit 101 causes the display unit 100 to display the extraction range in a display mode based on the determination result.SELECTED DRAWING: Figure 7

Description

Technical Field

[0001] The present invention relates to an image processing apparatus and method, and a program. Background Art

[0002] In video production sites, there are cases where shooting is performed using a plurality of imaging devices. Even when shooting the same subject, the color and luminance characteristics obtained from the imaging devices differ due to individual differences between the imaging devices and lenses used, differences in environmental illumination, and other factors. Color matching techniques using a chart exist to match the characteristics of captured images output from imaging devices.

[0003] Patent Document 1 discloses an image processing apparatus that extracts patch information from a captured chart image obtained by capturing a chart. In Patent Document 1, a transformation matrix is calculated from the coordinates of the four vertices of the chart and the coordinates of the four vertices of the chart in the captured image, and the center coordinates of all patches in the captured image are obtained from the center coordinates of each patch of the chart and the transformation matrix. Then, when all patches can be extracted without overlapping, the pixel values of the extraction range are acquired. Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Patent Laid-Open No. 2015-226219 Summary of the Invention Problem to be Solved by the Invention

[0005] However, pixel variation may occur in the extraction range due to positional displacement between the extraction range and the patch, compression noise, random noise, and the like. When performing color matching, if based on data including pixel variation, the accuracy of color matching decreases, and re-selection of the chart, re-acquisition of the image, and the like become necessary, resulting in a problem of reduced workability.

[0006] The present invention aims to facilitate the recognition of the appropriateness of specifying a range corresponding to a particular region within an image. [Means for solving the problem]

[0007] To achieve the above objective, the present invention provides a program for controlling an image processing apparatus, characterized in that the computer of the image processing apparatus is caused to execute: a first acquisition step of acquiring a captured image of a subject including a predetermined area; a control step of causing the captured image acquired in the first acquisition step to be displayed on a display unit; a designation step of specifying a plurality of extraction ranges for the subject based on user operation; a second acquisition step of acquiring the pixel values ​​of each extraction range specified in the designation step in the captured image; and a determination step of determining whether each extraction range satisfies predetermined conditions based on the pixel values ​​acquired in the second acquisition step, wherein the control step causes the plurality of extraction ranges to be displayed on the display unit in a display mode based on the determination result of the determination step. [Effects of the Invention]

[0008] According to the present invention, it is possible to easily recognize whether or not the specified extraction range corresponding to a particular region is appropriate. [Brief explanation of the drawing]

[0009] [Figure 1] This is a block diagram of the main components of the image processing device. [Figure 2] This diagram shows the image displayed on the display unit. [Figure 3] This figure shows an example of the display in the display section while specifying the extraction range. [Figure 4] This figure shows an example of the display in the display section after the extraction range has been specified. [Figure 5] This figure shows an example of how to display chart position information. [Figure 6] This is a diagram showing a single rectangular area. [Figure 7]This flowchart shows the process of plotting the extracted range. [Figure 8] This flowchart shows the process of plotting the extracted range. [Figure 9] This figure shows a modified version of the chart image. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings.

[0011] (First Embodiment) Figure 1 is a block diagram of the main parts of an image processing apparatus according to the first embodiment of the present invention. This image processing apparatus includes a display unit 100, a control unit 101, a pixel value extraction unit 102, and a pixel value determination unit 103. The image processing apparatus includes a CPU, ROM, RAM, etc., which are not shown. The ROM stores a control program executed by the CPU. The RAM provides a work area for the CPU to execute the control program. The functions of the control unit 101, the pixel value extraction unit 102, and the pixel value determination unit 103 are realized through the cooperation of the CPU, ROM, RAM, etc.

[0012] Figure 2 shows an image displayed on the display unit 100. The display unit 100 displays the captured image 110 and multiple chart position information 113. The captured image 110 is image data acquired by photographing a printed document (not shown) on which the original chart, which includes multiple measurement images called patches, is printed. The captured image 110 is acquired by the control unit 101, the pixel value extraction unit 102, and the pixel value determination unit 103, which serve as the first acquisition means.

[0013] The captured image 110 includes a chart image 111 corresponding to the chart described above. The chart image 111 includes a patch image 112 and a frame image 114. The frame image 114 is an image of the area of ​​the chart image 111 excluding the patch image 112. The patch image 112 is a chart element corresponding to a patch included in the chart.

[0014] The display unit 100 displays the input captured image 110 and the chart position information 113 output from the control unit 101. The chart position information 113 is displayed superimposed on the chart image 111.

[0015] Chart position information 113 is the "extraction range" that is targeted when extracting pixel values ​​for each patch image 112. The pixel values ​​within the extraction range become image information for determining whether or not there is pixel variation. The specific determination method will be described later. Determining whether or not there is pixel variation is one example of determining whether or not a predetermined condition is met.

[0016] The control unit 101 generates chart position information 113 based on the number of extractions (h,v), the extraction size (w), the user operation specifying the extraction range, and the pixel value determination information 115 output from the pixel value determination unit 103, and outputs it to the display unit 100. The control unit 101 also generates coordinate information 116 on the captured image 110 from the coordinates on the display unit 100 (explained in equation 4 below) and outputs it to the pixel value extraction unit 102.

[0017] First, the method for generating chart position information 113 will be explained. The extraction count (h,v) indicates the number of patch images 112 in the horizontal direction (h) and the number of patch images 112 in the vertical direction (v) within the chart image 111. In the example shown in Figure 2, the horizontal direction is 6 and the vertical direction is 4. The extraction size (w) is a numerical value indicating the width of the chart position information 113. Note that the extraction count and extraction size may be entered by the user in advance, or they may be stored in ROM as parameters for each chart beforehand.

[0018] Figure 3 shows an example of the display on the display unit 100 during the process of specifying the extraction range. In Figure 3, the center coordinates of the four corner patch images 112 are displayed. These center coordinates are the top-left patch coordinate 120, the top-right patch coordinate 121, the bottom-left patch coordinate 122, and the bottom-right patch coordinate 123.

[0019] The initial specification of the extraction range is done by determining the four vertices of rectangle 124. The four vertices of rectangle 124 are the specified coordinates Q1 (top left), Q2 (top right), Q3 (bottom left), and Q4 (bottom right).

[0020] The user can specify these designated coordinates using the mouse 119, etc. The method of specification is not limited, but as an example, the user first specifies one of the patch coordinates 120, 121, 122, or 123 as the starting point. At this point, the patch coordinates 120, 121, 122, and 123 do not necessarily coincide with the specified coordinates Q1, Q2, Q3, and Q4.

[0021] If the starting point is the top-left patch coordinate 120, the user will drag the mouse towards the bottom-right patch coordinate 123, which is diagonally opposite to patch coordinate 120. This will change the range of rectangle 124.

[0022] The rectangle 124 defines a matrix of chart position information 113 for the number of extractions (h,v) as the extraction range. During the specification process, the center coordinates of each of the chart position information 113 defined by rectangle 124 are coordinates 126. After the specification is complete, the center coordinates of each of the chart position information 113 defined by rectangle 124 are coordinates 130 (see Figure 4).

[0023] Coordinate values ​​(x) of coordinates 126 and 130 i, y j ) can be calculated using Equation 1, assuming the origin of the coordinate axes shown in Figure 3 is set as origin 127.

[0024]

number

[0025] As described above, in Equation 1, h is the number of samples taken horizontally, and v is the number of samples taken vertically. (x 1, y1) is the coordinate value of the top-left patch coordinate 120, (x h, y v ) indicates the coordinate value of the specified coordinate Q4 in the lower right.

[0026] Note that the starting point is not limited to the upper-left patch coordinate 120, and other vertices such as the upper-right patch coordinate 121 may be used as the starting point. In addition, it is not absolutely necessary to use any one of the four-corner patch coordinates as the starting point. For example, when the number of extractions is 4 in the horizontal direction and 2 in the vertical direction, the user may determine the patch coordinate serving as the starting point according to the number of extractions.

[0027] When the specified coordinate Q4 reaches the lower-right patch coordinate 123, the user releases the mouse pressing operation once. At this point, the initial specification of the extraction range is completed. Next, the user corrects the extraction range through the second and subsequent specifications. For example, the user finely adjusts the specified coordinate Q2 to match the upper-right patch coordinate 121 through mouse operation, and further finely adjusts the specified coordinate Q3 to match the lower-left patch coordinate 122. Thereby, as illustrated in FIG. 4, the shape and range of the rectangle 124 are changed.

[0028] FIG. 4 is a diagram showing a display example of the display unit 100 after completion of specifying at least one extraction range. As shown in FIG. 4, when the extraction range is ideally specified or corrected, the patch coordinates 120, 121, 122, 123 match the specified coordinates Q1, Q2, Q3, Q4. In addition, the coordinate 130 serving as the center coordinate of each piece of chart position information 113 matches the center coordinate of the patch image 112.

[0029] The control unit 101 generates the chart position information 113 based on the coordinate 130 and the extraction size. The coordinate value (X i, Y j ) can be obtained by Equations 2 and 3.

[0030]

Formula

[0031] The coordinate value (x 1, y1) in Equation 2 is the coordinate value of the upper-left specified coordinate Q1 on the rectangle 124, and the coordinate value (x h, y v) is the coordinate value of the specified coordinate Q4 in the lower right corner of rectangle 124. (X in equation 2) 1, Y1) is the coordinate value of the top-left coordinate 130, and in the example in Figure 4, it is equivalent to the coordinate value of the specified top-left coordinate Q1 on rectangle 124. (X in Equation 2) 4, Y4) is the coordinate value of the bottom right coordinate 130, and in the example of Figure 4, it is equivalent to the coordinate value of the specified bottom right coordinate Q4. The coordinate (X) in Equation 2 2, Y2) is the coordinate value of the upper right coordinate 130, and in the example in Figure 4, it is equivalent to the coordinate value of the upper right specified coordinate Q2. The coordinate (X) in Equation 2 3, Y3) is the coordinate value of the bottom left coordinate 130, and in the example in Figure 4, it is equivalent to the coordinate value of the specified bottom left coordinate Q3. Both coordinate values ​​of 130 are obtained when the user performs an operation.

[0032] The control unit 101 calculates the coordinate value (X) of coordinate 130. i, Y j Chart position information 113 can be generated from the (X) and the extraction size. For example, if the extraction size is w=10, the coordinate (X i -5 , Y j -5), (X i +5 , Y j The area whose outer shape is a rectangle passing through (+5) becomes the chart position information 113.

[0033] The control unit 101 changes the drawing method of the extraction range (chart position information 113) based on the pixel value determination information 115 output from the pixel value determination unit 103. The control unit 101 changes the display manner of the extraction range so that it can distinguish between cases where there is pixel variation in the chart position information 113 and cases where there is no pixel variation.

[0034] Figure 5 shows an example of the drawing of chart position information 113 in the display unit 100. In the example shown in Figure 5, some of the multiple chart position information 113 are misaligned with the corresponding patch image 112. As a result, some of the extracted pixel values ​​of the chart position information 113 are inconsistent.

[0035] For example, suppose chart position information 113-A is determined to have variations in pixel values, while chart position information 113-B is determined to have no variations in pixel values. In this case, the control unit 101 places a mark such as "×" within a rectangle on chart position information 113-A. No mark is placed on chart position information 113-B. This allows the presence or absence of pixel variations to be visually recognized.

[0036] However, the method of distinction is not limited to this. For example, at least one of the following may be used to differentiate between the extracted range determined to have no pixel variation (chart position information 113) and the extracted range determined to have pixel variation: color, mark, or frame shape. Alternatively, the rectangle (frame shape) may not be displayed for the extracted range determined to have pixel variation.

[0037] Next, we will explain how to generate coordinate information 116 (Figure 1) on the captured image.

[0038] Figures 6(a) and 6(b) show a single rectangular area 150. Each of the rectangular areas 150 corresponds to each drawing area of ​​the chart position information 113. Center coordinates (CX i, CY j ) is the center coordinate of the rectangular area 150. The coordinate information 116 on the captured image is the center coordinate (CX i, CY j ) and width CX w CY w It is defined as follows.

[0039] The resolution of the captured image 110 and the display resolution of the display unit 100 do not necessarily match. Therefore, a conversion from the coordinate system on the display unit 100 to the coordinate system on the captured image 110 is necessary. The horizontal resolution of the display unit 100 is H in , vertical resolution is V in The horizontal resolution of the captured image 110 is H out , vertical resolution is V out Let's assume this is the case. In this case, equation 4 makes it possible to convert from the coordinate system on the display unit 100 to the coordinate system on the captured image 110.

[0040]

number

[0041] The pixel value extraction unit 102 (Figure 1) acquires (extracts) pixel values ​​based on the input captured image 110 and the coordinate information 116 on the captured image input from the control unit 101. Here, the pixel value extraction unit 102 acquires pixel values ​​for each extraction range corresponding to the patch image 112 of the chart image 111 contained in the captured image 110.

[0042] In the example shown in Figure 6(a), the pixel value extraction unit 102 extracts pixel values ​​for each RGB color within the rectangular area 150, which is the extraction range, and performs this for all rectangular areas 150. The pixel value extraction unit 102 then outputs the pixel values ​​obtained for each extraction range and each RGB color as a pixel value group 117 (Figure 1) to the pixel value determination unit 103.

[0043] The pixel value determination unit 103 determines whether or not there is pixel variation based on the pixel value group 117 output from the pixel value extraction unit 102, and outputs the determination result as pixel value determination information 115 to the control unit 101. The method for generating the pixel value determination information 115 will be explained below.

[0044] The R, G, and B pixel values ​​of the extracted range (n,m) , G (n,m) B (n,m) Here, n and m are coordinates within the extraction range. The pixel value determination unit 103 determines the center pixel value R within the extraction range. c、 G c、 B c Using the threshold T, we determine whether equation 5, which includes the following three conditions, is true for the coordinates within the extraction range. Threshold T<|R c -R (n,m) | Threshold T<|G c -G (n,m) |...Formula 5 Threshold T<|B c -B (n,m) |

[0045] If at least one of the conditions in Equation 5 is met, it is determined that there is pixel variation. If none of the conditions are met, it is determined that there is no pixel variation. In other words, if the difference between the central pixel value and the non-central pixel value in the extraction range exceeds the threshold T for any of R, G, or B, it is determined that there is pixel variation in that extraction range.

[0046] The pixel value determination information 115 indicates the presence or absence of pixel variation at each coordinate 130. (i,j) This includes the information. If the pixel value determination unit 103 determines that there is pixel variation, it will set P in the pixel value determination information 115. (i,j) If set to =1 and it is determined that there is no pixel variation, then in the pixel value determination information 115, P (i,j) Set it to =0.

[0047] Note that threshold T may be set for each of R, G, and B, such as thresholds Tr, Tg, and Tb. In that case, the presence or absence of pixel variation for each of R, G, and B is determined by thresholds Tr, Tg, and Tb, and the P in the pixel value determination information 115 is determined. (i,j) Pr (i,j) Pg (i,j) Pb (i,j) This may be done separately for each color. If the pixel value determination information 115 is distinguished by RGB, it is possible to determine which color's pixel values ​​have been affected. Furthermore, the drawing method of the chart position information 113 may be changed depending on the color that has been affected by the pixel variation. For example, if only the RB pixel values ​​have been affected, the chart position information 113 may be drawn in magenta, and if only the GB pixel values ​​have been affected, the chart position information 113 may be drawn in cyan.

[0048] Alternatively, the presence or absence of pixel variation within the extraction range may be determined for each region into which the extraction range has been divided. For example, as shown in Figure 6(b), the rectangular range 150 may be divided into four quadrants passing through the center coordinates, and the presence or absence of pixel variation may be determined for each of the four quadrants 151, 152, 153, and 154. In this way, it is possible to determine which quadrant is experiencing pixel variation. Therefore, the user can reduce pixel variation by shifting the extraction range in the opposite direction from the quadrant where pixel variation is occurring, or by narrowing the extraction range. Note that the divided regions are not limited to quadrants.

[0049] Alternatively, the average value of the pixel values ​​within the extraction range may be calculated and this average value linked to the judgment result. For example, the pixel value judgment unit 103 averages the group of pixel values ​​117 input from the pixel value extraction unit 102 and outputs it as the pixel average value. This pixel average value can be used as input data for color matching between imaging devices. This pixel average value may also be output linked to the pixel value judgment information 115. This allows for a determination of whether or not to use the pixel average value data when performing color matching between imaging devices, based on whether or not there is pixel variation in the pixel average value data.

[0050] Next, we will explain the process from specifying the extraction range to drawing the extraction range, as described above, using Figure 7.

[0051] Figure 7 is a flowchart showing the extraction range drawing process. This process is achieved by the CPU loading a program stored in the ROM of the image processing device into RAM and executing it. This process starts, for example, when the user instructs the start of the extraction range drawing process.

[0052] In this explanation, we assume that the patch image 112 within the chart image 111 contained in the captured image 110 has uniform pixel values ​​(R=255, G=255, B=255). We also assume that the portion of the frame image 114 excluding the patch image 112 within the chart image 111 has uniform pixel values ​​(R=128, G=128, B=128). The threshold T used for pixel value determination is assumed to be 30.

[0053] In step S100, the control unit 101 starts specifying the extraction range on the captured image 110 displayed on the display unit 100 in response to user operation. First, the control unit 101 identifies the point initially specified by the user using the mouse as the starting point. For example, if the user places the cursor on the patch coordinate 120 in the upper left corner of the chart image 111 and presses the mouse 119, the control unit 101 identifies the patch coordinate 120 as the starting point (first point) (for example, the specified coordinate Q1 in Figure 3).

[0054] In step S101, the control unit 101 identifies the point specified by the user's mouse drag operation as the endpoint. Here, the user drags the mouse 119 toward the lower right patch coordinate 123, which is diagonally opposite to the starting point. The position where the mouse 119 is pressed may change from moment to moment. The control unit 101 identifies the current position where the mouse 119 is pressed as the endpoint (second point) (for example, the specified coordinate Q4 in Figure 3). This provisionally determines the rectangle 124.

[0055] In step S102, the control unit 101 uses equations 1 to 3 to determine the coordinate values ​​(X) of the coordinates 130 (or coordinates 126), which are the center coordinates of each of the chart position information 113 defined by the rectangle 124. i, Y j The control unit 101 calculates the coordinate values. Note that the endpoint is in a provisional state until the mouse 119 is released, and the control unit 101 continues to calculate the necessary coordinate values ​​while the mouse 119 is still being moved.

[0056] Thus, in steps S100 to S102, the control unit 101, acting as a designation means, specifies multiple extraction ranges on the captured image 110 displayed on the display unit 100 to correspond to multiple patches, based on user operation.

[0057] In step S103, the pixel value extraction unit 102, acting as a second acquisition means, extracts (acquires) pixel values ​​for each extraction range using Equation 4, and outputs these pixel values ​​as a pixel value group 117 to the pixel value determination unit 103. In step S104, the pixel value determination unit 103, acting as a determination means, performs pixel value determination using the pixel value group 117 output from the pixel value extraction unit 102. That is, the pixel value determination unit 103 determines whether or not predetermined conditions are met for each extraction range.

[0058] Consider the example in Figure 5. Chart image 111 is tilted, but rectangle 124 is not. When the mouse 119 is pressed near the patch coordinate 123 in the lower right, the area of ​​chart position information 113-A in the upper right spans the corresponding patch image 112 and frame image 114. In this case, from equation 5, the center pixel value R of chart position information 113-A, which is the extraction range, c、 G c、 B c The difference between the value and the threshold T is |255-128|=128, which exceeds the threshold T=32. In such cases, the pixel value determination unit 103 determines that there is pixel variation and sets the pixel value determination information 115 related to the chart position information 113-A to P (6,1) Set to =1 and output to the control unit 101.

[0059] On the other hand, the region of chart position information 113-B is contained within the corresponding patch image 112. In this case, from equation 5, the central pixel value R of chart position information 113-B, which is the extraction range, is obtained. c、 G c、 B c The difference between the value and the threshold T is |255-255|=0, which does not exceed the threshold T=32. Therefore, the pixel value determination unit 103 determines that there is no pixel variation and sets the pixel value determination information 115 related to the chart position information 113-B to P (6,4) Set to =0 and output to the control unit 101.

[0060] The pixel value determination unit 103 performs the same determination in other extraction ranges, generates pixel value determination information 115, and outputs it to the control unit 101.

[0061] In step S105, the pixel value determination unit 103 determines whether or not it has output the pixel value determination information 115 for all patch images to the control unit 101. If the pixel value determination unit 103 has not output the pixel value determination information 115 for all patch images, it returns to step S103; if it has output the pixel value determination information 115 for all patch images, it proceeds to step S106.

[0062] In step S106, the control unit 101, acting as a control means, displays the extraction range (chart position information 113) on the display unit 100 based on the pixel value determination information 115 output from the pixel value determination unit 103, that is, in a manner based on the determination result. If the chart position information 113 is already displayed, the display of the chart position information 113 is updated. The chart position information 113 displayed here corresponds to the extraction range in a provisional determination state (in the process of specification).

[0063] For example, the control unit 101 is P (i,j) Pixel value determination information 115 = 1, P (i,j) The pixel value determination information 115 for =0 is output to the display unit 100, and is different from it. As described above, as an example, P (i,j) The pixel value determination information 115, which is =1, is considered to be information for marking a "×" or similar symbol within the rectangle, P (i,j) The pixel value determination information 115, which is =0, is considered to be information that does not place a mark such as "×" within the rectangle. In addition, when the color is different, for example, P (i,j) The pixel value determination information 115 for =1 is a red rectangle, P (i,j) Pixel value determination information 115 where =0 is represented as a green rectangle. Based on the pixel value determination information 115, the control unit 101 overlays and draws multiple chart position information 113 onto the captured image 110.

[0064] In step S107, the control unit 101 determines whether the mouse 119 has been released. If the mouse 119 is still pressed, the endpoint has not been determined, so the control unit 101 returns to step S101. If the mouse 119 is released, the endpoint has been determined, so the process proceeds to step S107. At this point, the initial rectangle 124 is determined, and the initial extraction range specification is completed.

[0065] If the chart image 111 included in the captured image 110 is not tilted, specifying the extraction range can usually be completed in a single operation. However, as shown in the examples in Figures 3 and 5, the chart image 111 may be tilted, or the rectangle formed by the outline of the chart image 111 and the rectangle 124 may be misaligned.

[0066] Therefore, from step S108 onward, the control unit 101 fine-tunes the vertices of the rectangle 124 based on user input. Note that as a result of the vertex adjustment, the rectangle 124 may change into a quadrilateral with non-right angles, but for convenience, the term "rectangle" will continue to be used. The processing up to step S107 corresponds to the initial specification of the extraction range, and the processing from step S108 onward corresponds to the second and subsequent specification of the extraction range.

[0067] In step S108, the control unit 101 determines whether the user has initiated an operation to change the vertex positions of the rectangle 124. Here, the vertices that can be changed include the start point and the end point. If the mouse 119 is pressed within a predetermined range from any vertex, it is determined that an operation to change the vertex position has been initiated.

[0068] If the control unit 101 does not determine that an operation to change the vertex position has started, it terminates the process shown in Figure 7. If it determines that an operation to change the vertex position has started, it proceeds to step S109. The control unit 101 may also finalize the determination result in step S108 after a predetermined time has elapsed since the start of the determination. In other words, the process shown in Figure 7 may be terminated after a predetermined time has elapsed since the completion of the first or second specification of the extraction range.

[0069] In step S109, the control unit 101 obtains the coordinates of the modified vertices based on user operation. For example, as described above, the user moves a specified coordinate Q2 to match the upper right patch coordinate 121, or moves a specified coordinate Q3 to match the lower left patch coordinate 122, using mouse operation. Alternatively, the user moves the position of a specified coordinate Q1 or specified coordinate Q4. As a method of movement, the user drags the mouse 119 from the selected vertex toward the target point.

[0070] In steps S110 to S114, the control unit 101 performs the same processing as in steps S102 to S106. Since the chart position information 113 is already displayed, the display in step S114 corresponds to updating the display of the chart position information 113.

[0071] In step S115, the control unit 101 determines whether the mouse button 119 pressed on the vertex to be modified has been released. If the mouse button 119 has not been released, the position of the modified vertex has not been determined, so the control unit 101 returns to step S108. If the mouse button 119 has been released, the position of the modified vertex has been determined, so the control unit 101 terminates the process shown in Figure 7. Note that, similar to step S108, the control unit 101 may determine the determination result in step S115 after a predetermined time has elapsed since the start of the determination. In other words, the process shown in Figure 7 may be terminated after a predetermined time has elapsed since the completion of specifying the modification of the extraction range. Once the process shown in Figure 7 is completed, the extraction range is finally determined.

[0072] According to this embodiment, based on user operation, multiple extraction ranges (chart position information 113) are specified on the captured image 110 displayed on the display unit 100, and the presence or absence of pixel variation in the extraction range is determined from the pixel values ​​of the extraction range. The control unit 101 then displays the extraction range on the display unit 100 in a manner based on the determination result. For example, the control unit 101 displays the extraction range differently for extraction ranges with no pixel variation and extraction ranges with pixel variation. Therefore, the presence or absence of pixel variation in the extraction range becomes visually apparent. Thus, the user can easily recognize whether the specified extraction range corresponds to a specific area (corresponding to a patch).

[0073] In other words, in chart pixel value extraction to match the characteristics of the imaging device, visually indicating whether or not there is pixel variation within the extraction range during the extraction range specification operation can suppress rework such as re-selecting the chart or re-acquiring images.

[0074] Furthermore, once the pixel values ​​of the extraction range are obtained, if the extraction range is changed, the pixel values ​​of the changed extraction range are obtained again, and it is determined whether or not there is pixel variation in the changed extraction range (S108~S112). For example, as illustrated in Figure 4, the positions of the chart position information 113 and the corresponding patch image 112 match. This allows the extraction range to be appropriately corrected.

[0075] Furthermore, since the display of the extraction range begins after the initial selection of the extraction range but before the selection is completed (S106), the user can be informed of the appropriateness of the specified extraction range even while the selection process is still underway.

[0076] Furthermore, by determining the presence or absence of pixel variation within the extraction range for each R, G, and B color, or for each of the multiple regions into which the extraction range is divided, it becomes possible to recognize the appropriateness of specifying the extraction range in more detail.

[0077] Furthermore, by linking the average value of the pixel values ​​within the extraction range to the judgment result, it is possible to provide a basis for deciding whether or not to use the average pixel value when performing color matching between imaging devices.

[0078] (Second Embodiment) In the first embodiment, the display of the extraction range began after the initial selection of the extraction range started but before the selection was completed. However, as the endpoint changes due to mouse dragging operations, the display of the extraction range changes sequentially, which may cause screen flickering.

[0079] Therefore, in the second embodiment of the present invention, the control unit 101 does not perform drawing based on the pixel value determination information 115 until the endpoint is provisionally determined, because there is a high possibility of pixel variation occurring, and performs drawing based on the pixel value determination information 115 after the endpoint is provisionally determined.

[0080] Figure 8 is a flowchart showing the extraction range drawing process. The execution entity and start conditions for this process are the same as those for the extraction range drawing process shown in Figure 7.

[0081] The processing in steps S100 to S105 is the same as that described in Figure 7. In step S200, the control unit 101 displays the extracted range (chart position information 113) on the display unit 100 in a manner independent of the judgment result. For example, the control unit 101 draws the extracted range in the same manner as if there were no pixel variation in all extracted ranges. Therefore, all chart position information 113 is displayed in the same color and shape. This prevents visual flickering.

[0082] In step S201, the control unit 101 performs the same processing as in step S107 in Figure 7. If the mouse 119 is not pressed, the endpoint has not been determined, so the control unit 101 returns to step S101. If the mouse 119 is released, the endpoint has been determined, so the process proceeds to step S202. At this point, the initial rectangle 124 is determined, and the initial specification of the extraction range is completed.

[0083] In step S202, the control unit 101 displays the extracted range (chart position information 113) on the display unit 100 based on the pixel value determination information 115 output from the pixel value determination unit 103, that is, in a manner based on the determination result, similar to step S106. Although some illustrations are omitted, from step S203 onward, the control unit 101 performs the same processing as in steps S108 to S115.

[0084] According to this embodiment, the same effect as the first embodiment can be achieved in that the user can easily recognize whether or not the specified extraction range corresponding to a particular area is appropriate.

[0085] Furthermore, the control unit 101 displays the extraction range in a manner not based on the judgment result before the initial extraction range specification is completed, and displays the extraction range in a manner based on the judgment result after the initial extraction range specification is completed. This makes it possible to suppress visual flickering during the initial extraction range specification.

[0086] Furthermore, if the extraction range is changed by a second or subsequent specification, the control unit 101 displays the changed extraction range after the start of the second or subsequent specification but before the specification is completed, allowing the user to recognize whether the specified extraction range is appropriate.

[0087] In each of the above embodiments, the shape of the rectangle 124 defined by the initial provisional determination of the endpoint may be a predetermined quadrilateral whose interior angles are not right angles.

[0088] In each of the above embodiments, the multiple extraction ranges should be specified to include sequences corresponding to the sequences of multiple patches. For example, as shown in Figure 9(a), if the frame image 114 is wide, the number of extraction ranges may be greater than the number of patches, as shown in Figure 9(b).

[0089] In other words, pixel values ​​may be extracted including the portion of the frame image 114. The control unit 101 provides a virtual patch image 160 in addition to the patch image 112, in the area surrounding the frame image 114. The control unit 101 should specify the extraction range so that the number of extracted pixels is not 6 horizontally and 4 vertically corresponding to the patch image 112, but rather 8 horizontally and 6 vertically, including the virtual patch image 160. By performing a determination of pixel variation by extracting pixel values ​​in the portion of the virtual patch image 160, it becomes possible to obtain the average pixel value including the frame data. If a uniform area extends to the periphery of the chart image 111, the extraction range may be specified using a similar method to enable pixel value extraction.

[0090] (Other examples) Although the above embodiment describes an example using a chart that includes patches, the present invention is not necessarily limited to this form. For example, the present invention can be applied to any specific subject that includes a certain predetermined area, such as a structure having a specific pattern.

[0091] Furthermore, in this embodiment, pixel variation was used. That is, whether or not a predetermined condition is met was determined by determining the presence or absence of pixel variation. However, this method is not necessarily limited to this method. For example, the display of the extracted range may be controlled by determining the signal-to-noise ratio of the extracted range or by determining the presence or absence of edges. In this case as well, predetermined conditions based on the noise ratio and edge intensity can be set, and if, for example, the noise ratio or edge intensity is greater than a predetermined value, the same control as when pixel variation is determined to be present can be performed.

[0092] The present invention can also be realized by supplying a program that implements one or more of the functions of the above embodiments to a system or device via a network or a non-transient storage medium, and by having one or more processors in the computer of that system or device read and execute the program. The above program and the storage medium that stores the above program constitute the present invention. Furthermore, the present invention can also be realized by a circuit (e.g., an ASIC) that implements one or more functions.

[0093] Although the present invention has been described in detail above based on its preferred embodiments, the present invention is not limited to these specific embodiments, and various forms that do not depart from the spirit of the invention are also included in the present invention. Some of the above embodiments may be combined as appropriate.

[0094] This embodiment includes the following configuration (and methods, programs): (Composition 1) A first acquisition means for acquiring a captured image of a chart containing multiple patches, A control means for displaying the captured image acquired by the first acquisition means on a display unit, A specification means for specifying multiple extraction ranges to correspond to the multiple patches based on user operation, A second acquisition means for acquiring the pixel values ​​of each extraction range specified by the designation means in the captured image, The system includes a determination means that determines whether predetermined conditions are met for each extraction range based on the pixel values ​​obtained by the second acquisition means, The control means is characterized in that it causes the display unit to display the plurality of extraction ranges in a display manner based on the determination result by the determination means. (Configuration 2) The image processing apparatus according to Configuration 1, characterized in that the control means causes the display unit to display the plurality of extraction ranges, and the display mode differs between the extraction ranges that are determined to satisfy the predetermined conditions and the extraction ranges that are determined not to satisfy the predetermined conditions. (Composition 3) The image processing apparatus according to configuration 2, characterized in that the control means causes the display to change by changing at least one of the color, mark, or frame shape. (Composition 4) The image processing apparatus according to configuration 1 or 2, characterized in that the determination means determines whether the predetermined conditions are met by determining whether or not there is pixel variation. (Composition 5) The image processing apparatus according to configuration 4, characterized in that the determination means determines that there is pixel variation in the extraction range if the difference between the pixel value of the center and the pixel value of a pixel other than the center in each of the acquired extraction ranges exceeds a threshold, and determines that there is no pixel variation in the extraction range if the difference does not exceed the threshold. (Composition 6) The image processing apparatus according to any one of configurations 1 to 3, characterized in that, after acquiring the pixel values ​​of each extraction range, the plurality of extraction ranges are changed by the designation means, and the pixel values ​​of each extraction range after the change are acquired again. (Composition 7) The image processing apparatus according to any one of configurations 1 to 3 or 6, characterized in that the control means displays the plurality of extraction ranges after the initial specification of the plurality of extraction ranges has started but before the specification has been completed. (Composition 8) The image processing apparatus according to any one of configurations 1 to 3, 6, or 7, characterized in that the control means displays the plurality of extraction ranges in a manner not based on the determination result before the initial specification of the plurality of extraction ranges is completed, and displays the plurality of extraction ranges in a manner based on the determination result after the initial specification of the plurality of extraction ranges is completed. (Composition 9) The image processing apparatus according to configuration 8, characterized in that, if the plurality of extraction ranges are changed by a second or subsequent specification using the aforementioned designation means, the changed plurality of extraction ranges are displayed after the start of the second or subsequent specification but before the specification is completed. (Composition 10) The image processing apparatus according to any one of configurations 1 to 3, characterized in that the initial designation of the plurality of extraction ranges by the designation means is the designation of a quadrilateral whose vertices are a first point on the displayed captured image and a second point diagonally opposite to the first point. (Composition 11) The image processing apparatus according to configuration 10, characterized in that the second and subsequent designations by the designation means include a designation to correct the position of at least one of the four vertices of the quadrilateral. (Configuration 12) The image processing apparatus according to any one of configurations 1 to 3, characterized in that the control means displays the plurality of extraction ranges superimposed on the captured image in the display unit. (Composition 13) The image processing apparatus according to any one of configurations 1 to 3 or 12, characterized in that the plurality of extraction ranges are specified to include sequences corresponding to the sequences of the plurality of patches. (Composition 14) The image processing apparatus according to configuration 4 or 5, characterized in that the determination means determines whether or not there is pixel variation in the extraction range for each of the R, G, and B colors. (Composition 15) The image processing apparatus according to configuration 4 or 5, characterized in that the determination means determines whether or not there is pixel variation in the extraction range for each of the multiple regions obtained by dividing the extraction range into multiple parts. (Composition 16) The image processing apparatus according to any one of configurations 1 to 15, characterized in that the control means calculates the average value of the multiple pixel values ​​of the extraction range acquired by the second acquisition means and associates the average value with the determination result by the determination means. (Method 1) A first acquisition step involves acquiring an image of a chart containing multiple patches, A control step which causes the captured image acquired in the first acquisition step to be displayed on the display unit, A selection step that specifies multiple extraction ranges to correspond to the multiple patches based on user operation, A second acquisition step involves acquiring the pixel values ​​of each extraction range specified in the specified step in the captured image, The process includes a determination step of determining whether predetermined conditions are met for each extraction range based on the pixel values ​​obtained in the second acquisition step, The image processing method is characterized in that the control step causes the display unit to display the plurality of extraction ranges in a display manner based on the determination result of the determination step. (Program 1) A program for causing a computer to function as one of the means of an image processing apparatus described in any one of configurations 1 to 16. [Explanation of symbols]

[0095] 100 Display 101 Control Unit 110 Captured Images 102 Pixel value extraction unit 103 Pixel value determination unit 113 Chart Position Information

Claims

1. A program that controls an image processing device, The computer of the aforementioned image processing device, A first acquisition step involves acquiring a captured image of a subject that includes a predetermined area, A control step which causes the captured image acquired in the first acquisition step to be displayed on the display unit, A selection step in which multiple extraction ranges are specified for the subject based on user operation, A second acquisition step involves acquiring the pixel values ​​of each extraction range specified in the specified step in the captured image, A determination step is performed to determine whether predetermined conditions are met for each extraction range based on the pixel values ​​obtained in the second acquisition step. The control step is a program characterized by causing the display unit to display the plurality of extraction ranges in a display manner based on the determination result of the determination step.

2. The program according to claim 1, characterized in that the control step causes the display unit to display the plurality of extraction ranges, and the display mode is made different for the extraction ranges that are determined to satisfy the predetermined conditions and the extraction ranges that are determined not to satisfy the predetermined conditions.

3. The program according to claim 2, characterized in that the control step causes the display to be changed by changing at least one of the color, mark, or frame shape.

4. The program according to any one of claims 1 to 3, characterized in that the determination step determines whether the predetermined conditions are met by determining whether or not there is pixel variation.

5. The program according to claim 4, wherein the determination step determines that there is pixel variation in the extraction range if the difference between the central pixel value and the non-central pixel value in each of the acquired extraction ranges exceeds a threshold, and determines that there is no pixel variation in the extraction range if the difference does not exceed the threshold.

6. The program according to any one of claims 1 to 3, wherein the second acquisition step acquires the pixel values ​​of each extraction range, and if the plurality of extraction ranges are changed by the specification in the specification step, the pixel values ​​of each extraction range after the change are acquired again.

7. The program according to any one of claims 1 to 3, characterized in that the control step displays the plurality of extraction ranges after the initial specification of the plurality of extraction ranges has started but before the specification has been completed.

8. The program according to any one of claims 1 to 3, characterized in that the control step displays the plurality of extraction ranges in a manner not based on the determination result before the initial specification of the plurality of extraction ranges is completed, and displays the plurality of extraction ranges in a manner based on the determination result after the initial specification of the plurality of extraction ranges is completed.

9. The program according to claim 8, characterized in that if the multiple extraction ranges are changed by a second or subsequent specification in the aforementioned specification step, the program displays the changed multiple extraction ranges after the start of the second or subsequent specification but before the specification is completed.

10. The program according to any one of claims 1 to 3, characterized in that the initial specification of the plurality of extraction ranges by the specified step is the specification of a quadrilateral whose vertices are a first point on the displayed captured image and a second point diagonally opposite to the first point.

11. The program according to claim 10, characterized in that the second and subsequent designations in the aforementioned designation step include a designation to modify the position of at least one of the four vertices of the quadrilateral.

12. The program according to any one of claims 1 to 3, characterized in that the control step causes the display unit to display the plurality of extraction ranges superimposed on the captured image.

13. The subject including the aforementioned predetermined area is a chart containing multiple patches, The program according to any one of claims 1 to 3, characterized in that the plurality of extraction ranges are specified to include sequences corresponding to the sequences of the plurality of patches.

14. The program according to claim 4, characterized in that the determination step determines whether or not there is pixel variation in the extraction range for each of the R, G, and B colors.

15. The program according to claim 4, characterized in that the determination step determines whether or not there is pixel variation in the extraction range for each of the multiple regions obtained by dividing the extraction range into multiple parts.

16. The program according to any one of claims 1 to 3, characterized in that the control step calculates the average value of the multiple pixel values ​​of the extraction ranges acquired by the second acquisition step and associates the average value with the determination result of the determination step.

17. The program according to any one of claims 1 to 3, characterized in that the aforementioned designation step is a step of specifying a range to be used for color matching between a plurality of imaging devices.

18. A first acquisition step involves acquiring a captured image of a subject that includes a predetermined area, A control step which causes the captured image acquired in the first acquisition step to be displayed on the display unit, A selection step in which multiple extraction ranges are specified for the subject based on user operation, A second acquisition step involves acquiring the pixel values ​​of each extraction range specified in the specified step in the captured image, The process includes a determination step of determining whether predetermined conditions are met for each extraction range based on the pixel values ​​obtained in the second acquisition step, The image processing method is characterized in that the control step causes the display unit to display the plurality of extraction ranges in a display manner based on the determination result of the determination step.

19. A first acquisition means for acquiring a captured image of a subject that includes a predetermined area, A control means for displaying the captured image acquired by the first acquisition means on a display unit, A designation means for specifying multiple extraction ranges for the subject based on user operation, A second acquisition means for acquiring the pixel values ​​of each extraction range specified by the designation means in the captured image, The system includes a determination means that determines whether predetermined conditions are met for each extraction range based on the pixel values ​​obtained by the second acquisition means, The control means is characterized in that it causes the display unit to display the plurality of extraction ranges in a display manner based on the determination result by the determination means.

Images