Image Processing System

Abstract

The present invention suitably extracts polygonal sections from an image.　An image processing system 10 comprises: an input unit 11 which inputs an image including a subject to be extracted; a detection unit 12 which detects a section with the subject to be extracted in an in put image; a convex enclosure generation unit 13 which generates a convex enclosure serving as a polygon in the detected section; a vertex removal unit 14 which obtains a polygon having a preset number of vertices from the convex enclosure by repeating a vertex removal process which removes vertices corresponding to inner angles of the polygon; and an extraction unit 15 which extracts the above obtained polygon section from the input image, wherein the vertex removal process determines a new line on the basis of the area of a region enclosed by the new line passing through a vertex to be removed and two lines extending from two sides generating said vertex in the polygon and two sides adjacent to the vertex in the polygon, and generates a polygon with said vertex removed by using the newly determined line and the two extending lines.

Description

[Technical Field]

[0001] The present invention relates to an image processing system. [Background technology]

[0002] Conventionally, a method has been proposed for extracting a rectangular portion from an image based on a histogram of the number of black pixels for each direction of the image (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2004-30430 Summary of the Invention [Problem to be solved by the invention]

[0004] When capturing an image of a waybill (shipping label) attached to a package to be delivered, depending on the angle at which the image is captured, the portion of the waybill in the captured image may appear as a rectangle rather than a rectangle. Furthermore, such waybills may be chipped or wrinkled during the delivery process, and it is necessary to properly extract the part of the waybill even when chipped or wrinkled. The method described in Patent Document 1 does not take the above points into consideration, and for example, it is not always possible to properly extract the part of the waybill from the image.

[0005] An embodiment of the present invention has been made in view of the above, and has an object to provide an image processing system that can appropriately extract polygonal portions from an image. [Means for solving the problem]

[0006] In order to achieve the above object, an image processing system according to one embodiment of the present invention comprises an input unit that inputs an image including an extraction target, a detection unit that detects a portion of the extraction target from the image input by the input unit, a convex hull generation unit that generates a convex hull that becomes a polygon from the portion detected by the detection unit, a vertex reduction unit that obtains a polygon having a predetermined number of vertices from the convex hull generated by the convex hull generation unit by repeating a vertex reduction process that reduces vertices according to each interior angle of the polygon, and an extraction unit that extracts a portion of the polygon obtained by the vertex reduction unit from the image input by the input unit, wherein the vertex reduction process determines a new line based on the area of ​​a region surrounded by a new line passing through the vertex to be reduced, two edges that generate the vertex in the polygon, and two lines extending the edges of the polygon two lines adjacent to the vertex in the polygon, and generates a polygon after the vertices have been reduced using the new line that has been determined and the two extended lines.

[0007] In an image processing system according to an embodiment of the present invention, a polygonal portion having a predetermined number of vertices, not limited to a rectangle, is extracted from an image. Furthermore, in an image processing system according to an embodiment of the present invention, even if the extraction target in the image has chips, wrinkles, or the like, it is possible to extract an appropriate polygonal portion that includes all of the detected parts of the extraction target. In this way, the image processing system according to an embodiment of the present invention can appropriately extract a polygonal portion from an image. [Effects of the Invention]

[0008] According to one embodiment of the present invention, polygonal portions can be appropriately extracted from an image. [Brief explanation of the drawings]

[0009] [Figure 1] 1 is a diagram showing a configuration of an image processing system according to an embodiment of the present invention. [Figure 2] FIG. 1 is a diagram illustrating an overview of image processing in an image processing system. [Figure 3] 10A and 10B are diagrams illustrating examples of images used to detect the contour of a portion to be extracted. [Figure 4] FIG. 10 is a diagram showing an example of the contour of a portion to be extracted in an image. [Figure 5] FIG. 10 is a diagram showing an example of a quadrilateral that contacts a polygon from the outside. [Figure 6] FIG. 10 is a diagram showing an example of a quadrangle obtained from the contour of a portion to be extracted using a conventional method. [Figure 7] FIG. 10 is a diagram illustrating a vertex reduction process. [Figure 8] 10A and 10B are diagrams illustrating polygons in the process of generating a convex hull and reducing vertices. [Figure 9] FIG. 10 is a diagram showing an example of a rectangle with a minimum area obtained from a polygon. [Figure 10] FIG. 10 is a diagram illustrating the conversion of an image. [Figure 11] FIG. 10 is a diagram illustrating an example of calculation of the size of an image after conversion. [Figure 12] 3 is a flowchart showing a process executed in the image processing system according to the embodiment of the present invention. [Figure 13] 1 is a diagram illustrating a hardware configuration of an image processing system according to an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of an image processing system according to the present invention will be described in detail with reference to the drawings. In the description of the drawings, the same elements are given the same reference numerals and duplicated explanations will be omitted.

[0011] FIG. 1 shows an image processing system 10 according to this embodiment. The image processing system 10 is a system (device) that extracts (cuts out) a polygonal portion including an extraction target (subject) from an image including the extraction target. The image in question is, for example, an image of a waybill (shipping label) attached to a package to be delivered, as shown in FIG. 2(a). The extraction target is the part of the waybill. The image may be an image of the part of the waybill captured from any direction. Although the waybill has a rectangular shape, if it is captured from an oblique angle, the part of the waybill in the image will have a three-dimensional tilt. Therefore, the part will not appear rectangular in the image as shown in FIG. 2(a). Furthermore, the waybill may have chips, wrinkles, or the like, and in such cases, the part (outline) of the waybill in the image may not be rectangular. The image processing system 10 extracts a rectangular portion including the part of the waybill from such an image.

[0012] As shown in FIG. 2, the image processing system 10 extracts a rectangular portion including the invoice portion from the image. First, the image processing system 10 extracts the outline 100 of the invoice portion as shown in FIG. 2(b). As mentioned above, the outline 100 is not necessarily rectangular. Next, the image processing system 10 generates a rectangle 200 that contacts the outline 100 of the invoice portion from the outside as shown in FIG. 2(c), and extracts the rectangular portion 200 of the image. As will be described later, this rectangle 200 is determined to be an appropriate shape for extraction, including all of the extracted invoice portion while minimizing the inclusion of portions other than the invoice. The image processing system 10 then converts the extracted image into a rectangle as shown in FIG. 2(d). That is, an image of the object to be extracted that was not captured from the front is converted into an image that appears to be captured from the front. The image obtained in this manner can be used, for example, for managing invoices. In this case, character recognition may also be performed on the obtained image.

[0013] The image to be processed by the image processing system 10 is one in which the entire extraction object, for example, the entire invoice, is included in the image (the entire invoice is visible). In other words, an image in which part of the extraction object is not visible is not processed by the image processing system 10. Furthermore, it is preferable that the inclination of the image to be processed relative to the correct orientation of the part to be extracted (for example, the direction in which the characters on the invoice are correctly aligned) is within 45 degrees. In this case, a rectangular image with the correct orientation after conversion can be obtained. However, the inclination of the part to be extracted does not necessarily have to be as described above.

[0014] The polygon extracted by the image processing system 10 does not necessarily have to be a rectangle, but may be any polygon with a preset number of vertices (for example, a pentagon or a triangle). The extracted object in the image does not have to be an invoice, but may be anything that is expected to have a polygonal shape when extracted. For example, a slip or document other than the invoice may be the extracted object. The extracted image may also be used to identify objects in the photograph or determine the type of document, rather than for information management.

[0015] The image processing system 10 is realized by a computer such as a smartphone, a PC (personal computer), a server device, etc. The image processing system 10 may also be realized by a plurality of computers, that is, a computer system.

[0016] Next, the functions of the image processing system 10 according to this embodiment will be described. As shown in Fig. 1, the image processing system 10 includes an input unit 11, a detection unit 12, a convex hull generation unit 13, a vertex reduction unit 14, an extraction unit 15, and a conversion unit 16.

[0017] The input unit 11 is a functional unit that inputs an image including an extraction target. For example, the input unit 11 reads out and inputs an image that has been previously stored in a database of the image processing system 10. Alternatively, the input unit 11 may acquire and input an image through a user operation on the image processing system 10 (for example, an operation to capture an image of an object corresponding to the extraction target). Alternatively, the input unit 11 may receive and input an image from another device. Furthermore, the input unit 11 may input an image by a method other than those described above. The input unit 11 outputs the input image to the detection unit 12 and the extraction unit 15.

[0018] The detection unit 12 is a functional unit that detects the portion to be extracted from the image input by the input unit 11. For example, the detection unit 12 detects the portion to be extracted from the image as follows: The detection unit 12 inputs an image from the input unit 11. The detection unit 12 uses conventional technology to detect the contour of the portion to be extracted from the image. Specifically, it performs Otsu's binarization or the like to convert the image into a black and white image, and obtains the contour of the region with the largest area. This method can be used when there is a large difference in brightness between the extraction target and the background.

[0019] Alternatively, an input image such as that shown in Figure 3(a) can be converted into the HSV color space to obtain an image with H (hue) extracted, as shown in Figure 3(b). The image with H (hue) extracted is then subjected to the above-mentioned binarization process to obtain a black-and-white image, as shown in Figure 3(c). The contours of the area to be extracted are detected from the resulting black-and-white image. This method can be used when the difference in brightness between the object to be extracted and the background is not large, but the colors of the object to be extracted and the background are different.

[0020] The detection unit 12 uses the above method to detect, as the portion to be extracted, for example, all pixels that make up the outline of the portion to be extracted and the order of those pixels. That is, the detection unit 12 detects the portion to be extracted in raster format (as a raster image). Alternatively, the detection unit 12 detects, as the portion to be extracted, the coordinates on the image of the vertices that make up the outline of the portion to be extracted and the line segments connecting those coordinates. That is, the detection unit 12 detects the portion to be extracted in vector format (as a vector image). Note that the detection method for the portion to be extracted and the information indicating the portion to be extracted are not limited to those described above, and any other method can be used. The detection unit 12 outputs information indicating the detected portion to be extracted to the convex hull generation unit 13.

[0021] The convex hull generation unit 13 is a functional unit that generates a polygonal convex hull of the portion detected by the detection unit 12. The convex hull generation unit 13 generates a convex hull as follows: The convex hull generation unit 13 receives contour information of the portion to be extracted from the detection unit 12 as information indicating the portion. The contour information is information indicating the pixels constituting the contour and their order (raster format information), or information indicating the coordinates of the vertices constituting the contour and the line segments connecting those coordinates (vector format information). A contour represented by such information, i.e., a contour treated as digital data, can be considered a polygon regardless of its shape. FIG. 4(b) shows a pixel-by-pixel enlargement of a portion of the raster format contour image shown in FIG. 4(a). When the image is enlarged as shown in FIG. 4(b), the pixels are arranged in a grid pattern. When the contour is represented by raster format information like this, it can be considered a polygon by, for example, connecting the centers of the pixels constituting the contour in order with a line 300. When the contour is represented by vector format information as described above, the contour is originally a polygon. The convex hull generating unit 13 first generates the above polygon from the information input from the detecting unit 12 .

[0022] Next, the convex hull generation unit 13 converts the generated polygon into a convex hull and generates it. For example, for a vertex whose interior angle of the polygon exceeds 180 degrees (for example, vertex 310 in FIG. 5(c) whose interior angle is 225 degrees), the convex hull generation unit 13 connects the vertices adjacent to that vertex with a line (for example, line 320 in FIG. 5(c)), and uses the connecting line as the edge of the convex hull to be generated. The convex hull generation unit 13 generates new edges of the convex hull until there are no more vertices whose interior angle of the polygon exceeds 180 degrees. The convex hull generation unit 13 uses vertices whose interior angle of the polygon does not exceed 180 degrees as vertices of the convex hull. In this way, the convex hull generation unit 13 generates a convex hull that is an n-sided polygon (n is an integer greater than or equal to 3).

[0023] The convex hull generation by the convex hull generation unit 13 is performed on the coordinate axes of the image. Furthermore, the generation of the convex hull by the convex hull generation unit 13 does not necessarily have to be performed in the manner described above, and may be performed in any manner. The convex hull generation unit 13 outputs information indicating the generated convex hull to the vertex reduction unit 14. The information indicating the convex hull includes information indicating the position of the convex hull on the image.

[0024] The vertex reduction unit 14 is a functional unit that obtains a polygon having a preset number of vertices from the convex hull generated by the convex hull generation unit 13 by repeating a vertex reduction process that reduces vertices corresponding to each interior angle of the polygon. The vertex reduction process determines a new line based on the area of ​​a region surrounded by a new line passing through the vertex to be reduced, two sides of the polygon that generate the vertex, and two lines extending from the sides two sides adjacent to the vertex in the polygon, and generates a polygon after the vertices have been reduced using the determined new line and the two extended lines. The vertex reduction process reduces the vertex corresponding to the largest interior angle of the polygon, and the new line may be determined so as to minimize the area.

[0025] The polygon generated by the vertex reduction unit 14 is the portion extracted in the image processing system 10. Algorithms exist for obtaining a quadrilateral that is conditionally bounded by an arbitrary n-gon from the outside. For example, one such algorithm obtains an axis-aligned bounding box (AABB), which is a rectangle that is bounded from the outside along the vertical and horizontal axes. This algorithm can obtain an AABB simply by calculating the maximum and minimum values ​​of the top, bottom, left, and right of the n-gon. Another such algorithm obtains an oriented bounding box (OBB), which is a rectangle that is bounded from the outside. However, both methods result in the portions of the quadrilateral that are not part of the n-gon becoming larger, and are not necessarily suitable for obtaining an extracted quadrilateral.

[0026] This will be explained using FIG. 5. For an n-gon (hexagon) 400 shown in FIG. 5(a), an AABB quadrangle 410 is shown in FIG. 5(b), and an OBB quadrangle 410 is shown in FIG. 5(c). In either case, the portions of the quadrangle 410 other than the n-gon 400 that are included in the quadrangle 410 are large, and therefore are not necessarily suitable as quadrangles to be extracted. In this embodiment, the vertex reduction unit 14 obtains a quadrangle by a method that reduces the portions of the quadrangle 410 other than the n-gon 400, as shown in FIG. 5(d).

[0027] On the other hand, it is also possible to obtain an extracted rectangle directly from the contour of the extraction target detected by the detection unit 12 (without generating a convex hull as described above). For example, the following method simply approximates the contour with four straight lines. One method treats each pixel constituting the contour as a single point and approximates it with four straight lines using the least squares method or similar. When using the least squares method, four straight lines are drawn, using the sum of the squares of the distances from all points to the nearest straight line as a reference. Another method uses an algorithm for simplifying broken lines, such as the Dauglas-Peucker algorithm. While these methods cannot obtain a rectangle that is tangent to the contour of the extraction target from the outside, they can obtain a rectangle. However, with the above method, if the extraction target 100 contains noise, such as a partially turned-up invoice as shown in FIG. 6, part of the extraction target 100 will extend beyond the extracted rectangle 420. This noise in the extraction target 100 significantly affects the resulting rectangle 420. Therefore, it is better to generate a rectangle that contacts the extraction target 100 from the outside, as in this embodiment.

[0028] The vertex reduction unit 14 performs a vertex reduction process to reduce one vertex by expanding an n-gon, thereby obtaining an n-1-gon. The vertex reduction unit 14 repeatedly performs the vertex reduction process to obtain a polygon having a preset number of vertices (a quadrangle in the above-mentioned invoice example) from the convex hull generated by the convex hull generation unit 13. The vertex reduction unit 14 obtains the quadrangle as follows:

[0029] The vertex reduction unit 14 receives information indicating an n-gon, which is a convex hull, from the convex hull generation unit 13. As shown in FIG. 7(a), the vertex reduction unit 14 selects the vertex 510 associated with the largest interior angle of the n-gon 500 indicated by the received information as the vertex to be reduced. Next, as shown in FIG. 7(b), the vertex reduction unit 14 extends two edges 520 adjacent to the vertex 510 to be reduced toward the vertex. The vertex reduction unit 14 determines the line 530 so that the area of ​​the region (the two triangular regions shown in FIG. 7(b)) surrounded by the line 530 passing through the vertex 510 to be reduced, the two edges 540 that generate the vertex 510 in the polygon 500, and the two edges 520 adjacent to the vertex 510 is minimized. The only variable of the line 530 is the slope x. Once x is determined, the positions of the three vertices of the two triangles are determined. Therefore, the area of ​​the region (the sum of the areas of the two triangles) can be calculated using the formula for x. We need to find the value of x that minimizes this. Note that line 530 may overlap with either of the two sides 540 that generate vertex 510. In that case, the areas of the two triangles described above become the area of ​​a single triangle.

[0030] 7(c), the vertex reducing unit 14 eliminates the selected vertex 510, and sets the determined straight line 530 and the portions of the two extended straight lines 521 up to where they intersect with the straight line 530 as the sides of the n-1 polygon 501 from which the vertices have been reduced, thereby obtaining the n-1 polygon 501. The vertex reducing unit 14 repeats the above vertex reducing process until the resulting polygon is a polygon having a preset number of vertices (a rectangle in the above-mentioned invoice example).

[0031] The generation of a polygon (a rectangle in the above-mentioned invoice example) by the vertex reduction unit 14 is performed on the coordinate axes of the image. Furthermore, the vertex reduction of an n-gon by the vertex reduction unit 14 does not necessarily have to be performed as described above, as long as it reduces the vertices corresponding to each interior angle of the polygon and is performed based on the above-mentioned area. The vertex reduction unit 14 outputs information indicating the obtained polygon to the extraction unit 15. The information indicating the polygon includes information indicating the position of the polygon on the image.

[0032] The detection of the portion to be extracted by the detection unit 12, the process of generating a convex hull by the convex hull generation unit 13, and the generation of a polygon by the vertex reduction unit 14 will be described with reference to FIG. 8. As shown in FIG. 8(a), the detection unit 12 detects a contour 600 to be extracted. Then, as shown in FIG. 8(b), the convex hull generation unit 13 generates a hexagon 610, which is a convex hull of the contour 600. Then, as shown in FIG. 8(c), the vertex reduction unit 14 reduces the vertex 611 corresponding to the largest interior angle of the hexagon 610 to generate a pentagon 620. Then, as shown in FIG. 8(d), the vertex reduction unit 14 reduces the vertex 621 corresponding to the largest interior angle of the pentagon 620 to generate a quadrangle 630.

[0033] Note that the above-described method does not guarantee that an n-1-gon that touches an n-gon from the outside and has the smallest area can be obtained. For example, consider the pentagon shown in FIG. 9. Note that the wavy lines in the figure indicate sufficiently long straight lines. For the pentagon shown in FIG. 9, the above-described method does not result in a rectangle with the smallest area. In the pentagon shown in FIG. 9, vertex 701 is selected as the vertex to be reduced, but because the two edges that generate vertex 701 are sufficiently long, the areas of the two triangles created are also large. The rectangle whose edges are indicated by the dashed lines is the rectangle that touches pentagon 700 from the outside and has the smallest area.

[0034] In reality, capturing an image of the extraction target in such a shape requires capturing the image from almost exactly the side. Therefore, if the photographer intends to capture the extraction target, it is unlikely that the extraction target will end up in such a shape in the image. Furthermore, the objective of this embodiment is not to obtain a rectangle with the smallest area, but to obtain a rectangle suitable for conversion into an image that appears to have been captured from the front. Therefore, even if a distorted rectangle is obtained by emphasizing the smallest area, the objective will not be achieved.

[0035] The extraction unit 15 is a functional unit that extracts, from the image input by the input unit 11, a portion of a polygon obtained by repeated vertex reduction processing by the vertex reduction unit 14. The extraction unit 15 inputs an image from the input unit 11. The extraction unit 15 inputs information indicating a polygon from the vertex reduction unit 14. The extraction unit 15 extracts (cuts out) a portion of the image that is indicated by the information indicating the polygon. The extraction unit 15 outputs the extracted portion of the image to the conversion unit 16.

[0036] The conversion unit 16 is a functional unit that converts (corrects) the polygonal portion extracted by the extraction unit 15 into a predetermined polygonal shape. The conversion unit 16 may calculate the converted size (corrected size) based on the size of the polygonal portion extracted by the extraction unit 15. For example, in the above-mentioned example of the invoice, the conversion unit 16 converts the non-rectangular quadrangle 200 portion extracted by the extraction unit 15, as shown in FIG. 2(c), into a rectangular image, as shown in FIG. 2(d). This conversion corrects the tilt (e.g., the three-dimensional tilt) of the extracted polygonal image. Note that the number of vertices of the polygon before and after conversion, i.e., the value of n in an n-gon, is the same.

[0037] The conversion unit 16 receives the extracted polygonal image from the extraction unit 15. The conversion unit 16, for example, performs a scaling process on the input original image using coordinate transformation to convert it into a predetermined polygonal shape without losing any of the original image. The size of the converted image (e.g., aspect ratio) is preset at the time of conversion. This setting may be performed by a user's operation on the image processing system 10, for example, depending on the size of the shipping label. As shown in FIG. 10, the coordinate transformation is performed, for example, by setting the center of gravity of the image 800 extracted by the extraction unit 15 as the origin, setting the vertex with the largest sum of the x-coordinate and the y-coordinate as the upper right vertex of the converted image 810, and setting the vertex with the smallest sum of the x-coordinate and the y-coordinate as the lower left vertex of the converted image 810. The coordinate transformation is performed by a conventional method, for example, projective transformation of the image. As described above, if the inclination of the portion to be extracted is within 45 degrees, the relative positional relationships of the upper right, upper left, lower left, and lower right from the center are maintained even after conversion to a rectangle. The conversion may also be performed by methods other than those described above.

[0038] Furthermore, the conversion unit 16 may calculate the post-conversion size based on the size of the image input from the extraction unit 15, and convert the image to the calculated post-conversion size. For example, as shown in Fig. 11(a), the conversion unit 16 may calculate the length of each side of the image 800 input from the extraction unit 15, and use the average of the lengths of the opposing sides as the vertical and horizontal lengths of the post-conversion image 810, as shown in Fig. 11(b). Furthermore, the post-conversion size may be calculated by a method other than the above.

[0039] The conversion unit 16 outputs the image obtained by the conversion. For example, the conversion unit 16 outputs the image to a database of the image processing system 10 for storage. Alternatively, the conversion unit 16 may transmit the image to another device for output. The conversion unit 16 may also output the image by a method other than the above. The functions of the image processing system 10 according to this embodiment have been described above.

[0040] Next, the process executed by the image processing system 10 according to this embodiment (the operation method performed by the image processing system 10) will be described with reference to the flowchart of FIG.

[0041] In this process, first, the input unit 11 inputs an image including an extraction target (S01). Next, the detection unit 12 detects the portion of the extraction target, specifically the outline of the extraction target, from the image input by the input unit 11 (S02). Next, the convex hull generation unit 13 generates a convex hull that forms a polygon of the outline of the extraction target (S03). Next, the vertex reduction unit 14 performs a vertex reduction process to reduce vertices corresponding to each interior angle of the polygon (S04). The vertex reduction process (S04) by the vertex reduction unit 14 is repeated until the number of vertices of the polygon reaches a predetermined number (4 in the invoice example described above). Furthermore, the initial vertex reduction process is performed on the convex hull generated by the convex hull generation unit 13.

[0042] Next, the extraction unit 15 extracts a polygonal portion (a quadrangle in the above-mentioned invoice example) obtained by repeating the vertex reduction process by the vertex reduction unit 14 from the image input by the input unit 11 (S05). Next, the conversion unit 16 converts the polygonal portion extracted by the extraction unit 15 into a predetermined polygonal shape (a rectangle in the above-mentioned invoice example) (S06). Next, the conversion unit 16 outputs the converted image (S07). The above is the process executed by the image processing system 10 according to this embodiment.

[0043] In this embodiment, a polygonal portion having a preset number of vertices, not limited to a rectangle, is extracted from an image. For example, in the invoice example described above, any rectangular portion is extracted. Furthermore, in this embodiment, even if the extraction target in the image has chips, wrinkles, or the like, and the contour of the extraction target is not straight or clear, an appropriate polygonal portion that includes all of the detected extraction target can be extracted. Specifically, the polygon to be extracted can be set to a shape that includes all of the extraction target while minimizing the inclusion of parts other than the extraction target, and is appropriate for extraction. In this way, this embodiment allows appropriate extraction of a polygonal portion from an image.

[0044] Furthermore, the determination of the portion to be extracted according to this embodiment can be realized without using a method (e.g., Mask R-CNN) that detects the shape of an object using an AI (artificial intelligence) model generated by machine learning. Therefore, this embodiment can extract an image without the need to prepare learning data to be used for machine learning. Furthermore, while the above-mentioned learning-based method cannot detect an extraction target that differs from the learning data, this embodiment determines the polygon to be extracted on a rule-based basis as described above, and therefore can extract any extraction target.

[0045] Furthermore, the vertex reduction process of this embodiment may be a process for reducing the vertex associated with the largest interior angle of a polygon, and may determine a new line so as to minimize the area used to determine reduction. This configuration allows the vertex reduction process to be performed appropriately and reliably, and as a result, appropriate polygonal portions can be extracted from the image. However, the vertex reduction process need not necessarily be performed as described above, as long as it reduces vertices corresponding to each interior angle of the polygon and is performed based on the above area.

[0046] Furthermore, polygonal portions extracted from an image as in this embodiment may be converted into a preset polygonal shape, such as a rectangle. This configuration allows for a highly useful image to be obtained. For example, a non-rectangular invoice portion captured in the image can be converted into a rectangular image as if it were captured from the front.

[0047] Furthermore, when converting an image, the size after conversion may be calculated based on the size of the polygonal portion extracted by the extraction unit. With this configuration, the converted image can be obtained without the need to set the image size in advance. However, image conversion is not necessarily performed, and the image processing system 10 does not necessarily have to include the conversion unit 16. In this case, the extraction unit 15 may simply output the extracted image.

[0048] The block diagrams used to explain the above embodiments show functional blocks. These functional blocks (components) are realized by any combination of at least one of hardware and software. Furthermore, the method for realizing each functional block is not particularly limited. That is, each functional block may be realized using a single device that is physically or logically coupled, or may be realized using two or more physically or logically separated devices that are directly or indirectly connected (for example, using wires, wirelessly, etc.) and these multiple devices. The functional block may also be realized by combining the single device or multiple devices with software.

[0049] Functions include, but are not limited to, judgment, determination, judgment, calculation, computation, processing, derivation, investigation, search, confirmation, reception, transmission, output, access, resolution, selection, election, establishment, comparison, assumption, expectation, regard, broadcasting, notifying, communicating, forwarding, configuring, reconfiguring, allocating, mapping, and assignment. For example, a functional block (component) that performs transmission is called a transmitting unit or transmitter. As mentioned above, there are no particular limitations on how these functions are implemented.

[0050] For example, the image processing system 10 according to an embodiment of the present disclosure may function as a computer that performs information processing according to the present disclosure. Fig. 13 is a diagram illustrating an example of a hardware configuration of the image processing system 10 according to an embodiment of the present disclosure. The image processing system 10 described above may be physically configured as a computer device including a processor 1001, a memory 1002, a storage 1003, a communication device 1004, an input device 1005, an output device 1006, a bus 1007, etc.

[0051] In the following description, the term "apparatus" can be interpreted as a circuit, a device, a unit, etc. The hardware configuration of the image processing system 10 may be configured to include one or more of the apparatuses shown in the drawings, or may be configured to exclude some of the apparatuses.

[0052] Each function in the image processing system 10 is realized by loading predetermined software (programs) onto hardware such as the processor 1001 and memory 1002, causing the processor 1001 to perform calculations, control communication via the communication device 1004, and control at least one of reading and writing data in the memory 1002 and storage 1003.

[0053] The processor 1001 controls the entire computer by running, for example, an operating system. The processor 1001 may be configured as a central processing unit (CPU) including an interface with peripheral devices, a control device, an arithmetic unit, a register, etc. For example, each function in the image processing system 10 described above may be realized by the processor 1001.

[0054] Furthermore, the processor 1001 reads programs (program codes), software modules, data, etc. from at least one of the storage 1003 and the communication device 1004 into the memory 1002 and executes various processes in accordance with these. The programs used are those that cause a computer to execute at least some of the operations described in the above-described embodiments. For example, each function in the image processing system 10 may be implemented by a control program stored in the memory 1002 and running on the processor 1001. While the above-described various processes have been described as being executed by one processor 1001, they may also be executed simultaneously or sequentially by two or more processors 1001. The processor 1001 may be implemented by one or more chips. The programs may also be transmitted from a network via a telecommunications line.

[0055] The memory 1002 is a computer-readable recording medium and may be configured, for example, by at least one of a read-only memory (ROM), an erasable programmable read-only memory (EPROM), an electrically erasable programmable read-only memory (EEPROM), a random access memory (RAM), etc. The memory 1002 may also be called a register, a cache, a main memory (primary storage device), etc. The memory 1002 can store executable programs (program codes), software modules, etc. for performing information processing according to an embodiment of the present disclosure.

[0056] Storage 1003 is a computer-readable recording medium and may be, for example, at least one of an optical disk such as a CD-ROM (Compact Disc ROM), a hard disk drive, a flexible disk, a magneto-optical disk (e.g., a compact disk, a digital versatile disk, a Blu-ray® disk), a smart card, a flash memory (e.g., a card, a stick, a key drive), a floppy disk, a magnetic strip, etc. Storage 1003 may also be referred to as an auxiliary storage device. The storage medium provided in image processing system 10 may be, for example, a database, a server, or other appropriate medium including at least one of memory 1002 and storage 1003.

[0057] The communication device 1004 is hardware (transmission / reception device) for communicating between computers via at least one of a wired network and a wireless network, and is also called, for example, a network device, a network controller, a network card, or a communication module.

[0058] The input device 1005 is an input device (for example, a keyboard, a mouse, a microphone, a switch, a button, a sensor, etc.) that receives input from the outside. The output device 1006 is an output device (for example, a display, a speaker, an LED lamp, etc.) that outputs to the outside. The input device 1005 and the output device 1006 may be integrated into one device (for example, a touch panel).

[0059] Furthermore, each device, such as the processor 1001 and the memory 1002, is connected by a bus 1007 for communicating information. The bus 1007 may be configured using a single bus, or may be configured using different buses between each device.

[0060] Furthermore, the image processing system 10 may be configured to include hardware such as a microprocessor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a programmable logic device (PLD), or a field programmable gate array (FPGA), and some or all of the functional blocks may be realized by the hardware. For example, the processor 1001 may be implemented using at least one of these pieces of hardware.

[0061] The order of the procedures, sequences, flowcharts, etc. of each aspect / embodiment described in this disclosure may be changed unless it is consistent. For example, the methods described in this disclosure present elements of various steps using an example order, and are not limited to the particular order presented.

[0062] Input and output information may be stored in a specific location (for example, memory) or may be managed using a management table. Input and output information may be overwritten, updated, or added to. Output information may be deleted. Input information may be sent to another device.

[0063] The determination may be made based on a value represented by one bit (0 or 1), a Boolean value (true or false), or a numerical comparison (e.g., comparison with a predetermined value).

[0064] Each aspect / embodiment described in this disclosure may be used alone, in combination, or switched depending on the implementation. Furthermore, notification of predetermined information (e.g., notification that "X is true") is not limited to being done explicitly, but may be done implicitly (e.g., by not notifying the predetermined information).

[0065] Although the present disclosure has been described in detail above, it is clear to those skilled in the art that the present disclosure is not limited to the embodiments described herein. The present disclosure can be implemented in modified and altered forms without departing from the spirit and scope of the present disclosure as defined by the claims. Therefore, the description of the present disclosure is intended to be illustrative and does not have any limiting meaning on the present disclosure.

[0066] Software shall be construed broadly to mean instructions, instruction sets, code, code segments, program code, programs, subprograms, software modules, applications, software applications, software packages, routines, subroutines, objects, executable files, threads of execution, procedures, functions, etc., whether referred to as software, firmware, middleware, microcode, hardware description language, or otherwise.

[0067] Software, instructions, information, etc. may also be transmitted or received over a transmission medium. For example, if software is transmitted from a website, server, or other remote source using wired technologies (such as coaxial cable, fiber optic cable, twisted pair, Digital Subscriber Line (DSL)), and / or wireless technologies (such as infrared, microwave), then these wired and / or wireless technologies are included within the definition of transmission media.

[0068] As used in this disclosure, the terms "system" and "network" are used interchangeably.

[0069] Furthermore, the information, parameters, etc. described in this disclosure may be expressed using absolute values, may be expressed using relative values ​​from a predetermined value, or may be expressed using other corresponding information.

[0070] As used in this disclosure, the terms "determining" and "determining" may encompass a wide variety of actions. "Determining" and "determining" may include, for example, judging, calculating, computing, processing, deriving, investigating, looking up, searching, inquiring (e.g., searching in a table, database, or other data structure), ascertaining, and the like. "Determining" and "determining" may also include receiving (e.g., receiving information), transmitting (e.g., sending information), input, output, accessing (e.g., accessing data in memory), and the like. Furthermore, "judgment" and "decision" can include regarding resolving, selecting, choosing, establishing, comparing, etc. as having been "judged" or "decided." In other words, "judgment" and "decision" can include regarding some action as having been "judged" or "decided." Furthermore, "judgment (decision)" can be interpreted as "assuming," "expecting," "considering," etc.

[0071] The terms "connected," "coupled," or any variation thereof, refer to any direct or indirect connection or coupling between two or more elements, and may include the presence of one or more intermediate elements between two elements that are "connected" or "coupled" to each other. The coupling or connection between elements may be physical, logical, or a combination thereof. For example, "connected" may be read as "access." As used in this disclosure, two elements may be considered to be "connected" or "coupled" to each other using one or more wires, cables, and / or printed electrical connections, as well as electromagnetic energy having wavelengths in the radio frequency range, microwave range, and optical (both visible and invisible) range, as some non-limiting and non-exhaustive examples.

[0072] As used in this disclosure, the phrase "based on" does not mean "based only on," unless expressly stated otherwise. In other words, the phrase "based on" means both "based only on" and "based at least on."

[0073] As used in this disclosure, any reference to an element using a designation such as "first," "second," etc. does not generally limit the quantity or order of those elements. These designations may be used in this disclosure as a convenient method of distinguishing between two or more elements. Thus, a reference to a first and a second element does not imply that only two elements may be employed or that the first element must in some way precede the second element.

[0074] When used in this disclosure, the terms "include," "including," and variations thereof are intended to be inclusive, similar to the term "comprising." Furthermore, when used in this disclosure, the term "or" is not intended to be an exclusive or.

[0075] In this disclosure, where articles are added by translation, such as a, an, and the in English, the disclosure may include that the nouns following these articles are in the plural form.

[0076] In the present disclosure, the term "A and B are different" may mean "A and B are different from each other." The term may also mean "A and B are each different from C." Terms such as "separate" and "coupled" may also be interpreted in the same way as "different." [Explanation of symbols]

[0077] 10...image processing system, 11...input unit, 12...detection unit, 13...convex hull generation unit, 14...vertex reduction unit, 15...extraction unit, 16...conversion unit, 1001...processor, 1002...memory, 1003...storage, 1004...communication device, 1005...input device, 1006...output device, 1007...bus.

Claims

1. an input unit for inputting an image including an extraction target; a detection unit that detects a portion to be extracted from the image input by the input unit; a convex hull generation unit that generates a convex hull that is a polygon of the part detected by the detection unit; a vertex reduction unit that obtains a polygon having a predetermined number of vertices from the convex hull generated by the convex hull generation unit by repeating a vertex reduction process that reduces vertices corresponding to each interior angle of the polygon; an extraction unit that extracts a portion of the polygon obtained by the vertex reduction unit from the image input by the input unit; Equipped with The vertex reduction process is a process in which a new line passing through the vertex to be reduced is determined based on the area of ​​the region surrounded by the new line, the two edges that generate the vertex in the polygon, and two lines extending the edges two edges away from the vertex in the polygon, and the determined new line and the two extended lines are used to generate a polygon after the vertex has been reduced.

2. 2. The image processing system according to claim 1, wherein the vertex reduction process is a process for reducing the vertex associated with the largest interior angle of a polygon, and the new line is determined so as to minimize the area.

3. 3. The image processing system according to claim 1, further comprising a conversion unit that converts the polygonal portion extracted by the extraction unit into a predetermined polygonal shape.

4. The image processing system according to claim 3 , wherein the conversion unit calculates the converted size based on the size of the polygonal portion extracted by the extraction unit.

Images