Image processing device, image processing method, program, and storage medium

Abstract

To provide an image processing device, an image processing method, and a program which more suitably perform noise removal of image data.SOLUTION: In an image processing system, a server 30 as an image processing device includes: an acquisition unit 33 which acquires image data produced by reading a manuscript; a specifying unit 34 which specifies object regions including a predetermined object, in the image data; a removal unit 36 which removes noise having a size smaller than that specified by a first threshold value from the object regions; and a setting unit 35 which sets the first threshold value for each of the object regions specified by the specifying unit 34 according to a size of the object regions.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to an image processing device, an image processing method, and Law, P Program and storage media Regarding. [Background technology]

[0002] In recent years, the character recognition accuracy of OCR (Optical Character Recognition / Reader) has improved through the use of AI. However, if a document contains noise such as dirt, dust, scratches, show-through, or background patterns, the character recognition accuracy of OCR may decrease. Therefore, there is a need to remove noise that affects character recognition accuracy. Patent Document 1 proposes an image processing device that identifies a print area included in a scanned image generated by scanning a receipt or invoice, and removes from the identified print area any noise areas that meet predetermined noise conditions. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-168856 Summary of the Invention [Problem to be solved by the invention]

[0004] When removing noise from image data, it is necessary to remove only the noise without erroneously removing objects such as characters or graphics. For example, the above-mentioned prior art proposes recognizing a print area that satisfies the condition that its area is smaller than a predetermined minimum character size as a noise area and removing it from the print area. However, if the size of the noise that can be removed is small, it may not be possible to remove the noise that should be removed, and the desired noise removal results may not be obtained.

[0005] The present invention provides a technique for more appropriately removing noise from image data. [Means for solving the problem]

[0006] According to one aspect of the present invention, an acquisition means for acquiring image data generated by reading a document; a specifying means for specifying an object region including a predetermined object in the image data; a removal means for removing noise having a size smaller than a size specified by a first threshold from the object region; a setting unit that sets the first threshold value for each of the object regions identified by the identifying unit in accordance with the size of the object region. picture, the setting means sets the first threshold value in accordance with the size of the object region and a predetermined condition other than the size; the object area is a rectangular area surrounding the predetermined object, the predetermined condition is a condition regarding the ratio of the long side to the short side of the rectangle, the setting means sets the first threshold value in accordance with the size of the object region; The setting means changes the set first threshold value so that the first threshold value becomes smaller as the ratio approaches 1. An image processing device is provided. [Effects of the Invention]

[0007] According to the present invention, noise removal from image data can be performed more appropriately. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 2A is a schematic diagram of an image processing system according to an embodiment, and a functional block diagram of a server. [Figure 2] 10 is a flowchart showing an example of the operation of the entire system. [Figure 3] 10A and 10B are flowcharts showing an example of processing by a server. [Figure 4] 6A and 6B are diagrams for explaining how a character area is identified in scanned image data. [Figure 5] FIG. 10 is a diagram for explaining setting of a noise removal threshold value according to the size of an object region. [Figure 6]FIG. 10 is a diagram showing a specific example of an object area CA for explaining the flow of noise removal processing. [Figure 7] 10A and 10B are diagrams showing specific examples of situations in which erroneous removal of characters occurs. [Figure 8] 10 is a flowchart showing an example of processing by a server. [Figure 9] FIG. 10 is a diagram showing a specific example of removing an isolated point. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments will be described in detail with reference to the accompanying drawings. Note that the following embodiments do not limit the scope of the invention claimed. Although multiple features are described in the embodiments, not all of these multiple features are necessarily essential to the invention, and multiple features may be combined arbitrarily. Furthermore, in the accompanying drawings, the same reference numerals are used to designate the same or similar components, and redundant explanations will be omitted.

[0010] <<First embodiment>> <System Overview> FIG. 1(a) is a schematic diagram of an image processing system SY (hereinafter, referred to as system SY) according to one embodiment. The system SY includes a scanner 10, a communication device 20, and a server 30. For example, in the system SY, the scanner 10 scans an original document and generates image data, which it then transmits to the communication device 20. The communication device 20 transmits the image data received from the scanner 10 to the server 30 via a network such as the Internet. The server 30 performs noise reduction processing, described below, on the received image data, and provides the image data resulting from the processing to a user's terminal (e.g., the communication device 20). In this way, the server 30 is a server capable of providing users with a cloud service that performs noise reduction processing on image data. Specific configuration examples of each element are described below.

[0011] The scanner 10 is an image reading device that can optically read an original document, generate image data, and output the data to an external device. The scanner 10 is connected to a communication device 20 via a wired or wireless connection, and transmits the image data to the communication device 20. The scanner 10 may be a multifunction peripheral (MFP) that has multiple functions, such as an image forming function and a fax function, in addition to the function of a scanner.

[0012] The communication device 20 transmits the image data received from the scanner 10 to the server 30. The communication device 20 may be, for example, a personal computer (PC), a smartphone, a tablet PC, a notebook PC, a mobile phone, or an information processing device having similar processing capabilities and capable of communicating with the outside. The communication device 20 includes a control unit 21, an operation reception unit 23, a display 24, and a communication IF (Interface) 25.

[0013] The control unit 21 performs overall control of the communication device 20. The control unit 21 includes a CPU (Central Processing Unit) 21a, a ROM (Read Only Memory) 21b, and a RAM (Random Access Memory) 21c. The CPU 21a (processor) executes arithmetic processing in accordance with programs stored in the ROM 21b, using the RAM 21c as a work area, thereby realizing various functions of the communication device 20. In this embodiment, the programs stored in the ROM 21b include an application program for uploading image data generated by the scanner 10 reading a document to the server 30.

[0014] The operation reception unit 23 receives user operations for implementing various processes. The operation reception unit 23 includes, for example, physical buttons, a touch panel, a mouse, a keyboard, etc. The display 24 displays a UI (User Interface) for receiving operations for implementing various processes, and displays various information such as the results of executed processes.

[0015] The communication IF 25 is an interface for communicating with an external device. The communication IF 25 communicates with the external device via at least one of wired communication and wireless communication. In this embodiment, the communication device 20 is not only connected to the scanner 10 via the communication IF 25, but also connected to the Internet via the communication IF 25. In addition, the communication IF 25 may be capable of communicating via USB (Universal Serial Bus), Ethernet (registered trademark), wireless LAN (wireless local area network), NFC (Near Field Communication), Bluetooth (registered trademark), infrared communication, etc.

[0016] The scanner 10 and the communication device 20 may be independent devices as shown in Fig. 1(a), or they may be a single device. Specifically, the scanner 10 may be a multifunction peripheral that includes the communication device 20 and has a function for communicating with external devices via the Internet.

[0017] The server 30 performs noise removal processing (described later) on the image data received from the communication device 20. In other words, the server 30 is a specific example of an image processing device. The server 30 is realized, for example, by one or more information processing devices functioning as servers on the Internet. The server 30 includes a control unit 31 and a communication IF 37.

[0018] The control unit 31 includes a CPU 31a, a ROM 31b, and a RAM 31c. The CPU 31a (processor) executes arithmetic processing in accordance with a program 32 stored in the ROM 31b, using the RAM 31c as a work area, thereby realizing various functions of the server 30. In this embodiment, the programs stored in the ROM 31b include a program for executing a noise removal process, which will be described later. The communication IF 37 performs data communication with external devices via the Internet.

[0019] Fig. 1(b) is a functional block diagram of the server 30. The CPU 31a reads a program stored in the ROM 31b into the RAM 31c and executes it, thereby realizing the functions of the functional units shown in Fig. 1(b).

[0020] The acquisition unit 33 acquires image data generated by the scanner 10 reading a document from the communication device 20 via the Internet. The identification unit 34 identifies an object area containing a predetermined object in the image data acquired by the acquisition unit 33. The setting unit 35 sets a threshold value for the size of noise to be removed for each object area identified by the identification unit 34. The removal unit 36 ​​removes noise smaller than the threshold value set by the setting unit 35 from the image data. Note that the functional units are merely examples, and the functions of each functional unit may be integrated, or the function of one functional unit may be divided into multiple functional units.

[0021] The functions of the control unit 21 of the communication device 20 and the control unit 31 of the server 30 can be realized by either hardware or software. For example, the functions of the control unit 21 and the control unit 31 may each be realized by one or more CPUs using one or more memories to execute a predetermined program, as described above. Alternatively, the functions of the control unit 21 and the control unit 31 may each be realized by a known semiconductor device, such as a PLD (programmable logic device) or an ASIC (application-specific semiconductor integrated circuit). Alternatively, processing may be performed by a combination of a CPU and a hardware circuit. Furthermore, although the control unit 21 and the control unit 31 are shown here as single elements, the control unit 21 and the control unit 31 may each be divided into two or more elements as necessary.

[0022] <System processing example> 2 is a flowchart showing an example of the operation of the entire system SY. In detail, the flowchart shows an example of the operation when the user operates the communication device 20 to cause the scanner 10 to generate image data of the document, and then the user performs OCR processing of the document using a cloud service provided by the server 30. Below, each step will be simply referred to as S21, etc.

[0023] In S21, the control unit 21 of the communication device 20 transmits a document reading instruction to the scanner 10. The control unit 21 transmits the instruction to the scanner 10 via the communication IF 25 based on the reception of a user operation by the operation reception unit 23.

[0024] In S11, the scanner 10 receives an instruction to scan a document from the communication device 20. In S12, the scanner 10 executes a document scanning operation to generate image data of the document (hereinafter, sometimes referred to as scanned image data). The scanned document may include at least one or more objects. Here, examples of objects include letters, numbers, marks, pictures, symbols, and figures. In one embodiment, an object is visually distinguishable from its background and has some meaning or information attached to it. Specific examples of documents include homework sheets, meeting materials, and forms. Documents are not limited to printed characters and figures, but may also include handwritten characters and figures. Thereafter, in S13, the scanner 10 transmits the scanned image data generated in S12 to the communication device 20.

[0025] In S22, the control unit 21 receives the scanned image data from the scanner 10 via the communication IF 25. Thereafter, in S23, the control unit 21 transmits the scanned image data received in S22 to the server 30 via the communication IF 25. For example, the control unit 21 executes an application program for uploading the scanned image data to the server 30, and causes the display 24 to display a screen for transmitting the scanned image data. Then, upon receiving instructions from the user on the transmission screen to select the scanned image data to be transmitted and to perform noise reduction processing, the control unit 21 transmits the target scanned image data to the server 30.

[0026] In S31, the control unit 31 of the server 30 receives the read image data transmitted from the communication device 20 via the communication IF 37. That is, the acquisition unit 33 of the control unit 31 acquires the image data to be subjected to noise removal.

[0027] In S32, the identification unit 34 of the control unit 31 identifies an object region that includes a predetermined object in the image data acquired in S31. In this embodiment, the identification unit 34 identifies a character region that includes text as the object region. This will be described in detail later.

[0028] In S33, the setting unit 35 of the control unit 31 sets a threshold value for the size of noise to be removed (hereinafter, sometimes referred to as a size threshold value) for each object region identified in S31. In this embodiment, the noise size is the two-dimensional size of the noise, and the size threshold value is a threshold value for the dimensions of the noise in a predetermined direction (in the following explanation, the short side direction and long side direction of the object region). Details of the processing will be described later.

[0029] In S34, the removal unit 36 ​​of the control unit 31 performs noise removal. Specifically, the removal unit 36 ​​removes noise smaller than the size specified by the threshold set in S33 from the object region. This will be described in detail later.

[0030] In S35, the control unit 31 performs OCR processing on the image data after noise removal (hereinafter, may be referred to as post-removal image data). Since the OCR processing can use known technology, a description thereof will be omitted.

[0031] In S36, the control unit 31 transmits the processing result. Specifically, the control unit 31 transmits the image data that has been subjected to the OCR processing in S35 to the communication device 20 via the communication IF 37 as the processing result.

[0032] In S24, the control unit 21 of the communication device 20 receives the OCR processed image data as a processing result from the server 30. Thereafter, in S25, the control unit 21 notifies the user that the OCR processing has been completed by, for example, displaying a message to that effect on the display 24.

[0033] <Server 30 processing example> (Identifying the object area) A specific example of processing by the server 30 will be described below. Fig. 3(a) is a flowchart showing a specific example of the processing of S32 in Fig. 2. Fig. 4 is a diagram for explaining how a character area is identified in scanned image data by the processing of the flowchart in Fig. 3(a). Various methods have been proposed for identifying a character area, such as a rule-based method or a method using machine learning such as deep learning, but here we will explain an example of a rule-based method.

[0034] In S321, the specifying unit 34 generates binary image data (image data in which each pixel value is expressed in two gradations) of the read image. Since a known technique can be used to generate the binary image data, a detailed description will be omitted. As an example, the binary image data is generated by dividing each pixel into dark pixels (black, etc.) and light pixels (white, etc.) by comparing the pixel value of each pixel with a predetermined threshold.

[0035] In S322, the identification unit 34 extracts a cluster of dark pixels (hereinafter, sometimes referred to as a dark pixel cluster) from the binary image data. A dark pixel cluster is essentially a collection of dark pixels that exist within a predetermined rectangular area. The identification unit 34 determines that two dark pixels belong to a common dark pixel cluster if the distance between them is within a predetermined range. The identification unit 34 repeatedly compares the dark pixels and identifies a rectangle that includes all the dark pixels that should be included in the common dark pixel cluster as a single dark pixel cluster. The dark pixel cluster may be the smallest rectangle that includes all the dark pixels that should be included in the common dark pixel cluster, or it may be a rectangle that is a predetermined size larger than the smallest rectangle. Through this process, for example, a word or sentence written on the same line is determined to be a single dark pixel cluster. The dark pixel cluster may be an object region within the scanned image data. In other words, the object region within the scanned image data is identified by extracting the dark pixel cluster.

[0036] The left side of Fig. 4 shows dark pixels BL present in the binary image data DA generated from the scanned image data in S321. Here, each pixel constituting a character in the binary image data DA is a dark pixel BL. The right side of Fig. 4 also shows the dark pixel block (i.e., object area CA) extracted in S322 as a solid-line rectangle.

[0037] In this way, the specifying unit 34 performs binarization processing on the read image data to generate binary image data, and specifies the object region by extracting dark pixel blocks from the generated binary image data.

[0038] In addition, in a method using machine learning, for example, input data and training data are linked, and a trained model is generated by training the model using a predetermined algorithm, and a character region as an object region is detected using the trained model. Examples of the predetermined algorithm include an SVM (Support Vector Machine), a decision tree, or a neural network.

[0039] In this embodiment, for example, binary image data obtained by binarizing scanned image data obtained by scanning a document is used as input data, and character regions included in the binary image data are identified and output. The deviation between the output result and data (teaching data) indicating the actual correct character region is then calculated. The parameters of the learning model are updated to reduce the deviation.

[0040] (Setting the size threshold for noise removal) Fig. 5 is a diagram for explaining the setting of a size threshold for noise removal according to the size of the object area in S33 of Fig. 2. Fig. 5 shows, for a plurality of object areas CA of different sizes, their vertical width ST and the size SC of the smallest character type (a period in the figure) assumed when the vertical width ST is assumed to be the character height. Specifically, when the vertical width ST is 4 pt, the size SC is 0.4 pt, and when the vertical width ST is 28 pt, the size SC is 2.8 pt. Fig. 5 also shows the size threshold S for each of the plurality of object areas CA.

[0041] As described above, the setting unit 35 of the control unit 31 sets the size threshold S for each object region identified by the identification unit 34, depending on the size of the object region. In this embodiment, the setting unit 35 reduces the size threshold S as the size of the object region decreases.

[0042] In the example of FIG. 5, the size threshold S is set according to the dimension of the short side of the rectangular object area CA, which affects the size of the object area. Specifically, the size threshold S is set to one-tenth the dimension of the short side of the object area CA. For example, if the object area CA of interest is a rectangle with its short side vertical and its long side horizontal, the size threshold S is set to one-tenth the dimension of the vertical width of the object area CA. This allows noise smaller than a period when the vertical width of the rectangle is assumed to be the character height to be targeted for removal. In other words, the setting unit 35 reduces the size threshold S so that the size specified by the size threshold S is smaller than the size of the smallest character type (e.g., a period) in the font size of the characters included in the object area. If the object area is a character string, it is estimated that the short side of the rectangle represents the character height and the long side represents the direction in which the characters are arranged. Therefore, by setting the size threshold S based on the dimension of the short side, it is possible to set the size threshold S to match the characters that make up the character string.

[0043] As will be described in detail later, in this embodiment, isolated points that fit within a square whose one side is the size threshold S set as described above are removed as noise. In other words, the size threshold S here can be said to be a threshold for the vertical and horizontal dimensions of isolated points, which will be described later.

[0044] In this embodiment, a common size threshold S for both the short and long sides is set according to the dimension of the short side of the object region rectangle, which affects the size of the object region. However, other methods may be used to set the size threshold for noise removal according to the size of the object region. For example, a size threshold for the short side direction may be set according to the dimension of the short side of the object region, and a size threshold for the long side direction may be set according to the dimension of the long side of the object region. Alternatively, a size threshold may be set only for the short side of the object region. Setting the size threshold in this manner makes it possible to remove noise caused by, for example, linear stains that are long in the long side direction.

[0045] In addition, in this embodiment, the size threshold S is set to 1 / 10 of the dimension of the short side of the object area CA, but taking into account size measurement errors, etc., the size threshold S may also be set to, for example, 1 / 10 to 1 / 15 of the dimension of the short side of the object area CA.

[0046] (Removing noise from object areas) Fig. 3(b) is a flowchart showing a specific example of the processing in S34 in Fig. 2. Fig. 6 is a diagram showing a specific example of an object area CA to explain the processing flow in S34.

[0047] In S341, the removal unit 36 ​​of the control unit 31 performs binarization processing on the object area CA identified by the identification unit 34 in S32. As with S321, known techniques can be used as the binarization processing method. Note that here, the binarization processing may be performed not only on the object area CA but also on the entire read image data. Furthermore, if there is binarized image data generated by performing binarization processing on the read image data to identify the object area CA in S32, the binarized data of the target object area CA may be obtained from that binarized image data.

[0048] In S342, the removal unit 36 ​​of the control unit 31 detects isolated points in the object area CA binarized in S341. Many methods for detecting isolated points have been proposed, so detailed explanations are omitted here. One example is to shift a predetermined area by one pixel in raster scan order, and if the number of dark pixels in that area is equal to or less than a threshold, detect the dark pixels in the area as isolated points. The predetermined area may be, for example, a square area with a side length equal to the size threshold S set in S33. Alternatively, in addition to the aforementioned square area (denoted as square area A), isolated points may be detected using a square area B that contains square area A and has the same center point. For example, if the number of dark pixels in square area A is equal to or less than a threshold THA and the number of dark pixels in square area B is equal to or less than a threshold THB, the dark pixels in square area A may be detected as isolated points.

[0049] The top row of Fig. 6 shows an example of an object area CA binarized in S341. The middle row of Fig. 6 shows isolated point areas RA that surround the isolated points detected in S342 in the object area CA detected in S341. Here, the isolated point areas RA are the smallest rectangular areas that contain the isolated points. For ease of explanation, the isolated point areas RA are surrounded by long dashed two-dot lines in the figure. The "Size" shown in the figure indicates the size of each isolated point area RA. The figure includes two isolated point areas RA that are 2 points in size (the dimension of one side) and one isolated point area RA that is 1 point in size. Of these, the 2-point isolated point area RA should not be removed because it is an element that constitutes a character, while the 1-point isolated point area RA is a stain and needs to be removed.

[0050] In S343, the removal unit 36 ​​of the control unit 31 determines whether the size of the isolated point region RA detected in S342 is smaller than the size threshold S set in step S120. If the size of the isolated point region RA is smaller than the size threshold S, the removal unit 36 ​​proceeds to S344; otherwise, the removal unit 36 ​​ends the flowchart. In this embodiment, if the isolated point region RA fits within a square whose one side dimension is the size threshold S, the removal unit 36 ​​determines that the size of the isolated point region RA is smaller than the size specified by the size threshold S.

[0051] In S344, the removal unit 36 ​​of the control unit 31 removes isolated points contained in isolated point areas RA that are determined to be smaller than the size threshold S. Note that known techniques can be used to remove the isolated points. For example, the removal unit 36 ​​removes the isolated points by rewriting the pixel values ​​of the isolated points in the isolated point areas RA with light pixel values. In the example at the bottom of Figure 6, the 1-point isolated point area RA is removed because it is smaller than the size threshold S (2 points in this example), while the 2-point isolated point area RA is left unremoved because it is not smaller than the size threshold.

[0052] As described above, according to this embodiment, the size threshold S is set for each object area CA identified by the identification unit 34 according to the size of the object area CA. Setting the size threshold S according to the size of the object area CA allows for more appropriate noise removal from image data. From another perspective, the removal unit 36 ​​removes noise contained in the object area CA with an upper limit of a size according to the size of the object area CA. Since the upper limit of the size of the noise to be removed is based on the size of the object area CA, erroneous removal of characters and the like can be suppressed. Furthermore, by reducing the size threshold S as the object area CA becomes smaller, erroneous removal of elements constituting characters can be suppressed when the size of the characters contained in the object area CA is small. Furthermore, when the size of the characters contained in the object area CA is large, even relatively large noise can be removed.

[0053] <<Second embodiment>> The second embodiment will be described below. In the first embodiment, a size threshold is set for each identified object region according to the size of the object region. Here, if the object region is a region containing graphics and characters, the size threshold may not reflect the size of the characters contained in the object region. In such cases, the size threshold may be set to a value larger than an appropriate value for the size of the characters contained in the object region. As a result, elements that make up the characters may be erroneously removed during noise removal. Therefore, in this embodiment, noise removal is performed according to the attributes of the objects contained in the object region. Hereinafter, components similar to those in the first embodiment will be assigned the same reference numerals and descriptions will be omitted.

[0054] FIG. 7 shows a specific example of a situation in which erroneous character removal occurs when an object area includes a character and a graphic. Here, for object area CA1 including graphic Sh and character Ch1 in FIG. 7, and object area CA2 including character Ch2 and noise N, a size threshold S (3 pt) is set based on the vertical width (30 pt) of each object area. In this case, it is possible to remove noise N contained in isolated point area RA that is smaller than size threshold S. However, because the size (2 pt) of isolated point area RA including character Ch2 is smaller than size threshold S, there is a risk of erroneous removal of character Ch2. Therefore, in this embodiment, the size threshold S is changed based on predetermined conditions using the process described below.

[0055] FIG. 8 is a flowchart showing a specific example of the process of S33 in FIG. In S331, the setting unit 35 sets the size threshold S according to the size of the object region. The process in S331 corresponds to the process in S33 in the first embodiment.

[0056] In S332, the setting unit 35 changes (resets) the size threshold S set in S331 based on a predetermined condition.

[0057] The predetermined condition may be, for example, as follows: The setting unit 35 may change the size threshold S based on the area of ​​each object region. For example, if the area of ​​the object region is larger than a predetermined threshold, the size threshold S may be set smaller than the value set in S331. As an example, if the area of ​​the object region is larger than the predetermined threshold, the setting unit 35 may determine a new size threshold SN by dividing the size threshold S set in S331 by a predetermined value (>1).

[0058] If an object area on a document is too large, for example, half the size of the document, it is unlikely to be a text area. Therefore, if the area of ​​the object area is equal to or greater than the threshold, it can be determined that the object area under consideration is not a text area. If the object area is not a text area, the size threshold can be reset to a size smaller than the size threshold set in S311 to prevent erroneous removal of text.

[0059] The predetermined condition may be, for example, as follows: The setting unit 35 may change the size threshold S based on the aspect ratio of each object region. For example, if the height of a rectangular object region of interest is the short side, the aspect ratio of the object region may be set to 1:A (>1). In this case, the setting unit 35 may set the new size threshold SN to a value obtained by subtracting 1 / A from the size threshold S. If the object region includes a character region, the character region is likely to be composed of multiple characters, and therefore the aspect ratio of the character region is unlikely to be 1:1. Therefore, the closer the aspect ratio of the object region is to 1:1, i.e., the closer the ratio of the long side to the short side of the object region is to 1, the less likely it is that the object region is a character region. Therefore, by resetting the size threshold to a size smaller than the size threshold set in S311 depending on the possibility that the object region is not a character region, erroneous removal of characters can be suppressed.

[0060] The predetermined condition may be a combination of the two conditions described above, or may be a combination of other conditions. For example, the predetermined condition may be the ratio of the number of dark pixels to the number of pixels in the object region. In other words, if the ratio of dark pixels is high, it can be assumed that the object region contains a filled-in figure or the like. Therefore, if the ratio of the number of dark pixels to the number of pixels in the object region is equal to or greater than a threshold, the size threshold may be set to a smaller value.

[0061] As described above, according to this embodiment, when it is determined that an object region contains graphics, the size threshold is set smaller than when the object region is composed of only text. In other words, the size threshold, which is set according to the size of the object region, is changed according to the attributes of the object region, which are conditions other than size. This allows for more appropriate noise removal from image data according to the attributes of the object region.

[0062] Specifically, a size threshold is set for each object region of interest based on the size of the object region identified from the binary image data of the scanned image data, as well as on certain conditions, such as the aspect ratio and area of ​​the object region. This allows the size threshold to be reset when the object region contains something other than text, thereby preventing erroneous removal of text contained in graphic regions.

[0063] <Other embodiments> In the above embodiment, the server 30 performs noise removal on the image data. However, the scanner 10, which has an OCR function, may perform noise removal as pre-processing of the OCR process. Alternatively, the communication device 20, which can communicate with the scanner 10, may perform noise removal on the image data. When the scanner 10 or the communication device 20 performs noise removal on the image data, a program to be executed by the control unit (not shown) of the scanner 10 or the control unit 21 of the communication device 20 may be provided via the Internet or the like. Furthermore, the scanner 10 or the communication device 20 may execute such a program to realize the above process without communicating with an external server or the like via the Internet.

[0064] In the above embodiment, image processing is performed on scanned image data generated by scanning a document using the scanner 10 or a multifunction peripheral having multiple functions in addition to the scanner function. However, scanned image data may also include image data generated by capturing an image of a document using a device with a photographing function, such as a personal computer (PC), smartphone, tablet PC, or notebook PC. In other words, capturing an image of a document using an imaging device such as a camera may also be included in document reading.

[0065] Furthermore, in the above embodiment, the scanned image data is binarized in S341. However, noise removal processing can also be performed without binarization (i.e., on a multi-valued image). This is because if the non-text areas (background areas) of a document are close to the text color, binarization processing can result in the text and background having the same pixel values, potentially resulting in the loss of text information. Therefore, when processing a multi-valued image, the isolated point area can be removed from the object area by replacing it with the pixel values ​​of the background surrounding the isolated point area. When performing isolated point removal on a multi-valued image, the background of the scanned image does not necessarily have a predetermined fixed pixel value (e.g., a light pixel value in a binarized image). Therefore, differences in pixel values ​​between the replaced isolated point area and the surrounding area can adversely affect OCR character recognition accuracy, similar to noise. Therefore, by extracting background pixels from the surrounding pixel values ​​of the isolated point area, the adverse effect of isolated point removal using predetermined fixed pixel values ​​on OCR character recognition accuracy can be reduced.

[0066] For example, the removal unit 36 ​​of the control unit 31 removes isolated points from the object area by overwriting pixel values ​​of isolated point areas smaller than a size threshold with pixel values ​​of the background extracted from the surrounding area of ​​the isolated point area. FIG. 9 illustrates a specific example of isolated point removal. FIG. 9 illustrates an example in which an isolated point area RA is removed by filling it with surrounding pixels. Here, the removal unit 36 ​​removes isolated points from the object area by overwriting isolated point areas RA smaller than a size threshold S with predetermined fixed pixel values ​​(e.g., light pixel values ​​of a binarized image) such as pixel values ​​of the background area RB. Here, the background area RB is an area that has the same shape and size as the isolated point area RA and borders the isolated point area RA on one side. In more detail, the removal unit 36 ​​performs a process of overwriting any pixel value of the isolated point area RA with the pixel value of the corresponding point in the background area RB for all pixels in the isolated point area RA, thereby removing the isolated points. As an example of corresponding points, point A1 corresponds to point B2, point A2 corresponds to point B1, point A3 corresponds to point B4, point A4 corresponds to point B3, and point OA corresponds to point OB. In this way, the removal unit 36 ​​can rewrite the isolated points to be removed into the background.

[0067] In the above embodiment, the noise size is the two-dimensional size of the noise, and the size threshold is a threshold for the dimension of the noise in a predetermined direction. However, the definitions of the noise size and the size threshold are not limited. For example, the noise size may be defined as the diagonal length of the smallest rectangle that contains the noise, and the size threshold may be defined as the value for the diagonal length. Alternatively, the noise size may be defined as the area of ​​the noise (isolated point) itself or the area of ​​the smallest rectangle that contains the noise, and the size threshold may be defined as the value for the area.

[0068] In addition, in the above embodiment, noise is removed from the scanned image data as preprocessing for OCR processing, but processing may be performed with the primary purpose of removing noise from the scanned image data. For example, before the communication device 20 stores the scanned image data acquired from the scanner 10, the communication device 20 may perform the noise removal processing according to the above embodiment, and store the image data from which noise has been removed in a storage area of ​​the communication device 20 as image data of the document.

[0069] The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0070] The invention is not limited to the above-described embodiments, and various changes and modifications can be made without departing from the spirit and scope of the invention. Accordingly, the following claims are appended to apprise the public of the scope of the invention.

Claims

1. an acquisition means for acquiring image data generated by reading a document; a specifying means for specifying an object region including a predetermined object in the image data; a removal means for removing noise having a size smaller than a size specified by a first threshold from the object region; a setting unit that sets the first threshold value for each of the object regions identified by the identifying unit in accordance with the size of the object region, the setting means sets the first threshold value in accordance with the size of the object region and a predetermined condition other than the size; the object area is a rectangular area surrounding the predetermined object, the predetermined condition is a condition regarding the ratio of the long side to the short side of the rectangle, the setting means sets the first threshold value in accordance with the size of the object region; the setting means changes the set first threshold value so that the first threshold value becomes smaller as the ratio approaches 1; 1. An image processing device comprising:

2. An acquisition means for acquiring image data generated by reading a document; a specifying means for specifying an object region including a predetermined object in the image data; a removal means for removing noise having a size smaller than a size specified by a first threshold from the object region; a setting unit that sets the first threshold value for each of the object regions identified by the identifying unit in accordance with the size of the object region, the setting means sets the first threshold value in accordance with the size of the object region and a predetermined condition other than the size; the predetermined condition is a condition regarding the area of ​​the object region, the setting means sets the first threshold value in accordance with the size of the object region; the setting means changes the set first threshold value when the area is larger than the second threshold value so that the set first threshold value is smaller than when the area is equal to or smaller than the second threshold value; 1. An image processing device comprising:

3. 3. The image processing device according to claim 1, the setting means sets the first threshold to a smaller value as the size of the object region becomes smaller; 1. An image processing device comprising:

4. 3. The image processing device according to claim 1. the object area is a rectangular area surrounding the predetermined object, the setting means sets the first threshold to a smaller value as the dimension of the short side of the rectangle becomes smaller; 1. An image processing device comprising:

5. 3. The image processing device according to claim 1, the predetermined object is a character, the setting means reduces the first threshold value so that the size specified by the first threshold value is smaller than the size of the smallest character type in the font size of characters included in the object region; 1. An image processing device comprising:

6. 6. The image processing device according to claim 1, The removal means is Detecting isolated points as noise included in the object region in the binarized data obtained by binarizing the image data; removing the outliers smaller than a size specified by the first threshold; 1. An image processing device comprising:

7. an acquisition step of acquiring image data generated by reading a document; a specifying step of specifying an object region including a predetermined object in the image data; a removing step of removing noise having a size smaller than a size specified by a first threshold from the object region; a setting step of setting the first threshold value for each of the object regions identified in the identifying step in accordance with the size of the object region, the setting step sets the first threshold value in accordance with the size of the object region and a predetermined condition other than the size; the object area is a rectangular area surrounding the predetermined object, the predetermined condition is a condition regarding the ratio of the long side to the short side of the rectangle, the setting step sets the first threshold value according to a size of the object region; the setting step changes the set first threshold value so that the first threshold value becomes smaller as the ratio approaches 1; An image processing method comprising:

8. An acquisition step of acquiring image data generated by reading a document; a specifying step of specifying an object region including a predetermined object in the image data; a removing step of removing noise having a size smaller than a size specified by a first threshold from the object region; a setting step of setting the first threshold value for each of the object regions identified in the identifying step in accordance with the size of the object region, the setting step sets the first threshold value in accordance with the size of the object region and a predetermined condition other than the size; the predetermined condition is a condition regarding the area of ​​the object region, the setting step sets the first threshold value according to a size of the object region; The setting step changes the set first threshold value when the area is larger than a second threshold value so that the set first threshold value is smaller than when the area is equal to or smaller than the second threshold value. An image processing method comprising:

9. A program for causing at least one computer to function as each means of an image processing device described in any one of claims 1 to 6.

10. A computer-readable storage medium storing a program for causing at least one computer to function as each means of an image processing device described in any one of claims 1 to 6.

Images