Image processing device, image processing method, and program

The image processing apparatus enhances orientation detection accuracy by determining document type and selecting the appropriate method, addressing the issue of reduced accuracy in conventional systems.

JP7947933B2Active Publication Date: 2026-11-11RICOH CO LTD
View PDF 3 Cites 0 Cited by

Patent Information

Application Number
JP2022185736
Authority / Receiving Office
JP · JP
Patent Type
Patents
Current Assignee / Owner
Filing Date
2022-11-21
Publication Date
2026-11-11
Estimated Expiration
2042-11-21

AI Technical Summary

Technical Problem

Conventional top-bottom identification methods for documents do not select the appropriate method based on the document's characteristics, leading to reduced accuracy in orientation detection, especially when multiple document types are scanned together.

Method used

An image processing apparatus that determines the document type based on OCR setting information or a pre-trained deep learning model, and selects the appropriate orientation identification method accordingly, such as OCR or AI-based methods, to enhance accuracy.

Benefits of technology

Accurately identifies the top and bottom orientation of documents by selecting the appropriate method based on document type, improving orientation detection accuracy.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure 0007947933000001
    Figure 0007947933000001
  • Figure 0007947933000002
    Figure 0007947933000002
  • Figure 0007947933000003
    Figure 0007947933000003
Patent Text Reader

Abstract

To accurately identify the top and bottom of a document.SOLUTION: An image processing device includes a document type discrimination unit that discriminates the type of a document read by a scanner, a method setting unit that sets a top and bottom identification method for discriminating the top and bottom of a target image obtained by reading the document by the scanner on the basis of the type of document, and a top and bottom identification unit that discriminates the top and bottom of the target image according to the top and bottom identification method.SELECTED DRAWING: Figure 4
Need to check novelty before this filing date? Find Prior Art

Description

TECHNICAL FIELD

[0001] The present invention relates to an image processing apparatus, an image processing method, and a program.

BACKGROUND ART

[0002] When a document is read by a scanner, the document may be read with its top and bottom reversed, or with its top and bottom oriented sideways. There is a technique for automatically identifying the top-bottom orientation of a document read in such a state and correcting the orientation to a normal top-bottom orientation (hereinafter also referred to as "top-bottom identification").

[0003] For example, Patent Document 1 discloses an invention that identifies top-bottom orientation using Optical Character Recognition (OCR). Further, for example, Patent Document 2 discloses an invention that identifies top-bottom orientation using a Deep Neural Network (DNN) model based on deep learning.

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0004] However, the conventional techniques have a problem that a top-bottom identification method is not selected according to the characteristics of a document. For example, a document containing a large amount of characters such as a text or a form can be identified for its top-bottom orientation with high accuracy by top-bottom identification using OCR. On the other hand, for a document containing almost no characters such as a photograph or a drawing, top-bottom identification using a DNN can identify the top-bottom orientation with higher accuracy.

[0005] In view of the above technical problems, an embodiment of the present invention aims to accurately identify the top-bottom orientation of a document.

MEANS FOR SOLVING THE PROBLEMS

[0006] An image processing apparatus according to one embodiment of the present invention comprises: a document type determination unit for determining the type of document scanned by a scanner; a method setting unit for setting a top / bottom identification method for determining the top / bottom of a target image scanned from a document based on the document type; and a top / bottom identification unit for determining the top / bottom of a target image according to the top / bottom identification method. The document type discrimination unit determines the type of document based on the setting information, which is set when scanning the document with the scanner, indicating whether or not to perform character recognition on the target image. ru. [Effects of the Invention]

[0007] According to one embodiment of the present invention, the top and bottom orientation of a document can be accurately identified. [Brief explanation of the drawing]

[0008] [Figure 1] This figure shows an example of the scanning direction for a document. [Figure 2] This is a block diagram showing an example of the hardware configuration of an image forming apparatus in one embodiment. [Figure 3] This is a block diagram showing an example of the functional configuration of an image forming apparatus in one embodiment. [Figure 4] This is a block diagram showing an example of the functional configuration of the method selection unit in one embodiment. [Figure 5] This figure shows an example of the document type discrimination result in the first embodiment. [Figure 6] This figure shows an example of the setting values ​​for the orientation detection method in the first embodiment. [Figure 7] This is a flowchart showing an example of the method selection process in the first embodiment. [Figure 8] This figure shows an example of the document type discrimination result in the second embodiment. [Figure 9] This figure shows an example of the setting values ​​for the orientation detection method in the second embodiment. [Figure 10] This flowchart shows an example of the method selection process in the second embodiment. [Modes for carrying out the invention]

[0009] The embodiments of this invention will be described in detail below with reference to the drawings. Note that components having the same function are numbered the same in the drawings, and redundant explanations are omitted.

[0010] [overview] Traditionally, when scanning documents, the document may be scanned upside down or sideways. Figure 1 shows an example of the scanning direction of a document. For example, a document that should be scanned in "north" in Figure 1 may be scanned in "south" (i.e., upside down), or in "east" or "west" (i.e., sideways).

[0011] In Figure 1, an example is shown where top and bottom are represented by directions (north, east, west, south). However, any label can be used to represent top and bottom as long as the relationship between the orientation and the label is consistent. For example, the orientation could be represented by an index such as 0 to 3.

[0012] Various orientation recognition methods are used to determine the top and bottom of a document. For example, there is an orientation recognition method based on the processing results of OCR (hereinafter also called "OCR orientation recognition"). Orientation recognition methods that do not use OCR include, for example, an orientation recognition method that utilizes line rectangles, or an orientation recognition method based on the prediction results of DNN (hereinafter also called "AI orientation recognition").

[0013] OCR is not a standard feature included with scanners and may be offered as an optional feature. Therefore, OCR orientation detection cannot be used if the scanner does not have OCR. On the other hand, orientation detection that does not use OCR, such as AI orientation detection (hereinafter also referred to as "non-OCR orientation detection"), can be used regardless of whether OCR is installed or not. Conversely, if OCR is installed, both OCR orientation detection and non-OCR orientation detection can be used.

[0014] Cases where a plurality of orientation detection methods are available may occur not only at the time of device introduction. For example, although the AI-based orientation detection provided as standard was used without installing the OCR option at the initial stage of introduction, a plurality of orientation detection methods become available after the OCR option is installed midway. When the OCR option is installed, it is expected that use cases in which orientation detection is performed simultaneously with OCR will increase. In that case, the number of cases in which documents that are easy to process with OCR are scanned increases, and there is a possibility that orientation detection by OCR can detect the orientation with higher accuracy than AI-based orientation detection.

[0015] However, if a mechanism for automatically switching the orientation detection method when the OCR option is installed is not provided, and the user forgets the operation of manually switching the orientation detection method, the AI-based orientation detection will still be applied even after the OCR option is installed. In this case, some documents have good compatibility with orientation detection by OCR but poor compatibility with AI-based orientation detection, which may degrade not only the accuracy of orientation detection but also the accuracy of OCR.

[0016] In order to avoid the above problem, a method of switching the setting value of the orientation detection method according to whether the OCR option is installed can be considered. However, such a method does not take into account the case where the OCR option is installed but the user intentionally performs orientation detection without applying OCR.

[0017] For example, even if the OCR option is installed, it is possible to select not to perform OCR as before the installation. Therefore, depending on the document and the intended use, the user may intentionally choose to perform orientation detection without performing OCR. For example, when scanning a document that is not suitable for OCR (such as a document containing many photographs or a document in a language not supported by OCR), orientation detection by OCR will be used even though the document is originally unsuitable for OCR, leading to deterioration in the accuracy of orientation detection. In addition, since OCR is applied even when there is no target area for OCR, another problem of increased processing time and file size also occurs.

[0018] Furthermore, in actual use, it is also conceivable that documents suitable for OCR and documents not suitable for OCR are mixed and scanned. However, even if a plurality of orientation identification methods are available, only a single unique orientation identification method can be used for the scanned documents (that is, the orientation identification method cannot be switched for each individual document). For example, even if a plurality of documents are scanned at one time, if OCR orientation identification is set, only OCR orientation identification will be used. Therefore, it is not possible to mix documents that should use OCR orientation identification and documents that should use AI orientation identification and scan them all at once.

[0019] Each orientation identification method has different strengths (in other words, different tendencies of documents that it can identify with high accuracy). For example, OCR orientation identification can identify the orientation with high accuracy for documents written in languages supported by OCR. In addition, AI orientation identification can identify the orientation with high accuracy for documents in languages not supported by OCR, or photographs and the like that contain almost no text. Therefore, even with a method that switches the orientation identification method according to whether the OCR option is installed or not, when scanning a plurality of documents with different tendencies at one time, the orientation identification accuracy may be degraded for some documents.

[0020] An embodiment of the present invention aims to, when a plurality of orientation identification methods are available, select an appropriate orientation identification method according to the characteristics of a document, so as to improve the orientation identification accuracy for scanned documents.

[0021] [First Embodiment] The first embodiment of the present invention is an image forming apparatus that reads a document with a scanner and outputs a scanned image. The image forming apparatus in the present embodiment has a function of identifying the orientation of the image of the document read by the scanner, and correcting the orientation of the scanned image to match the orientation of the document.

[0022] The image forming apparatus in the present embodiment determines the type of the document read by the scanner (hereinafter also referred to as "document type"), and identifies the orientation of the image using an orientation identification method corresponding to the document type.

[0023] In this embodiment, the type of document is determined based on setting information indicating whether or not to use OCR (hereinafter also referred to as "OCR setting information"). OCR is a function generally used for documents with a large amount of text. Therefore, if the user sets the document to use OCR, it is highly likely that the document is suitable for OCR orientation recognition. On the other hand, if the user sets the document not to use OCR, it is highly likely that the document is not suitable for OCR orientation recognition. Accordingly, by selecting the orientation recognition method according to the OCR setting information, the orientation recognition method can be appropriately selected.

[0024] <Hardware Configuration> The hardware configuration of the image forming apparatus in this embodiment will be described with reference to Figure 2. Figure 2 is a block diagram showing an example of the hardware configuration of the image forming apparatus in this embodiment.

[0025] As shown in Figure 2, the image forming apparatus 100 includes a controller 1010, a short-range communication circuit 1020, an engine control unit 1030, an operation panel 1040, and a network interface 1050.

[0026] Of these, the controller 1010 includes the main components of the computer: the CPU 1001, system memory (MEM-P) 1002, northbridge (NB) 1003, southbridge (SB) 1004, ASIC (Application Specific Integrated Circuit) 1005, local memory (MEM-C) 1006, HDD controller 1007, and HD 1008, with the NB 1003 and ASIC 1005 connected by an AGP (Accelerated Graphics Port) bus 1021.

[0027] Of these, the CPU 1001 is a control unit that performs overall control of the image forming apparatus 100. The NB 1003 is a bridge for connecting the CPU 1001 with the MEM-P 1002, SB 1004, and AGP bus 1021, and includes a memory controller that controls reading and writing to the MEM-P 1002, as well as a PCI (Peripheral Component Interconnect) master and an AGP target.

[0028] MEM-P1002 consists of ROM 1002a, which is a memory for storing programs and data that realize the various functions of the controller 1010, and RAM 1002b, which is used for program and data deployment and drawing during memory printing. The programs stored in RAM 1002b may be configured to be provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, CD-R, or DVD.

[0029] SB1004 is a bridge for connecting NB1003 to PCI devices and peripheral devices. ASIC1005 is an integrated circuit (IC) for image processing applications that has hardware elements for image processing and acts as a bridge connecting the AGP bus 1021, PCI bus 1022, HDD 1007, and MEM-C 1006, respectively. This ASIC1005 consists of a PCI target and AGP master, an arbiter (ARB) that forms the core of ASIC1005, a memory controller that controls MEM-C 1006, multiple DMACs (Direct Memory Access Controllers) that perform image data rotation using hardware logic, and a PCI unit that performs data transfer via PCI bus 1022 between the scanner unit 1031 and the printer unit 1032. Note that ASIC1005 may also be connected to a USB (Universal Serial Bus) interface or an IEEE1394 (Institute of Electrical and Electronics Engineers 1394) interface.

[0030] MEM-C1006 is local memory used as a copy image buffer and a code buffer. HD1008 is storage for storing image data, font data used during printing, and forms. HD1008 controls data reading or writing to it according to the control of CPU1001. The AGP bus 1021 is a bus interface for graphics accelerator cards proposed to speed up graphics processing, and by directly accessing MEM-P1002 with high throughput, the graphics accelerator card can be made faster.

[0031] Furthermore, the short-range communication circuit 1020 is equipped with a short-range communication circuit 1020a. The short-range communication circuit 1020 is a communication circuit such as NFC or Bluetooth.

[0032] Furthermore, the engine control unit 1030 is composed of a scanner unit 1031 and a printer unit 1032. The operation panel 1040 includes a panel display unit 1040a, such as a touch panel, which displays current settings and selection screens and accepts input from the operator, and an operation panel 1040b, which consists of a numeric keypad that accepts setting values ​​for image formation conditions such as density settings and a start key that accepts a copy start command. The controller 1010 controls the entire image forming apparatus 100, for example, controlling drawing, communication, and input from the operation panel 1040. The scanner unit 1031 or the printer unit 1032 includes an image processing section, such as error diffusion and gamma conversion.

[0033] Furthermore, the image forming apparatus 100 can be sequentially switched and selected between the document box function, copy function, printer function, and facsimile function using the application switching key on the operation panel 1040. When the document box function is selected, it enters document box mode; when the copy function is selected, it enters copy mode; when the printer function is selected, it enters printer mode; and when the facsimile mode is selected, it enters facsimile mode.

[0034] Furthermore, the network interface 1050 is an interface for data communication using a communication network. The short-range communication circuit 1020 and the network interface 1050 are electrically connected to the ASIC 1005 via the PCI bus 1022.

[0035] <Functional Configuration> The functional configuration of the image forming apparatus in this embodiment will be described with reference to Figure 3. Figure 3 is a block diagram showing an example of the functional configuration of the image forming apparatus in this embodiment.

[0036] The image forming apparatus in this embodiment is a digital color image forming apparatus. As shown in Figure 3, the digital color image forming apparatus 100 comprises a scanner 1, an image processing device 2, an HDD 3, a plotter 4, and an image file format conversion device 5. Furthermore, the digital color image forming apparatus 100 functions as the scanner 1, image processing device 2, HDD 3, plotter 4, and image file format conversion device 5 by executing a program.

[0037] Scanner 1 is a device that reads image data from a document. Scanner 1 sends the read image data to image processing device 2.

[0038] The image processing device 2 determines whether the pixel or pixel block of interest in the image read by the scanner 1 is a character area or a non-character area (i.e., a picture), and whether it is chromatic or achromatic, and performs color reproduction processing appropriate to the original document according to the results of these determinations.

[0039] The image processing device 2 consists of a gamma correction unit 21, a region detection unit 22, a data interface unit 23, a color processing / UCR unit 24, and a printer correction unit 25. The image processing device 2 performs processing to obtain a copied image and a scanned image.

[0040] The gamma correction unit 21 is a means of applying a one-dimensional conversion to each signal in order to equalize the gradation balance of each color in the data read by the scanner 1 (8 bits each for r, g, and b after A / D conversion). For the purpose of this explanation, the converted signal is assumed to be a density linear signal (RGB signal: the signal value representing white is set to 0). The output of the gamma correction unit 21 is sent to the region detection unit 22 and the data interface unit 23.

[0041] The region detection unit 22 includes a character determination unit and a color determination unit (not shown). The character determination unit determines whether the pixel or pixel block of interest in the image is a character region or a non-character region (i.e., a picture). The color determination unit determines whether the pixel or pixel block of interest in the image is chromatic or achromatic.

[0042] The data interface unit 23 is an HDD management interface for temporarily saving the image data processed by the gamma correction unit 21 and the judgment results from the region detection unit 22 to the HDD 3. The image data processed by the gamma correction unit 21 and the judgment results from the region detection unit 22 are sent from the data interface unit 23 to the color processing / UCR unit 24.

[0043] The color processing / UCR unit 24 is a means for selecting color processing or UCR (Under Color Removal) processing based on the determination result from the region detection unit 22.

[0044] The printer correction unit 25 receives the c, m, y, and Bk image signals from the color processing / UCR unit 24, performs gamma correction and dithering processing considering the printer characteristics, and then sends the image to the plotter 4.

[0045] HDD3 temporarily stores the image data processed by the gamma correction unit 21 and the determination results from the region detection unit 22. The image data temporarily stored in HDD3 is read out by the image file format conversion device 5.

[0046] The plotter 4 is a transfer printing unit. The plotter 4 transfers the image data output by the image processing device 2 onto the image forming medium.

[0047] The image file format conversion device 5 reads the image data processed by the gamma correction unit 21 (hereinafter also referred to as the "target image") which has been temporarily stored on the HDD 3, and performs orientation recognition on the image data. Using the result obtained from orientation recognition, the image file format conversion device 5 converts the image data into a file format such as PDF (Portable Data Format) or Office Open XML Document format used in Microsoft Word (registered trademark).

[0048] The image file format conversion device 5 consists of a format selection unit 51, an orientation identification unit 52, and a file format conversion unit 53. The image file format conversion device 5 performs processing for orientation identification. More precisely, the processing performed by the image file format conversion device 5 is to perform orientation identification on the image data and then convert the file format using the orientation identification result.

[0049] The method selection unit 51 selects the orientation identification method to be applied by the subsequent orientation identification unit 52. The orientation identification method selected by the method selection unit 51 is input to the orientation identification unit 52 along with the image data read from the HDD3.

[0050] The orientation identification unit 52 applies preprocessing to the image data read from the HDD3 according to the orientation identification method selected by the method selection unit 51, and performs orientation identification according to the orientation identification method selected by the method selection unit 51. Preprocessing according to the orientation identification method includes binarization or scaling. The orientation identification result will be one of north, east, west, or south. The orientation identification result obtained by the orientation identification unit 52 is output to the file format conversion unit 53.

[0051] The file format conversion unit 53 uses the orientation identification result output from the orientation identification unit 52 to convert the image data read from the HDD3 into a file format such as PDF or Office Open XML Document.

[0052] The functional configuration of the method selection unit in this embodiment will be described with reference to Figure 4. Figure 4 is a block diagram showing an example of the functional configuration of the method selection unit in this embodiment.

[0053] The format selection unit 51 consists of a document type discrimination unit 511, a format setting unit 512, and a format update unit 513.

[0054] The document type determination unit 511 determines the document type of the image data read from the HDD3. The document type determination result determined by the document type determination unit 511 is input to the format setting unit 512 along with the image data read from the HDD3. Note that if the image forming apparatus 100 does not have multiple orientation recognition methods, document type determination does not need to be performed. If document type determination is not performed, only the image data read from the HDD3 is input to the format setting unit 512.

[0055] The type and number of types of documents identified by the document type discrimination unit 511 are arbitrary, but examples include classifying documents into those with a lot of text and numbers, such as documents and forms, and those with little text and numbers, such as photographs and drawings. Alternatively, it is also possible to classify documents into more detailed types.

[0056] The method for identifying the document type is arbitrary and can be determined according to the document types and number of documents to be identified. In this embodiment, setting information indicating whether or not to use OCR is used as a criterion for determining the document type, and an example of classifying documents into "documents with many characters or numbers" and "documents with few characters or numbers" is described.

[0057] Figure 5 shows an example of the document type discrimination result in this embodiment. As shown in Figure 5, in this embodiment, when the OCR setting information indicates that OCR should be used, the document type is identified as "a document with many characters or numbers." On the other hand, when the OCR setting information indicates that OCR should not be used, the document type is identified as "a document with few characters or numbers." This is because OCR has the characteristic of being generally used for documents with many characters.

[0058] The format setting unit 512 sets the orientation identification method pre-associated with each document type based on the document type identification result determined by the document type identification unit 511. The setting value of the orientation identification method set by the format setting unit 512 is input to the format update unit 513 along with the image data read from the HDD3.

[0059] For each type of document that can be output by the document type discrimination unit 511, it is necessary to pre-associate the orientation recognition method to be set. Figure 6 shows an example of the setting value for the orientation recognition method in this embodiment. As shown in Figure 6, in this embodiment, if the document type is a "document with many characters and numbers", the orientation recognition method is set to OCR orientation recognition. On the other hand, if the document type is a "document with few characters and numbers", the orientation recognition method is set to AI orientation recognition. For "documents with few characters and numbers", a non-OCR orientation recognition method other than AI orientation recognition may be associated.

[0060] Furthermore, if there are three or more types of documents that can be output, a common orientation recognition method may be assigned to multiple document types. Also, if the image forming apparatus 100 does not have the capability to utilize multiple orientation recognition methods, the method setting unit 512 only needs to set the only orientation recognition method that is installed.

[0061] The format update unit 513 presents the user with the top / bottom identification method setting value set by the format setting unit 512 based on the document type identification result, as well as the available top / bottom identification method options, and updates the top / bottom identification method setting value according to the user's selection. However, updating the top / bottom identification method setting value is not mandatory. If the user does not select another top / bottom identification method, the format update unit 513 outputs the top / bottom identification method setting value set by the format setting unit 512 as is.

[0062] The orientation identification method settings updated (or not updated) by the method update unit 513 are input to the orientation identification unit 52 along with the image data read from the HDD3.

[0063] <Processing Procedure> The method selection process performed by the method selection unit 51 in this embodiment will be described with reference to Figure 7. Figure 7 is a flowchart showing an example of the method selection process in this embodiment.

[0064] In step S1, the document type discrimination unit 511 receives the image data input to the method selection unit 51. This image data is the image data processed by the gamma correction unit 21 and temporarily stored in the HDD 3. Next, the document type discrimination unit 511 acquires the OCR setting information. The OCR setting information is set by the user when scanning the document with the scanner 1.

[0065] Next, the document type determination unit 511 determines the document type of the document scanned by scanner 1 based on the acquired OCR setting information. Then, the document type determination unit 511 sends the image data and the document type determination result to the format setting unit 512.

[0066] In this embodiment, as shown in Figure 5, when the OCR setting information indicates that OCR should be used, the document type is identified as "a document with many characters and numbers." On the other hand, when the OCR setting information indicates that OCR should not be used, the document type is identified as "a document with few characters and numbers."

[0067] In step S2, the format setting unit 512 receives image data and the document type determination result from the document type determination unit 511. Next, the format setting unit 512 determines whether the document type determination result is a "document with many characters or numbers". If the document type determination result is a "document with many characters or numbers" (YES), the format setting unit 512 proceeds to step S3. On the other hand, if the document type determination result is a "document with few characters or numbers" (NO), the format setting unit 512 proceeds to step S4.

[0068] In step S3, the format setting unit 512 sets the orientation recognition method to OCR orientation recognition. This is because if the document type is a "document with many characters and numbers," there is a high possibility that the orientation can be accurately recognized by OCR orientation recognition. Next, the format setting unit 512 sends the image data and the orientation recognition method setting value to the format update unit 513.

[0069] In step S4, the format setting unit 512 sets the orientation recognition method to AI orientation recognition. This is because, when the document type is a "document with few characters or numbers," AI orientation recognition is more likely to be able to recognize the orientation accurately. Note that in step S4, it may also be set to a non-OCR orientation recognition method other than AI orientation recognition. Next, the format setting unit 512 sends the image data and the orientation recognition method setting value to the format update unit 513.

[0070] In step S5, the method update unit 513 receives image data and orientation recognition method settings from the method setting unit 512. Next, the method update unit 513 presents the orientation recognition method settings and the available orientation recognition method options to the user. For example, the method update unit 513 displays a screen on the operation panel 1040 of the image forming apparatus 100 that allows the user to select an orientation recognition method.

[0071] In step S6, the method update unit 513 determines whether the user has changed the orientation identification method. If the user has changed the orientation identification method (in other words, the user has performed an operation on the operation panel 1040 to select an orientation identification method different from the set value), the method update unit 513 proceeds to step S7. On the other hand, if the user has not changed the orientation identification method, the method update unit 513 proceeds to step S8.

[0072] In step S7, the method update unit 513 obtains the orientation identification method selected by the user. Next, the method update unit 513 updates the orientation identification method setting value with the newly obtained orientation identification method.

[0073] In step S8, the format update unit 513 outputs the image data and the orientation recognition method setting to the format selection unit 51. The image data and orientation recognition method setting output from the format selection unit 51 are input to the orientation recognition unit 52.

[0074] <Effects> In this embodiment, the image forming apparatus 100 sets an orientation identification method for identifying the top and bottom of an image based on the type of document scanned by the scanner 1. For example, when multiple orientation identification methods are available, the appropriate orientation identification method differs depending on the type of document. By selecting the appropriate orientation identification method according to the type of document, the accuracy of orientation identification is improved. Therefore, according to the image forming apparatus 100 in this embodiment, the top and bottom of a document can be identified with high accuracy.

[0075] In this embodiment, the image forming apparatus 100 determines the type of document according to setting information indicating whether or not to use OCR. If it is set to use OCR, there is a high probability that the document is suitable for OCR orientation recognition, and if it is set not to use OCR, there is a high probability that the document is not suitable for OCR orientation recognition. Therefore, according to the image forming apparatus 100 in this embodiment, an orientation recognition method suitable for the type of document can be selected.

[0076] [Second Embodiment] In the first embodiment, an example was described in which the type of document is determined based on setting information indicating whether or not OCR is to be used. In the second embodiment, an example was described in which the type of document is determined using a pre-trained model that has been trained in advance by deep learning.

[0077] In this embodiment, the document type output from the trained model may be used directly as the document type discrimination result. Alternatively, it may be combined with document type discrimination using the OCR setting information described in the first embodiment. For example, if the document type discrimination result using the OCR setting information differs from the document type discrimination result using the trained model, the system may be configured to prioritize one of the document type discrimination results.

[0078] When scanning multiple documents in a single scan, OCR may not be able to control whether or not OCR is applied to each document. Therefore, in the document type discrimination method using OCR setting information described in the first embodiment, if multiple documents are scanned at once, document type discrimination cannot be applied to each document.

[0079] On the other hand, using DNN for document type discrimination has the advantage of being able to discriminate the document type for each document even when multiple documents are scanned at once. In other words, the image forming apparatus 100 in this embodiment can switch to the appropriate orientation discrimination method for each document even when multiple documents are scanned at once.

[0080] <Functional Configuration> In this embodiment, the document type determination unit 511 determines the document type of the image data read from the HDD3 based on the prediction results of a trained model. If the image data read from the HDD3 contains multiple documents, the document type determination unit 511 determines the document type for each document included in the image data.

[0081] The trained model in this embodiment is a machine learning model that has learned the relationship between an image representing the content of a document and the type of document it is. In this embodiment, the structure of the trained model is a neural network model (DNN) based on deep learning. That is, the trained model is a model that determines the type of document based on features extracted from an image representing the content of the document. However, the trained model can be any model that is capable of determining the type of document based on an image.

[0082] Figure 8 shows an example of the document type discrimination result in this embodiment. As shown in Figure 8, the trained model in this embodiment takes image data read from a document by scanner 1 as input and outputs a document type discrimination result that represents "document," "form," "photograph," or "graphic." However, the document type to be discriminated can be set arbitrarily and is not limited to these.

[0083] In this embodiment, the orientation recognition method to be set is pre-associated with each type of document output by the trained model. Figure 9 shows an example of the orientation recognition method settings in this embodiment. As shown in Figure 9, in this embodiment, when the document type is "document" or "form," the orientation recognition method is set to OCR orientation recognition. On the other hand, when the document type is "photograph" or "graphic," the orientation recognition method is set to AI orientation recognition. Non-OCR orientation recognition other than AI orientation recognition may be associated with "photograph" or "graphic."

[0084] <Processing Procedure> The method selection process performed by the method selection unit 51 in this embodiment will be described with reference to Figure 10. Figure 10 is a flowchart showing an example of the method selection process in this embodiment.

[0085] In step S11, the document type discrimination unit 511 receives the image data input to the method selection unit 51. Next, the document type discrimination unit 511 acquires a trained model. In this embodiment, the trained model identifies the document type as either "document," "form," "photograph," or "graphic," as shown in Figure 8.

[0086] Next, the document type discrimination unit 511 inputs the image data into the acquired trained model to determine the document type of the document scanned by the scanner 1. The document type discrimination unit 511 may perform preprocessing, such as scaling, before inputting the image data into the trained model. The document type discrimination unit 511 then sends the image data and the document type discrimination result to the format setting unit 512.

[0087] In step S12, the format setting unit 512 receives image data and the document type determination result from the document type determination unit 511. Next, the format setting unit 512 determines the document type determination result. If the document type determination result is "document" or "form," the format setting unit 512 proceeds to step S13. On the other hand, if the document type determination result is "photograph" or "graphic," the format setting unit 512 proceeds to step S14.

[0088] In step S13, the method setting unit 512 sets the orientation recognition method to OCR orientation recognition. This is because when the document type is "document" or "form," the document contains many characters, and there is a high possibility that the orientation can be accurately recognized by OCR orientation recognition. Next, the method setting unit 512 sends the image data and the orientation recognition method setting value to the method update unit 513.

[0089] In step S14, the format setting unit 512 sets the orientation recognition method to AI orientation recognition. This is because, when the document type is a "photograph" or "graphic," the document contains fewer characters, and AI orientation recognition is more likely to be able to accurately recognize the orientation. Note that in step S14, it may also be set to a non-OCR orientation recognition method other than AI orientation recognition. Next, the format setting unit 512 sends the image data and the orientation recognition method setting value to the format update unit 513.

[0090] The processes from step S15 to step S18 are the same as the processes from step S5 to step S8 in the first embodiment.

[0091] <Effects> In this embodiment, the image forming apparatus 100 determines the type of document by inputting the image scanned by the scanner 1 into a trained model. By utilizing machine learning, the type of document can be determined in detail, and the orientation recognition method can be appropriately set according to the type of document. Furthermore, if multiple documents are included in a single scan, the type of document can be determined for each document, so even if documents suitable for different orientation recognition methods are mixed together, the appropriate orientation recognition method can be set for each document. Therefore, the image forming apparatus 100 in this embodiment can accurately identify the top and bottom of a document.

[0092] [supplement] Each of the functions of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (digital signal processors), FPGAs (field programmable gate arrays), and conventional circuit modules designed to execute each of the functions described above.

[0093] Although embodiments of the present invention have been described in detail above, the present invention is not limited to these embodiments, and various modifications or changes are possible within the scope of the gist of the present invention as described in the claims. [Explanation of Symbols]

[0094] 100 Image forming apparatus 1 Scanner 2 Image Processing Device 3 HDD 4 plotters 5. Image file format conversion device 21 Gamma Correction Unit 22 Region detection unit 23 Data Interface Section 24-color processing / UCR section 25 Printer Correction Unit 51 Method Selection Section 52. Top and bottom identification unit 53 File Format Conversion Section 511 Original type discrimination section 512 Method setting section 513 Method update section [Prior art documents] [Patent Documents]

[0095] [Patent Document 1] Japanese Patent Publication No. 2007-324648 [Patent Document 2] Japanese Patent Publication No. 2022-90536

Claims

1. A document type identification unit that determines the type of document scanned by the scanner, A method setting unit sets an orientation identification method for identifying the orientation of a target image scanned by the scanner based on the type of the original document, A top / bottom identification unit that identifies the top / bottom of the target image according to the top / bottom identification method, Equipped with, The document type determination unit determines the type of document based on setting information set by the user when scanning the document with the scanner, which indicates whether or not to perform character recognition on the target image. Image processing device.

2. An image processing apparatus according to claim 1, The document type determination unit determines the type of each document when multiple documents are scanned by the scanner. Image processing device.

3. An image processing apparatus according to claim 1 or 2, The system further includes a system update unit that presents the orientation identification method set by the system setting unit to the user and updates the orientation identification method according to the user's operation. Image processing device.

4. The image processing device Procedures for identifying the type of document scanned with a scanner, A procedure for setting an orientation identification method for identifying the top and bottom of a target image scanned by the scanner based on the type of the aforementioned document, A procedure for identifying the top and bottom of the target image according to the top and bottom identification method, Execute, The aforementioned determination procedure involves determining the type of document based on setting information, which is set by the user when scanning the document with the scanner, indicating whether or not to perform character recognition on the target image. Image processing methods.

5. In the image processing device, Procedures for identifying the type of document scanned with a scanner, A procedure for setting an orientation identification method for identifying the top and bottom of a target image scanned by the scanner based on the type of the aforementioned document, A procedure for identifying the top and bottom of the target image according to the top and bottom identification method, Make it run, The aforementioned determination procedure involves determining the type of document based on setting information, which is set by the user when scanning the document with the scanner, indicating whether or not to perform character recognition on the target image. program.

Citation Information

Patent Citations

  • Image reading system and control method thereof

    JP2007324648A

  • Image processing system, image processing device, and control method

    JP2022090536A

  • Image processing apparatus, image processing method, and program

    JP2022147074A