Image processing device, image reading device, image processing method, program, and image processing system

Abstract

The goal is to obtain output images with settings appropriate for the subject without performing a preview scan. [Solution] The system comprises a reading unit that reads a subject, a reading image of the subject read in full color, a first determination unit that determines a first setting from a plurality of setting values ​​based on the reading image and a first trained model, and an image conversion unit that converts the reading image into an output image using the first setting determined by the first determination unit.

Description

[[Technical Field]]

[0001] The present invention relates to an image processing apparatus, an image reading apparatus, an image processing method, a program, and an image processing system. [[Background Art]]

[0002] Conventionally, in an image reading apparatus that reads a document using a setting selected from a plurality of setting values to obtain an output image, a technique for automatically determining a setting according to the document is known.

[0003] Patent Document 1 discloses that image data obtained by preview scanning of a document is input to a trained model, a setting is derived based on the obtained output value, and main scanning is performed using the derived setting. [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] However, according to the conventional technique, in order to obtain an output image with a setting corresponding to a document (hereinafter sometimes referred to as a subject), it is necessary to perform preview scanning before main scanning, which poses a problem that it takes time and effort.

[0005] The present invention has been made in view of the above, and an object of the present invention is to obtain an output image with a setting corresponding to a subject without performing preview scanning. [[Means for Solving the Problem]]

[0006] In order to solve the above-mentioned problem and achieve the object, the present invention includes: a first determining unit that determines a first setting from a plurality of setting values based on a read image obtained by reading a subject in full color by a reading unit that reads the subject and a first trained model; and an image converting unit that converts the read image into an output image using the first setting determined by the first determining unit. [[Effect of the Invention]]

[0007] According to the present invention, it is possible to obtain an output image with settings appropriate to the subject without performing a preview scan. [Brief explanation of the drawing]

[0008] [Figure 1] Figure 1 is a side view showing an overview of an image reading device according to the first embodiment. [Figure 2] Figure 2 is a block diagram showing an example configuration of an image reading device according to the first embodiment. [Figure 3] Figure 3 shows an example of the functional configuration of the processing unit according to the first embodiment. [Figure 4] Figure 4 shows an example of training data according to the first embodiment. [Figure 5] Figure 5 shows an example of a display screen shown on the control panel. [Figure 6] Figure 6 is a flowchart showing an example of the processing procedure according to the first embodiment. [Figure 7] Figure 7 is a flowchart showing an example of the procedure for accumulating learning data according to the first embodiment. [Figure 8] Figure 8 shows an example of the functional configuration of the processing unit according to the second embodiment. [Figure 9] Figure 9 shows an example of a color region detected by the detection unit. [Figure 10] Figure 10 shows an example of training data according to the second embodiment. [Figure 11] Figure 11 is a flowchart showing an example of a processing procedure according to the second embodiment. [Figure 12] Figure 12 is a flowchart showing an example of the procedure for accumulating learning data according to the second embodiment. [Figure 13] Figure 13 shows an example of the functional configuration of the processing unit according to the third embodiment. [Figure 14] Figure 14 is a flowchart showing an example of a processing procedure according to the third embodiment. [Figure 15]Figure 15 shows an example of the functional configuration of the processing unit according to the fourth embodiment. [Figure 16] Figure 16 shows an example of training data used to train the second trained model. [Figure 17] Figure 17 is a flowchart showing an example of a processing procedure according to the fourth embodiment. [Modes for carrying out the invention]

[0009] Embodiments of the image processing apparatus, image reading apparatus, image processing method, program, and image processing system will be described in detail below with reference to the attached drawings.

[0010] (First Embodiment) Figure 1 is a side view showing an overview of the image reading device 10 according to the first embodiment. The image reading device 10 is, for example, a sheet-through type and comprises a reading unit body 100 (flatbed scanner) and an automatic document feeder (ADF) 102.

[0011] The scanning unit body 100 comprises a contact glass 104, a reference whiteboard 106, a first carriage 108, a second carriage 110, a lens 118, an image sensor 122 provided on a light-receiving element substrate 120, a scanner motor 124, and an operation panel 125. The first carriage 108 has a light source 109 and a mirror 112. The second carriage 110 has mirrors 114 and 116. The scanning unit body 100 is also provided with a scanning window 134 for scanning the subject P, which is the original document transported by the automatic document feeder 102.

[0012] An automatic document feeder 102 is disposed on an upper part of a reading unit main body 100, and automatically feeds and conveys a subject P. The automatic document feeder 102 includes a document tray 130, a conveyance drum 132, a paper discharge roller 136, a paper discharge tray 138, and the like. Then, the automatic document feeder 102 conveys the subject P placed on the document tray 130 toward the conveyance drum 132, and the conveyance drum 132 conveys the subject P toward a reading window 134. The subject P is exposed by a light source 109 when passing through the reading window 134. Reflected light from the subject P is folded back by a mirror 112 of a first carriage 108 and mirrors 114 and 116 of a second carriage 110, passes through a lens 118, and forms a reduced image on a light receiving surface of an image sensor 122 on a light receiving element substrate 120.

[0013] In addition, in flatbed reading in which the subject P is fixed on a contact glass 104 and the first carriage 108 and the second carriage 110 (hereinafter sometimes collectively referred to as "carriages") are scanned to read the subject P, the subject P on the contact glass 104 is irradiated by the light source 109 from below the contact glass 104. Reflected light from the subject P is folded back by the mirror 112 of the first carriage 108 and the mirrors 114 and 116 of the second carriage 110, passes through the lens 118, and forms a reduced image on the light receiving surface of the image sensor 122 on the light receiving element substrate 120. At this time, in the image reading apparatus 10, the first carriage 108 moves at a speed V in the sub-scanning direction of the subject P, the second carriage 110 moves in conjunction with the first carriage 108 at a speed of 1 / 2V, which is half the speed of the first carriage 108, and reads the entire subject P.

[0014] An operation panel 125 includes a touch panel or the like that displays setting values of the image reading apparatus 10, an image reading start button, and the like, and accepts input data from a user, an image reading start instruction, and the like. The touch panel or the like accepts touch inputs from a user, and the user can use a finger, a pen, or the like to perform operations such as inputting a numerical value into an input box displayed on a screen, selecting a pull-down menu, and toggling a check box on / off. In addition, the operation panel 125 may include input means such as a numeric keypad, a trackball, and a touchpad.

[0015] Next, a configuration example of the image reading apparatus 10 according to the present embodiment will be described in detail. FIG. 2 is a block diagram showing a configuration example of the image reading apparatus 10 according to the first embodiment. The image reading apparatus 10 includes a light receiving element substrate 120, a storage section 220, an image processing substrate 230, and a CPU (Central Processing Unit) 240.

[0016] The light receiving element substrate 120 photoelectrically converts the focused reflected light, processes the obtained read image data as described later, and outputs the processed data as an output image. The storage section 220 is constituted by an HDD (Hard Disk Drive), a memory, or the like, and stores various data.

[0017] The image processing substrate 230 performs various types of image processing on the output image.

[0018] The CPU 240 controls each section constituting the image reading apparatus 10.

[0019] The light receiving element substrate 120 includes an image sensor 122 and a processing section 300. As described above, the image sensor 122 reads an image formed by reduction on the light receiving surface to generate a read image. The processing section 300 processes the read image and outputs an output image.

[0020] The image sensor 122 is, for example, a CMOS (Complementary Metal Oxide Semiconductor) linear image sensor, and reads the subject P in full color. The image sensor 122 includes, for example, three color sensors (line image sensors): an R (Red) sensor, a G (Green) sensor, and a B (Blue) sensor.

[0021] Note that the processing section 300 is an example of an image processing apparatus that processes images, and the image sensor 122 is an example of a reading section that reads the subject P. Further, the image reading apparatus 10 according to the present embodiment is an image reading apparatus including the image processing apparatus (processing section 300) and the reading section (image sensor 122).

[0022] Figure 3 shows an example of the functional configuration of the processing unit 300 according to the first embodiment. As shown in Figure 3, the processing unit 300 includes a receiving unit 310, a first decision unit 320, a second decision unit 330, a first trained model 321, a storage unit 350, and an image conversion unit 370. In this embodiment, each part included in the functional configuration may be referred to as a functional unit.

[0023] The reception unit 310 receives user input data. For example, the reception unit 310 controls the operation panel 125 to receive user input data and instructions to start image reading.

[0024] The first decision unit 320 determines a first setting from a plurality of setting values ​​based on the read image of the subject P read in full color and the first trained model 321. Here, the plurality of setting values ​​are values ​​that indicate the color setting of the output image that the processing unit 300 converts and outputs from the read image. For example, 1 represents full color, 2 represents monochrome, and 3 represents two colors. That is, the output image output by the processing unit 300 is a full-color image when the setting value is 1, a monochrome image when it is 2, and a two-color image when it is 3. Here, a two-color image is, for example, a binary image in which the read image has been binarized into two colors, white and black.

[0025] The first pre-trained model 321 is an AI (Artificial Intelligence) model (machine learning model) that has been trained using training data that includes a set of settings set by the user for the subject P and a read image of the subject P read in full color. In this embodiment, the first pre-trained model 321 can be used to determine the color settings corresponding to the subject P.

[0026] Machine learning is a technique that enables computers to acquire human-like learning abilities. It involves computers autonomously generating algorithms necessary for data identification and other judgments from pre-programmed training data, and then applying these algorithms to new data to make predictions. The learning method for machine learning can be supervised learning, unsupervised learning, semi-supervised learning, reinforcement learning, or deep learning, or a combination of these methods; the learning method for machine learning is not restricted.

[0027] Figure 4 shows an example of training data according to the first embodiment. As shown in Figure 4(a), each training data 400 includes a color setting 410 and a read image 420. The color setting 410 is a numerical value such as 1 (full color), 2 (monochrome), or 3 (two colors), and the read image 420 is image data read in full color. The first trained model 321 is an AI model that has been trained using multiple such training data.

[0028] Figure 4(b) shows examples of multiple training data sets used for machine learning of the first trained model 321. For example, training data No. 1 is a read image aaa.jpg with a color setting value of 1, and training data No. 2 is a read image bbb.bmp with a color setting value of 2. The "Read Image" column in Figure 4(b) shows the file name and data format of the read image. Here, jpg is JPEG (Joint Photographic Experts Group) format, bmp is bitmap format, and png is PNG (Portable Network Graphics) format, but files in other data formats may also be used.

[0029] The second determination unit 330 determines a second setting from multiple setting values ​​based on the first setting determined by the first determination unit 320 and the input data received by the reception unit 310. The first setting is presented to the user, for example, by being displayed on the operation panel 125.

[0030] Figure 5 shows an example of the display screen 500 shown on the control panel 125. As shown in Figure 5, the display screen 500 has a setting value area 510 and a confirmation button 520. The setting value area 510 displays the first setting as the initial value. In the example in Figure 5, "Monochrome" is displayed as the first setting.

[0031] The setting area 510 in Figure 5 is a pull-down menu, and the user can change the setting value displayed in the setting area 510 to a second setting different from the first setting by operating the pull-down menu. The user can also input the setting value displayed in the setting area 510 as the second setting by pressing the confirmation button 520.

[0032] Thus, if the reception unit 310 receives confirmation that the setting value in the setting value area 510 has been changed and the confirmation button 520 has been pressed, the second determination unit 330 determines a setting value different from the first setting as the second setting. On the other hand, if the reception unit 310 receives confirmation that the setting value in the setting value area 510 has not been changed and the confirmation button 520 has been pressed, the second determination unit 330 determines the first setting as the second setting. In this embodiment, the first setting is presented to the user, and the user is given the option to use the first setting as the color setting or change to the second setting, thereby determining a color setting that corresponds to the subject P.

[0033] The storage unit 350 stores training data for further training the first trained model 321. The storage unit 350 stores training data, for example, by storing a set of second settings and read images in the memory unit 220. In this embodiment, by storing the set of second settings and read images as new training data and further training the first trained model 321 using the stored training data, the performance of the first decision unit 320 in determining the first settings can be improved.

[0034] The image conversion unit 370 converts the read image into an output image using the second setting determined by the second determination unit 330. If the color setting is "full color", the image conversion unit 370 generates the read image as the output image as is. If the color setting is "monochrome", the image conversion unit 370 converts the read image into a monochrome image, and if the color setting is "two-color", it converts the read image into a binary image, for example, black and white, and generates it as the output image.

[0035] In addition, the functional configuration of this embodiment may be a first configuration in which the processing unit 300 does not include a receiving unit 310, a second determination unit 330, and a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the first setting determined by the first determination unit 320, and no learning data for additional learning is stored. Furthermore, the functional configuration of this embodiment may be a second configuration in which the processing unit 300 does not include a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the second setting, and no learning data for additional learning is stored.

[0036] Figure 6 is a flowchart showing an example of the processing procedure according to the first embodiment. First, the first determination unit 320 determines the first setting from the read image (step S100), and the reception unit 310 receives the user's input data (step S101).

[0037] Next, the second determination unit 330 determines the second setting based on the first setting and the input data (step S102). Then, the image conversion unit 370 converts the read image into an output image using the second setting (step S103), and the storage unit 350 stores the read image and the second setting (step S104).

[0038] If the functional configuration of this embodiment is the first configuration, the processes in steps S101, S102, and S104 in Figure 6 are omitted, and in step S103, the image conversion unit 370 converts the read image into an output image using the first setting. Also, if the functional configuration of this embodiment is the second configuration, the process in step S104 in Figure 6 is omitted.

[0039] Figure 7 is a flowchart showing an example of the procedure for accumulating training data according to the first embodiment. Here, we will explain the accumulation of training data for training the first trained model 321. The reception unit 310 receives user settings, which are color settings set by the user according to the subject P (step S110). For example, when the user performs normal reading processing with the image reading device 10, the user inputs the desired color settings according to the subject P as user settings into the operation panel 125, and the reception unit 310 receives the user settings input into the operation panel 125.

[0040] Next, the image conversion unit 370 converts the read image into an output image using user settings (step S111), and the storage unit 350 stores the set of read image and user settings as training data (step S112). In this way, the storage of training data for the first trained model 321 can be performed in parallel with the normal reading process of the image reading device 10. Note that the storage of training data for the first trained model 321 may be performed without the normal reading process of the image reading device 10. In that case, the process in step S111 in Figure 7 is omitted.

[0041] Thus, according to this embodiment, an output image can be obtained with settings appropriate to the subject without performing a preview scan.

[0042] In the above description, the processing unit 300 is provided on the photodetector substrate 120, but the processing unit 300 may be provided outside the photodetector substrate 120. For example, the processing unit 300 may be configured to be provided on the image processing substrate 230. Typically, the circuit size of the image processing substrate 230 is larger than that of the photodetector substrate 120 and is designed with ample margin, so the processing unit 300 can be mounted without increasing the circuit size of the image processing substrate 230.

[0043] (Second Embodiment) In the second embodiment, the first determination unit 320 determines the first setting based on the color region in the read image. In the following description of the second embodiment, the parts that overlap with the first embodiment will be omitted, and the parts that differ from the first embodiment will be described.

[0044] Figure 8 shows an example of the functional configuration of the processing unit 300 according to the second embodiment. The difference from the first embodiment is that the processing unit 300 further includes a detection unit 340 for detecting color regions, and the color region is used in the first determination unit 320, the first learned model 321, and the storage unit 350.

[0045] The detection unit 340 detects color regions in the read image. A color region is a region of pixels whose saturation is greater than a predetermined value. For example, if the pixel value is represented by three values, R, G, and B, the detection unit 340 detects regions where the magnitudes of each value are different as color regions. The detection unit 340 may also detect regions where the brightness of the pixels is greater than a predetermined value as color regions. In this way, the detection unit 340 detects regions of color photographs, regions of colored figures or characters, etc., contained in the read image as color regions. This detection also identifies regions other than color regions (monochrome regions), such as regions of monochrome photographs and regions of colorless figures or characters.

[0046] Figure 9 shows an example of a color region detected by the detection unit 340. The read image in Figure 9 includes both color regions and areas of colorless text, etc. In this example, the detection unit 340 detects a rectangular region as the color region. Here, the X and Y axes are coordinate axes of a two-dimensional coordinate system representing the position on the read image, and the rectangle is represented by the position of the top-left vertex (x, y), width w, and height h. In this embodiment, such a rectangle is represented in the form (x, y, w, h).

[0047] Note that the color region detected by the detection unit 340 is not limited to a rectangle, but may be a color region of any shape. In that case, the shape of the color region can be represented, for example, by a binary image in which a rectangle encompassing the color region and the value of each pixel within the rectangle are 1 (a pixel inside the color region) or 0 (a pixel outside the color region).

[0048] The first decision unit 320 determines a first setting based on the read image, the first trained model 321, and the color region detected by the detection unit 340. Here, the first trained model 321 is an AI model that has been trained using training data including setting values ​​set by the user for the subject P, a read image of the subject P read in full color, and the color region detected by the detection unit 340. In this embodiment, by using the first trained model 321, it is possible to determine a color setting corresponding to the subject P and its color region.

[0049] Figure 10 shows an example of training data according to the second embodiment. The difference from Figure 4 is that, as shown in Figure 10(a), the training data 400 further includes a color region 430.

[0050] Figure 10(b) shows examples of multiple training data sets used for machine learning of the first trained model 321. As shown in Figure 10(b), each training data set contains color region data detected by the detection unit 340 in the format (x, y, w, h). For example, training data set No. 1 includes multiple color regions (0, 0, 100, 100), (30, 20, 400, 500), etc., and training data set No. 2 includes the color region (10, 20, 300, 200).

[0051] The storage unit 350 stores training data for further training the first trained model 321. The difference from the first embodiment is that the storage unit 350 stores data including the read image, color region, and second setting set as training data for further training.

[0052] Furthermore, the functional configuration of this embodiment may also be a third configuration in which the processing unit 300 does not include a receiving unit 310, a second determination unit 330, and a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the first setting, and no learning data for additional learning is stored. Alternatively, the functional configuration of this embodiment may also be a fourth configuration in which the processing unit 300 does not include a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the second setting, and no learning data for additional learning is stored.

[0053] Figure 11 is a flowchart showing an example of a processing procedure according to the second embodiment. The difference from Figure 6 is that a step S200 for detecting a color area is added, and the color area is used in steps S201 and S205. The processing in steps S202 to S204 in Figure 11 is the same as in steps S101 to S103 in Figure 6, so the explanation is omitted.

[0054] The detection unit 340 detects a color region from the read image (step S200), and the first determination unit 320 determines a first setting using the first trained model 321 based on the read image and the color region (step S201).

[0055] Furthermore, the storage unit 350 stores the read image, color region, and second setting set as training data for additional learning (step S205).

[0056] Furthermore, if the functional configuration of this embodiment is the third configuration, the processes in steps S202, S203, and S205 in Figure 11 are omitted, and in step S204, the image conversion unit 370 converts the read image into an output image using the first setting. Also, if the functional configuration of this embodiment is the fourth configuration, the process in step S205 in Figure 11 is omitted.

[0057] Figure 12 is a flowchart showing an example of the training data accumulation procedure according to the second embodiment. Here, we will explain the accumulation of training data for training the first trained model 321. The difference from Figure 7 is that a step S212 for detecting color regions is added, and the color regions are used in step S213. The processing in steps S210 and S211 in Figure 12 is the same as in steps S110 and S111 in Figure 7, so the explanation is omitted.

[0058] The detection unit 340 detects color regions from the read image (step S212), and the storage unit 350 stores the data, including the read image, color regions, and user-defined sets, as training data (step S213). Note that the storage of training data for the first trained model 321 may be performed without performing the normal reading process of the image reading device 10. In that case, the process in step S211 of Figure 12 is omitted.

[0059] Thus, according to this embodiment, an output image can be obtained with settings appropriate to the subject without performing a preview scan. Furthermore, by using the color area detected from the scanned image, settings appropriate to the subject can be determined more appropriately.

[0060] (Third embodiment) The third embodiment adds a setting to convert image data not included in the color region of the read image into monochrome as a color setting. In other words, this setting converts image data included in the monochrome region (regions other than the color region) into monochrome (hereinafter, this setting will be referred to as "monochrome region processing"). In the following description of the third embodiment, the explanation of parts that overlap with the second embodiment will be omitted, and the parts that differ from the second embodiment will be explained.

[0061] Figure 13 shows an example of the functional configuration of the processing unit 300 according to the third embodiment. The difference from the second embodiment is that the image conversion unit 370 includes a monochrome area processing unit 371, and the setting value for monochrome area processing is included among the multiple setting values ​​processed by each functional unit. The multiple setting values ​​are, for example, 1 (full color), 2 (monochrome), 3 (two colors), and 4 (monochrome area processing).

[0062] The monochrome area processing unit 371 converts image data contained in the monochrome area of ​​the read image to monochrome when the color setting is a value indicating monochrome area processing (for example, 4). Then, the image conversion unit 370 outputs an output image in which the image data in the color area remains in full color and the image data in the monochrome area has been converted to monochrome when either the first setting or the second setting is 4. Note that the value indicating monochrome area processing is just one example of a predetermined setting value.

[0063] Furthermore, the functional configuration of this embodiment may also be a fifth configuration in which the processing unit 300 does not include a reception unit 310, a second determination unit 330, and a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the first setting, and no learning data for additional learning is stored. Also, the functional configuration of this embodiment may be a sixth configuration in which the processing unit 300 does not include a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the second setting, and no learning data for additional learning is stored.

[0064] Figure 14 is a flowchart showing an example of a processing procedure according to the third embodiment. The difference from Figure 11 is that the process of converting the read image into an output image has been changed to the process in steps S304 to S306. The processes in steps S300 to S303 and S307 in Figure 14 are the same as steps S200 to S203 and S205 in Figure 11, so their explanation is omitted.

[0065] If the second setting is not a predetermined setting value (a value indicating monochrome area processing) (step S304: No), the image conversion unit 370 converts the read image into an output image in the same manner as in step S204 of Figure 11. That is, the image conversion unit 370 converts the read image into an output image in which the entire image is either a full-color image, a monochrome image, or a binary image, according to the second setting (step S305).

[0066] On the other hand, if the second setting is a predetermined setting value (a value indicating monochrome area processing) (step S304: Yes), the image conversion unit 370 uses the monochrome area processing unit 371 to convert the read image into an output image in which the color areas are full-color images and the areas other than the color areas are monochrome images (step S306).

[0067] Furthermore, if the functional configuration of this embodiment is the fifth configuration, the processes in steps S302, S303, and S307 in Figure 14 are omitted, and in steps S305 to S306, the image conversion unit 370 converts the read image into an output image using the first setting. Also, if the functional configuration of this embodiment is the sixth configuration, the process in step S307 in Figure 14 is omitted.

[0068] Thus, according to this embodiment, an output image can be obtained with settings appropriate to the subject without performing a preview scan. Furthermore, a setting can be used to convert image data in areas other than the color region to monochrome.

[0069] (Fourth embodiment) The fourth embodiment improves image quality using machine learning on image data included in the color region. Generally, scanned images obtained by scanners, etc., suffer from degradation such as changes in color and distortion from the subject P (original). Such degradation occurs due to factors such as scanning accuracy, dirt such as fingerprints and dust on the glass surface or original, and machine vibrations. Furthermore, if the original itself is degraded with stains or dirt, degradation of the scanned image is unavoidable even with high-precision scanning. Therefore, in this embodiment, degradation of the scanned image is reduced by improving image quality using machine learning on image data included in the color region. In the following description of the fourth embodiment, the explanation of parts that overlap with the second embodiment will be omitted, and the parts that differ from the second embodiment will be explained.

[0070] Figure 15 shows an example of the functional configuration of the processing unit 300 according to the fourth embodiment. The difference from the first embodiment is that the processing unit 300 further includes an image improvement unit 360.

[0071] The image improvement unit 360 performs image data improvement processing (correction processing) based on the image data contained in the color region and the second trained model 361. More specifically, the image improvement unit 360 takes the image data contained in the color region detected by the detection unit 340 as input and uses the second trained model 361 to improve at least one of the color changes and distortions of the image data. The improved read image (improved image) processed by the image improvement unit 360 is transmitted to the image conversion unit 370. The image conversion unit 370 then converts the improved image into an output image instead of the read image.

[0072] The second pre-trained model 361 is an AI model obtained by any machine learning method. The second pre-trained model 361 is, for example, a pre-trained neural network whose parameters have been adjusted using backpropagation. Note that the second pre-trained model 361 may be a model other than a neural network.

[0073] Next, we will explain the machine learning of the second pre-trained model 361. The second pre-trained model 361 is a model that has been trained to recognize degradation and image changes by comparing the raw data used to generate the original document with the scanned image obtained by scanning the original document in full color. Here, raw data refers to the first-stage acquired digital data, which has not undergone processes such as printing or scanning. Furthermore, degradation includes changes in color, distortion, contrast, and sharpness, and image changes include changes such as the presence or absence of smudges.

[0074] Figure 16 shows an example of training data used to train the second trained model 361. As shown in Figure 16(a), each training data 600 includes raw data 610, a corresponding read image 620, and a color region 630 in the read image 620.

[0075] Figure 16(b) shows examples of multiple training datasets used for machine learning of the second pre-trained model 361. For example, training dataset No. 1 includes the raw data aa0.bmp, its corresponding read image aaa.jpg, and multiple color regions (0,0,100,100), (30,20,400,500), etc., in aaa.jpg. Training dataset No. 2 includes the raw data bb0.bmp, its corresponding read image bbb.jpg, and the color regions (10,20,300,200) in bbb.jpg.

[0076] The image enhancement unit 360 uses a second trained model 361, which has been machine-learned as described above, to obtain an improved image that suppresses the effects of scanning accuracy, dirt such as fingerprints and dust attached to the glass surface or original document, machine vibrations, etc.

[0077] In the above example, the image improvement unit 360 performs improvement processing using the second trained model 361. However, it is also possible to perform improvement processing using generative artificial intelligence (generative AI or generative AI) instead of the second trained model 361. In this case, the generative AI performs improvement processing such as improving image quality by removing dirt, or generating an image with dirt removed.

[0078] Furthermore, the functional configuration of this embodiment may also be a seventh configuration in which the processing unit 300 does not include a reception unit 310, a second determination unit 330, and a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the first setting, and no learning data for additional learning is stored. Also, the functional configuration of this embodiment may be an eighth configuration in which the processing unit 300 does not include a storage unit 350. In that case, the image conversion unit 370 converts the read image into an output image using the second setting, and no learning data for additional learning is stored.

[0079] Figure 17 is a flowchart showing an example of a processing procedure according to the fourth embodiment. The difference from Figure 11 is the addition of step S401, which performs image improvement, and the use of the improved image instead of the read image in steps S402, S405, and S406. The processing in steps S400, S403, and S404 in Figure 17 is the same as in steps S200, S202, and S203 in Figure 11, so the explanation is omitted.

[0080] The image improvement unit 360 performs image improvement in the color region (step S401), and the first determination unit 320 determines a first setting based on the improved image and the color region (step S402).

[0081] Furthermore, the image conversion unit 370 converts the improved image into an output image using the second setting (step S405), and the storage unit 350 stores the data, including the improved image, the color region, and the set of the second setting, as training data for additional learning (step S406).

[0082] Furthermore, if the functional configuration of this embodiment is the seventh configuration, the processes in steps S403, S404, and S406 in Figure 17 are omitted, and in step S405, the image conversion unit 370 converts the read image into an output image using the first setting. Also, if the functional configuration of this embodiment is the eighth configuration, the process in step S406 in Figure 17 is omitted.

[0083] Thus, according to this embodiment, an output image can be obtained with settings appropriate to the subject without performing a preview scan. Furthermore, by processing the image data in the color region to improve it and using the improved image with reduced degradation, it is possible to determine settings more appropriately for the subject.

[0084] The programs executed by the image processing apparatus of each embodiment described above are provided as installable or executable files recorded on a computer-readable recording medium such as a CD-ROM, flexible disk (FD), CD-R, or DVD (Digital Versatile Disk).

[0085] Furthermore, the program executed by the image processing device of each embodiment may be stored on a computer connected to a network such as the Internet and provided by downloading it via the network. Alternatively, the program executed by the image processing device of each embodiment may be provided or distributed via a network such as the Internet. In addition, some functions of the processing unit 300 that process using AI or generation AI, such as the image improvement unit 360 and the image conversion unit 370, may be stored on a computer connected to a network such as the Internet and made available to the image processing device via the network. In this case, an image processing system including the image processing device and an external computer may execute the image processing of each embodiment.

[0086] Furthermore, the programs for each embodiment may be pre-installed and provided in ROM or the like.

[0087] The program executed in the image processing apparatus of each embodiment has a modular configuration that includes the above-described parts (reception unit 310, first determination unit 320, etc.), and in actual hardware, the CPU (processor) reads the program from the recording medium and executes it, thereby loading the above-described parts onto the main memory and generating them.

[0088] Each function of the embodiments described above can be realized by one or more processing circuits. Hereinafter, "processing circuit" as used herein includes processors programmed to execute each function by software, such as processors implemented by electronic circuits, as well as devices such as ASICs (Application Specific Integrated Circuits), DSPs (Digital Signal Processors), FPGAs (Field Programmable Gate Arrays), and conventional circuit modules designed to execute each function described above.

[0089] Although various embodiments of the present invention have been described above, these embodiments are presented as examples only and are not intended to limit the scope of the invention. These novel embodiments can be implemented in a variety of other forms, and various omissions, substitutions, and modifications can be made without departing from the spirit of the invention. These novel embodiments and their variations are included in the scope and spirit of the invention, as well as in the claims of the invention and its equivalents. Furthermore, components from different embodiments and modifications may be combined as appropriate.

[0090] Examples of the present invention are as follows: <1> A reading unit reads the subject, and a first determination unit determines a first setting from a plurality of setting values ​​based on the read image obtained by the reading unit, in full color, and a first trained model. An image conversion unit converts the read image into an output image using the first setting determined by the first determination unit, This is an image processing apparatus characterized by having the following features. <2> A reception unit that receives user input data, A second determination unit determines a second setting from the plurality of setting values ​​based on the first setting determined by the first determination unit and the input data received by the reception unit, Furthermore, The image conversion unit is configured to convert the read image into an output image using the second setting determined by the second determination unit. <1> This is the image processing device described above. <3> The system further includes a storage unit for storing training data to further train the first trained model, The storage unit stores the read image and the second setting. <2> This is the image processing device described above. <4> The system further includes a detection unit for detecting color regions in the read image, The first determination unit determines the first setting based on the color region detected by the detection unit. <2> This is the image processing device described above. <5> The system further includes a detection unit for detecting color regions in the read image, The first determination unit determines the first setting based on the color region detected by the detection unit. <3> This is the image processing device described above. <6> The image conversion unit, when the second setting is a predetermined setting value, converts the image data not included in the color region of the read image into monochrome. <4> or <5> This is the image processing device described above. <7> The system further comprises an image improvement unit that performs correction processing on the image data based on the image data included in the color region and a second trained model. <4> or <5> This is the image processing device described above. <8> The image improvement unit further comprises an image correction unit that performs image data correction processing based on the image data included in the color region and a generative artificial intelligence system. <4> or <5> This is the image processing device described above. <9> The second trained model is a model that has been trained using the raw data used to generate the subject and the read image in which the subject is read in full color. <7> This is the image processing device described above. <10> The image improvement unit corrects at least one of the color changes and distortions of the image data. <7> or <8> This is the image processing device described above. <11> The subject is a manuscript. A reading unit for reading the aforementioned document subject, The aforementioned <1> ~ <10> An image processing device described in any one of the following, This is an image reading device that has [a certain feature]. <12> An image processing method performed by an image processing device, A first determination step involves determining a first setting from multiple setting values ​​based on a reading image obtained by a reading unit that reads the subject in full color and a first trained model, and An image conversion step is performed to convert the read image into an output image using the first setting determined in the first determination step, This is an image processing method characterized by including [a specific element]. <13> A detection step for detecting a color region in the read image, Image improvement step, which performs image correction processing on the image data based on the image data included in the color region and a second trained model, The above further includes <12> This is the image processing method described in [reference]. <14> A detection step for detecting a color region in the read image, An image improvement step which performs image correction processing based on the image data included in the color region and a generative artificial intelligence, The above further includes <12> This is the image processing method described in [reference]. <15> The image improvement step corrects at least one of the color changes and distortions of the image data. <13> or <14> This is the image processing method described in [reference]. <16> Computers, A reading unit reads the subject, and a first determination means determines a first setting from a plurality of setting values ​​based on the read image obtained by the reading unit and a first trained model, An image conversion means that converts the read image into an output image using the first setting determined by the first determination means, This is a program that makes it function as such. <17> A detection means for detecting a color region in the read image, Image improvement means that performs correction processing on the image data based on the image data included in the color region and a second trained model, The above further comprises <16> This is the program described in [the document]. <18> A detection means for detecting a color region in the read image, Image improvement means that performs image correction processing based on image data included in the color region and a generative artificial intelligence system, The above further comprises <16> This is the program described in [the document]. <19> The image improvement means corrects at least one of the color changes and distortions of the image data. <17> or <18> This is the program described in [the document]. <20> The aforementioned <1> ~ <6> An image processing device described in any one of the following, An image processing system comprising: an image processing unit provided outside the image processing device and receiving image data relating to the read image from the image processing device via the Internet, The image improvement unit performs correction processing on the image data based on artificial intelligence or generative artificial intelligence. The aforementioned image processing device is an image processing system characterized by receiving image data corrected by the image improvement unit via the Internet. [Explanation of Symbols]

[0091] 10 Image reading device 120 Photodetector substrate 122 Image Sensors 125 Control Panel 220 Storage section 230 Image Processing Board 240 CPU 300 Processing Unit 310 Reception Department 320 First Decision Section 321 First pre-trained model 330 Second Decision Section 340 Detection unit 350 Storage Unit 360 Image Improvement Department 361 Second pre-trained model 370 Image Conversion Unit 371 Monochrome Area Processing Unit [Prior art documents] [Patent Documents]

[0092] [Patent Document 1] Japanese Patent Publication No. 2020-17839

Claims

1. A reading unit reads the subject, and a first determination unit determines a first setting from a plurality of setting values ​​based on the read image obtained by the reading unit, in full color, and a first trained model. An image conversion unit converts the read image into an output image using the first setting determined by the first determination unit, An image processing apparatus characterized by comprising:

2. A reception unit that receives user input data, A second determination unit determines a second setting from the plurality of setting values ​​based on the first setting determined by the first determination unit and the input data received by the reception unit, Furthermore, The image processing apparatus according to claim 1, wherein the image conversion unit is configured to convert the read image into an output image using the second setting determined by the second determination unit.

3. The system further includes a storage unit for storing training data to further train the first trained model, The image processing apparatus according to claim 2, wherein the storage unit stores the read image and the second setting.

4. The system further includes a detection unit for detecting color regions in the read image, The image processing apparatus according to claim 2, wherein the first determination unit determines the first setting based on the color region detected by the detection unit.

5. The system further includes a detection unit for detecting color regions in the read image, The image processing apparatus according to claim 3, wherein the first determination unit determines the first setting based on the color region detected by the detection unit.

6. The image processing apparatus according to claim 4, wherein the image conversion unit converts image data not included in the color region of the read image into monochrome when the second setting is a predetermined setting value.

7. The image processing apparatus according to claim 4, further comprising an image improvement unit that performs correction processing on the image data based on the image data included in the color region and a second trained model.

8. The image processing apparatus according to claim 4, further comprising an image improvement unit that performs image correction processing based on image data included in the color region and a generative artificial intelligence system.

9. The image processing apparatus according to claim 7, wherein the second trained model is a model trained using raw data used to generate the subject and a read image of the subject read in full color.

10. The image processing apparatus according to claim 7, wherein the image improvement unit corrects at least one of the color change and distortion of the image data.

11. The subject is a manuscript. A reading unit for reading the aforementioned document, An image processing apparatus according to any one of claims 1 to 10, An image reading device having [a certain feature].

12. An image processing method performed by an image processing device, A first determination step involves determining a first setting from a plurality of setting values ​​based on a reading image obtained by a reading unit that reads the subject in full color and a first trained model, An image conversion step is performed to convert the read image into an output image using the first setting determined in the first determination step, An image processing method characterized by including

13. A detection step for detecting a color region in the read image, An image improvement step which performs image correction processing on the image data based on the image data included in the color region and a second trained model, The image processing method according to claim 12, further comprising:

14. A detection step for detecting a color region in the read image, An image improvement step which performs image correction processing based on the image data included in the color region and a generative artificial intelligence, The image processing method according to claim 12, further comprising:

15. The image processing method according to claim 13, wherein the image improvement step corrects at least one of the color change and distortion of the image data.

16. Computers, A reading unit reads the subject, and a first determination means determines a first setting from a plurality of setting values ​​based on the read image obtained by the reading unit and a first trained model, An image conversion means that converts the read image into an output image using the first setting determined by the first determination means, A program that makes it function as such.

17. A detection means for detecting a color region in the read image, Image improvement means that performs correction processing on the image data based on the image data included in the color region and a second trained model, The program according to claim 16, further comprising:

18. A detection means for detecting a color region in the read image, Image improvement means that performs correction processing of the image data based on the image data included in the color region and a generative artificial intelligence system, The program according to claim 16, further comprising:

19. The program according to claim 17, wherein the image improvement means corrects at least one of the color change and distortion of the image data.

20. An image processing apparatus according to any one of claims 1 to 6, An image processing system comprising: an image processing unit provided outside the image processing device and receiving image data relating to the read image from the image processing device via the Internet, The image improvement unit performs correction processing on the image data based on artificial intelligence or generative artificial intelligence. The image processing device is an image processing system characterized by receiving image data corrected by the image improvement unit via the Internet.

Images