Image inspection system, image inspection device, and image inspection method

Abstract

The present invention provides an image inspection system, an image inspection apparatus, and an image inspection method that can easily and appropriately detect image defects. [Solution] The image inspection system comprises a first image defect identification unit that reads an image formed on a recording medium and performs a first identification process to detect image defects from the formed image and identify the type of image defect, and a second image defect identification unit that performs a second identification process to further identify the type of image defect using a trained model based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect was detected, and image formation conditions related to the image defect.

Description

TECHNICAL FIELD

[0001] The present disclosure relates to an image inspection system, an image inspection apparatus, and an image inspection method for inspecting an image formed on a recording medium.

BACKGROUND ART

[0002] Conventionally, in an image forming apparatus (image forming system) that forms an image on a recording medium such as paper, image diagnosis is performed in which the recording medium on which the image is formed is read, and the read result is analyzed to identify an image defect and present the defect to a user. In recent years, technologies have been proposed that detect not only the presence or absence of image defects, but also the types of defects such as streaks, fireflies, spots, smudges, and unevenness.

[0003] For example, Patent Document 1 discloses an inspection system that determines the presence or absence of a defect in a first inspection step, and when it is determined that a defect exists in the first inspection step, identifies the type of the defect using a neural network in a second inspection step.

PRIOR ART DOCUMENTS

PATENT DOCUMENTS

[0004]

PATENT DOCUMENT 1

SUMMARY OF THE INVENTION

PROBLEMS TO BE SOLVED BY THE INVENTION

[0005] On the other hand, in recent years, user needs for image quality have been increasing. Therefore, it is difficult to satisfy user needs by merely notifying the type of defect. For example, in order to satisfy high user needs, it is necessary to have a response from an expert with high printed matter inspection capability, who can visually check the paper on which the image is formed and identify finer image defects in comparison with the above-mentioned image defects.

[0006] However, relying on experts for this task increases operational costs and securing expert personnel is not easy. Therefore, there was a challenge in appropriately and easily detecting image defects to satisfy user needs.

[0007] The purpose of this disclosure is to provide an image inspection system, an image inspection apparatus, and an image inspection method that can easily and appropriately detect image defects. [Means for solving the problem]

[0008] The image inspection system related to this disclosure is A first image defect identification unit reads an image formed on a recording medium and performs a first identification process to detect image defects from the formed image and identify the type of the image defect, A second image defect identification unit performs a second identification process to further identify the type of image defect using a trained model, based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect is detected, and image formation conditions relating to the image defect. It is equipped with.

[0009] The image inspection device related to this disclosure is An information acquisition unit that acquires image defect information including information indicating the type of image defect contained in an image formed on a recording medium, a defective image containing the image defect, and image formation conditions related to the image defect. An image defect identification unit identifies the type of image defect using a trained model based on the image defect information, the defective image, and the image formation conditions. It is equipped with.

[0010] The image inspection method related to this disclosure is: A first identification process is performed to detect image defects from the formed image obtained by reading the image formed on the recording medium and to identify the type of the image defect. A second identification process is performed to further identify the type of image defect using a trained model, based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect is detected, and image formation conditions related to the image defect. [Effects of the Invention]

[0011] According to this disclosure, image defects can be easily and appropriately detected. [Brief explanation of the drawing]

[0012] [Figure 1] Figure 1 is a schematic diagram showing an example of the configuration of the image inspection system according to this embodiment. [Figure 2] Figure 2 is a schematic diagram showing an example of the configuration of an image forming apparatus according to this embodiment. [Figure 3] Figure 3 is a block diagram showing the main part of the control system of the image forming apparatus according to this embodiment. [Figure 4] Figure 4 is a functional block diagram showing an example of the configuration of the control unit shown in Figure 3. [Figure 5] Figure 5 is a schematic diagram showing an example of the configuration of the medical imaging apparatus according to this embodiment. [Figure 6] Figure 6 is a functional block diagram showing an example of the configuration of the control unit shown in Figure 5. [Figure 7] Figure 7 is a schematic diagram showing an example of the configuration of an information terminal according to this embodiment. [Figure 8] Figure 8 is a sequence diagram showing an example of the inspection process flow in the image inspection system according to this embodiment. [Figure 9] Figure 9 is a schematic diagram showing an example of a formed image. [Figure 10] Figure 10 is a schematic diagram showing an example of image defect information. [Figure 11] Figure 11 is a schematic diagram showing an example of image formation conditions. [Figure 12] Figure 12 is a schematic diagram showing other examples of image formation conditions. [Figure 13]Figure 13 is a schematic diagram showing still another example of image forming conditions. [Figure 14] Figure 14 is a schematic diagram for explaining identification of the type of image defect by the second specific processing. [Figure 15] Figure 15 is a schematic diagram showing an example of a summary page of a report. [Figure 16] Figure 16 is a schematic diagram showing an example of a detail page of a report. [Figure 17] Figure 17 is a schematic diagram showing an example of a report in a case where erroneous detection has occurred. [Figure 18] Figure 18 is a schematic diagram for explaining display of a plurality of image defects with overlapping regions. [Modes for Carrying Out the Invention]

[0013] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The present disclosure is not limited to the following embodiments, and various modifications can be made without departing from the gist of the present disclosure. Further, in each drawing, components denoted by the same reference numerals are identical or equivalent thereto, and this is common throughout the entire specification. The image forming system according to the present embodiment is capable of forming an image on a sheet such as a sheet of paper (recording medium).

[0014] [Configuration of Image Inspection System 1000] Figure 1 is a schematic diagram showing an example of the configuration of the image inspection system 1000 according to the present embodiment. As shown in Figure 1, the image inspection system 1000 includes an image forming apparatus 1, an image diagnostic apparatus 2, and an information terminal 3. In the image inspection system 1000, the image forming apparatus 1, the image diagnostic apparatus 2, and the information terminal 3 are connected to each other via the Internet, or a network 4 such as a LAN (Local Area Network) or a WAN (Wide Area Network).

[0015] Image forming apparatus 1 forms an image on a recording medium (paper) and performs a first inspection by executing a first identification process to identify the types of image defects contained in the formed image. Image diagnostic apparatus 2 is, for example, a server and performs a second inspection by executing a second identification process on the image defects identified by the first identification process. Information terminal 3 is, for example, a PC (Personal Computer) and gives instructions to image forming apparatus 1 and displays reports on image defects identified by image diagnostic apparatus 2.

[0016] (Image forming apparatus 1) Figure 2 is a schematic diagram showing an example of the configuration of the image forming apparatus 1 according to this embodiment. Figure 3 is a block diagram showing the main part of the control system of the image forming apparatus 1 according to this embodiment. As shown in Figure 2, the image forming apparatus 1 is composed of an image forming unit 100, a paper feeding unit 200, an inspection unit 300, and a paper discharge unit 400, etc.

[0017] The image forming unit 100 is a color image forming apparatus that uses an intermediate transfer method utilizing electrophotographic process technology. Specifically, the image forming unit 100 first transfers the toner images of each CMYK color formed on the photoreceptor to an intermediate transfer surface, superimposes the four toner images on the intermediate transfer surface, and then secondarily transfers them to paper to form an image. CMYK refers to C (cyan), M (magenta), Y (yellow), and K (black). Furthermore, in the following description, the process of image formation by the image forming unit 100 may be referred to as "printing."

[0018] The image forming unit 100 employs a tandem system in which photoreceptors corresponding to the four CMYK colors are arranged in series in the direction of travel of the intermediate transfer body, and the toner images of each color are sequentially transferred to the intermediate transfer body in a single step.

[0019] As shown in Figure 3, the image forming apparatus 1 includes an image processing unit 10, a forming unit 20, a fixing unit 30, a paper transport unit 40, an operation display unit 50, a communication unit 71, a storage unit 72, a control unit 500, a paper feeding unit 200, an inspection unit 300, and a paper discharge unit 400, etc.

[0020] The image processing unit 10 includes circuits and other components that perform image processing on the input image data according to initial settings or user settings. The forming unit 20 is controlled based on the image data that has undergone image processing.

[0021] The forming unit 20 includes an image forming unit 21, an intermediate transfer unit 22, and a secondary transfer unit 23, etc. The image forming unit 21 forms an image using colored toners of each color component based on image data from the image processing unit 10. For example, it forms an image using colored toners of each color component: Y (yellow), M (magenta), C (cyan), and K (black).

[0022] The image forming unit 21 includes, for each color, an exposure device, a developing device, a photoreceptor drum, a charging device, and a drum cleaning device. Since known technologies can be used for the exposure device, developing device, photoreceptor drum, charging device, and drum cleaning device, a detailed explanation is omitted here.

[0023] The intermediate transfer unit 22 includes an intermediate transfer belt, etc. The intermediate transfer belt is stretched in a loop shape across multiple support rollers and travels in the direction of arrow A. The support rollers include backup rollers, primary transfer rollers, and drive rollers, but these are not shown in the illustration here. The intermediate transfer belt is pressed against the photoreceptor drum, thereby transferring the toner image from the photoreceptor drum to the intermediate transfer belt.

[0024] The intermediate transfer unit 22 may also be configured to include a belt cleaning device having a plate-shaped belt cleaning blade or the like that slides against the surface of the intermediate transfer belt. The belt cleaning device removes any remaining transfer toner or the like that from the surface of the intermediate transfer belt after secondary transfer.

[0025] The secondary transfer unit 23 includes a secondary transfer roller and the like. The secondary transfer roller is pressed against the intermediate transfer belt. This creates a secondary transfer nip between the intermediate transfer belt and the secondary transfer roller.

[0026] In the secondary transfer unit 23, when paper is transported to the secondary transfer nip, the toner images of each color carried on the intermediate transfer belt are transferred onto the paper all at once. The paper on which the toner images have been transferred is then transported toward the fuser unit 30 by the secondary transfer rollers.

[0027] Furthermore, the secondary transfer unit 23 may be a belt-type secondary transfer unit having a secondary transfer belt stretched across multiple support rollers, instead of a roller-type secondary transfer unit having a secondary transfer roller or the like.

[0028] The fixing unit 30 includes a fixing roller and a pressure roller. The fixing roller is heated to a predetermined fixing temperature, and the pressure roller forms a fixing nip that grips and transports the paper between itself and the fixing roller. In this fixing unit 30, the toner image is fixed to the paper by heating and pressurizing the paper that has been transported after secondary transfer of the toner image using the fixing nip.

[0029] The paper transport unit 40 includes a transport path unit 41, etc. The transport path unit 41 consists of, for example, a path for transporting paper, a plurality of transport rollers, and a drive motor that rotates these transport rollers. The transport path unit 41 transports the paper supplied from the paper feeding unit 200 to the secondary transfer unit 23 and the fixing unit 30, and then transports it to the paper discharge unit 400.

[0030] In the transport path section 41, a pair of registration rollers 41a is provided upstream of the secondary transfer unit 23 in the paper transport direction D. The pair of registration rollers 41a adjusts the transport timing and orientation (angle) of the paper transported to the secondary transfer unit 23.

[0031] The operation display unit 50 is composed of, for example, a liquid crystal display (LCD) with a touch panel, and functions as both a display unit and an operation unit. The display unit displays various operation screens, image status displays, and the operating status of each function in accordance with display control signals input from the control unit 500. The operation unit is equipped with various operation keys such as a numeric keypad and a start key, accepts various input operations from the user, and outputs operation signals to the control unit 500.

[0032] The paper feeding unit 200 is located upstream of the image forming unit 100 in the paper transport direction D and is connected to the image forming unit 100. The paper feeding unit 200 has a plurality of paper trays 201, 202, and 203, and each paper tray 201, 202, and 203 holds sheets of paper (standard paper or special paper) identified based on basis weight or size, etc., according to a preset type. Based on instructions from the image forming unit 100, the paper feeding unit 200 supplies the instructed paper to the image forming unit 100.

[0033] The inspection unit 300 is positioned between the image forming unit 100 and the paper discharge unit 400 in the paper transport direction D. The inspection unit 300 is connected to the image forming unit 100 on the upstream side and to the paper discharge unit 400 on the downstream side.

[0034] The inspection unit 300 inspects the condition of the paper on which the image has been formed by the image forming unit 100. The inspection unit 300 has sensors 301 and 302 that read the paper transported from the image forming unit 100.

[0035] The sensor 301 is positioned above the path through which the paper is transported and facing the paper being transported. This configuration ensures that the sensor 301 reads the surface (top) of the paper being transported.

[0036] The sensor 302 is positioned below the paper transport path and facing the paper being transported. This configuration allows the sensor 302 to read the back (bottom) side of the transported paper.

[0037] Sensors 301 and 302 may include, for example, CCDs (Charge Coupled Devices) or CIS (Contact Image Sensors). Alternatively, line sensors extending over a width greater than or equal to the width of the paper (the width in the direction perpendicular to the paper transport direction D), such as a line scanner or a line sensor camera, may be used as sensors 301 and 302.

[0038] Sensors 301 and 302 read the image formed on the paper. Here, the inspection unit 300 is configured to have sensors 301 and 302 that read both sides (front and back) of the paper, as an example, but it is not limited to this configuration, and for example, it may have a configuration that has only sensor 301 that reads one side (front) of the paper.

[0039] The inspection unit 300 reads the image formed on the paper using sensors 301 and 302 and acquires the read image as the formed image. The inspection unit 300 also acquires paper information, including the basis weight and size of the paper on which the image is formed. The inspection unit 300 transmits the acquired formed image and paper information to the control unit 500, which will be described later.

[0040] The paper discharge unit 400 is located downstream of the inspection unit 300 in the paper transport direction D and is connected to the image forming unit 100 via the inspection unit 300. The paper discharge unit 400 has a paper discharge tray 401 and a purge tray 402, and discharges the transported paper into the paper discharge tray 401 or the purge tray 402.

[0041] The output tray 401 ejects paper on which an image has been formed in the image forming unit 100. The purge tray 402 ejects paper on which, for example, an image defect has been detected as a result of inspection by the inspection unit 300. Paper ejection to the output tray 401 and the purge tray 402 is controlled by the control unit 500.

[0042] If the size of the paper read by the inspection unit 300 does not meet the predetermined conditions, the paper output unit 400 may output the paper that does not meet the predetermined conditions from the purge tray 402, as indicated by the dashed arrow in Figure 2.

[0043] The control unit 500 is connected to the image processing unit 10, forming unit 20, fixing unit 30, paper transport unit 40, operation display unit 50, communication unit 71, storage unit 72, paper feeding unit 200, inspection unit 300, and paper discharge unit 400. The control unit 500 issues various instructions to each of these units and executes predetermined processes.

[0044] The control unit 500 includes a CPU (Central Processing Unit) 501, a ROM (Read Only Memory) 502, and a RAM (Random Access Memory) 503, among others. The CPU 501 reads a program corresponding to the processing content from the ROM 502, loads it into the RAM 503, and works in cooperation with the loaded program to centrally control the operation of each block of the image forming apparatus 1. At this time, various data such as a LUT (Look Up Table) stored in the storage unit 72 are referenced. The storage unit 72 is composed of, for example, a non-volatile semiconductor memory (so-called flash memory) or a hard disk drive.

[0045] The control unit 500 transmits and receives various types of data between the medical imaging device 2 and the information terminal 3, which are connected to a network 4 such as a LAN or WAN, via the communication unit 71. The communication unit 71 is composed of, for example, a communication control card such as a LAN card.

[0046] In this embodiment, the control unit 500 receives a print job from, for example, the information terminal 3, and forms an image on the paper, which is the recording medium, based on the received print job. A print job is information that includes print instructions for forming a predetermined image on the paper. In this embodiment, the print job includes information for generating an inspection image.

[0047] The inspection image is used to check the condition of the paper, the ink ejection status, etc., and is, for example, a single-color image, a so-called solid-color image. However, the inspection image is not limited to this example; for example, it may be a multi-color image using different colors for each predetermined area.

[0048] Furthermore, the control unit 500 performs a first identification process based on various information, including the formed image and paper information, received from the inspection unit 300. Details of the first identification process will be described later. In addition, the control unit 500 transmits image defect information, which includes information indicating the type of image defect identified by the first identification process, a defect image, which is an image including the region where the defect was detected, and image formation conditions to the image diagnostic device 2 via the communication unit 71. The image formation conditions indicate the environmental conditions when forming an image on the paper.

[0049] Figure 4 is a functional block diagram showing an example of the configuration of the control unit 500 in Figure 3. As shown in Figure 4, the control unit 500 includes an information acquisition unit 511, an image defect identification unit 512, a defective image extraction unit 513, an image formation condition acquisition unit 514, a printing control unit 515, an inspection control unit 516, and a communication control unit 517.

[0050] The information acquisition unit 511 acquires various types of information transmitted from the image diagnostic device 2 or information terminal 3 connected to the network 4, or from various parts of the image forming apparatus 1, via the communication unit 71. For example, the information acquisition unit 511 acquires print jobs transmitted from the information terminal 3. The information acquisition unit 511 also acquires formed images on paper that have been read by the inspection unit 300.

[0051] The image defect identification unit 512 detects image defects contained in the formed image on the paper acquired by the information acquisition unit 511. The image defect identification unit 512 then identifies the type of image defect detected. The image defect identification unit 512 corresponds to the "first image defect identification unit" of this disclosure.

[0052] The defect image extraction unit 513 extracts the region containing the image defect identified by the image defect identification unit 512 from the formed image and generates a defect image.

[0053] The image formation condition acquisition unit 514 acquires image formation conditions. The image formation conditions indicate environmental conditions when forming an image on paper, and include, for example, at least one of "temperature and humidity," "paper information," "position information," "toner information," "screen information," and "coverage information."

[0054] "Temperature and humidity" refers to the temperature and humidity around the image forming apparatus 1 when forming an image on paper. "Paper information" refers to information about the paper on which the image is formed, such as the basis weight and size. "Position information" refers to information indicating the position on the paper of the area where the image defect occurred. "Toner information" refers to information about the toner used when forming the area where the image defect occurred. "Screen information" refers to information about the screen used in the area where the image defect occurred. "Coverage information" refers to information about the cumulative usage amount of components (electrode strip, photoreceptor) over a predetermined period for the area where the image defect occurred.

[0055] The print control unit 515 controls the image forming unit 100 to form an image on the paper based on the print job acquired by the information acquisition unit 511. The inspection control unit 516 controls the inspection unit 300 to read the formed image on the paper.

[0056] The communication control unit 517 controls the communication unit 71 to exchange various types of information with the image diagnostic device 2 or information terminal 3 connected to the network 4. For example, the communication control unit 517 controls the communication unit 71 to transmit image defect information, defective images, and image formation conditions to the image diagnostic device 2.

[0057] (Medical imaging device 2) Figure 5 is a schematic diagram showing an example of the configuration of the medical imaging apparatus 2 according to this embodiment. As shown in Figure 5, the medical imaging apparatus 2 includes a control unit 700, a communication unit 271, and a storage unit 272. The medical imaging apparatus 2 corresponds to the "image inspection apparatus" of this disclosure.

[0058] The control unit 700 includes a CPU 701, ROM 702, and RAM 703, among other components. The CPU 701 reads a program corresponding to the processing content from the ROM 702, loads it into the RAM 703, and works in cooperation with the loaded program to centrally control the operation of each block of the image diagnostic device 2.

[0059] Furthermore, the control unit 700 transmits and receives various types of data between the image forming apparatus 1 and the information terminal 3, which are connected to a network 4 such as a LAN or WAN, via the communication unit 271. The communication unit 271 is composed of, for example, a communication control card such as a LAN card.

[0060] In this embodiment, the control unit 700 performs a second identification process using the learned model 280 stored in the storage unit 272. Details of the second identification process will be described later. The control unit 700 also creates a report including the results of the second identification process and transmits the created report to the information terminal 3 via the communication unit 271.

[0061] The storage unit 272 is composed of, for example, a non-volatile semiconductor memory (so-called flash memory) or a hard disk drive. In this embodiment, the storage unit 272 pre-stores a trained model 280. The trained model 280 is pre-generated by machine learning such as deep learning, using training data in which defective images are input data and the types of image defects are output data.

[0062] Figure 6 is a functional block diagram showing an example of the configuration of the control unit 700 in Figure 5. As shown in Figure 6, the control unit 700 includes an information acquisition unit 711, an image defect identification unit 712, a report creation unit 713, and a communication control unit 714.

[0063] The information acquisition unit 711 acquires various types of information transmitted from the image forming apparatus 1 or information terminal 3 connected to the network 4 via the communication unit 271. For example, the information acquisition unit 711 acquires image defect information, defective images, and image forming conditions transmitted from the image forming apparatus 1.

[0064] The image defect identification unit 712 identifies the type of image defect contained in the defective image based on the image defect information, defective image, and image formation conditions acquired by the information acquisition unit 711. Specifically, the image defect identification unit 712 uses a trained model 280 stored in the storage unit 272 to output the type of image defect when given image defect information, a defective image, and image formation conditions as input. The image defect identification unit 712 corresponds to the "second image defect identification unit" of this disclosure.

[0065] The report generation unit 713 creates a report containing information about the identified image defects based on the results of the processing performed by the image defect identification unit 712. Specific details of the report will be described later.

[0066] The communication control unit 714 controls the communication unit 271 to exchange various types of information with the image forming apparatus 1 or information terminal 3 connected to the network 4. For example, the communication control unit 714 controls the communication unit 271 to transmit the report created by the report creation unit 713 to the information terminal 3.

[0067] (Information terminal 3) Figure 7 is a schematic diagram showing an example of the configuration of the information terminal 3 according to this embodiment. As shown in Figure 7, the information terminal 3 includes a control unit 900, a communication unit 371, a storage unit 372, and an input / output unit 373.

[0068] The control unit 900 controls the entire information terminal 3. For example, the control unit 900 controls various communications exchanged between the image forming apparatus 1 and the image diagnostic apparatus 2 via the communication unit 371, as well as user input and output to the outside via the input / output unit 373.

[0069] The control unit 900 includes a CPU 901, ROM 902, and RAM 903, among other components. The CPU 901 reads a program corresponding to the processing content from the ROM 902, loads it into the RAM 903, and works in cooperation with the loaded program to centrally control the operation of each block of the information terminal 3.

[0070] Furthermore, the control unit 900 transmits and receives various types of data between the image forming apparatus 1 and the image diagnostic apparatus 2, which are connected to a network 4 such as a LAN or WAN, via the communication unit 371. The communication unit 371 is composed of, for example, a communication control card such as a LAN card.

[0071] The memory unit 372 stores various types of data. For example, the memory unit 372 stores reports received from the medical imaging device 2.

[0072] The input / output unit 373 includes an output device such as a liquid crystal display or an organic EL display, and an input device such as a keyboard, a mouse, or a touch panel superimposed on the screen of the display device. The input / output unit 373 corresponds to the "display unit" in this disclosure.

[0073] The input / output unit 373 displays various information on the display device, which is the output device, and converts user input operations to the input device into operation signals and outputs them to the control unit 900.

[0074] [Inspection process] Figure 8 is a sequence diagram showing an example of the inspection process flow in the image inspection system 1000 according to this embodiment.

[0075] First, in step S31, the control unit 900 of the information terminal 3 generates a print job for forming an inspection image on the recording medium, which is paper, based on instructions from the user. Then, the control unit 900 transmits the generated print job to the image forming apparatus 1 via the communication unit 371 (sequence SEQ1). As a result, the image forming apparatus 1 receives the print job from the information terminal 3 via the communication unit 71.

[0076] When the information acquisition unit 511 of the control unit 500 in the image forming apparatus 1 acquires a print job, in step S11, the print control unit 515 controls the forming unit 20, etc., and forms an inspection image on the paper based on the acquired print job.

[0077] In step S12, the inspection control unit 516 controls the inspection unit 300 from reading the inspection image formed on the paper. As a result, the inspection unit 300 uses sensors 301 and 302 to read the inspection image formed on the paper and outputs the resulting formed image. The information acquisition unit 511 acquires the formed image output from the inspection unit 300.

[0078] Figure 9 is a schematic diagram showing an example of a formed image. The formed image is an inspection image formed on a paper recording medium, which is then read by the inspection unit 300. In this case, the inspection image is, for example, a monochrome image, a so-called solid color image.

[0079] In this example, the formed image contains three image defects. In Figure 9, Image01, Image02, and Image03, which are enlarged in callouts, are the defective images described later.

[0080] In step S13 of Figure 8, the image defect identification unit 512 performs predetermined image processing on the acquired formed image to detect image defects, which are defects contained in the formed image. The image defect identification unit 512 then performs a first identification process to perform a first inspection to identify the type of image defect detected. The image defect identification unit 512 also generates image defect information, which is information about the identified image defect.

[0081] Figure 10 is a schematic diagram showing an example of image defect information. The image defect information includes, for each identified image defect, the type of image defect and information about the region where the image defect occurred.

[0082] As shown in Figure 10, the image defect information includes information about one or more image defects detected from a single formed image. For each image defect, the image defect information includes, for example, a "diagnosis ID," "defect type," "region information," and "image information."

[0083] The "Diagnostic ID" is a unique number assigned to each formed image in which image defects are detected. If multiple image defects are detected in a single formed image, the same diagnostic ID is assigned. In this example, the same diagnostic ID "11111" is assigned to three image defects.

[0084] The "type of defect" is the type of image defect identified from the formed image. The types of image defects identified by the first identification process include at least one of the following: "streaks," "fireflies," "spots," "smudges," and "unevenness." In this example, the first and second image defects are identified as "fireflies," and the third type of image defect is identified as "smudges."

[0085] "Region information" refers to information about the region where an image defect has occurred. The region where an image defect has occurred is, for example, the x-coordinate, y-coordinate, and area of ​​the image defect. Note that the x-coordinate and y-coordinate are, for example, the coordinates of the top-left corner of the defective image extracted by the defective image extraction unit 513.

[0086] "Image information" is the file name of the defective image from which the region containing the image defect has been extracted. The defective image is an image extracted from the formed image by the defective image extraction unit 513 (see Figure 9). In this example, the file names of the defective images corresponding to the first to third image defects are "Image01", "Image02", and "Image03", respectively.

[0087] Next, in step S14 of Figure 8, the defect image extraction unit 513 extracts an image of the region containing the image defect from the formed image acquired by the information acquisition unit 511 as a defect image. Also, in step S15, the image formation condition acquisition unit 514 acquires the image formation conditions, which are one of the pieces of information necessary for the second specific processing described later, using a sensor (not shown) provided in the image forming apparatus 1.

[0088] Figure 11 is a schematic diagram showing an example of image formation conditions. Image formation conditions refer to the environmental conditions when an image is formed on paper. In this example, various types of information are associated with the diagnostic ID, and "paper information" and "environmental information" are associated with the diagnostic ID as image formation conditions.

[0089] "Paper information" refers to information about the paper on which the inspection image is formed. In this example, the paper information includes "type," "basis weight," "width," and "height." "Type" is the type of paper on which the inspection image is formed, for example, "plain paper" or "high-quality paper." "Basis weight" is the basis weight of the paper on which the inspection image is formed. "Width" and "height" are the dimensions of the paper on which the inspection image is formed.

[0090] "Environmental information" refers to information about the environment in which the inspection image is formed on the paper. In this example, the environmental information is "temperature" and "humidity".

[0091] Figure 12 is a schematic diagram illustrating another example of image formation conditions. In this example, screen information is associated with a diagnostic ID as an image formation condition.

[0092] Screen information refers to information about the screen used for the examination image. Here, it is assumed that the same screen is used for the entire examination image. However, if different screens are used for specific regions of the examination image, the screen information may be associated with both the diagnostic ID and the defective image indicating the region where the image defect occurs.

[0093] Figure 13 is a schematic diagram illustrating yet another example of image formation conditions. In this example, toner information is associated with the diagnostic ID as an image formation condition. Toner information is information about the toner used in the examination image. Note that if the examination image has multiple colors, different colored toners may be used for each predetermined area. Therefore, in addition to the diagnostic ID, toner information is also associated with the defect image, which indicates the area where the image defect occurred.

[0094] Furthermore, image formation conditions include, for example, positional information indicating the location of the region where the image defect occurred. As shown in Figure 10, positional information is included in the image defect information, so its explanation is omitted here.

[0095] Returning to Figure 8, the communication control unit 517 transmits image defect information, defective images, and information indicating image formation conditions to the image diagnostic device 2 via the communication unit 71 (sequence SEQ2). As a result, the image diagnostic device 2 receives image defect information, defective images, and information indicating image formation conditions from the image forming device 1 via the communication unit 271.

[0096] When the information acquisition unit 711 of the control unit 700 in the diagnostic imaging device 2 acquires information indicating image defect information, a defective image, and image formation conditions, in step S21, the image defect identification unit 712 performs a second identification process based on the acquired information and conducts a second inspection to identify the type of image defect.

[0097] In the second identification process performed during the second inspection, image defect information, defective images, and image formation conditions are input to a pre-trained model 280 stored in the memory unit 272. The type of image defect is then output from the pre-trained model.

[0098] In this embodiment, the image defect identification unit 712 uses the trained model 280 stored in the storage unit 272 to output the type of image defect when it receives image defect information and image formation conditions for each defective image as input. The number of image defect types output at this time is greater than the number of image defect types identified by the image defect identification unit 512. That is, the image defects identified by the image defect identification unit 712 are classified into more types than the image defects identified by the image defect identification unit 512 in the image forming apparatus 1.

[0099] The types of image defects identified at this time include, for example, at least one of the following: "streaks," "fireflies," "spots," "scratches," "paper wrinkles," "toner stains," "dust," "other stains," "bleeding," "leading edge blurring," "trailing edge blurring," "moire," and "poor density."

[0100] Figure 14 is a schematic diagram illustrating the identification of image defect types by the second identification process. In the second identification process, a trained model is used to perform a more detailed identification of the image defect types identified in the first identification process. Specifically, for each defective image corresponding to the image defect detected in the first identification process, a score resulting from the output of a trained model for multiple image defect types is associated.

[0101] In this example, for the defective image "Image01," the highest score is 0.78 when the type of image defect is "scratch." Therefore, the type of image defect corresponding to the defective image "Image01" is identified as "scratch."

[0102] Furthermore, regarding the defective image "Image02," the highest score was 0.69 when the type of image defect was "spots." Therefore, the type of image defect corresponding to the defective image "Image02" is identified as "spots."

[0103] Furthermore, regarding the defective image "Image03," the highest score was 0.45 when the type of image defect was "paper wrinkles." Therefore, the type of image defect corresponding to the defective image "Image03" is identified as "paper wrinkles."

[0104] In step S22 of Figure 8, the report creation unit 713 creates a report based on the results of the second specific processing. Once the report is created, the communication control unit 714 transmits the created report to the information terminal 3 via the communication unit 271 (sequence SEQ3). As a result, the information terminal 3 receives the report from the image diagnostic device 2 via the communication unit 371.

[0105] When the control unit 900 of the information terminal 3 receives a report, in step S32, the control unit 900 displays the received report using the input / output unit 373.

[0106] The report displayed on information terminal 3 includes an overview page and a detail page. Figure 15 is a schematic diagram showing an example of the report's overview page. Figure 16 is a schematic diagram showing an example of the report's detail page.

[0107] As shown in Figure 15, the summary page primarily displays a general overview of the inspection results for image defects. Specifically, the summary page describes the date and time the inspection was performed, the diagnostic ID, and the final judgment result. In this example, it states that the inspection performed on April 1, 2024, ultimately found the paper or formed image to be abnormal.

[0108] The summary page also describes the types of image defects detected and identified in the first inspection (first identification process), and the types of image defects identified in the second inspection (second identification process) for those same image defects. In this example, it is described that the first inspection identified image defects that were "fireflies," "spots," and "stains," and that the second inspection identified each of these image defects as "paper wrinkles," "scratches," and "toner stains."

[0109] In this way, the summary page includes not only the types of image defects identified in the second examination, which is the final identification result, but also the types of image defects identified in the first examination, allowing users to confirm the cause of the image defect determination.

[0110] As shown in Figure 16, the details page displays detailed inspection results for image defects detected from the formed image. Specifically, the details page displays the entire formed image. In addition, the areas of image defects identified in the second inspection are marked, and labels indicating the type of image defect are added near the markings.

[0111] In this example, a circular marking A is applied to the area where the identified image defect type is "firefly (paper wrinkles)," and the label "firefly (paper wrinkles)" is written near marking A. Similarly, circular markings B and C are applied to the areas where the image defect types are "spots (scratches)" and "stains (toner stains)," respectively, and "spots (scratches)" and "stains (toner stains)" are written near markings B and C, respectively. The report generation unit 713 of the control unit 700 creates a report with these markings and labels superimposed on the formed image.

[0112] A label might contain, for example, two names: the name outside the parentheses indicates the type of image defect identified by the first inspection, and the name inside the parentheses indicates the type of image defect identified by the second inspection.

[0113] The size of the marking represents the area of ​​the image defect. It is preferable to make the marking larger as the area of ​​the image defect increases, so that the size of the image defect can be easily determined. Note that the marking is not limited to a circular shape; it may also be rectangular or of other shapes. Furthermore, for example, the area of ​​the region corresponding to the marking may be displayed on the label.

[0114] On the details page, when a marked area is selected, the image defect corresponding to the selected marking is displayed as an enlarged image, as shown in the callout in Figure 16. This enlarged display of the image defect allows the user to easily and thoroughly examine the condition of the defect.

[0115] Furthermore, if the types of image defects identified differ between the first and second inspections, this may be noted in the report so that the user can easily confirm that it is a false positive.

[0116] Figure 17 is a schematic diagram showing an example of a report in the event of a false positive. As shown in Figure 17, for example, if the first inspection identifies an image defect as a stain, but the second inspection identifies the image defect as a streak, the label for that image defect should display "Stain (False positive for streak)". In this case, the marking and label colors may also be displayed in colors different from those used under normal circumstances.

[0117] This visually demonstrates that the type of image defect identified in the first inspection was a false positive. Therefore, the user can easily confirm that the result of the first inspection was a false positive.

[0118] Furthermore, it is possible that the first inspection detects multiple image defects, the regions of these multiple image defects overlap, and the type of image defect identified by the second inspection is the same. In this case, the multiple image defects identified in overlapping regions may be combined and displayed as a single image defect in the report.

[0119] Figure 18 is a schematic diagram illustrating the representation of multiple overlapping image defects. This example shows a case where the first inspection identifies two overlapping image defects as "fireflies," and the second inspection identifies the two image defects as "blurs."

[0120] In this case, the report may be configured to merge two overlapping image defects and then display a report with markings and labels applied to the merged image defect region.

[0121] Furthermore, when multiple image defects with overlapping regions are merged, it has the advantage of making it easier for users to understand the areas of the image defects when reviewing the report. On the other hand, when multiple image defects with overlapping regions are not merged, it has the advantage of making it easier for users to understand the number of image defects detected when reviewing the report.

[0122] Thus, there are advantages to both combining multiple image defects and not combining them. Therefore, it is preferable to decide whether or not to combine multiple image defects depending on the format desired by the user.

[0123] Furthermore, while this example explains how the results of the first and second tests should be presented together in the report, this is not the only example. For example, if the results of the first and second tests are identical, only one test result may be described. Moreover, if the score from the second test is lower than a predetermined value and the reliability of the classification result is low, the result of the second test may be omitted.

[0124] As described above, in the image inspection system 1000 according to this embodiment, a first identification process is performed to identify the type of image defect detected from the formed image obtained by reading the image formed on the recording medium. Then, a second identification process is performed to further identify the type of image defect using the trained model 280, based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect was detected, and image formation conditions related to the image defect.

[0125] Thus, in this embodiment, the type of image defect is identified in two stages, and the type of image defect is identified using a pre-trained model 280 generated by machine learning. Therefore, image defects can be easily and appropriately detected without requiring the response of experts as in the past.

[0126] Although embodiments have been described above, this disclosure is not limited to the embodiments described above, and various modifications and applications are possible without departing from the gist of this disclosure. For example, the report created by the medical imaging apparatus 2 is not limited to being displayed on the information terminal 3, but may also be displayed on the display unit of the operation display unit 50 in the image forming apparatus 1. [Explanation of Symbols]

[0127] 1. Image forming apparatus 2. Diagnostic imaging equipment 3. Information terminals 4 Network 10 Image Processing Unit 20 Formation part 21 Image forming unit 22 Intermediate Transfer Unit 23 Secondary Transfer Unit 30 Fixing section 40 Paper transport section 41 Conveyor Path Section 41a Resist Roller 50 Operation display section 71 Communications Department 72 Memory section 100 Image forming unit 200 Paper feed section 201, 202, 203 Paper feed trays 271 Communications Department 272 Memory section 280 pre-trained models 300 Inspection Department 301, 302 sensors 371 Communications Department 372 Storage section 373 Input / output section 400 Paper output section 401 Paper Output Tray 402 Purge Tray 500, 700, 900 Control Unit 501, 701, 901 CPUs 502, 702, 902 ROM 503, 703, 903 RAM 511 Information Acquisition Department 512 Image Defect Identification Unit 513 Defective Image Extraction Unit 514 Image Formation Condition Acquisition Unit 515 Printing Control Unit 516 Inspection Control Unit 517 Communication Control Unit 711 Information Acquisition Department 712 Image Defect Identification Unit 713 Report Creation Department 714 Communication Control Unit 1000 Image Inspection Systems

Claims

1. A first image defect identification unit reads an image formed on a recording medium and performs a first identification process to detect image defects from the formed image and identify the type of the image defect, A second image defect identification unit performs a second identification process to further identify the type of image defect using a trained model, based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect is detected, and image formation conditions relating to the image defect. An image inspection system equipped with the following features.

2. The system further includes a report generation unit that generates a report including the type of image defect identified by the second specific processing, The image inspection system according to claim 1.

3. The system further includes a display unit for displaying the aforementioned report. The image inspection system according to claim 2.

4. The report includes the type of image defect identified by the second specific processing, The image inspection system according to claim 2.

5. The aforementioned report creation unit, The type of image defect identified by the first specific process and the type of image defect identified by the second specific process are associated and created. The image inspection system according to claim 2.

6. The aforementioned report creation unit, A report is created that includes the formed image and an enlarged image obtained by enlarging the defective image. The image inspection system according to claim 2.

7. The aforementioned report creation unit, The report is created by superimposing a label indicating the type of image defect identified by the second specific processing onto the formed image. The image inspection system according to claim 6.

8. The information terminal includes the aforementioned display unit, The image inspection system according to claim 3.

9. The image forming conditions include at least one of the type, basis weight, and size of the recording medium. The image inspection system according to claim 1.

10. The image forming conditions include screen information for the defective image, The image inspection system according to claim 1.

11. The image formation conditions include toner information at the time the defective image was formed. The image inspection system according to claim 1.

12. The image forming conditions include information indicating the position on the recording medium. The image inspection system according to claim 1.

13. The types of image defects identified by the second specific processing are greater than the types of image defects identified by the first specific processing. The image inspection system according to claim 1.

14. An image forming apparatus having the first image defect identification unit, An image inspection apparatus having the second image defect identification unit, Equipped with, The image inspection system according to claim 1.

15. An information acquisition unit that acquires image defect information including information indicating the type of image defect contained in an image formed on a recording medium, a defective image containing the image defect, and image formation conditions related to the image defect. An image defect identification unit identifies the type of image defect using a trained model based on the image defect information, the defective image, and the image formation conditions. Equipped with, Image inspection device.

16. A first identification process is performed to detect image defects from the formed image obtained by reading the image formed on the recording medium and to identify the type of the image defect. A second identification process is performed to further identify the type of image defect using a trained model, based on image defect information including information indicating the type of image defect, a defective image including the region where the image defect was detected, and image formation conditions related to the image defect. Image inspection methods.

Images