scanner

Abstract

To provide a technique for increasing the possibility of obtaining a reading result suitable for an image of an original.SOLUTION: The MFP1 can access the generation AI server 200 that uses the trained model 201 via the communication IF14, and the trained model 201 has been trained so as to be able to output a recommended parameter related to generation of scan data based on input data. When there is a specific scan instruction via the user IF13, the MFP1 outputs a question related to generation of scan data by using the user IF13, receives an answer to the question, and transmits question information in which the question and the answer are associated to the AI server 200. Thereafter, in a case where the recommended parameter is received from the generation AI server 200 via the communication IF14, the document is scanned in accordance with the recommended parameter, and the output based on the scan result is performed.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The technical field disclosed in this specification relates to a scanner that reads an image of a document and outputs image data. [Background technology]

[0002] Conventionally, there are known scanners that read a document in accordance with parameters corresponding to various items of scan settings and output image data of the read image. For example, Patent Document 1 discloses a scanner that allows a user to select the type of document image and changes the amount of show-through correction depending on the selected type of image. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Publication No. 2020-92375 Summary of the Invention [Problem to be solved by the invention]

[0004] The technology disclosed in Patent Document 1 determines the amount of show-through correction based on the type of image selected by the user, but scan setting parameters are not limited to the amount of show-through correction, so there is room for improvement in order to obtain scanning results that are appropriate for the image of the document. [Means for solving the problem]

[0005] A scanner made to solve this problem is a scanner comprising a reading engine, a user interface, and a communication interface, wherein the scanner is further capable of accessing a server that uses a trained model via the communication interface, and the trained model has been trained to be able to output recommended parameters for generating scan data based on input data, and the scanner is further configured to: when a specific scan instruction is received via the user interface, perform a question process that outputs a question regarding the generation of scan data using the user interface and receives an answer to the question, and a question sending process that sends question information in which the question and the answer are associated to the server via the communication interface; and after performing the question sending process, when the scanner receives the recommended parameters output by the trained model based on the question information from the server via the communication interface, scan a document using the reading engine in accordance with the recommended parameters received from the server, perform a recommended scan process that generates recommended scan data showing an image based on the scan results, and perform output based on the recommended scan data generated in the recommended scan process.

[0006] The scanner disclosed in this specification outputs a question related to scanning and transmits question information, which associates the question with the answer, to a server that uses a trained model. This allows the trained model to output parameters that match the question information. After receiving recommended parameters output from the trained model from the server, the scanner scans a document using the recommended parameters to generate and output recommended scan data. With this scanner, a user is likely to obtain output results that are appropriate for the image of the document, even if they do not have knowledge of scanning parameters.

[0007] A control method for realizing the functions of the scanner, a computer program, and a computer-readable storage medium storing the computer program are also novel and useful. [Effects of the Invention]

[0008] The technology disclosed in this specification realizes a technology that increases the likelihood of obtaining a reading result that is suitable for an image on a document. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is an explanatory diagram showing a schematic configuration of an MFP according to an embodiment of the present invention. [Figure 2] 10 is a flowchart showing the procedure of a scan execution process. [Figure 3] 10A and 10B are explanatory diagrams showing examples of a standby screen and an AI support selection screen. [Figure 4] 10 is a flowchart showing the procedure of automatic processing. [Figure 5] 10 is a flowchart showing the procedure of a confirmation process. [Figure 6] FIG. 10 is an explanatory diagram showing an example of a confirmation screen. [Figure 7] FIG. 10 is an explanatory diagram showing an example of a NG reason selection screen. [Figure 8] FIG. 10 is an explanatory diagram showing an example of a parameter screen. [Figure 9] 10 is a flowchart showing the procedure of interactive processing. [Figure 10] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 11] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 12] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 13] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 14] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 15] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 16] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 17] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 18] FIG. 10 is an explanatory diagram showing an example of a question screen. [Figure 19] FIG. 10 is an explanatory diagram showing an example of a question screen. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of a scanner will be described in detail with reference to the accompanying drawings. This specification discloses a multifunction peripheral (hereinafter referred to as "MFP") having various functions including an image reading function and a communication function.

[0011] As shown in Fig. 1, the MFP 1 of this embodiment includes a controller 10 including a CPU 11 and a memory 12. The MFP 1 also includes a user interface (hereinafter referred to as "user IF") 13, a communication interface (hereinafter referred to as "communication IF") 14, a print engine 15, and a reading engine 16, which are electrically connected to the controller 10. The MFP 1 is an example of a scanner. Note that the controller 10 in Fig. 1 is a general term for hardware and software used to control the MFP 1, and does not necessarily represent a single piece of hardware actually present in the MFP 1.

[0012] The CPU 11 of the MFP 1 executes various processes in accordance with programs read from the memory 12 and based on user operations. The memory 12 of the MFP 1 stores various programs including an operating system (hereinafter referred to as "OS") 21, and various data including scan parameters 22, parameter range information 23, and question selection information 24. The memory 12 is also used as a work area when various processes are executed. The buffer provided in the CPU 11 is also an example of the memory 12. Details of the programs and data will be described later.

[0013] Note that an example of memory 12 is not limited to a ROM, RAM, HDD, etc. built into MFP 1, but may also be a storage medium that is readable and writable by CPU 11. For example, an external memory such as a USB memory or HDD connected to MFP 1, or a memory or HDD provided in a device connected to MFP 1 via communication IF 14 are also examples of memory.

[0014] Computer-readable storage media are non-transitory media. In addition to the above examples, non-transitory media also include recording media such as CD-ROMs and DVD-ROMs. Non-transitory media are also tangible media. On the other hand, electrical signals carrying programs downloaded from servers on the Internet are computer-readable signal media, which is a type of computer-readable medium, but are not included in non-transitory computer-readable storage media.

[0015] The user IF 13 includes, for example, a touch panel. The touch panel includes hardware for displaying a screen for notifying the user of information and hardware for accepting operations by the user. The user IF 13 may include a combination of a display and hardware buttons.

[0016] The communication IF 14 includes hardware for communicating with external devices. The communication IF 14 includes functions compatible with communication standards such as Wi-Fi (registered trademark), Ethernet (registered trademark), and USB. The MFP 1 may be equipped with multiple communication IFs 14 compatible with multiple communication standards.

[0017] Print engine 15 includes a configuration for printing an image on a print medium such as a sheet. The image formation method of print engine 15 is, for example, an electrophotographic method or an inkjet method. Print engine 15 may be configured to be capable of multi-color printing, or may be configured to perform only monochrome printing.

[0018] The reading engine 16 includes a configuration for scanning a document placed on the flatbed or a document placed on a document feeder and transported to a reading position, and generating scan data that is the scan result. The MFP1 of this embodiment is equipped with the reading engine 16 that can perform both color reading, which reads a document as a color image, and monochrome reading, which reads a document as a monochrome image.

[0019] As shown in FIG. 1, for example, the MFP 1 can connect to the Internet 100 via a communication IF 14 and can access a generation AI server 200 via the Internet 100. The generation AI server 200 includes a trained model 201. The generation AI server 200 is an example of a server that uses a trained model. The generation AI server 200 may be a dedicated server provided by a vendor of the MFP 1, or may be a server of an AI provider that is publicly available on the Internet 100. An example of an AI provider is OpenAI.

[0020] The trained model 201 has been trained in advance using various data so that it can output response data based on input data, for example, so that it can output recommended parameters for generating scan data. For example, a trained model on a server provided by an AI provider is trained using big data. Therefore, it is expected that it can analyze input data and identify recommended parameters with high accuracy. The trained model 201 may be a model prepared by a vendor of the MFP 1, for example.

[0021] The generation AI server 200 may be provided with an API for instructing the trained model 201 to output recommended parameters. For example, the MFP 1 may be able to instruct the trained model 201 to output recommended parameters by using the API provided in the generation AI server 200 for instructing the trained model 201 to output recommended parameters.

[0022] Alternatively, the generation AI server 200 may accept the instruction in the form of a prompt, i.e., in the form of a string. For example, the MFP 1 may instruct the trained model 201 to output recommended parameters by inputting input data and a prompt instructing the generation AI server 200 to output recommended parameters determined based on the input data.

[0023] Next, the procedure for scanning by the MFP1 will be described. The following processes basically refer to CPU processing in accordance with instructions written in a program. In other words, processes such as "judging," "extracting," "selecting," "calculating," "deciding," "identifying," "acquiring," "receiving," and "control" in the following description represent CPU processing. CPU processing also includes hardware control using an OS API. In this specification, the operation of each program will be described without mentioning the OS. In other words, in the following description, a statement to the effect that "Program B controls Hardware C" may also mean that "Program B controls Hardware C using the OS API." Furthermore, CPU processing in accordance with instructions written in a program may be described in abbreviated terms. For example, it may be described as "performed by the CPU." Furthermore, CPU processing in accordance with instructions written in a program may be described in abbreviated terms, such as "performed by Program A."

[0024] In this specification, the terms "notification," "alert," "notification," "reply," "response," and "answer" are used not only to mean the transmission of information to a person, but also to mean communication or exchange of information between devices or between components within a device. Note that the components within a device include software.

[0025] Note that "obtaining" is used as a concept that does not require a request. In other words, the process of receiving data without the CPU requesting it is also included in the concept of "the CPU obtaining data." Furthermore, "data" in this specification is represented as a bit string that can be read by a computer. Data with the same substantial meaning but different formats will be treated as the same data. The same applies to "information" in this specification. Furthermore, "requesting" and "instructing" are concepts that indicate outputting information indicating a request or an instruction to the other party. Furthermore, information indicating a request or an instruction will also be simply referred to as a "request" or "instruction."

[0026] Furthermore, the process by a CPU to determine whether information A indicates event B is sometimes conceptually described as "determining whether event B is true from information A." The process by a CPU to determine whether information A indicates event B or event C is sometimes conceptually described as "determining whether event B or event C is true from information A."

[0027] In addition, in this specification, a setting item may be simply referred to as a "setting." A setting value may be simply referred to as a "setting." A setting value may also be referred to as a "parameter." Furthermore, storing a setting value in a memory or the like may be simply referred to as a "setting." An operation for setting or an input for setting may be simply referred to as a "setting."

[0028] The procedure for the scan execution process for reading an original document and outputting scanned data using the MFP 1 will be described with reference to the flowchart in Fig. 2. This scan execution process is executed by the CPU 11 of the MFP 1, for example, when the power of the MFP 1 is turned on. Note that the MFP 1 can execute various processes in addition to the scan execution process shown in Fig. 2.

[0029] When the MFP 1 is in standby mode, the CPU 11 causes the user IF 13 to display a standby screen (S101). For example, as shown in FIG. 3A, the CPU 11 displays a standby screen 50 including a copy icon 51, a scan icon 52, and an AI support button 53. In addition to the above, the standby screen 50 also displays a plurality of icons that can accept instructions for various functions that can be executed by the MFP 1. By accepting an operation on each icon, the MFP 1 accepts selection of a function corresponding to the operated icon.

[0030] The CPU 11 determines whether an operation on the copy icon 51 or the scan icon 52 has been accepted (S102). If it is determined that neither an operation on the copy icon 51 nor an operation on the scan icon 52 has been accepted (S102: NO), the CPU 11 determines whether an operation on the AI ​​support button 53 has been accepted (S103). The copy icon 51 is an icon that accepts an instruction to start a copy procedure, and the copy procedure includes the execution of a scan. The scan icon 52 is an icon that accepts an instruction to start a scan procedure. Note that the various icons may also be referred to as buttons. Also, the various buttons may also be referred to as icons.

[0031] The AI ​​support button 53 is a button that accepts the selection of whether to enable or disable AI support. When AI support is enabled, the CPU 11 can use the trained model 201 to execute a scan based on recommended parameters output from the generation AI server 200. For example, as shown in FIG. 3(A), when the AI ​​support button 53 is set to "on," AI support is enabled.

[0032] When it is determined that an operation on the AI ​​support button 53 has been received (S103: YES), the CPU 11 switches AI support between "on" and "off" (S104). That is, if the AI ​​support was "on", the CPU 11 switches it to "off", and if the AI ​​support was "off", the CPU 11 switches it to "on".

[0033] After S104, or if it is determined that an operation on the AI ​​support button 53 has not been accepted (S103: NO), the CPU 11 waits until it accepts either an operation on the copy icon 51 or the scan icon 52, or an operation on the AI ​​support button 53. The MFP 1 can also accept instructions other than those related to scanning procedures, and can operate based on the accepted instructions. After the operation is completed, the CPU 11 proceeds to S101 and displays the standby screen 50.

[0034] When it is determined that an operation on the copy icon 51 or the scan icon 52 has been accepted (S102: YES), the CPU 11 determines whether AI support is "on" or not (S106). When an operation on the copy icon 51 or the scan icon 52 is accepted while AI support is "on", the CPU 11 can perform scanning with support from the AI.

[0035] If it is determined that AI support is not "on" (S106: NO), the CPU 11 executes a scan according to the conventional procedure (S108). That is, the CPU 11 uses the scan parameters 22 stored in the memory 12 to cause the reading engine 16 to execute a scan and generate scan data. That is, if the CPU 11 receives an operation on the copy icon 51 or the scan icon 52 while AI support is "off," the CPU 11 can execute a scan without AI support.

[0036] The scanning parameters 22 are parameters used as defaults for various setting items related to scanning. The scanning parameters 22 are a set of multiple parameters including scanning parameters and parameters related to image processing of the scanned results. The scanning parameters include, for example, a parameter indicating the reading resolution and a parameter indicating whether color scanning is performed. The image processing parameters include, for example, parameters related to contrast and brightness, and parameters related to correction processes such as thin line correction, background color correction, and tilt correction. In other words, the "scan data" generated by executing a scan is data obtained by performing image processing according to the parameters on the scanned results obtained by scanning according to the parameters. The scanning parameters 22 may further include a parameter indicating the type of image of the original and a parameter indicating the file format to be output.

[0037] The scan parameters 22 may be values ​​stored at the time of shipment of the MFP 1, or may be values ​​that can be changed by an administrator or user operation after shipment. The MFP 1 may also be capable of storing different values ​​as the scan parameters 22 for each output destination.

[0038] If it is determined that AI support is "ON" (S106: YES), the CPU 11 causes the user IF 13 to display an AI support selection screen (S111). For example, as shown in FIG. 3(B), the CPU 11 causes the AI ​​support selection screen 60 to be displayed, which includes an "Auto (no confirmation)" button 61, an "Auto (confirmation required)" button 62, an "Interactive" button 63, and an execute button 65.

[0039] The "Automatic (without confirmation)" button 61 is a button for selecting a scanning procedure in which a scan is performed using recommended scanning parameters output from the trained model 201, and scan data is generated and output. The "Automatic (with confirmation)" button 62 is a button for selecting a scanning procedure in which a scan is performed using recommended scanning parameters output from the trained model 201, and the generated scan data is output after the user has confirmed it. The "Interactive" button 63 is a button for selecting a scanning procedure that includes a procedure in which questions about the scan are asked of the user before the scan is performed, and the answers are received. Details of each procedure will be described later.

[0040] Then, the CPU 11 waits until it receives an operation on one of the buttons 61 to 63 and an operation on the execute button 65. Note that the execute button 65 is ready to receive an operation when one of the buttons 61 to 63 is selected. When an operation on the execute button 65 is received, the CPU 11 determines which of the buttons 61 to 63 the user has selected (S112).

[0041] When it is determined that the "Automatic (without confirmation)" button 61 or the "Automatic (with confirmation)" button 62 has been selected by the user (S112: Auto), the CPU 11 executes the automatic process (S114). Receiving an operation on the execute button 65 when the "Automatic (without confirmation)" button 61 or the "Automatic (with confirmation)" button 62 is selected is an example of a predetermined scan instruction. An operation on the execute button 65 when the "Automatic (without confirmation)" button 61 is selected is an example of a second type of scan instruction, and an operation on the execute button 65 when the "Automatic (with confirmation)" button 62 is selected is an example of a first type of scan instruction.

[0042] The procedure for the automatic process will be described with reference to the flowchart in Fig. 4. In the automatic process, the CPU 11 reads out the scanning parameters 22 stored in the memory 12 (S201). Then, the CPU 11 executes scanning using the read out scanning parameters 22 (S202). Specifically, the CPU 11 causes the reading engine 16 to read an image of a document set on the flatbed or document feeder, and generates scan data.

[0043] The CPU 11 transmits various data including the scan data generated in S202, the scan parameters 22 used for the scan in S202, and the parameter range information 23 to the generation AI server 200 via the communication IF 14 (S205). That is, in S205, the CPU 11 analyzes the various data using the trained model 201 and instructs the generation AI server 200 to return recommended parameters within the range of the parameter range information 23. For convenience, the data transmitted by the CPU 11 in S205 is also referred to as transmitted data.

[0044] The CPU 11 may instruct the generation AI server 200 to respond with recommended parameters by sending transmission data to the generation AI server 200 via a dedicated API. The CPU 11 may also instruct the generation AI server 200 to respond with recommended parameters by sending the transmission data and a prompt indicating the instruction to the generation AI server 200. Note that the scan data sent from the MFP1 to the generation AI server 200 may be the raw data obtained as a result of reading by the reading engine 16, i.e., RAW data, or may be data after processing such as format conversion by the MFP1.

[0045] The parameter range information 23 (see FIG. 1) is information indicating the settable range of a parameter for each setting item used in scanning. The parameter range information 23 is an example of the settable range of a parameter. The setting items used in scanning include, for example, color, resolution, contrast, brightness, image type, fine line correction, background color correction, and tilt correction. The parameter range information 23 is stored in advance in the memory 12. The CPU 11 may instruct the generation AI server 200 to respond with recommended parameters within the range of the parameter range information 23 by sending the information via a dedicated API, or by sending the transmission data and a prompt indicating the instruction.

[0046] The color item indicates whether or not color scanning is performed, and two configurable parameters for the color item are color and monochrome. The resolution item indicates the scanning resolution, and three configurable parameters for the resolution item are 200 dpi, 300 dpi, and 600 dpi. The contrast and brightness items are items related to correction of the scanning results, and each configurable parameter has a value from -5 to 5 in increments of 1. The image type item indicates the type of document image, and four configurable parameters are automatic, text, photo, and receipt. The thin line correction, background color correction, and skew correction items indicate whether or not to perform each correction on the scanning results, and each configurable parameter has two configurable parameters: yes and no. Note that the setting items and their value ranges of the parameter range information 23 described above are merely examples and are not limited to these.

[0047] By passing parameter range information 23 indicating the settable range of parameters to the generation AI server 200 and requesting recommended parameters corresponding to that settable range, the possibility of the generation AI server 200 recommending inappropriate parameters can be reduced.

[0048] Note that the generation AI server 200 may perform various processes on each piece of data received from the MFP1 without significantly altering its content before inputting it to the trained model 201. In this specification, the case where each piece of data transmitted by the MFP1 is input to the trained model 201 after undergoing various processes is also included in the category of the MFP1 inputting data to the trained model 201.

[0049] The generation AI server 200 can analyze the input data using the trained model 201 and generate answer data including information on recommended parameters. The generation AI server 200 outputs the generated answer data to the MFP 1. As a result, the MFP 1 receives information indicating the recommended parameters from the generation AI server 200 via the communication IF 14.

[0050] As with input data, the generation AI server 200 may perform various processes on the data output by the trained model 201 without significantly altering its content, and then transmit the data to the MFP 1. In this specification, the case where the MFP 1 receives data that has been subjected to various processes after being output by the trained model 201 is also included in the category of the MFP 1 receiving data output by the trained model.

[0051] After transmitting the transmission data to the generation AI server 200 in S205, the CPU 11 determines whether appropriate recommended parameters have been received as a response from the generation AI server 200 (S207). If the response received from the generation AI server 200 includes recommended parameters and includes parameters within the range of the parameter range information 23, the CPU 11 determines that appropriate recommended parameters have been received (S207: YES).

[0052] On the other hand, if no response has been received from the generation AI server 200, or if the response received from the generation AI server 200 is an inappropriate response, such as a response that does not include recommended parameters or a response that includes parameters outside the range of the parameter range information 23, the CPU 11 determines that appropriate recommended parameters have not been received (S207: NO). Even if a response from the generation AI server 200, i.e., a response from the trained model 201, is received, there is a possibility that the response does not comply with the instructions in the transmitted data. Therefore, if the received response does not comply with the instructions in the transmitted data transmitted in S205, the CPU 11 determines that appropriate recommended parameters have not been received, thereby reducing the possibility of performing a scan using inappropriate parameters.

[0053] If it is determined that appropriate recommended parameters have not been received (S207: NO), the CPU 11 determines whether to determine that the process has failed (S208). If it is determined that the process has not failed (S208: NO), the CPU 11 returns to the determination of S207.

[0054] If the CPU 11 determines that appropriate recommended parameters have not been received from the generation AI server 200 (S207: NO), it may resend the transmission data sent in S205 to the generation AI server 200 before making the determination in S208. Alternatively, if the CPU 11 determines that a failure has not been determined in S208 (S208: NO), it may resend the transmission data sent in S205 to the generation AI server 200 before making the determination in S207. Even if the same transmission data is resent to the generation AI server 200, i.e., re-input into the trained model 201, there is a possibility that the trained model 201 will output an answer that is different from the previous one. In other words, by resending the transmission data, there is a possibility that the trained model 201 will output an answer that includes appropriate recommended parameters.

[0055] If the CPU 11 determines that the request has failed (S208: YES), it determines that it has failed to obtain appropriate recommended parameters with the support of the generation AI server 200 (S209) and discards information such as the generated scan data. The CPU 11 may determine that the request has failed, for example, if a predetermined time has passed since the transmission of the transmission data in S205 without receiving a response from the generation AI server 200, or if only an inappropriate response has been received after a predetermined number of transmissions. The CPU 11 may also determine that the request has failed if, for example, the CPU 11 has performed the determination of whether to determine a failure in S208 a predetermined number of times without receiving appropriate recommended parameters after the transmission in S205. If the CPU 11 determines that the response received from the generation AI server 200 does not include appropriate recommended parameters, it may determine that the request has failed without resending the transmission data or repeating the determination in S207.

[0056] If it is determined that appropriate recommended parameters have been received (S207: YES), the CPU 11 determines whether the received recommended parameters are the same as the scan parameters 22 used for the scan in S202 (S211). Note that the recommended parameters are multiple parameters used for scanning, similar to the scan parameters 22, and are information including parameters corresponding to each of multiple setting items used for scanning.

[0057] If it is determined that there is a setting item for which the received recommended parameters and the scan parameters 22 have different parameters (S211: NO), the CPU 11 uses the received recommended parameters to have the reading engine 16 read the image of the document and generate scan data (S212).

[0058] After S212, or if it is determined that the recommended parameters and the scan parameters 22 are the same (S211: YES), the CPU 11 determines whether to request user confirmation (S215). If the recommended parameters and the scan parameters 22 are the same, there is a high possibility that the scan results will not change even if the scan is performed again. If the parameters are the same, the CPU 11 does not perform the rescan in S212, thereby avoiding unnecessary scanning.

[0059] In S215, the CPU 11 determines whether the user's selection received on the AI ​​support selection screen 60 (see FIG. 3(B)) was the "Leave it to me (without confirmation)" button 61 or the "Leave it to me (with confirmation)" button 62. If it is determined that the "Leave it to me (with confirmation)" button 62 was selected (S215: YES), the CPU 11 executes confirmation processing (S216).

[0060] The procedure of the confirmation process will be described with reference to the flowchart in Fig. 5. In the confirmation process, the CPU 11 causes the user IF 13 to display a confirmation screen (S301). The CPU 11 causes, for example, the confirmation screen 70A shown in Fig. 6(A). The CPU 11 may also cause the confirmation screen 70B shown in Fig. 6(B).

[0061] 6A includes an image 71 of the scan result shown in the scan data generated in S212 or the scan data generated in S202 when the parameters are the same, and a button 72 for receiving an instruction to display the confirmation screen 70B. Also, the confirmation screen 70B shown in FIG. 6B includes parameter information 73 showing the recommended parameters used in S212 or the scan parameters 22 used in S202 when the parameters are the same, and a button 74 for receiving an instruction to display the confirmation screen 70A.

[0062] That is, when the "Leave it to me (with confirmation)" button 62 is operated on the AI ​​support selection screen 60, the user can check the image shown in the generated scan data in image 71 on confirmation screen 70A, and can check the parameters used for the scan in parameter information 73 on confirmation screen 70B. Because the user can check both the image and the parameters, it is easy for the user to determine whether the results are acceptable.

[0063] Furthermore, both the confirmation screen 70A and the confirmation screen 70B include an OK button 75, a redo button 76, and a cancel button 77. The OK button 75 is a button that accepts an instruction to output the scan data. The redo button 76 is a button that accepts an instruction to redo the reception of parameters recommended by the trained model 201. The cancel button 77 is a button that accepts an instruction to cancel the procedure for generating scan data with AI support.

[0064] The user checks the displayed confirmation screen 70A or 70B and operates the OK button 75, the redo button 76, or the cancel button 77. The CPU 11 determines whether the operation of the OK button 75, the redo button 76, or the cancel button 77 has been accepted (S302). If it is determined that the operation of the OK button 75 has been accepted (S302: OK), the CPU 11 determines OK (S303) and ends the confirmation process.

[0065] That is, a user who considers the scan data acceptable requests output of the scan data by operating the OK button 75. When the OK button 75 is operated, the CPU 11 outputs the scan data generated in S212, or the scan data generated in S202 if the parameters are the same. Operating the OK button 75 is an example of an output instruction. The confirmation screen 70A or the confirmation screen 70B, which can accept an operation on the OK button 75, is both an example of an output confirmation screen.

[0066] If it is determined that an operation on the retry button 76 has been accepted (S302: Retry), the CPU 11 causes the user IF 13 to display a NG reason selection screen (S311), accepts the user's selection (S312), and generates reason information based on the accepted selection.

[0067] 7, the CPU 11 displays a NG reason selection screen 80 including a plurality of options 81 indicating the reason why the scan data is judged to be unacceptable, and an OK button 82. The user selects an option 81 indicating the reason why the scan data is judged to be unacceptable, and operates the OK button 82. Each option 81 corresponds to a parameter setting item used for scanning. Based on the option 81 selected in S312, the CPU 11 generates reason information including an instruction to correct the parameter of the setting item corresponding to the option 81. The reason information is information indicating the correction content for each setting item related to correcting the scan data, and S312 is an example of a correction content reception process.

[0068] The reason information may be information indicating the option 81 selected in S312, or may be information determined by the CPU 11 based on the selected option 81, for example, information about a setting item corresponding to the option 81. The CPU 11 may be able to accept the selection of multiple options 81 on the NG reason selection screen 80, or may be able to accept input of the OK button 82 without selecting any option.

[0069] Specifically, when "1. The characters are blurred" is selected on the NG reason selection screen 80, the CPU 11 generates reason information including a correction instruction to strengthen the parameters of the contrast item or an instruction to perform thin line correction. When "2. The image is dark" is selected, the CPU 11 generates reason information including an instruction to strengthen the brightness item. When "3. The image is tilted" is selected, the CPU 11 generates reason information including an instruction to perform tilt correction. When "4. The image is dirty" is selected, the CPU 11 generates reason information including an instruction to perform background color correction. When "5. The lines are broken" is selected, the CPU 11 generates reason information including an instruction to increase the resolution. When "6. Not color" is selected, the CPU 11 generates reason information including an instruction to change the color item to color reading.

[0070] The NG reason selection screen 80 may also include options that indicate instructions in the opposite direction to these options 81, such as "image is too bright" or "not monochrome." By displaying options 81 corresponding to each setting item of the scanning parameters and accepting the user's selection, the CPU 11 can generate reason information that includes clear instructions for correction.

[0071] The CPU 11 transmits various data including the scan data, scan parameters, and reason information to the generation AI server 200 via the communication IF 14 (S315). That is, in S315, the CPU 11 analyzes the various data using the trained model 201 and instructs the trained model 201 to return recommended parameters that reflect the reason information. S315 is an example of an update process. When the redo button 76 is operated, the CPU 11 does not output based on the scan data generated in S212 or the scan data generated in S202 when the parameters were the same, but instead transmits the various data to the generation AI server 200 again. Operating the redo button 76 is an example of a correction instruction. For convenience, the data transmitted by the CPU 11 in S315 is also referred to as redo data.

[0072] The CPU 11 may instruct the generation AI server 200 to respond with recommended parameters that reflect the reason information by sending retry data to the generation AI server 200 via a dedicated API. The CPU 11 may instruct the generation AI server 200 to respond with recommended parameters that reflect the reason information by sending retry data and a prompt indicating the instruction to the generation AI server 200. Note that if the trained model 201 is configured to output recommended parameters that reflect the reason information if the input data includes reason information, the API and prompt used by the CPU 11 in S315 may be the same as the API and prompt used by the CPU 11 in S205. Furthermore, the API and prompt used by the CPU 11 in S315 may be different from the API and prompt used by the CPU 11 in S205 and may be an API and prompt that clearly instructs the generation AI server 200 to reflect the reason information.

[0073] In S315, the CPU 11 transmits, as the scan data, the scan data generated in S212, or the scan data generated in S202 if the parameters are the same. In S315, the CPU 11 transmits, as the scan parameters, the recommended parameters used in S212, or the scan parameters 22 used in S202 if the parameters are the same. The reason information transmitted to the generation AI server 200 in S315 is an example of correction instruction data. Furthermore, redo data including reason information may be considered an example of correction instruction data. Furthermore, a portion of the redo data including reason information may be considered an example of correction instruction data.

[0074] In S315, the CPU 11 transmits retry data including the reason information received by the user's operation to the generation AI server 200, which increases the likelihood of receiving recommended parameters that are more in line with the user's intentions than previously from the generation AI server 200. Note that the CPU 11 may transmit retry data including the parameter range information 23 to the generation AI server 200 in S315.

[0075] Then, after transmitting the retry data to the generation AI server 200, the CPU 11 determines whether or not appropriate recommended parameters have been received as a response from the generation AI server 200 (S317). If it determines that appropriate recommended parameters have not been received (S317: NO), the CPU 11 determines whether or not to determine failure (S318). S317 is the same determination as S207, and S318 is the same determination as S208.

[0076] If it is determined that appropriate recommended parameters have been received (S317: YES), the CPU 11 determines whether the currently received recommended parameters are the same as the recommended parameters previously received in S207 (S319). If it is determined that the generation AI server 200 failed to obtain appropriate recommended parameters (S318: YES), or if it is determined that the received recommended parameters are the same as the previously received parameters (S319: YES), or if it is determined that the user has pressed the cancel button 77 on the confirmation screen 70A or 70B (S302: Cancel), the CPU 11 determines that the process has failed (S321), discards the generated scan data and the received recommended parameter information, and ends the confirmation process. Pressing the cancel button 77 is an example of a cancel instruction.

[0077] If the CPU 11 receives the same parameters as the previously recommended parameters, it may notify the user that the same parameters have been recommended and ask the user whether to cancel the scan or output the previous scan data. If the received recommended parameters are the same as the previous ones, the CPU 11 may resend the same redo data as in S315 to the generation AI server 200 and request again recommended parameters that reflect the reason information.

[0078] If it is determined that parameters that are not the same as those received last time (S319: NO), the CPU 11 performs a scan using the recommended parameters received this time (S322). That is, the CPU 11 transmits the recommended parameters received last time from the generation AI server 200 and the scan data obtained by scanning using the previous recommended parameters to the generation AI server 200, and when it receives recommended parameters obtained based on these, it scans again using the current recommended parameters. In particular, because the reason information is transmitted to the generation AI server 200, there is a high possibility that an output result close to the user's intention will be obtained.

[0079] After S322, the CPU 11 again proceeds to S301, where it displays the confirmation screen 70 based on the scan data generated by the scan in S322 on the user IF 13 (S301).The CPU 11 then again accepts operation of any of the OK button 75, the redo button 76, and the cancel button 77 (S302).

[0080] The CPU 11 performs confirmation by the user until it determines that the process is OK (S303) or that the process is failed (S321), and when it determines that the process is OK or failed, it ends the confirmation process and returns to the automatic process.

[0081] The recommended parameters received from the generation AI server 200 may not necessarily be in line with the user's intentions. When the "Automatic (with confirmation)" button 62 is selected on the AI ​​support selection screen 60 (see FIG. 3(B)), the user can ultimately select whether or not to output based on the scan data, thereby avoiding output based on scan data that the user does not intend. On the other hand, when the "Automatic (without confirmation)" button 61 is selected, the MFP 1 does not display the confirmation screen 70 (see FIG. 6) or accept instructions, thereby reducing the user's effort. The MFP 1 displays the "Automatic (without confirmation)" button 61 and the "Automatic (with confirmation)" button 62 and accepts the user's selection of whether or not to output after confirmation, thereby providing a high degree of processing flexibility.

[0082] Returning to the description of the automatic processing in FIG. 4, the CPU 11 determines whether or not the confirmation processing in S216 has determined that the data has been OK (S221). If it is determined that the data has been OK (S221: YES), the CPU 11 causes the user IF 13 to display a parameter screen (S225). For example, as shown in FIG. 8, the CPU 11 causes a parameter screen 90 to be displayed, which includes scan parameters 91 used to generate the scan data determined to be OK, a save button 92, and a do not save button 93. Then, the CPU 11 accepts operation on the save button 92 or the do not save button 93.

[0083] The CPU 11 determines whether or not an operation on the save button 92 on the parameter screen 90 has been accepted (S226). If it is determined that an operation on the save button 92 has been accepted (S226: YES), the CPU 11 saves each parameter being displayed as the scan parameters 91 in the memory 12 as the scan parameters 22 (S227). Note that the CPU 11 may display all parameters for all items related to the scan on the parameter screen 90, or may display only parameters that differ from the scan parameters 22 stored in the memory 12.

[0084] The parameters stored in memory 12 as scan parameters 22 are read out in S201 during the next scan, as described above, and used to execute the scan. In other words, S227 is a process of setting the scan parameters 91 used to generate the output based on the scan data that has been determined to be OK as the parameters to be used in executing the first scan in S202. The user's selection received by operating the Save button 92 or the Do not save button 93 on the parameter screen 90 is an example of a setting selection of whether to set the parameters as scan parameters.

[0085] If the recommended parameters used to generate the scan data determined to be OK are saved as scan parameters 22, the MFP 1 will use those parameters in the next scan in S202. Therefore, particularly if the next scan is of a similar document image, there is a high possibility that good scan results will be obtained from the start in the next scan in S202. As a result, the possibility of rescanning in S212 is reduced. Furthermore, since the MFP 1 accepts a selection of whether or not to save the recommended parameters as scan parameters 22, the user can select whether or not to use the current recommended parameters in the next scan in S202 onward.

[0086] In addition, if the scan parameters 91 used to generate the scan data determined to be OK match the scan parameters 22 read out in S201, the CPU 11 may not display the parameter screen 90 or accept the selection of whether to save or not.

[0087] After S227, or when it is determined that an operation on the do not save button 93 has been accepted (S226: NO), the CPU 11 ends the automatic process. Furthermore, when it is determined that the result of the confirmation process in S216 is not OK (S221: NO), the CPU 11 determines that the process has failed (S209) and ends the automatic process.

[0088] Returning to the description of the scan execution process in Fig. 2, after the automatic process of S114 is completed, the CPU 11 determines whether the automatic process has determined OK (S118). If it determines that the automatic process has determined failure rather than OK (S118: NO), the CPU 11 notifies the failure (S121) and terminates the scan execution process. In this case, the CPU 11 does not output the scan data.

[0089] If it is determined that the scan data is OK (S118: YES), the CPU 11 outputs the scan data that has been determined to be OK (S122). Also, after performing the conventional process (S108), the CPU 11 outputs the generated scan data (S122). The CPU 11 saves image data based on the scan data or prints the scan data based on the user's instructions.

[0090] Specifically, when an operation on the scan icon 52 on the standby screen 50 (see FIG. 3A) is accepted in S102, the CPU 11 accepts, via the user IF 13, in S122, a designation of a storage location to which the scan data will be output and a designation of a file format for saving image data based on the scan data. The CPU 11 then converts the image data based on the scan data into the designated file format and outputs the converted image data to the designated output destination. Note that the CPU 11 may accept the designation of the storage location and the designation of the file format before displaying the AI ​​support selection screen 60 (see FIG. 3B).

[0091] Output destinations for scanned data include, for example, a USB memory attached to the MFP 1, an information processing device connected to the MFP 1, an email recipient indicated by an email address registered on the MFP 1, and cloud storage to which the data can be uploaded from the MFP 1. File formats include, for example, JPEG, PDF, TIFF, and RAW. The ability to specify the output destination and file format makes it easy for users to use saved scanned data.

[0092] Furthermore, if an operation on the copy icon 51 on the standby screen 50 (see FIG. 3A) is accepted in S102, the CPU 11 causes the print engine 15 to perform printing based on the scan data in S122.

[0093] After S121 or S122, the CPU 11 ends the scan execution process. The CPU 11 may, for example, proceed to S101 and display the standby screen 50.

[0094] Next, the procedure when an operation on the execute button 65 is accepted while the "interactive" button 63 is selected on the AI ​​support selection screen 60 (see FIG. 3(B)) displayed in S111 will be described. When an operation on the execute button 65 is accepted and it is determined that the "interactive" button 63 is selected (S112: interactive), the CPU 11 executes interactive processing (S116). Accepting an operation on the execute button 65 while the "interactive" button 63 is selected is an example of a specific scan instruction.

[0095] The procedure for the interactive processing will be described with reference to the flowchart of FIG. 9. In the interactive processing, the CPU 11 displays a question screen on the user IF 13 based on the question selection information 24 (see FIG. 1) stored in the memory 12 (S401). The question selection information 24 includes a plurality of questions corresponding to each setting item of the scanning parameters and a plurality of options serving as answers to each question. The question selection information 24 may include a question regarding the characteristics of the image of the original document or a question prompting the user to select a file format to be output based on the scanned data. Furthermore, one question may correspond to a plurality of setting items.

[0096] For example, as shown in FIG. 10, the CPU 11 displays a question screen 101 including one question 101a, multiple options 101b that are answers to the question, and a Next button 101c that accepts an instruction to proceed to the next question. The question 101a on the question screen 101 corresponds to the color item among the scanning parameters, and asks whether color is required. The question 101a is an example of a color question. The options 101b include three types of answers: "color," "black and white," and "I don't know."

[0097] The CPU 11 accepts a selection from a plurality of options 101b on the currently displayed question screen 101 (S402). S402 is an example of question processing. When one of the options 101b is selected, the CPU 11 becomes able to accept an operation on the Next button 101c. When the selection is confirmed by pressing the Next button 101c (S402: YES), the CPU 11 stores question information indicating a combination of the question displayed in S401 and the answer accepted for that question (S403).

[0098] A user who requires color image data as the scan result can select "Color" from option 101b, and question information indicating a desire for color scanning will be stored. The question and its answering options are displayed on the screen, making it easy for the user to answer the question. Note that the selection from multiple options may be multiple-selectable, or may be single-choice, or whether multiple selection is possible may vary depending on the question.

[0099] The question information stored in S403 is sent to the generation AI server 200 in a later procedure and is used when the trained model 201 determines recommended parameters. For example, if the question information includes information indicating that "color" has been selected on the question screen 101, it is highly likely that the parameter for the color setting item among the recommended parameters will be "color." In other words, by sending question information including whether color is required, it is expected that the recommended parameters received from the generation AI server 200 will include parameters for appropriate color settings.

[0100] The multiple options for each question include an option of "I don't know," which indicates that the answer is unknown. "I don't know" is an example of unknown information. Requiring answers to all questions can be a burden for the user. By being able to select "I don't know" for questions that the user does not know, the burden on the user is reduced. When "I don't know" is selected, the CPU 11 may store information indicating that "I don't know" has been selected as question information, or may not store information regarding the question for which "I don't know" has been selected.

[0101] When the Next button 101c is operated, the CPU 11 determines whether or not there are any remaining questions (S405). The CPU 11 displays the multiple questions included in the question selection information 24 one by one on the question screen and accepts answers one by one in turn. If it is determined that there are remaining questions (S405: YES), the CPU 11 proceeds to S401, displays the next question on the question screen, accepts user input, and stores the accepted information. The CPU 11 repeats S401 to S403 until answers are obtained for all questions included in the question selection information 24.

[0102] By repeatedly displaying a question screen including a question and options for each question and accepting answers, the user can easily understand the content of the question and easily answer each question. Furthermore, by providing multiple questions, the user's intention becomes clearer, increasing the possibility of obtaining more suitable recommended parameters. Note that, instead of switching screens for each question, the CPU 11 may display a screen that displays multiple questions at once. Furthermore, instead of providing the next button 101c, the CPU 11 may automatically move to the next question screen when one of the options 101b is selected. Furthermore, a back button may be provided on the question screen, and the CPU 11 may return to the previous question screen when the back button is operated.

[0103] The question selection information 24 may include a usage question prompting the user to select the usage of the scan data. In this case, the CPU 11 displays a question screen 102 including various usage options 102b, as shown in FIG. 11, and accepts the user's selection. The trained model 201 is likely to determine, for example, the scan resolution, file format, and whether or not to perform image correction, depending on the usage of the scan data. By transmitting question information including the usage of the scan data, the recommended parameters received from the generation AI server 200 can be expected to include appropriate resolution, file format, whether or not to perform image correction, etc.

[0104] The question selection information 24 may also include a type question prompting the user to select the type of image to be scanned. In this case, the CPU 11 displays a question screen 103 including various image types as options 103b, as shown in FIG. 12, and accepts the user's selection. The trained model 201 is likely to determine, for example, whether to perform color scanning and the type of image correction, depending on the type of image to be scanned. By transmitting question information including the image type, the recommended parameters received from the generation AI server 200 can be expected to include appropriate image type, resolution, color selection, whether or not to perform image correction, etc.

[0105] The question selection information 24 may also include a brightness question prompting the user to answer about the brightness level or a contrast question prompting the user to answer about the contrast level. In this case, the CPU 11 displays question screens 104 and 105 including options 104b and 105b regarding brightness and contrast, as shown in FIGS. 13 and 14, and accepts the user's selection. The trained model 201 is likely to determine the brightness and contrast levels based on the user's selection. By transmitting question information including the brightness and contrast levels, the recommended parameters received from the generation AI server 200 are expected to include appropriate brightness and contrast levels.

[0106] The question selection information 24 may also include a format question prompting the user to select a file format to be output based on the scanned data. In this case, the CPU 11 displays a question screen 106 including various file formats as options 106b, as shown in Fig. 15, and accepts the user's selection. If the format question shown in Fig. 15 is not included, the CPU 11 may determine the file format based on the answer to another question.

[0107] The question selection information 24 may also include a correction question prompting the user to answer whether various types of image correction are necessary. In this case, the CPU 11 may display a question screen inquiring about the necessity of various types of correction and accept the user's selection, as shown in, for example, FIGS. 16 to 19 . As question screens for various correction questions, the CPU 11 may display, for example, a question screen 107 including the necessity of noise cancellation as an option 107b, a question screen 108 including the necessity of thin line correction as an option 108b, a question screen 109 including the necessity of background color correction as an option 109b, and a question screen 110 including the necessity of tilt correction as an option 110b. By accepting the necessity of each type of image correction and transmitting question information including the necessity of the image correction, the recommended parameters received from the generation AI server 200 can be expected to include information about the presence or absence of appropriate image correction.

[0108] 19, the CPU 11 displays a screen including an end button 110c instead of a next button as the question screen for the last question stored in the question selection information 24. This makes it easy for the user to know that this is the last question.

[0109] If it is determined that there are no remaining questions, that is, that question information for all questions has been stored (S405: NO), the CPU 11 transmits various data including the stored question information and parameter range information 23 to the generation AI server 200 (S411). The parameter range information 23 includes information indicating the setting items of the parameters used for scanning and is an example of item information. That is, in S411, the CPU 11 analyzes various data using the trained model 201 and instructs the trained model 201 to respond with recommended parameters within the range of the parameter range information 23. S411 is an example of a question transmission process. For convenience, the data transmitted by the CPU 11 in S411 will also be referred to as question transmission data.

[0110] The CPU 11 may instruct the generation AI server 200 to provide recommended parameters that reflect the question information by sending question transmission data to the generation AI server 200 via a dedicated API. The CPU 11 may instruct the generation AI server 200 to provide recommended parameters that reflect the question information by sending question transmission data and a prompt indicating the instruction to the generation AI server 200. Note that if the trained model 201 is configured to output recommended parameters that reflect the question information if the input data includes question information, the API and prompt used by the CPU 11 in S411 may be the same as the API and prompt used by the CPU 11 in S205. Furthermore, the API and prompt used by the CPU 11 in S411 may be different from the API and prompt used by the CPU 11 in S205 and may be an API and prompt that clearly instructs the generation AI server 200 to reflect the question information.

[0111] After transmission, the CPU 11 determines whether appropriate recommended parameters have been received as a response from the generation AI server 200 (S412). If it determines that appropriate recommended parameters have not been received (S412: NO), the CPU 11 determines whether to determine failure (S413). If it determines that failure has been determined (S413: YES), the CPU 11 determines failure (S414), discards the question information, and ends the interactive processing. S412 is the same determination as S207 in the automatic processing (see FIG. 4), and S413 is the same determination as S208 in the automatic processing.

[0112] If it is determined that appropriate recommended parameters have been received (S412: YES), the CPU 11 causes the reading engine 16 to read the image of the document using the received recommended parameters and generate scan data (S421). S421 is an example of recommended scan processing, and the scan data generated in S421 is an example of recommended scan data.

[0113] After the scan is performed, the CPU 11 executes a confirmation process (S422) to have the user confirm the generated scan data. The confirmation process of S422 is the process shown in FIG. 5 and is the same as the confirmation process of S216. Even in interactive processing, the CPU 11 displays the scanned image and recommended parameters on the user IF 13 and outputs them after receiving an output instruction from the user. This prevents the scanned results from being output unintended by the user.

[0114] Then, the CPU 11 determines whether the user's confirmation result in the confirmation process is OK (S423). If the user's confirmation result is not OK and it is determined to be a failure (S423: NO), the CPU 11 determines that the generation AI server 200 failed to obtain appropriate recommended parameters (S414), discards the generated scan data and received information on recommended parameters, and ends the interactive processing.

[0115] If it is determined that the user's confirmation result is OK (S423: YES), the CPU 11 causes the user IF 13 to display the parameter screen 90 (see FIG. 8) (S425), and receives the user's selection as to whether or not to save the recommended parameters (S426).

[0116] If it is determined that an operation on the save button 92 has been accepted (S426: YES), the CPU 11 saves each parameter being displayed as the scan parameters 91 in the memory 12 as the scan parameters 22 (S427). After S427, or if it is determined that an operation on the do not save button 93 has been accepted (S426: NO), the CPU 11 ends the interactive processing. S425 to S427 are the same processing as S225 to S227 in the automatic processing.

[0117] In interactive processing, before performing a scan, the MFP1 outputs a question about the scan, accepts an answer to the question, and transmits the question and the answer to the generation AI server 200 that uses the trained model 201, thereby receiving recommended parameters determined by the trained model 201 from the generation AI server 200. According to this procedure, unlike the automatic processing, the MFP1 transmits question information based on the user's answer to the question about the scan to the generation AI server 200 without performing a scan in advance, and it is expected that appropriate recommended parameters corresponding to the question information will be received. In other words, by the user providing answers that correspond to the image of the document, the likelihood of obtaining an output result based on parameters appropriate for the image of the document increases.

[0118] Returning to the explanation of the scan execution process in Fig. 2, after the interactive process in S116 is completed, the CPU 11 performs the same process as after the automatic process is completed. That is, if it is determined that the scan has failed (S118: NO), the CPU 11 notifies the failure (S121) and terminates the scan execution process. If it is determined that the scan has been successful (S118: YES), the CPU 11 outputs the scan data for which the scan has been successful (S122).

[0119] As described above in detail, when the automatic scan mode is selected, the MFP 1 scans a document and transmits the scan data generated by the scan, along with the scan parameters 22 used in the scan, to the generation AI server 200 that uses the trained model 201. Because the trained model 201 has been trained to be able to output recommended parameters, the MFP 1 is likely to be able to receive the recommended parameters from the generation AI server 200. After receiving the recommended parameters determined by the trained model 201 from the generation AI server 200, the MFP 1 rescans the document according to the received recommended parameters to generate scan data and performs output based on the scan data. This makes it highly likely that an output result based on parameters appropriate for the document image can be obtained without the user having to input information such as the type of image.

[0120] Furthermore, when the interactive mode is selected, the MFP 1 outputs a question about scanning, accepts an answer, and transmits question information associating the question and answer to the generation AI server 200 that uses the trained model 201. Because the trained model 201 has been trained to be able to output recommended parameters, the MFP 1 is likely to be able to receive the recommended parameters from the generation AI server 200. After receiving the recommended parameters determined by the trained model 201 from the generation AI server 200, the MFP 1 scans a document using the recommended parameters to generate scan data and performs output based on the scan data. As a result, even a user who has no knowledge of scanning parameters is likely to obtain an output result appropriate for the document.

[0121] The embodiments are merely examples and do not limit the present invention in any way. Therefore, the technology disclosed in this specification can naturally be improved and modified in various ways without departing from the spirit of the present invention. For example, the scanner is not limited to the MFP 1, and may be a copy machine or a fax machine as long as it has an image reading function and a communication function.

[0122] Furthermore, the display format in the embodiment is not limited to the illustrated example. For example, the types, number, and shapes of icons displayed on the standby screen 50 shown in FIG. 3(A) are not limited to the illustrated example. For example, the AI ​​support button 53 may be omitted. The MFP 1 may always perform processing with AI support, or may display copy icons and scan icons with AI support and copy icons and scan icons without AI support on the standby screen 50. Furthermore, the AI ​​support selection screen 60 shown in FIG. 3(B) may be a screen that can accept user selections, and the shapes of the buttons, etc. are not limited to the format shown in this figure.

[0123] Furthermore, in the embodiment, the MFP 1 transmits parameter range information 23 to the generation AI server 200 in S205 of the automatic processing (see FIG. 4) and S411 of the interactive processing (see FIG. 9), but this does not have to be the case. For example, if the recommended parameters received from the generation AI server are outside the range of the parameter range information 23, the MFP 1 may transmit the parameter range information 23 and request the recommended parameters again. For example, if the MFP 1 determines NO in S207 of FIG. 4, the MFP 1 may transmit transmission data including the parameter range information 23 to the generation AI server 200. For example, if the MFP 1 determines NO in S412 of FIG. 9, the MFP 1 may transmit question transmission data including the parameter range information 23 to the generation AI server 200.

[0124] Also, the determination in S211 of the automatic process (see FIG. 4) and the determination in S319 of the confirmation process (see FIG. 5) may be omitted. In other words, when the MFP 1 receives recommended parameters, it may scan again using the received recommended parameters even if the parameters are the same as those used previously.

[0125] Furthermore, in the confirmation process (see FIG. 5), the MFP 1 switches between displaying confirmation screen 70A of FIG. 6(A) and confirmation screen 70B of FIG. 6(B) in response to a button operation, but the MFP 1 may display a single screen that includes both image 71 and parameter information 73. Also, cancel button 77 on the confirmation screen may be omitted. For example, the MFP 1 may repeatedly request recommended parameters until OK button 75 is operated. Furthermore, in the embodiment, confirmation processing is always performed in interactive processing, but it need not be performed, and the MFP 1 may accept a user's choice as to whether or not to confirm.

[0126] In the embodiment, the process for accepting a user's request involves displaying the NG reason selection screen 80 (see FIG. 7) and question screens 101-110 (see FIGS. 10-19) on the user IF 13 and accepting a selection from each option, but this is not limitative. The MFP 1 may, for example, output a question by voice and accept an answer by voice input. In this case, the MFP 1 may transmit voice data to the generation AI server 200. The MFP 1 may also accept a free-form entry rather than being limited to a selection from options.

[0127] Furthermore, the questions and answer options shown on question screens 101 to 110 (see FIGS. 10 to 19) are merely examples and are not limiting. For example, if "color" is selected on question screen 101, MFP 1 may further output a question about color (hue, saturation, etc.).

[0128] Also, steps S225 to S227 of the automatic processing (see FIG. 4) and steps S425 to S427 of the interactive processing (see FIG. 9) may be omitted. That is, the MFP 1 does not have to save the recommended parameters in memory 12, or may always save them. Even if the recommended parameters are saved, the MFP 1 may save them after outputting based on the scan data. The MFP 1 may also be able to accept a selection as to whether or not to save the recommended parameters as part of the scan settings before receiving the recommended parameters.

[0129] Furthermore, in the embodiment, the MFP 1 is described as having both automatic processing (see FIG. 4) and interactive processing (see FIG. 9), but it need not have automatic processing. In that case, the AI ​​support selection screen 60 (see FIG. 3(B)) may include only the "Interactive" button 63. Alternatively, the MFP 1 may execute interactive processing when AI support is "on" and copy or scan is selected.

[0130] Furthermore, in the embodiment, the MFP 1 generates scan data using its own reading engine 16 and transmits the generated scan data to the generation AI server 200 via its own communication IF 14. However, the generation of scan data and the transmission of scan data may be performed by different devices. For example, scan data generated by a scanner may be received by a terminal device separate from the scanner, and transmitted to the generation AI server 200 via the terminal device's communication IF 14.

[0131] Furthermore, in the interactive processing of the embodiment, a question and answer options are displayed on the user IF 13 of the MFP 1, and the user is asked to select one. However, the provision of the question and the acquisition of the answer may be performed by a device other than the MFP 1. For example, a terminal device incorporating a predetermined application program may ask a question, acquire question information, and send the acquired question information to the MFP 1. Alternatively, the terminal device may send the question information to the generation AI server 200.

[0132] Furthermore, the generation AI server 200 is not limited to one that has the trained model 201, and may be able to use the trained model 201 of another server. In this case, the generation AI server 200 passes the various data it has received to the other server that has the trained model 201, and transmits answer data to the MFP1 based on the answer from the trained model 201.

[0133] In addition, in the embodiment, a configuration using the trained model 201 of the generation AI server 200 is exemplified, but instead of the trained model 201, it can also be applied to a program created based on the coding of a programmer.

[0134] Furthermore, in any flowchart or sequence diagram disclosed in the embodiments, the execution order of multiple processes in any multiple steps can be changed or executed in parallel as desired, as long as no contradictions occur in the processing content.

[0135] The processes disclosed in the embodiments may be executed by hardware such as a single CPU, multiple CPUs, or ASIC, or a combination thereof. The processes disclosed in the embodiments may be realized in various ways, such as a recording medium on which a program for executing the processes is recorded, or a method. [Explanation of symbols]

[0136] 1 MFP 13 User Interface 14 Communication Interface 15 Print Engine 16 reading engines 200 Generation AI Server 201 trained models

Claims

1. A reading engine and A user interface; a communication interface; A scanner comprising: The scanner further comprises: A server that uses a trained model is accessible via the communication interface, and the trained model has been trained to be able to output recommended parameters for generating scan data based on input data; The scanner further comprises: When a specific scan instruction is given via the user interface, a query process that uses the user interface to output a question regarding the generation of scan data and receive an answer to the question; a question transmission process for transmitting question information in which the question and the answer are associated with each other to the server via the communication interface; Run The scanner further comprises: After executing the question transmission process, when the recommended parameters output by the trained model based on the question information are received from the server via the communication interface, executes a recommended scan process to scan an original using the reading engine in accordance with the recommended parameters received from the server and generate recommended scan data representing an image based on the scan results; performing an output based on the recommended scan data generated in the recommended scan process; A scanner configured to:

2. 2. A scanner according to claim 1, In the question sending process, transmitting the question information and the parameter settable range to the server via the communication interface; The scanner After executing the question transmission process, when the recommended parameters corresponding to the settable range output by the trained model based on the question information and the settable range are received from the server via the communication interface, the recommendation scan process is executed. A scanner configured to:

3. 2. A scanner according to claim 1, The scanner after the recommended scan process is executed, an output confirmation screen capable of receiving an output instruction to perform output based on the recommended scan data generated by the recommended scan process is displayed using the user interface, and the output confirmation screen includes an image of the scan result indicated by the recommended scan data and the recommended parameters received from the server; When the output instruction is received via the output confirmation screen, output is performed based on the recommended scan data generated by the recommended scan processing. A scanner configured to:

4. 4. A scanner according to claim 3, The scanner After the recommended scan process is executed, the output confirmation screen capable of accepting the output instruction and the correction instruction is displayed using the user interface; When the output instruction is received via the output confirmation screen, output is performed based on the recommended scan data generated by the recommended scan processing; when the correction instruction is received via the output confirmation screen, output based on the recommended scan data generated in the recommended scan process is not performed, and an update process is executed, the update process being a process of transmitting the recommended parameters used in the recommended scan process and the recommended scan data generated in the recommended scan process to the server via the communication interface; The scanner further comprises: After the update process is executed, when the recommended parameters output by the trained model based on the recommended parameters and the recommended scan data are received from the server via the communication interface, the recommended scan process is executed, and in the recommended scan process after the update process is executed, the recommended scan data is generated according to the recommended parameters received from the server after the update process is executed; Even when the recommended scan process is executed after the update process is executed, the output confirmation screen is displayed using the user interface, and when the output instruction is received via the output confirmation screen, output is performed based on the recommended scan data generated in the recommended scan process after the update process is executed. A scanner configured to:

5. 5. A scanner according to claim 4, The scanner when the correction instruction is received via the output confirmation screen, output based on the recommended scan data is not performed, and a correction content receiving process is executed before the update process is executed, the correction content receiving process being a process of receiving correction content for the recommended scan data via the user interface; In the update process after the execution of the correction content reception process, the recommended parameters used in the scan in the recommended scan process, the recommended scan data generated in the recommended scan process, and correction instruction data based on the correction content received in the correction content reception process are transmitted to the server via the communication interface, The scanner further comprises: After the correction content acceptance process and the update process are executed, when the recommended parameters, the recommended scan data, and the recommended parameters output by the trained model based on the correction instruction data are received from the server via the communication interface, the recommended scan process is executed, and in the recommended scan process after the correction content acceptance process and the update process are executed, the recommended scan data is generated in accordance with the recommended parameters received from the server after the correction content acceptance process and the update process are executed; Even when the recommended scan process is executed after the correction content acceptance process and the update process are executed, the output confirmation screen is displayed using the user interface, and when the output instruction is accepted via the output confirmation screen, output is performed based on the recommended scan data generated in the recommended scan process after the correction content acceptance process and the update process are executed. A scanner configured to:

6. 2. A scanner according to claim 1, In the question processing, using the user interface to display a question screen including the question regarding scanning and a plurality of options serving as the answer to the question, and accepting a selection from the plurality of options via the question screen as the answer to the question; In the question sending process, transmitting the question information, in which the question including the plurality of options and the option selected as the answer to the question are associated with each other, to the server via the communication interface; A scanner configured to:

7. 2. A scanner according to claim 1, In the question processing, outputting a plurality of questions regarding the scan using the user interface and accepting an answer for each of the questions; In the question sending process, transmitting the question information, in which the plurality of questions and the answers for each of the questions are associated with each other, to the server via the communication interface; A scanner configured to:

8. 8. A scanner according to claim 7, In the question processing, displaying a question screen including one of the plurality of questions and a plurality of options serving as the answer to the one question, and accepting a selection from the plurality of options via the question screen as the answer to the one question; after receiving the answer to the one question, displaying a next question screen corresponding to a next question among the plurality of questions to which the answer has not been received, and receiving the answer to the next question via the next question screen; repeating the display of the question screen for each question and the reception of the answer to the question displayed on the question screen until there is no longer a need for questions; A scanner configured to:

9. 8. A scanner according to claim 7, In the question processing, outputting the question corresponding to at least one of a plurality of setting items used for scanning; In the question sending process, transmitting the question information, in which the plurality of questions and the answers for each of the questions are associated with each other, and item information, indicative of the plurality of setting items, to the server via the communication interface; The scanner After executing the question transmission process, when the recommended parameters corresponding to the plurality of setting items output by the trained model based on the question information and the item information are received from the server via the communication interface, the recommendation scan process is executed. A scanner configured to:

10. 8. A scanner according to claim 7, In the question processing, outputting a color question as one of the plurality of questions, prompting the user to answer whether color is required; In the question sending process, transmitting the question information, including whether or not color is required as an answer to the color question, to the server via the communication interface; A scanner configured to:

11. 8. A scanner according to claim 7, In the question processing, outputting a use question prompting the user to answer a use of the scan data as one of the plurality of questions; In the question sending process, transmitting the question information, including information indicating that the use of the scan data is an answer to the use question, to the server via the communication interface; A scanner configured to:

12. 8. A scanner according to claim 7, In the question processing, outputting a type question as one of the plurality of questions, the type of image to be scanned; In the question sending process, transmitting the question information, including the type of the image to be scanned being the answer to the type question, to the server via the communication interface; A scanner configured to:

13. 8. A scanner according to claim 7, In the question processing, outputting a brightness question as one of the plurality of questions, the brightness question prompting a response regarding a desire for brightness; In the question sending process, transmitting the question information, including that the brightness of the image shown in the scan data is an answer to the brightness question, to the server via the communication interface; A scanner configured to:

14. 8. A scanner according to claim 7, In the question processing, outputting a contrast question as one of the plurality of questions, the contrast question prompting a response regarding a desire for contrast; In the question sending process, transmitting the question information, including that the contrast of the image shown in the scan data is an answer to the contrast question, to the server via the communication interface; A scanner configured to:

15. 8. A scanner according to claim 7, In the question processing, outputting a format question as one of the plurality of questions, the format question prompting the user to answer a file format of the scan data; In the question sending process, transmitting the question information, including the file format of the scan data being an answer to the format question, to the server via the communication interface; A scanner configured to:

16. 8. A scanner according to claim 7, In the question processing, outputting a correction question as one of the plurality of questions, the correction question prompting the user to answer whether image correction is necessary; In the question sending process, transmitting the question information, including whether or not image correction is required for the scan data as an answer to the correction question, to the server via the communication interface; A scanner configured to:

17. 8. A scanner according to claim 7, In the question processing, As the answer to the question, unknown information indicating that the answer is unknown can be accepted, In the question sending process, transmitting, via the communication interface, to the server, the question information to which the valid answer is not associated, for the question to which the unknown information has been received as the answer; A scanner configured to:

18. 2. A scanner according to claim 1, The scanner A designation of a storage location to be an output destination can be accepted via the user interface; The scanner further comprises: After executing the query transmission process, when the recommended parameters are received from the server via the communication interface, the recommended scan process is executed, and image data based on the recommended scan data generated in the recommended scan process is saved in the specified save location as an output based on the recommended scan data. A scanner configured to:

19. 19. A scanner according to claim 18, The scanner A file format for saving the image data can be specified via the user interface; The scanner further comprises: After executing the query transmission process, when the recommended parameters are received from the server via the communication interface, the recommended scan process is executed, and as an output based on the recommended scan data, the image data based on the recommended scan data generated in the recommended scan process is converted into a specified file format, and the converted image data is saved in the specified save location. A scanner configured to:

20. 2. A scanner according to claim 1, Equipped with a print engine, The scanner further comprises: after executing the query transmission process, when the recommended parameters are received from the server via the communication interface, the recommended scan process is executed, and printing based on the recommended scan data generated in the recommended scan process is performed using the print engine as an output based on the recommended scan data. A scanner configured to:

Images