Image processing apparatus, image processing system, method for controlling image processing apparatus, and program

Abstract

To provide a mechanism that allows a user to quickly reach an operation screen on which the user can execute a desired operation.SOLUTION: An image forming apparatus 150 comprises: document scanning means 300 that can acquire first scanned data and second scanned data; operating means (operating unit 112) that allows users to execute an operation for a first document; identification means (authentication unit 121) that identifies each one of the plurality of users operating the operating means; clustering means 302 that performs clustering of classifying the first scanned data into one cluster of a plurality of clusters set to each user; and document recommended menu setting means 304 that links the details of the operation on the operating means executed by each user for the first scanned data, to the cluster into which the first scanned data is classified.SELECTED DRAWING: Figure 5

Description

[Technical field]

[0001] The present invention relates to an image processing apparatus, an image processing system, and a control method and program for an image processing apparatus. [Background technology]

[0002] In recent years, as image forming devices have become more multifunctional, operation menus displayed on the operation unit of the image forming device tend to be diverse and complicated. For example, in an image forming device having a copy function, a scan function, and a fax transmission function, the operation for executing these functions is performed on the operation unit of the image forming device, so that the operation menu displayed on the operation unit inevitably becomes many. In addition, when each operation menu is selected on the operation unit, a submenu is further displayed on the operation unit, and the user is required to select the submenu. Therefore, it takes time for the user to reach the menu for executing the desired function, or it becomes difficult to reach the menu. Patent Document 1 discloses an image forming device that determines the type of document scanned (scanned) to obtain image data, and stores an operation menu (recommended function) corresponding to each type of document. In the image forming device described in Patent Document 1, when a document is scanned, if the document is a document for which a recommended operation menu is stored, the recommended operation menu can be displayed on the operation unit. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] JP 2011-15348 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, in the image forming device described in Patent Document 1, although a recommended operation menu can be displayed for each type of document, when there are multiple users using the image forming device, the recommended operation menu that each user wants to operate may differ. For example, when the recommended operation menu for an invoice is fax transmission, one user operates fax transmission to fax the scanned data of the invoice. In contrast, another user operates email transmission instead of fax transmission to email the scanned data of the invoice. Therefore, the other user must search for the email transmission operation screen, resulting in a problem that the email transmission operation screen cannot be quickly reached.

[0005] The present invention has been made in view of the above problems, and has an object to provide a mechanism for quickly arriving at an operation screen that enables a desired operation. [Means for solving the problem]

[0006] In order to achieve the above object, the image processing device of the present invention is characterized in that it comprises a scanning means capable of obtaining first scan data of a first original by scanning a first original and obtaining second scan data of the second original by scanning a second original separate from the first original, an operation means capable of performing a predetermined operation on the first original scanned by the scanning means, an identification means for identifying a plurality of users operating the operation means for each user, a clustering means for performing clustering to classify the first scan data into one of a plurality of clusters set for each user according to the content of the first original, a linking means for linking the operation content performed by the operation means for each user on the first original to the cluster into which the first scan data was classified, an inference means for inferring which of the plurality of clusters the second scan data will be classified, and a control means for controlling display of the operation content linked to the cluster inferred by the inference means to which the second scan data is classified. Effect of the Invention

[0007] According to the present invention, it is possible to quickly arrive at an operation screen that enables a desired operation. [Brief description of the drawings]

[0008] [Figure 1A] 1 is a block diagram showing a hardware configuration of an image forming apparatus according to a first embodiment. [Figure 1B] FIG. 4 is a diagram showing an example of a screen displayed on the image forming apparatus. [Diagram 2] FIG. 2 is a diagram illustrating an example of an original to be scanned by a scanner of the image forming apparatus. [Diagram 3] 1 is a diagram for explaining an original scanning means, a clustering means, and an original recommendation menu setting means (linking means). FIG. [Figure 4] 6 is a flowchart showing a process in a learning mode executed by the image forming apparatus. [Diagram 5] FIG. 13 is a diagram for explaining an inference means. [Figure 6] FIG. 13 is a diagram for explaining a recommended menu display unit. [Figure 7] 10 is a flowchart showing a process in an inference mode executed by the image forming apparatus. [Figure 8] 13 is a diagram for explaining an original scanning unit, a clustering unit, and an original recommendation menu setting unit of an image forming apparatus according to a third embodiment. FIG. [Figure 9] FIG. 11 is a block diagram showing the overall configuration of an image processing system according to a third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Each embodiment of the present invention will be described in detail below with reference to the drawings. However, the configurations described in each of the following embodiments are merely examples, and the scope of the present invention is not limited by the configurations described in each embodiment. For example, each part constituting the present invention can be replaced with any configuration that can exert a similar function. In addition, any configuration may be added. In addition, any two or more configurations (features) of each embodiment can be combined.

[0010] First Embodiment Hereinafter, the first embodiment will be described with reference to FIG. 1A to FIG. 7. FIG. 1A is a block diagram showing a hardware configuration of an image forming apparatus according to the first embodiment. FIG. 1B is a diagram showing an example of a screen displayed on the image forming apparatus. The image forming apparatus 150 shown in FIG. 1A is an MFP to which an image processing apparatus capable of image processing is applied, and has a copy function, a scan function, a fax transmission function, an e-mail transmission function, and the like. The image forming apparatus 150 has a controller unit 100, an operation unit (operation means) 112, a USB storage 114, an authentication unit (identification means) 121, a scanner 170 which is an image input device, and a printer 195 which is an image output device. The controller unit 100 controls the printer 195 to print out image data read by the scanner 170. The controller unit 100 has a CPU (control means) 101, a RAM 102, a ROM 103, a storage 104, an image bus I / F 105, an operation unit I / F 106, a system bus 107, and a GPU 109. The controller unit 100 also has an image bus 108, a network I / F 110, a USB host I / F 113, an RTC 115, a device I / F 120, a scanner image processing unit 180, and a printer image processing unit 190. In the controller unit 100, the CPU 101 to the operation unit I / F 106, the GPU 109, the network I / F 110, the USB host I / F 113, and the RTC 115 are connected to each other so as to be able to communicate with each other via a system bus 107. Also, the image bus I / F 105, the device I / F 120, the scanner image processing unit 180, and the printer image processing unit 190 are connected to each other so as to be able to communicate with each other via the image bus 108.

[0011] The CPU 101 starts up an operating system (OS) by a boot program stored in the ROM 103. The CPU 101 executes various processes by executing programs stored in the storage 104 on this OS, that is, controls the operation of each unit. The ROM 103 stores setting values ​​for various settings of the image forming apparatus 150. As described later, the setting values ​​include an inference flag indicating whether the image forming apparatus 150 operates in a learning mode or an inference mode. In the image forming apparatus 150, a switching image 1000 (see FIG. 1B) for switching between the learning mode and the inference mode is displayed on the operation unit 112. The switching image 1000 includes a learning mode button 1001, an inference mode button 1002, an OK button 1003, and a cancel button 1004. The learning mode button 1001 is operated, that is, pressed, when the image forming apparatus 150 is operated in the learning mode. This disables the inference flag. Then, by operating the OK button 1003 after operating the learning mode button 1001, the image forming apparatus 150 operates in the learning mode. The inference mode button 1002 is operated when operating the image forming apparatus 150 in the inference mode. This enables the inference flag. Then, by operating the OK button 1003 after operating the inference mode button 1002, the image forming apparatus 150 operates in the inference mode. The cancel button 1004 is operated when canceling the operation of the learning mode button 1001 or the inference mode button 1002. In this way, the image forming apparatus 150 has a learning function and an inference function. The GPU 109 is a computing unit capable of performing processing specialized for neural network calculation. The GPU 109 can also handle calculation processing together with the CPU 101. The GPU 109 functions as a clustering means 302, a manuscript recommendation menu setting means 304, and an inference means 500, which will be described later, either alone or together with the CPU 101. Furthermore, the image forming apparatus 150 may have, for example, a TPU instead of the GPU 109. Furthermore, the image forming apparatus 150 may have an NPU, which together with the CPU 101 may be responsible for the arithmetic processing.

[0012] The RAM 102 is used as a working area for the CPU 101 and the GPU 109. The RAM 102 is also used as an image memory area for temporarily storing image data. The storage 104 stores programs, image data, information on the paper feed capacity including the maximum number of sheets that can be fed by the image forming apparatus 150, and the like. The programs include, for example, programs for causing the CPU 101 (computer) to execute each unit and each means (control method of the image processing apparatus) of the image forming apparatus 150. The operation unit I / F 106 is an interface with the operation unit 112 having a touch panel. The operation unit I / F 106 outputs image data displayed on the operation unit 112 to the operation unit 112. The operation unit I / F 106 also transmits information input by a user via the operation unit 112 to the CPU 101. The network I / F 110 is an interface for connecting the image forming apparatus 150 to a LAN. The USB host I / F 113 is an interface unit capable of communicating with the USB storage 114. The USB host I / F 113 is an output unit for storing data stored in the storage 104 in the USB storage 114. The USB host I / F 113 also inputs the data stored in the USB storage 114 to the CPU 101. The USB storage 114 is an external storage device that stores data, and is detachable from the USB host I / F 113.

[0013] Further, an authentication unit 121 is connected to the USB host I / F 113. The image forming apparatus 150 is used by a plurality of users, that is, is shared by a plurality of users. The authentication unit 121 can identify a plurality of users who use the image forming apparatus 150, that is, who operate the operation unit 112, for each user (identification process). In this embodiment, the authentication unit 121 is, for example, a card reader. In this case, for example, by holding an authentication card A owned by a user A over the card reader, subsequent operations on the image forming apparatus 150 are deemed to have been performed by the user A. Thereafter, for example, by holding an authentication card B owned by a user B over the card reader, subsequent operations on the image forming apparatus 150 are deemed to have been performed by the user B, not the user A. Thus, in this embodiment, the image forming apparatus 150 has a learning function and an inference function, and is shared by a plurality of users. In contrast, for example, a smartphone has a learning function and an inference function, but unlike the image forming apparatus 150, it is not shared by multiple people, that is, it is dedicated to one user. Note that, although the authentication unit 121 is a card reader in this embodiment, it is not limited to this and may be, for example, a device that performs biometric authentication using fingerprints or veins. In addition to the USB storage 114 and the authentication unit 121, multiple USB devices can be connected to the USB host I / F 113. The RTC 115 controls the time in the image forming apparatus 150. The time information controlled by the RTC 115 is used to obtain the time when performing auto-shutdown, record the time of job input, and the like.

[0014] The image bus I / F 105 is a bus bridge for connecting the system bus 107 and an image bus 108 that transfers image data at high speed and converting the data format. The image bus 108 complies with standards such as PCI bus or IEEE1394. The device I / F 120, the scanner image processing unit 180, and the printer image processing unit 190 are connected to the image bus 108. The device I / F 120 is connected to a scanner 170 and a printer 195. The device I / F 120 performs synchronous / asynchronous conversion of image data. The scanner 170 is an original scanning means (scanning means) 300 that scans an original to obtain scan data of the original. The scan data is used by a clustering means 302 and an inference means 500, which will be described later. The printer 195 prints and outputs the image data on a printing paper or the like. The scanner image processing unit 180 performs correction, processing, and editing on the scan data. A printer image processor 190 performs correction, resolution conversion, and the like on the image data to be printed in accordance with a printer 195 .

[0015] FIG. 2 is a diagram showing an example of a document scanned by a scanner of an image forming apparatus. Document 250 shown in FIG. 2(a) is an invoice requesting payment of a usage fee. Document 252 shown in FIG. 2(b) is an equipment borrowing application form for requesting borrowing of equipment. Document 254 shown in FIG. 2(c) is a printed matter on which the contents of an e-mail are printed. Scanner 170 can scan each of documents 250, 252, and 254 as a first document (scanning process). This allows first scan data of each first document to be acquired. Clustering means 302 can perform clustering to classify the first scan data of each first document into one of a plurality of clusters set for each user according to the contents of each first document (clustering process). For example, for user A, a total of four clusters, namely a first cluster 311, a second cluster 312, a third cluster 313, and a fourth cluster 314, are set as described below. When user A operates image forming apparatus 150 and first scan data is acquired by scanner 170, the first scan data is classified into one of first cluster 311 to fourth cluster 314 by clustering means 302. When clustering, clustering means 302 extracts a feature amount from the first scan data. Then, clustering is performed based on this feature amount. As the feature amount, for example, the following two can be mentioned in this embodiment, but are not limited thereto. The first feature is a semantic vector generated by reading character information by a reading means such as OCR and generating the read character information by a natural language processing model. The second feature is image information (ruled lines, illustrations, etc.) excluding characters extracted as feature points and vectorized.

[0016] FIG. 3 is a diagram for explaining the document scanning means, the clustering means, and the document recommendation menu setting means (linking means). FIG. 3(a) shows the operation of the document scanning means 300, the clustering means 302, and the document recommendation menu setting means 304 when a user A operates the image forming apparatus 150 in the learning mode (learning phase) state to scan the first document (scan input by user A). FIG. 3(b) shows the operation of the document scanning means 300, the clustering means 302, and the document recommendation menu setting means 304 when a user B operates the image forming apparatus 150 in the learning mode state to scan the first document (scan input by user B). In addition, whether the operator who scanned the first document is user A or user B is identified by the operator holding an authentication card over the authentication unit 121 before scanning the first document.

[0017] In FIG. 3(a), first, the user A operates the image forming apparatus 150 to scan the first document by the document scanning means 300. As a result, the first scan data of the first document is acquired. Then, as described above, the clustering means 302 performs clustering to classify the first scan data into one of four clusters set for the user A based on the feature amount according to the contents of the first document. Note that the number of clusters set for the user A is four in this embodiment, but is not limited thereto, and may be, for example, one to three or five or more. This clustering is performed using the k-means method. In the k-means method, in the first step, any four points are randomly determined in a vector space of the same dimension as the feature vector, which is the feature amount extracted from the first scan data, and each point is set as a cluster point representing each of the four clusters. In the second step, the Euclidean distance from the point on the feature vector of the first scan data to each of the four cluster points is calculated. Then, the first scan data is assigned to the cluster point with the shortest Euclidean distance, i.e., the cluster that represents the closest cluster point among the four cluster points, which is the point on the feature vector. In the third step, each cluster point is moved to the center of gravity of the point on the feature vector of the first scan data that belongs to the cluster that represents the cluster point. The second and third steps are executed until each cluster point converges. Similarly, when another first original is scanned and input, the second and third steps are executed. In this embodiment, the clustering in the clustering means 302 is performed using the k-means method, but is not limited to this, and may be performed using, for example, a mixed Gaussian distribution or spectral clustering.

[0018] As shown in FIG. 3(a), a first cluster 311, a second cluster 312, a third cluster 313, and a fourth cluster 314 are set as four clusters. Three copies of the first original (first scan data) belong to the first cluster 311. One copy of the first original belongs to the second cluster 312. Two copies of the first original belong to the third cluster 313. Three copies of the first original belong to the fourth cluster 314. The assignment of the first original to each cluster is performed by the above-mentioned clustering (clustering means 302). In addition, in the operation unit 112, multiple types of operations can be performed on the first original scanned by the original scanning means 300 (operation process). In this embodiment, the operation contents in the operation unit 112 include execution of a copy function, execution of a scan function, execution of a fax transmission function, and execution of an email transmission function. After the clustering by the clustering means 302, the document recommendation menu setting means (linking means) 304 links the operation contents performed by the user A on the first document on the operation unit 112 to the cluster into which the first scan data is classified (linking process). The first cluster 311 is linked to the execution of the copy function as the operation contents. The second cluster 312 is linked to the execution of the scan function, that is, the creation of PDF data as the operation contents. The third cluster 313 is linked to the execution of the fax transmission function as the operation contents. The fourth cluster 314 is linked to the execution of the email transmission function as the operation contents. In this embodiment, the operation contents for the first document include, but are not limited to, the execution of the copy function, the scan function, the fax transmission function, and the email transmission function. The operation contents for the first document may include at least one of the execution of the copy function, the scan function, the fax transmission function, and the email transmission function.

[0019] In FIG. 3(b), similarly to FIG. 3(a), first, the user B operates the image forming apparatus 150, and the first original is scanned by the original scanning means 300. Next, the clustering means 302 performs clustering to classify the first scan data into one of the first cluster 321, the second cluster 322, the third cluster 323, and the fourth cluster 324 set for the user B. Note that the number of clusters set for the user B is four in this embodiment, but is not limited thereto, and may be, for example, one to three or five or more. One copy of the first original belongs to the first cluster 321. Three copies of the first original belong to the second cluster 322. Two copies of the first original belong to the third cluster 323. Three copies of the first original belong to the fourth cluster 324. After the clustering by the clustering means 302, the original recommendation menu setting means 304 links the operation content performed by the user A on the first original on the operation unit 112 to the cluster into which the first scan data has been classified. The first cluster 321 is linked to the execution of a copy function as the operation content. The second cluster 322 and the third cluster 323 are linked to the execution of a scan function, that is, the creation of PDF data as the operation content. Note that the operation content of the second cluster 322 and the operation content of the third cluster 323 are both PDF data creation, but the destinations where the PDF data is saved are different from each other. The fourth cluster 324 is linked to the execution of an email sending function as the operation content.

[0020] Fig. 4 is a flow chart showing the process in the learning mode executed by the image forming apparatus. As shown in Fig. 4, in step S401, CPU 101 judges whether or not the first original has been placed on scanner 170. If it is judged that the first original has been placed as a result of the judgment in step S401, the process proceeds to step S402. On the other hand, if it is judged that the first original has not been placed as a result of the judgment in step S401, the process remains in step S401 and waits.

[0021] In step S402, CPU 101 controls document scanning means 300 in accordance with the operation of a scan start button (not shown) on operation unit 112 to scan the first document placed on scanner 170 in step S401. This allows first scan data to be acquired. After the scan start button is operated, one of the following functions is executed: copy function, scan function, fax function, or e-mail function. The execution of the function is then stored in storage 104, for example, as the operation content for the first document. In step S402, CPU 101 can identify the operator of the scan start button based on the authentication result in authentication unit 121. After step S402 is executed, the process proceeds to step S403.

[0022] In step S403, CPU 101 determines whether the inference flag is disabled, i.e., whether the learning mode is set, based on the result of the operation on switching image 1000 (see FIG. 1B). If it is determined in step S403 that the inference flag is disabled, the process proceeds to step S404. On the other hand, if it is determined in step S403 that the inference flag is not disabled, the process ends.

[0023] In step S404, CPU 101 controls clustering means 302 to perform the above-mentioned clustering on the first scan data acquired in step S402. As a result, the first scan data (first original) belongs to one of the clusters. After step S404 is executed, the process proceeds to step S405.

[0024] In step S405, CPU 101 calls up the operation content for the first document stored in step S401.

[0025] In step S406, the CPU 101 controls the manuscript recommendation menu setting unit 304 to associate the operation content for the first manuscript called up in step S405 with the cluster into which the first scan data is classified. As a result, the operation content for the first manuscript is set as a recommendation menu.

[0026] The learning mode described above provides a trained model to be used in the inference mode. In the learning mode, the first manuscript is used, whereas in the inference mode, a second manuscript different from the first manuscript is used. For each user, the first manuscript and the second manuscript are different manuscripts, but the contents of the first manuscript and the second manuscript may be the same (including similar) or different. The trained model includes, for example, a neural network, and is set for each operator (user). The neural network calculation is performed by the GPU 109. In addition, the machine learning algorithm is preferably deep learning, but is not limited thereto, and may be, for example, a support vector machine, a logistic regression, a decision tree, or the like. In the flowchart shown in FIG. 4, a process for determining whether or not the time is up may be provided between step S401 and step S402. In this case, for example, the configuration may be such that the process proceeds to the subsequent steps only if step S402 is executed within 60 seconds after the first manuscript is placed in step S401.

[0027] FIG. 5 is a diagram for explaining the inference means. FIG. 5(a) shows the operation of the inference means 500 when the user A operates the image forming apparatus 150 in the inference mode (inference phase) state to scan the second original (scan input by the user A). FIG. 5(b) shows the operation of the inference means 500 when the user B operates the image forming apparatus 150 in the inference mode (inference phase) state to scan the second original (scan input by the user B). The second scan data obtained by scanning the second original is an inference target for inferring which cluster the second scan data is classified into by the trained model. In addition, whether the operator who scanned the second original is the user A or the user B is identified by the operator holding an authentication card over the authentication unit 121 before scanning the first original.

[0028] In FIG. 5(a), first, the user A operates the image forming apparatus 150, whereby the second document is scanned by the document scanning means 300 (scanning step). As a result, second scan data of the second document is acquired. Next, the inference means 500 infers whether the second scan data is classified into any one of the first cluster 311 to the fourth cluster 314 set for the user A, using a trained model set for the user A (inference step). In this inference, as a first step, in a vector space of the same dimension as the feature vector, which is a feature amount extracted from the second scan data, the Euclidean distances from the points on the feature vector of the second scan data to the four cluster points are calculated. In the example shown in FIG. 5(a), the Euclidean distance between the second scan data and the first cluster 311 is 35.64. The Euclidean distance between the second scan data and the second cluster 312 is 0.85. The Euclidean distance between the second scan data and the third cluster 313 is 10.02. The Euclidean distance between the second scan data and fourth cluster 314 is 0.27. In the second step, the second scan data is assigned to a cluster that represents the cluster point with the shortest Euclidean distance. In this case, the second scan data (second original) will belong to fourth cluster 314. When determining which cluster the second scan data belongs to, a threshold value may be set for the Euclidean distance. In this case, the cluster to which the second scan data belongs can be determined based on the magnitude relationship between the Euclidean distance and the threshold value.

[0029] In FIG. 5(b), similarly to FIG. 5(a), first, the user B operates the image forming apparatus 150 to scan the second original by the original scanning means 300. Next, the inference means 500 infers whether the second scan data is classified into any one of the first cluster 321 to the fourth cluster 324 set for the user B, using the trained model set for the user B. In the example shown in FIG. 5(b), the Euclidean distance between the second scan data and the first cluster 321 is 10.24. The Euclidean distance between the second scan data and the second cluster 322 is 117.35. The Euclidean distance between the second scan data and the third cluster 323 is 3.86. The Euclidean distance between the second scan data and the fourth cluster 324 is 331.97. In this case, the second scan data (second original) belongs to the third cluster 323.

[0030] 6 is a diagram for explaining the recommended menu display means. The CPU 101 functions as a recommended menu display means 600 that performs control to display, as a recommended menu, operation contents linked to a cluster inferred by the inference means 500 to classify the second scan data (control step). The recommended menu is displayed on the operation unit 112. The recommended menu display means 600 is also prepared for each user. Here, the case of user A will be described as a representative example.

[0031] FIG. 6(a) is a diagram showing an example of a screen displayed on the operation unit 112 when the inference unit 500 infers that the second document input by the user A to the document scanning unit 300 belongs to the fourth cluster 314. As shown in FIG. 3(a), a plurality of operation contents are linked to the fourth cluster 314. Specifically, two overlapping operation contents "operation is ScanToEmail, format is PDF, destination is me, Mr. X, Mr. Y" and one operation "operation is ScanToEmail, format is PDF, destination is me, Mr. X, Mr. Z" are linked. In this case, the recommended menu display unit 600 can select at least one operation content from the plurality of operation contents and display it on the operation unit 112. A screen 600A shown in FIG. 6(a) includes a first candidate operation content 601 and a second candidate operation content 602. In this embodiment, the recommended menu display means 600 sets the first candidate operation content 601 in descending order of the operation content with the greatest number of overlaps as "the operation is ScanToEmail, the format is PDF, and the destinations are myself, Mr. X, and Mr. Y." The recommended menu display means 600 also sets the second candidate operation content 602 as "the operation is ScanToEmail, the format is PDF, and the destinations are myself, Mr. X, and Mr. Z." Note that the number of candidates is two in this embodiment, but is not limited to this and may be, for example, one or three or more.

[0032] FIG. 6(b) is a diagram showing an example of a screen displayed on the operation unit 112 when correcting the first candidate on the screen shown in FIG. 6(a). The screen 600B shown in FIG. 6(b) functions as a change unit for changing the operation content. This screen 600B includes an operation 603, a format 604, a destination 605, a button 606, and a button 607. Note that the configuration of the screen 600B is not limited to this, and may include other items, for example. In the operation 603, the operations "Send To Email", "FAX", and "COPY" can be selected and set. In the format 604, the file formats "PDF", "JPG", and "TIFF" can be selected and set. In the destination 605, "You", "Mr. X", and "Mr. Y" can be deleted from the destination, or a destination from the address book can be added. By operating (pressing) button 606, all of the settings in the fourth cluster 314, in which "operation is ScanToEmail, format is PDF, and destinations are myself, Mr. X, and Mr. Y", can be replaced with the corrected settings. This allows the changed operation contents to be continuously displayed thereafter. Specifically, if next time user A inputs a second document into the document scanning means 300 and the inference means 500 infers that it belongs to the fourth cluster 314, the setting items corrected last time will be displayed as the first candidate. Furthermore, by operating button 607, the continued display of the changed operation contents is restricted.

[0033] Fig. 7 is a flow chart showing the process in the inference mode executed by the image forming apparatus. As shown in Fig. 7, in step S701, CPU 101 judges whether or not a second original has been placed on scanner 170. If it is judged that a second original has been placed as a result of the judgment in step S701, the process proceeds to step S702. On the other hand, if it is judged that a second original has not been placed as a result of the judgment in step S701, the process remains in step S701 and waits.

[0034] In step S702, CPU 101 controls document scanning means 300 in accordance with the operation of a scan start button (not shown) on operation unit 112 to scan the second document placed on scanner 170 in step S701. This allows second scan data to be acquired. Also, in step S702, CPU 101 can identify the operator of the scan start button based on the authentication result in authentication unit 121. After executing step S702, the process proceeds to step S703.

[0035] In step S703, CPU 101 determines whether the inference flag is enabled, i.e., whether the inference mode is enabled, based on the result of the operation on switching image 1000 (see FIG. 1B). If it is determined in step S703 that the inference flag is enabled, processing proceeds to step S704. From this point on, the inference mode is executed. On the other hand, if it is determined in step S703 that the inference flag is not enabled, processing ends. In this case, the learning mode can be executed.

[0036] In step S704, the CPU 101 controls the inference means 500 to perform the above-mentioned inference on the second scan data acquired in step S702, that is, to infer which cluster the data belongs to.

[0037] In step S705, CPU 101 controls recommended menu display means 600 to set the most popular recommended menu in the cluster to which the second scan data belongs as the first candidate, and the second most popular recommended menu as the second candidate, as described above. Then, the first candidate and the second candidate are displayed on operation unit 112, respectively.

[0038] As described above, the image forming device 150 is shared by multiple users (here, user α and user β). In this case, even if user α and user β each obtain the same scan data, user α may fax the scan data using the fax transmission function, while user β may email the scan data using the email transmission function. In this way, the operation contents that the user wants to operate may differ for each user, so there is a risk that it may be difficult to quickly reach the desired operation screen. Therefore, the image forming device 150 can perform learning on the first scan data for each user to generate a trained model. Then, inference can be made on the second scan data according to the trained model for each user. As a result, a screen for executing the fax transmission function can be displayed for user α, and a screen for executing the email transmission function can be displayed for user β. In this way, the image forming device 150 can quickly reach an operation screen that enables each user to perform a desired operation, and is therefore a device that excels in user support (operation support).

[0039] <Second embodiment> Hereinafter, the second embodiment will be described with reference to FIG. 8, but the differences from the above-mentioned embodiment will be mainly described, and the description of the similar matters will be omitted. This embodiment is the same as the first embodiment except for the difference in clustering. FIG. 8 is a diagram for explaining the document scanning means, clustering means, and document recommended menu setting means of the image forming apparatus according to the third embodiment. For example, when the number of recommended menus in a certain cluster becomes relatively large, the recommended menu suitable for the user may not necessarily be displayed on the operation unit 112. In this embodiment, the total number of operation contents linked to each cluster is restricted. When the total number of operation contents linked to each cluster exceeds a threshold value, the clustering means 302 performs clustering again. This clustering again can generate a sub-cluster with a smaller number of recommended menus. Then, in the inference mode, the sub-clusters can be used to increase the possibility of displaying a recommended menu suitable for the user.

[0040] Here, a specific example is taken in which the threshold for the total number of operation contents associated with each cluster is set to 3. As shown in FIG. 8, when the operation contents (recommended menus) of the second cluster 312 exceed the threshold (3) and become four types, clustering is performed again. As a result, a first sub-cluster 801, a second sub-cluster 802, a third sub-cluster 803, and a fourth sub-cluster 804 are generated. The first sub-cluster 801 to the fourth sub-cluster 804 are all one type with the number of operation contents being less than the threshold. From now on, in the inference mode, the use of the operation contents of the second cluster 312 is restricted, and instead, the operation contents of the first sub-cluster 801 to the fourth sub-cluster 804 can be used. As a result, in the inference mode, it is possible to display operation contents (recommended menus) that are suitable for the user. The method of re-clustering is not particularly limited, and for example, k-means or the like can be used.

[0041] <Third embodiment> Hereinafter, the third embodiment will be described with reference to Fig. 9, focusing on the differences from the above-mentioned embodiments, and the description of the similarities will be omitted. This embodiment is similar to the first embodiment, except that the clustering means 302, the manuscript recommendation menu setting means 304, the inference means 500, and the recommendation menu display means 600 are arranged in different devices. Fig. 9 is a block diagram showing the overall configuration of an image processing system according to the third embodiment.

[0042] As shown in FIG. 9, the image processing system 900 includes an image forming apparatus 150 and a server 901 communicably connected to the image forming apparatus 150. The image forming apparatus 150 includes a document scanning unit 300, an operation unit 112, an authentication unit 121, and the like. In this embodiment, the clustering unit 302, the document recommended menu setting unit 304, the inference unit 500, and the recommended menu display unit 600 are arranged in the server 901. That is, the server 901 includes the clustering unit 302, the document recommended menu setting unit 304, the inference unit 500, and the recommended menu display unit 600. This allows, for example, a simple configuration of the image forming apparatus 150. In addition, a single server 901 can be shared by a plurality of image forming apparatuses 150. Note that the programs for causing a computer to execute each unit or each unit of the image forming apparatus 150 and the server 901 may be stored in the image forming apparatus 150 or in the server 901. Furthermore, this program may be stored in a distributed manner in the image forming apparatus 150 and the server 901. Furthermore, the server 901 may be located in Japan or abroad.

[0043] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to the above-mentioned embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. The present invention can also be realized by supplying a program that realizes one or more functions of the above-mentioned embodiments to a system or device via a network or storage medium, and having one or more processors of a computer in the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions.

[0044] The disclosure of each embodiment includes the following configurations, methods, and programs. (Configuration 1) A scanning means capable of acquiring first scan data of a first document by scanning a first document, and acquiring second scan data of the second document by scanning a second document different from the first document; an operation means for executing a predetermined operation on the first document scanned by the scanning means; an identification means for identifying a plurality of users who operate the operation means for each of the users; a clustering unit that performs clustering to classify the first scan data into one of a plurality of clusters set for each user according to the content of the first document; a linking means for linking the operation content performed by each user on the first document with the cluster into which the first scan data is classified; an inference means for inferring whether the second scan data is classified into any one of the plurality of clusters; and a control means for controlling display of the operation content linked to the cluster inferred by the inference means to classify the second scan data. (Configuration 2) Equipped with copy, scan, fax, and email functions; 2. The image processing device according to claim 1, wherein the operation content includes at least one of execution of the copy function, execution of the scan function, execution of the fax transmission function, and execution of the email transmission function. (Configuration 3) The image processing apparatus according to configuration 1 or 2, wherein the clustering means extracts features from the first scan data and performs the clustering based on the features. (Configuration 4) The image processing device according to any one of configurations 1 to 3, wherein the clustering means performs the clustering using one of k-means, a mixture of Gaussian distributions, and spectral clustering. (Configuration 5) The image processing device described in any one of configurations 1 to 4, characterized in that the clustering means performs the clustering again when the total number of the operation contents linked to each cluster by the linking means is regulated. (Configuration 6) The image processing apparatus according to any one of configurations 1 to 5, wherein the linking means is capable of linking a plurality of the operation contents to each of the clusters for each of the users. (Configuration 7) The image processing device according to configuration 6, wherein the control means selects and displays at least one of the plurality of operation contents. (Configuration 8) The image processing device according to configuration 6 or 7, wherein the control means displays a first candidate for the operation content and a second candidate for the operation content from among a plurality of the operation contents. (Configuration 9) The plurality of operation contents include operation contents that overlap with each other, 9. The image processing device according to configuration 8, wherein the control means selects the first candidate and the second candidate in descending order of overlap number. (Configuration 10) The image processing device according to any one of configurations 1 to 9, further comprising a change means for changing the operation content displayed by the control means. (Configuration 11) The image processing device according to configuration 10, wherein the control means continues to display the operation content changed by the change means. (Configuration 12) The image processing apparatus according to any one of configurations 1 to 11, wherein the operation means is configured to be able to display the operation content by the control means. (Configuration 13) An image processing system including an image processing device capable of image processing and a server communicably connected to the image processing device, The image processing device includes: a scanning means capable of acquiring first scan data of a first document by scanning a first document, and acquiring second scan data of the second document by scanning a second document different from the first document; an operation means for executing a predetermined operation on the first document scanned by the scanning means; an identification means for identifying a plurality of users who operate the operation means for each of the users, The server, a clustering unit that performs clustering to classify the first scan data into one of a plurality of clusters set for each user according to the content of the first document; a linking means for linking the operation content performed by each user on the first document with the cluster into which the first scan data is classified; an inference means for inferring whether the second scan data is classified into any one of the plurality of clusters; and a control means for controlling the image processing device to display the operation content linked to the cluster inferred by the inference means to which the second scan data is classified. (Method 1) A method for controlling an image processing device capable of image processing, comprising the steps of: a scanning step for scanning a first document to obtain first scan data of the first document and for scanning a second document different from the first document to obtain second scan data of the second document; an operation step capable of executing a predetermined operation on the first document scanned in the scanning step; an identification step of identifying a plurality of users who perform the operation in the operation step for each of the users; a clustering step of classifying the first scan data into one of a plurality of clusters set for each user according to the content of the first document; a linking step of linking the operation content in the operation step performed for each user on the first document to the cluster into which the first scan data is classified; an inference step of inferring which of the plurality of clusters the second scan data is classified into; and a control step of controlling to display the operation content linked to the cluster inferred in the inference step that the second scan data is classified. (Program 1) A program for causing a computer to execute each of the means of the image processing device according to any one of configurations 1 to 12. [Explanation of symbols]

[0045] 101 CPU (control means) 150 Image forming device 112 Operation unit (operation means) 121 Authentication unit (identification means) 300 Document scanning means 302 Clustering Methods 304 Manuscript recommendation menu setting means (linking means) 500 Reasoning means 600 Recommended menu display method

Claims

1. a scanning means capable of acquiring first scan data of a first document by scanning a first document, and acquiring second scan data of the second document by scanning a second document different from the first document; an operation unit capable of executing a predetermined operation on the first document scanned by the scanning unit; an identification means for identifying a plurality of users who operate the operation means for each of the users; a clustering unit that performs clustering to classify the first scan data into one of a plurality of clusters that are set for each user according to the content of the first document; a linking means for linking the operation content performed by each user on the first document with the cluster into which the first scan data is classified; an inference means for inferring whether the second scan data is classified into any one of the plurality of clusters; and a control means for controlling display of the operation content linked to the cluster inferred by the inference means to classify the second scan data.

2. It has copy, scan, fax and email functions.

2. The image processing apparatus according to claim 1, wherein the operation content includes at least one of execution of the copy function, execution of the scan function, execution of the fax transmission function, and execution of the mail transmission function.

3. 2. The image processing apparatus according to claim 1, wherein the clustering means extracts a feature amount from the first scan data, and performs the clustering based on the feature amount.

4. 2. The image processing apparatus according to claim 1, wherein the clustering means performs the clustering using one of k-means, a mixture of Gaussian distributions, and spectral clustering.

5. 2 . The image processing apparatus according to claim 1 , wherein the clustering means performs the clustering again when the total number of the operation contents linked to each cluster by the linking means is restricted.

6. 2. The image processing apparatus according to claim 1, wherein the linking means is capable of linking a plurality of the operation contents to each of the clusters for each of the users.

7. 7. The image processing apparatus according to claim 6, wherein the control means selects and displays at least one of the plurality of operation contents.

8. 7. The image processing apparatus according to claim 6, wherein the control means displays a first candidate of the operation contents and a second candidate of the operation contents from among the plurality of operation contents.

9. The plurality of operation contents include operation contents that overlap with each other, 9. The image processing apparatus according to claim 8, wherein the control means selects the first candidate and the second candidate in descending order of overlap number.

10. 2. The image processing apparatus according to claim 1, further comprising a change unit for changing the operation content displayed by the control unit.

11. 11. The image processing apparatus according to claim 10, wherein the control means continues to display the operation content changed by the change means.

12. 2. The image processing apparatus according to claim 1, wherein the operation means is configured so that the operation contents can be displayed by the control means.

13. An image processing system including an image processing device capable of image processing and a server communicably connected to the image processing device, The image processing device includes: a scanning means capable of acquiring first scan data of a first document by scanning a first document, and acquiring second scan data of the second document by scanning a second document different from the first document; an operation unit capable of executing a predetermined operation on the first document scanned by the scanning unit; an identification means for identifying a plurality of users who operate the operation means for each of the users, The server, a clustering unit that performs clustering to classify the first scan data into one of a plurality of clusters that are set for each user according to the content of the first document; a linking means for linking the operation content performed by each user on the first document with the cluster into which the first scan data is classified; an inference means for inferring whether the second scan data is classified into any one of the plurality of clusters; and a control means for controlling the image processing device to display the operation content linked to the cluster inferred by the inference means to which the second scan data is classified.

14. 1. A method for controlling an image processing device capable of image processing, comprising: a scanning step for scanning a first document to obtain first scan data of the first document and for scanning a second document different from the first document to obtain second scan data of the second document; an operation step capable of executing a predetermined operation on the first document scanned in the scanning step; an identification step of identifying a plurality of users who perform the operation in the operation step for each of the users; a clustering step of classifying the first scan data into one of a plurality of clusters set for each user according to the content of the first document; a linking step of linking the operation content in the operation step performed for each user on the first document to the cluster into which the first scan data is classified; an inference step of inferring which of the plurality of clusters the second scan data is classified into; A control method for an image processing device, comprising: a control process for controlling display of the operation content linked to the cluster inferred in the inference process to classify the second scan data.

15. 2. A program for causing a computer to execute each of the means of the image processing apparatus according to claim 1.

Images