Image processing apparatus, control method of image processing apparatus, and storage medium

Abstract

The invention provides an image processing apparatus, a control method of the image processing apparatus, and a storage medium. An image processing apparatus may classify image data to storage locations using a generative AI service. An image of a document is read and image data is generated, a prompt for causing a generative AI to execute an image analysis for extracting a predetermined character string from the image data is received, the image data and the prompt are transmitted to the generative AI, the character string extracted by the image analysis performed by the generative AI is acquired, and control is performed. And storing the image data in a folder indicated by the folder path including the character string that has been acquired.

Description

Technical Field

[0001] The present invention relates to an image processing apparatus, a control method of the image processing apparatus, and a storage medium. Background Art

[0002] An image processing apparatus that reads an image of a document and generates image data is known. A technique disclosed in Japanese Patent Application Laid-Open No. 2020-86717 is proposed as a technique for storing such image data. In Japanese Patent Application Laid-Open No. 2020-86717, the generated image data is stored in a folder whose name has characters obtained by performing character recognition processing on the generated image data. This enables classification of the image data to an appropriate storage location for each character included in the generated image data.

[0003] On the other hand, a generative AI service that performs image analysis processing using generative AI is used. In the generative AI service, a user needs to input a freely selected prompt.

[0004] However, in the technique of Japanese Patent Application Laid-Open No. 2020-86717 described above, since character recognition processing is performed by the image processing apparatus, character recognition processing using a generative AI service is not assumed. Summary of the Invention

[0005] The present invention provides an image processing apparatus, a control method of the image processing apparatus, and a storage medium that can classify image data to a storage location using a generative AI service.

[0006] According to an aspect of the present invention, there is provided an image processing apparatus including: a reading unit configured to read an image of a document and generate image data; an operation unit configured to receive a prompt for causing a generative AI to perform image analysis for extracting a predetermined string from the image data; and at least one memory and at least one processor serving as: a sending unit configured to send the image data and the prompt to the generative AI; an acquisition unit configured to acquire a string extracted by the image analysis performed by the generative AI; and a control unit configured to control to store the image data in a folder indicated by a folder path including the acquired string.

[0007] According to the present invention, it is possible to classify image data to a storage location using a generative AI service.

[0008] Other features of the present invention will become apparent from the following description of exemplary embodiments with reference to the accompanying drawings. Brief Description of the Drawings

[0009] Figure 1is a block diagram showing a configuration example of a system including an MFP as an image processing apparatus according to the present embodiment.

[0010] Figure 2 schematically shows Figure 1 a block diagram of the hardware configuration of the MFP in

[0011] Figure 3 schematically shows Figure 1 a block diagram of the hardware configuration of the user terminal in

[0012] Figure 4 schematically shows Figure 1 a block diagram of the hardware configuration of the generative AI server in

[0013] Figure 5 schematically shows Figure 1 a block diagram of the hardware configuration of the storage server in

[0014] Figure 6 is a sequence diagram as follows, which shows a process of storing the scanned image data generated by scanning a document using the MFP in Figure 1 to a storage location determined based on the analysis result of image analysis performed by the generative AI server.

[0015] Figure 7 is Figure 2 a screen transition diagram of the operation unit in

[0016] Figure 8 shows Figure 1 a flowchart of a scanning control process executed by the MFP in

[0017] Figure 9 shows an example of a job execution button setting screen for registering a job execution button displayed on the main screen in Figure 7 DETAILED DESCRIPTION OF THE EMBODIMENTS

[0018] Hereinafter, embodiments of the present invention will be described in detail with reference to the drawings. In the present embodiment, a multifunction peripheral (MFP) having a plurality of functions such as a printing function, a scanning function, and a fax function will be described as an example of an image processing apparatus.

[0019] Figure 1 is a block diagram showing a configuration example of a system including an MFP 200 as an image processing apparatus according to the present embodiment. In this system, as Figure 1 ​As shown, the MFP 200 and the user terminal 300 are communicably connected to the generative AI server 400 and the storage server 500 via the network 100. It should be noted that the network 100 can be the Internet or a local area network (LAN). The network 100 can be a wired network or a wireless network.

[0020] The MFP 200 has the function of scanning and digitizing paper originals. The MFP 200 sends the scanned image data generated by scanning the paper originals to the storage server 500. The user terminal 300 is an electronic device such as a personal computer, a smartphone, or a tablet PC owned by the user. The user terminal 300 communicates with the MFP 200 via the network 100 and can perform function settings of the MFP 200 from a browser or the like operating on the user terminal 300.

[0021] The generative AI server 400 communicates with the MFP 200 via the network 100 and receives the scanned image data generated by the MFP 200 and a prompt used as an instruction for performing image analysis for extracting a predetermined string from the scanned image data. The generative AI server 400 interprets the received prompt and returns an answer (response) to the prompt to the MFP 200.

[0022] The storage server 500 is a network attached storage (NAS) or the like. The storage server 500 is a server that manages files according to a file sharing protocol such as the Common Internet File System (CIFS) or the Network File System (NFS).

[0023] Figure 2 is a block diagram schematically showing Figure 1 the hardware configuration of the MFP 200 in Figure 2 In, the MFP 200 includes a CPU 201, a ROM 202, a RAM 203, a memory 204, an operation unit 205, a printing unit 206, a reading unit 207, and a communication unit 208. These components are interconnected via a bus 209.

[0024] The CPU 201 controls the overall operation of the MFP 200. The CPU 201 reads the control program stored in the ROM 202 or the memory 204 into the RAM 203 and executes the control program read into the RAM 203 to perform various controls such as reading control and printing control. The ROM 202 stores the control program that the CPU 201 can execute. The RAM 203 is the main storage memory and is used as a temporary storage area for expanding the work area and various control programs stored in the ROM 202 and the memory 204. The memory 204 stores image data, print data, various programs, and various types of setting information. It should be noted that in this embodiment, a flash memory is assumed to be the memory 204, but storage devices such as a solid state drive (SSD) or a hard disk drive (HDD) can be used. In addition, an embedded multimedia card (eMMC) can be used as the memory 204. It should be noted that in the MFP 200 according to this embodiment, one CPU 201 uses one RAM 203 to perform various types of processing shown in the sequence described later, but the present invention is not limited to this configuration. For example, multiple CPUs, RAMs, ROMs, and memories can cooperate with each other to perform various types of processing shown in the sequence described later. In addition, a hardware circuit such as an ASIC or an FPGA can be used to perform a certain type of processing.

[0025] For example, the operation unit 205 is a display unit such as a touch panel or hard keys. The operation unit 205 displays information to the user and detects input from the user. For example, the operation unit 205 receives a prompt input from the user. The printing unit 206 prints the image data (print data) stored in the RAM 203 on the recording paper fed from the paper feed cassette. The reading unit 207 reads the image of the original document, and the CPU 201 converts the image into scanned image data (such as binary data). The scanned image data generated based on the image read by the reading unit 207 is sent to an external device or printed on the recording paper by the printing unit 206.

[0026] The communication unit 208 is connected to the network 100. The communication unit 208 sends the image data to an external device on the network 100 and receives the print data from the user terminal 300. For example, the communication unit 208 sends the image data and the prompt to the generative AI server 400. As a method of sending and receiving via the network 100, sending and receiving using email and file sending using other protocols (such as FTP, SMB, WEBDAV, etc.) can be performed. In addition, various types of setting data can be sent and received via the network 100 by accessing the HTTP communication of the user terminal 300.

[0027] Figure 3 is schematically shown Figure 1Block diagram of the hardware configuration of the user terminal 300 in. In Figure 3 In, the user terminal 300 includes a CPU 301, a ROM 302, a RAM 303, a memory 304, a display unit 305, and a communication unit 306. These components are interconnected via a bus 307.

[0028] The CPU 301 controls the overall operation of the user terminal 300. The CPU 301 reads the control program stored in the ROM 302 or the memory 304 into the RAM 303 and executes the control program read into the RAM 303 to perform various types of processing for controlling the operation of the user terminal 300. The ROM 302 stores the control program that the CPU 301 can execute. The RAM 303 is the main storage memory and is used as a temporary storage area for expanding the work area and various control programs stored in the ROM 302 and the memory 304. The memory 304 stores application data, various programs, and various types of setting information. The display unit 305 is, for example, a display device (such as a liquid crystal panel) that displays the image data processed by the CPU 301. The communication unit 306 transmits and receives data to and from the generative AI server 400, the storage server 500, and the MFP 200 via the network 100.

[0029] Figure 4 Schematically shows Figure 1 Block diagram of the hardware configuration of the generative AI server 400 in. In Figure 4 In, the generative AI server 400 includes a CPU 401, a ROM 402, a RAM 403, a communication unit 404, and an HDD 405. These components are interconnected via a bus 406.

[0030] The CPU 401 uses the control program stored in the ROM 402 or the learning model of the generative AI stored in the HDD 405 to execute the processing for controlling the operation of generating an appropriate response. The learning model includes an image analysis function and generates text or an image in response to an input prompt. For example, when a natural language question such as "What is shown in the image?" is input as a prompt, the learning model performs image analysis corresponding to the question and generates an answer to the question. For example, the answer includes the name of the object shown in the image.

[0031] The ROM 402 stores control programs. The RAM 403 is used as a temporary storage area such as the main memory or work area of the CPU 401. The HDD 405 stores various types of data such as learning models or generative AI applications. The communication unit 404 exchanges data with various devices such as the user terminal 300 and the MFP 200. It should be noted that the communication unit 404 can perform wired communication using Ethernet (registered trademark) or can perform wireless communication such as Wi-Fi.

[0032] Figure 5 is a schematic diagram showing Figure 1 the hardware configuration of the storage server 500 in. In Figure 5 the storage server 500 includes a CPU 501, a ROM 502, a RAM 503, an HDD 504, and a communication unit 505. These components are interconnected via a bus 506.

[0033] The CPU 501 reads the control program stored in the ROM 502 into the RAM 503 and executes the control program read into the RAM 503 to perform various types of processing for controlling the operation of the storage server 500. The ROM 502 stores control programs. The RAM 503 is used as a temporary storage area such as the main memory or work area of the CPU 501. The HDD 504 stores files. The communication unit 505 receives files from the MFP 200 or the user terminal 300 via the Internet 100 and stores the received files in the HDD 504. The communication unit 505 also sends the files stored in the HDD 504 to the MFP 200 or the user terminal 300. It should be noted that the communication unit 505 can perform wired communication using Ethernet or can perform wireless communication such as Wi-Fi.

[0034] Figure 6 is a sequence diagram showing the following processing procedure: storing the scanned image data generated by scanning the original document using the Figure 1 MFP 200 in in the storage location determined based on the analysis result of the image analysis performed by the generative AI server 400. It should be noted that it is assumed that the main screen 701 in Figure 7 is displayed on the operation unit 205 of the MFP 200. The main screen 701 includes buttons corresponding to the functions of the MFP 200 (such as a copy button and a fax button), and job execution buttons (such as an "AI classification and send" button 702).

[0035] In Figure 6 first, when the user presses the "AI classification and send" button 702 on the main screen 701 (S601), the screen of the operation unit 205 of the MFP200 switches from the main screen 701 to the AI classification and send setting screen 703.

[0036] Next, the user performs scan settings on the AI classification transmission setting screen 703 (S602). Specifically, the user sets the read size 704, double-sided original 705, file format 706, segmentation of each page 707, save location 708 and file name 709 as scan settings. The read size 704 is a setting related to the size of the original to be scanned. The double-sided original 705 is a setting about whether to read both sides of the original to be scanned. The file format 706 is a setting related to the file format of the scanned image data to be generated by scanning. In the segmentation of each page 707, set on (ON) or off (OFF). In the case of setting off in the segmentation of each page 707, when scanning a multi-page original, a scanned image data including data of all pages is generated. On the other hand, in the case of setting on in the segmentation of each page 707, when scanning a multi-page original, a plurality of scanned images of multiple pages segmented for each specified number of pages are generated. The save location 708 is a setting related to the storage location of the scanned image data to be generated by scanning. The file name 709 is a setting related to the file name of the scanned image data to be generated by scanning. It should be noted that when the user selects the reset button 710, the information set on the AI classification transmission setting screen 703 is cleared.

[0037] For example, when the user selects the save location 708, the screen of the operation unit 205 of the MFP 200 switches from the AI classification transmission setting screen 703 to the save location prompt selection screen 713. On the save location prompt selection screen 713, the Figure 9 The prompt list registered on the job execution button setting screen 901 in . The prompt is a prompt for causing the generative AI server 400 to perform image analysis for extracting a predetermined character string to be used for determining a storage location from the scanned image data generated by scanning. Figure 7 , as examples, prompts such as “Tell the company name,” “Extract the name written in the name column,” and “Read the character string of the QR code” are displayed. When the user selects a prompt from the list displayed on the save location prompt selection screen 713, the screen of the operation unit 205 of the MFP 200 switches from the save location prompt selection screen 713 to the AI classification transmission setting screen 703. In the save location 708 of the AI classification transmission setting screen 703, the prompt selected by the user on the save location prompt selection screen 713 is set. In this way, instead of the folder path itself indicating the storage location of the scanned image data, a prompt for causing the generative AI server 400 to perform image analysis for extracting a predetermined character string to be used for determining the storage location is set in the save location 708.

[0038] In addition, when the user selects the file name 709, the screen of the operation unit 205 of the MFP 200 switches from the AI classification transmission setting screen 703 to a file name prompt selection screen (not shown). On the file name prompt selection screen, a list of prompts registered on the job execution button setting screen 901 described later is displayed. The prompt is as follows: a prompt for causing the generative AI server 400 to execute image analysis for extracting a predetermined string to be used for determining the file name from the scanned image data generated by scanning. When the user selects a prompt from the list displayed on the file name prompt selection screen, the screen of the operation unit 205 of the MFP 200 switches from the file name prompt selection screen to the AI classification transmission setting screen 703. In the file name 709 of the AI classification transmission setting screen 703, the prompt selected by the user on the file name prompt selection screen is set. In this way, instead of the file name itself of the scanned image data, a prompt for causing the generative AI server 400 to execute image analysis for extracting a predetermined string to be used for determining the file name is set in the file name 709. Figure 9 In the job execution button setting screen 901 described later, a list of prompts registered is displayed. The prompt is as follows: a prompt for causing the generative AI server 400 to execute image analysis for extracting a predetermined string to be used for determining the file name from the scanned image data generated by scanning. When the user selects a prompt from the list displayed on the file name prompt selection screen, the screen of the operation unit 205 of the MFP 200 switches from the file name prompt selection screen to the AI classification transmission setting screen 703. In the file name 709 of the AI classification transmission setting screen 703, the prompt selected by the user on the file name prompt selection screen is set. In this way, instead of the file name itself of the scanned image data, a prompt for causing the generative AI server 400 to execute image analysis for extracting a predetermined string to be used for determining the file name is set in the file name 709.

[0039] Next, the user issues an instruction to start scanning the original document (S603). In this embodiment, the user can issue an instruction to start scanning the original document by selecting the black and white start button 711 or the color start button 712 on the AI classification transmission setting screen 703.

[0040] The CPU 201 of the MFP 200 that has received the instruction to start scanning the original document controls the reading unit 207 to scan the set original document (S604). In S604, the original document is scanned based on the setting information of the reading size 704, double-sided original document 705, file format 706, and division of each page 707 on the AI classification transmission setting screen 703. Through this scanning, scanned image data is generated.

[0041] Next, the CPU 201 of the MFP 200 controls the communication unit 208 to send the scanned image data generated in S604 to the generative AI server 400 (S605). The CPU 201 also controls the communication unit 208 to send the prompt set on the AI classification transmission setting screen 703 to the generative AI server 400 (S606).

[0042] The CPU 401 of the generative AI server 400 that has received the scanned image data and the prompt uses the learning model stored in the HDD 405 to perform image analysis for extracting a predetermined string corresponding to the prompt from the scanned image data (S607). For example, when the received prompt is "indicate the company name", in the image analysis using the learning model, a predetermined string indicating the company name is extracted from the scanned image data.

[0043] The CPU 401 of the generative AI server 400 controls the communication unit 404 to send a predetermined string extracted from the scanned image data as an analysis result to the MFP 200 (S608).

[0044] Next, the CPU 201 of the MFP 200 determines the storage location of the scanned image data by using the analysis result received from the generative AI server 400 as part of the folder path (S609).

[0045] Next, the CPU 201 of the MFP 200 controls the communication unit 208 to send the scanned image data generated in S604 to the storage server 500 (S610). At this time, the storage location determined in S609 is specified as the storage location of the scanned image data. Therefore, in the storage server 500, the scanned image data generated in S604 is stored at the storage location determined in S609. After that, this process ends.

[0046] Figure 8 is a flowchart showing the process of the scan control process executed by the Figure 1 MFP 200 in. The scan control process is implemented by the CPU 201 of the MFP 200 reading the program stored in the ROM 202 into the RAM 203 and executing the program read into the RAM 203. The scan control process starts when the power of the MFP 200 is turned on and the main screen 701 is displayed on the operation unit 205.

[0047] In Figure 8 first, the CPU 201 determines whether the "AI classification and send" button 702 has been selected (S801). When it is determined in S801 that the "AI classification and send" button 702 has not been selected, this process returns to S801. When it is determined in S801 that the "AI classification and send" button 702 has been selected, this process proceeds to S802.

[0048] In S802, the CPU 201 determines whether a prompt as a selection candidate has been registered in the save location 708 and the file name 709. When it is determined in S802 that no prompt as a selection candidate has been registered in the save location 708 or the file name 709, this process proceeds to S803.

[0049] In S803, the CPU 201 causes the operation unit 205 to display the AI classification transmission setting screen 703, and this setting screen 703 includes an unregistered error notification indicating a prompt that is not registered as a selection candidate in the save location 708 or the file name 709. Next, the CPU 201 determines whether the "Return" button on the AI classification transmission setting screen 703 has been selected (S804). When it is determined in S804 that the "Return" button has not been selected, this process returns to S804. On the other hand, when it is determined in S804 that the "Return" button has been selected, this process returns to S801.

[0050] When it is determined in S802 that a prompt has been registered as a selection candidate in the save location 708 and the file name 709, this process proceeds to S805.

[0051] In S805, the CPU 201 causes the operation unit 205 to display the AI classification transmission setting screen 703. When the save location 708 or the file name 709 of the AI classification transmission setting screen 703 is selected, the CPU 201 causes the operation unit 205 to display a prompt selection screen corresponding to the selected item (S806). In the following description, as an example, it is assumed that the user has selected the save location 708, and the save location prompt selection screen 713 is displayed on the operation unit 205.

[0052] Next, the CPU 201 determines whether a prompt has been selected on the save location prompt selection screen 713 displayed on the operation unit 205 (S807). When it is determined in S807 that a prompt has not been selected on the save location prompt selection screen 713, this process returns to S806. On the other hand, when it is determined in S807 that a prompt has been selected on the save location prompt selection screen 713, this process proceeds to S808.

[0053] In S808, the CPU 201 determines whether an instruction to start original document scanning has been received. In S808, when the user has neither selected the black and white start button 711 nor the color start button 712 on the AI classification transmission setting screen 703, it is determined that an instruction to start original document scanning has not been received, and this process returns to S808. On the other hand, when the user selects the black and white start button 711 or the color start button 712 on the AI classification transmission setting screen 703, it is determined that an instruction to start original document scanning has been received, and this process proceeds to S809. It should be noted that in this embodiment, when it is determined that an instruction to start original document scanning has been received and a prompt is not set in at least one of the save location 708 and the file name 709, this process can return to S806. In S806, a prompt selection screen corresponding to the item where the prompt is not set is displayed on the operation unit 205.

[0054] In S809, the CPU 201 controls the reading unit 207 based on the information set on the AI classification transmission setting screen 703, and scans the set original document. Thereby, scanned image data is generated. It should be noted that when scanning a multi-page original document, according to the setting of the segmentation 707 of each page on the AI classification transmission setting screen 703, one scanned image data including data of all pages or multiple scanned image data segmented for each specified number of pages is generated.

[0055] Next, the CPU 201 converts the scanned image data generated in S809 into the file format set by the file format 706 of the AI classification transmission setting screen 703 (S810).

[0056] Next, the CPU 201 controls the communication unit 208 to send the scanned image data whose file format has been converted in S810 to the generative AI server 400 (S811). The CPU 201 also controls the communication unit 208 to send the prompt selected in S807 to the generative AI server 400 (S812). It should be noted that when multiple scanned image data are generated in S810, the processes of S811 and S812 are performed on each scanned image data.

[0057] Next, the CPU 201 determines whether an analysis result has been received from the generative AI server 400 (S813). In S813, when the status code in the response of the HTTP communication is an error or the main body information of the response includes a parameter indicating that information cannot be obtained, it is determined that an analysis result has not been received from the generative AI server 400. In this case, this process proceeds to S814.

[0058] In S814, the CPU 201 displays an acquisition error notification indicating that the analysis result cannot be received on the AI classification transmission setting screen 703. Next, the CPU 201 determines whether the "Return" button on the AI classification transmission setting screen 703 has been selected (S815). When it is determined in S815 that the "Return" button on the AI classification transmission setting screen 703 has not been selected, this process returns to S815. On the other hand, when it is determined in S815 that the "Return" button on the AI classification transmission setting screen 703 has been selected, this process returns to S801.

[0059] When it is determined in S813 that an analysis result has been received from the generative AI server 400, the CPU 201 generates a folder path indicating the storage location of the scanned image data based on the received analysis result (S816). In step S816, according to the Figure 9The folder path rule preset for job settings in the folder path rule setting column 910 of the job execution button setting screen 901 in [

[0060] Next, the CPU 201 displays the folder path generated in S814 on the AI classification transmission setting screen 703. The CPU 201 also displays on the AI classification transmission setting screen 703 buttons for selecting whether to use (confirm) or not use (cancel) the folder indicated by the folder path as the storage location (S817). It should be noted that a keyboard can be displayed on the operation unit 205 so that the user can edit the folder path generated in S814.

[0061] Next, the CPU 201 determines whether the folder indicated by the folder path has been confirmed as the storage location (S818). When it is determined in S818 that the folder indicated by the folder path has been canceled as the storage location, this process returns to S801. On the other hand, when it is determined in S818 that the folder indicated by the folder path has been confirmed as the storage location, this process proceeds to S819.

[0062] In S819, the CPU 201 sends the scanned image data whose file format has been converted in S810 to the storage server 500. At this time, the folder path confirmed in S818 is specified as the storage location of the scanned image data. Therefore, in the storage server 500, the scanned image data sent from the MFP 200 is saved in the folder indicated by the folder path confirmed in S818. After that, this process ends.

[0063] Figure 9 is shown for registration in Figure 7The figure shows an example of a job execution button setting screen 901 for the job execution button displayed on the main screen 701. When a user accesses the button setting page provided by the HTTP server (not shown) of the MFP 200 from the browser of the PC they are operating, the job execution button setting screen 901 is displayed. In Figure 9 As an example, information related to the "AI Classification and Sending" button 702 is set. It should be noted that in this embodiment, the structure for displaying the job execution button setting screen 901 on the display unit of the PC operated by the user will be described, but the present invention is not limited to this structure. For example, a screen corresponding to the job execution button setting screen 901 can be displayed on the operation unit 205 of the MFP 200.

[0064] In the logged-in user column 902, the name of the user who has logged in to the MFP 200 from the browser to set the job execution button is displayed. When setting the "AI Classification and Sending" button 702, a user with administrator privileges for the MFP 200 or an ordinary user with such setting permissions logs in to the MFP 200. When the logout button 903 is selected, the user logs out from the MFP 200, and a screen indicating that the user has logged out from the MFP 200 is displayed on the browser.

[0065] When the user selects the OK button 904, the MFP 200 stores the job settings set in columns 906 to 917 in the memory 204 and displays job execution buttons for executing jobs according to the job settings on the main screen 701. When the user selects the Cancel button 905, the MFP 200 clears the information set in columns 906 to 917 and ends the button setting.

[0066] In the button name column 906, the name of the job execution button displayed on the main screen 701 is set. The values set in the reading size selection column 907, the page splitting selection column 908, and the file format selection column 909 are used as the initial values of the reading size 704, the page splitting 707, and the file format 706 on the AI classification and sending setting screen 703, respectively.

[0067] In the folder path rule setting column 910, a folder path generation rule indicating the storage location of the scanned image data is set. In the AI analysis content setting columns 911 to 913, prompts for causing the generative AI server 400 to perform image analysis for extracting a string to be used for the folder path from the scanned image data are set. The prompts set in the AI analysis content setting columns 911 to 913 are displayed as options on the save location prompt selection screen 713. It should be noted that Figure 9 A structure showing that three prompts can be registered is shown as an example, but the number of prompts that can be registered is not limited to three.

[0068] Set the file name generation rule for the scanned image data in the file name rule setting column 914. In the AI analysis content setting columns 915 to 917, set the prompts for enabling the generative AI server 400 to perform image analysis for extracting the string to be used as the file name for the scanned image data from the scanned image data. The prompts set in the AI analysis content setting columns 915 to 917 are displayed as options on the file name prompt selection screen displayed on the operation unit 205 when the file name 709 is selected.

[0069] According to the above embodiment, scanned image data is generated by reading the image of the original document, and a prompt for enabling the generative AI server 400 to perform image analysis for extracting a predetermined string from the scanned image data is received. The scanned image data and the prompt are sent to the generative AI server 400, the analysis result of the image analysis performed by the generative AI server 400 is obtained, and control is performed to store the scanned image data in the folder indicated by the folder path including the string of the obtained analysis result. This can classify the image data to the storage location using the generative AI service.

[0070] In the above embodiment, the prompts to be sent to the generative AI server 400 are registered in association with the "AI classification send" button 702 which is a job execution button. This enables the user to instruct the generative AI server 400 to perform image analysis simply by pressing the "AI classification send" button 702 without inputting the prompts.

[0071] In the above embodiment, the folder path is displayed on the operation unit 205, and the user is enabled to select whether to store the scanned image data in the folder indicated by the folder path. This can prevent the scanned image data from being stored in an unexpected folder.

[0072] In the above embodiment, a keyboard for enabling the user to edit the folder path is displayed on the operation unit 205. This enables the user to change the storage location of the scanned image data with less trouble based on the folder path generated according to the analysis result of the image analysis performed by the generative AI server 400.

[0073] It should be noted that in the above embodiment, the configuration of directly sending the scanned image data to the generative AI server 400 in S811 has been described, but the present invention is not limited to being applied to this configuration. For example, after the scanned image data is stored in the storage server 500, the file path specifying the scanned image data can be notified to the generative AI server 400 in S811.

[0074] In the above-described embodiment, as an example, a configuration for determining the storage location of the scanned image data using the analysis result generated by the generative AI server 400 has been described. However, the analysis result generated by the generative AI server 400 can be used to determine the file name of the scanned image data.

[0075] In the present embodiment, when it is determined in S818 that the folder path displayed on the AI classification transmission setting screen 703 has been canceled as the storage location, a screen for editing a prompt may be displayed on the operation unit 205. On this screen, the prompt sent to the generative AI server 400 in S812 can be edited. For example, when the prompt sent to the generative AI server 400 in S812 is "Inform the company name", the user edits the prompt to "Inform the company name ending with 'Co., Ltd.'". That is, a prompt such as "ending with 'Co., Ltd.'" is added to a prompt such as "Inform the company name". Thereafter, this process returns to S811, and in S812, the prompt edited by the user is sent to the generative AI server 400. With this control, when the result desired by the user cannot be obtained in the image analysis performed by the generative AI server 400, the execution conditions of the image analysis can be changed, and the generative AI server 400 can perform the image analysis again. Therefore, the scanned image data can be controlled to be stored in the expected storage location.

[0076] In the present embodiment, a configuration has been described in which the generative AI server 400 performs image analysis for extracting a predetermined string from the scanned image data. However, the present invention is not limited to this configuration, and the MFP 200 may include a generative AI. For example, in S605 and S811, the CPU 201 of the MFP 200 inputs the scanned image data into a learning model that is stored in the memory 204 or the like and serves as a generative AI. Further, in S606 and S812, the CPU 201 inputs a prompt to the learning model. The learning model performs image analysis using these data as inputs. Such a configuration can also achieve the same effects as those of the above-described embodiment.

[0077] Other embodiments

[0078] Embodiments of the present invention can also be implemented by a computer of a system or apparatus that reads and executes computer-executable instructions (e.g., one or more programs) recorded on a storage medium (which may also be more fully referred to as a "non-transitory computer-readable storage medium") to perform one or more functions of the above-described embodiments and / or includes one or more circuits (e.g., an application specific integrated circuit (ASIC)) for performing one or more functions of the above-described embodiments, and embodiments of the present invention can be implemented by a method of, for example, reading and executing the computer-executable instructions from the storage medium by the computer of the system or apparatus to perform one or more functions of the above-described embodiments and / or controlling the one or more circuits to perform one or more functions of the above-described embodiments. The computer may include one or more processors (e.g., a central processing unit (CPU), a microprocessing unit (MPU)), and may include a network of separate computers or separate processors to read and execute the computer-executable instructions. The computer-executable instructions may be provided to the computer, for example, from a network or the storage medium. The storage medium may include, for example, a hard disk, a random access memory (RAM), a read only memory (ROM), a memory of a distributed computing system, an optical disc (such as a compact disc (CD), a digital versatile disc (DVD), or a Blu-ray disc (BD) TM ), a flash device, and a memory card, among others.

[0079] Although the present invention has been described with reference to exemplary embodiments, it should be understood that the present invention is not limited to the disclosed exemplary embodiments. The scope of the appended claims should be given the broadest interpretation so as to cover all such variations and equivalent structures and functions.

[0080] Embodiments of the present invention can also be implemented by the following method, that is, by providing software (including a computer program product of computer programs / instructions) that performs the functions of the above-described embodiments to a system or apparatus via a network or various storage media, and the method of the computer (central processing unit (CPU), microprocessing unit (MPU)) of the system or apparatus reading and executing the computer programs / instructions.

[0081] This application claims the priority of Japanese Patent Application No. 2024-009370 filed on January 25, 2024, the entire content of which is incorporated herein by reference.

Claims

1. An image processing apparatus, comprising: A reading unit configured to read an image of an original document and generate image data; An operation unit configured to receive a prompt for causing a generative AI to perform image analysis for extracting a predetermined string from the image data; And At least one memory and at least one processor, which function as the following units: A sending unit configured to send the image data and the prompt to the generative AI; An obtaining unit configured to obtain a string extracted by the image analysis performed by the generative AI; And A control unit configured to control to store the image data in a folder indicated by a folder path including the already obtained string.

2. The image processing apparatus according to claim 1, wherein A registration job execution button for executing a job of reading an image of an original document, generating image data, and storing the image data in the folder indicated by the folder path, and The prompt is registered in association with the job execution button.

3. The image processing apparatus according to claim 1, wherein, The at least one memory and the at least one processor further function as the following units: A display unit configured to display the folder path; And A selection unit configured to enable a user to select whether to store the image data in the folder indicated by the folder path.

4. The image processing apparatus according to claim 3, wherein, The at least one memory and the at least one processor further function as the following units: An editing unit configured to enable a user to edit the folder path.

5. The image processing apparatus according to claim 3, wherein When the user selects not to store the image data in the folder indicated by the folder path, the sending unit sends the image data and another prompt with supplementary information added to the prompt to the generative AI.

6. A control method for an image processing apparatus, the control method comprising the following steps: Reading an image of an original document and generating image data; Receiving a prompt for causing a generative AI to perform image analysis for extracting a predetermined string from the image data; Sending the image data and the prompt to the generative AI; Obtaining a string extracted by the image analysis performed by the generative AI; And Controlling to store the image data in a folder indicated by a folder path including the already obtained string.

7. A computer-readable non-transitory storage medium storing a program for causing a computer to execute a control method for an image processing apparatus, the control method comprising the following steps: Reading an image of an original document and generating image data; Receiving a prompt for causing a generative AI to perform image analysis for extracting a predetermined string from the image data; Sending the image data and the prompt to the generative AI; Obtaining a string extracted by the image analysis performed by the generative AI; And Controlling to store the image data in a folder indicated by a folder path including the already obtained string.

Images