Image processing system and control method thereof

Abstract

To provide a mechanism which improves the convenience of a user by preferably specifying information that a user wants to access when accessing information stored in a device by an operation through an interactive agent.SOLUTION: An image processing device that can manage image data by using multiple boxes in a box function includes: acquisition means which acquires a message and acquires feature information obtained from the message; specification means which specifies at least one box out of the multiple boxes on the basis of the feature information according to acquisition of one or more words corresponding to the intention to perform the box function from the message acquired by the acquisition means; and output means which outputs response information including the information about the box specified by the specification means.SELECTED DRAWING: Figure 2

Description

[Technical Field]

[0001] The present invention relates to an image processing system and a control method thereof. [Background technology]

[0002] In addition to the copy function, multifunction devices are equipped with a function (box function) that stores original images scanned by a scanner in a storage device such as an HDD (hard disk drive) inside the copier, enabling them to be reused. Image data stored in a box can be called up from the UI (user interface) and printed, or sent externally via a network. In this way, image data in a box can be used for a variety of purposes.

[0003] However, as the functions of electronic devices such as multifunction peripherals become more sophisticated, their operation becomes more complicated for users. Patent Document 1 proposes a dialogue agent that can give instructions and perform operations in natural language. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Japanese Patent Application Laid-Open No. 2014-222513 Summary of the Invention [Problem to be solved by the invention]

[0005] However, the above-mentioned conventional technology has the following problems. For example, in the box function, the area in which image data is stored is divided into multiple areas (boxes), each assigned a box number. Therefore, a user can use multiple areas with different box numbers depending on the purpose of the image data, for example. However, in the above-mentioned conventional technology, when operating boxes using a dialogue agent, the user may not remember when, in which of the multiple boxes, and what document was saved. As a result, the user ends up having to select the desired image file while checking the contents of the box on the UI, which requires complicated operations. This results in a problem of reduced usability due to the dialogue agent.

[0006] The present invention has been made in consideration of at least one of the above-mentioned problems, and aims to provide a mechanism that, when accessing information stored on a device through operations via a dialogue agent, can suitably identify the information that the user wishes to access, thereby improving user convenience. [Means for solving the problem]

[0007] The present invention is, for example, an image processing device capable of managing image data using a plurality of boxes in a box function, comprising: an extraction means for extracting feature information obtained from voice information acquired as input by a user; a specification means for specifying at least one box from the plurality of boxes based on the feature information in accordance with acquiring one or more words corresponding to an intention to execute the box function from the voice information; and a box specified by the specification means. Box Number and an output means for outputting response information including the When a plurality of boxes to be used by the user are identified based on the characteristic information, the output means outputs a predetermined box number to be preferentially presented to the user from among the plurality of boxes as the response information. It is characterized by the following. [Effects of the Invention]

[0008] According to the present invention, when accessing information stored in a device by operating via a dialogue agent, the information that the user wishes to access can be suitably identified, improving user convenience. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a diagram showing the hardware configuration of a multifunction peripheral according to an embodiment. [Figure 2] FIG. 1 is a diagram showing a system configuration according to an embodiment. [Figure 3] FIG. 10 is a diagram illustrating an example of an operation of a dialogue agent according to an embodiment. [Figure 4] FIG. 10 is a sequence diagram showing a box storage process according to an embodiment. [Figure 5] FIG. 10 is a diagram illustrating an example of an operation of a dialogue agent according to an embodiment. [Figure 6] FIG. 4 is a sequence diagram showing a box print process according to an embodiment. [Figure 7] FIG. 4 is a diagram showing an example of file management information according to an embodiment. [Figure 8] FIG. 10 is a diagram showing information for executing an action according to an embodiment. [Figure 9] FIG. 10 is a diagram illustrating an example of an operation of a dialogue agent according to an embodiment. [Figure 10] FIG. 10 is a sequence diagram showing a box transmission process according to an embodiment. [Figure 11] 10 is a flowchart illustrating an example of a processing procedure of a multifunction peripheral according to an embodiment. [Figure 12] 10 is a flowchart showing a storage process of a multifunction peripheral according to an embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0010] An embodiment of the present invention will be described below. The individual embodiments described below will be useful for understanding various concepts, such as higher-level, middle-level, and lower-level concepts, of the present invention. The technical scope of the present invention is defined by the claims and is not limited to the individual embodiments described below. The image processing device according to the embodiment will be described using a multifunction peripheral (digital multifunction peripheral / MFP / Multi Function Peripheral) as an example. However, the present invention can also be applied to electrophotographic image processing devices such as laser printers and fax machines without departing from the spirit of the present invention. Furthermore, the present invention does not necessarily have to be limited to image processing devices, and can also be applied to information processing devices that do not have image forming or image processing functions.

[0011] First Embodiment <Configuration of image processing device> A first embodiment of the present invention will be described below with reference to the accompanying drawings. First, the hardware configuration of a multifunction peripheral 10 according to this embodiment will be described with reference to FIG. 1. The multifunction peripheral 10 of this embodiment is assumed to have a scanner function, but may also be a multifunction peripheral without a scanner function. In addition, an image processing device to which the present invention is applied will be described as an example, but functions realized by the cooperation of each device in an image processing system described below with reference to FIG. 2 may be assigned to the image processing device or other devices in any manner. Therefore, it is not intended that the present invention be limited to the configuration of this embodiment described below.

[0012] The multifunction device 10 includes a printer unit 13, a scanner unit 14, an operation unit 15, a speaker 16, a microphone 17, and a controller 100. The controller 100 includes a CPU 101, a RAM 102, a ROM 103, a HDD 104, an image processing unit 105, a printer I / F 106, a scanner I / F 107, an operation unit I / F 108, an audio processing unit 109, a wireless LAN I / F 110, and a wired LAN I / F 111.

[0013] The CPU 101 reads out a control program stored in the ROM 103 and executes various processes for comprehensively controlling the operation of the multifunction peripheral 10. The ROM 103 stores the control program and the like. The RAM 102 is used as a temporary storage area such as the main memory and work area of ​​the CPU 101, and is also used as an image memory for storing image data that has undergone various image processing by the image processing unit 105 during printing, etc. The HDD 104 stores various data such as print data and scanned images. The HDD 104 includes multiple divided box areas (e.g., 100 boxes) described below and provides box functions. In the case of the multifunction peripheral 10, a single CPU 101 executes each process shown in the flowcharts described below, but other configurations are also possible. For example, a configuration in which multiple CPUs cooperate to execute each process shown in the flowcharts described below may be used.

[0014] Printer unit 13 is controlled by CPU 101 via printer I / F 106, and executes printing processing on sheets based on print data received from an external device and scanned images generated by scanner unit 14. Scanner unit 14 is controlled by CPU 101 via scanner I / F 107, and reads an original and generates scanned images (scanned image data). The scanned images generated by scanner unit 14 are printed on sheets by printer unit 13, or stored in a box area of ​​HDD 104.

[0015] The operation unit 15 includes a liquid crystal display with a touch panel function and a keyboard, and displays various images. A user can input instructions and information to the multifunction device 10 via the operation unit 15. The operation unit 15 is connected to the controller 100 via an operation unit I / F 108.

[0016] The speaker 16 is an output interface for transmitting a sound wave signal processed by the audio processing unit 109. The microphone 17 is an input interface for receiving a sound wave signal from the outside. The wired LAN I / F 111 connects to the LAN 12 via a LAN I / F connector and performs network communication with a destination PC. The wireless LAN I / F 110 performs wireless LAN communication with an external device (not shown) such as an access point via the wireless LAN communication unit 11, which is an antenna. The blocks within the controller 100 are connected by a system bus 112 so that they can send and receive data to and from each other. The microphone 17 is an input interface for receiving a sound wave signal from the outside. The speaker 16 is an output interface for transmitting a sound wave signal processed by the audio processing unit 109 within the controller 100 of the multifunction peripheral 10.

[0017] <System configuration> Next, the system configuration of this embodiment will be described with reference to FIG. 2. The multifunction peripheral 10 according to this embodiment has a dialogue agent that dialogues with a user via voice input or the like. The dialogue agent receives operation input from a user 50 in natural language via a microphone 17 and transmits the input to a cloud-based natural language processing and lexical analysis service (e.g., a server device) 30. Here, the configuration in which the multifunction peripheral 10 acquires the user's voice is not limited to the microphone 17 connected to the multifunction peripheral 10, but a microphone (not shown) built into the mobile terminal 40 may also be used. In this case, the multifunction peripheral 10 and the mobile terminal 40 are wirelessly connected via a wireless LAN or USB or via a wired connection 60, and the user's voice input to the mobile terminal 40 via the multifunction peripheral 10 is transmitted to the natural language processing and lexical analysis service 30. Alternatively, the voice from the mobile terminal 40 may be transmitted directly to the natural language processing and lexical analysis service 30.

[0018] In this embodiment, the natural language processing and lexical analysis service 30 will be described as a cloud-based service. However, this is not intended to limit the present invention, and the service may be provided on the multifunction peripheral 10, the mobile terminal 40, or the like. When the natural language processing and lexical analysis service 30 receives a user's operation in natural language from a dialogue agent on the multifunction peripheral 10 or the mobile terminal 40, the service performs natural language processing and analyzes the extracted lexical characters. Based on the result, the service communicates with the multifunction peripheral management service 20, which is a cloud-based service, and communicates with the multifunction peripheral 10 based on the communication from the natural language processing and lexical analysis service 30. The multifunction peripheral management service 20 is an example of a management server. The multifunction peripheral management service 20 is an intermediate function provided between the natural language processing and lexical analysis service 30 and multiple multifunction peripherals. It transmits commands from the multifunction peripheral management service 20 to a specific multifunction peripheral and notifies the natural language processing and lexical analysis service 30 of the response from the multifunction peripheral. The multifunction peripheral management service 20 may be provided on the same device as the natural language processing and lexical analysis service 30, or on a different device. In this embodiment, an example of operation will be described in which the microphone 17 connected to the multifunction peripheral 10 is used to acquire the user's voice.

[0019] <Dialogue Agent> Next, an example of the operation of the dialogue agent of this embodiment will be described with reference to FIG. 3. First, the user 50 operates the operation unit 15 of the multifunction peripheral 10 or speaks a predetermined word to the multifunction peripheral 10, thereby activating the dialogue agent and causing the multifunction peripheral 10 to enter dialogue mode. Then, with the dialogue agent on the multifunction peripheral 10 activated, the user 50 uses natural language voice to notify activation of the box function at 301. As described above, the box function provides the function of saving files to multiple box areas in the HDD 104 and reading files from a predetermined box area. 301 and 303 indicate information input by the user 50's voice. 302, 304, and 305 indicate natural language data that is played back as voice via the speaker 16 or displayed on the operation unit 15 in response to transmission from the natural language processing and lexical analysis service 30. The content shown in FIG. 3 may be displayed on the operation unit of the multifunction peripheral 10 or the mobile terminal 40.

[0020] The multifunction device 10 transmits the input natural language data 301 to the natural language processing and lexical analysis service 30. The natural language processing and lexical analysis service 30 analyzes the input natural language data and divides it into phrases and words. It then extracts nouns from each word and classifies them into proper nouns and other nouns. It then determines whether the classified nouns contain the pre-registered "information for executing an action."

[0021] FIG. 8 shows an example of information 800 for executing an action in this embodiment. For example, in this embodiment, a call name 801 for using the box function, such as "Box," and actions 802 for the box function, such as "Save, Print, Send," are registered in advance. Furthermore, additional information 803 indicating time, such as "month, day, hour, minute, today, yesterday, or days ago," and additional information 804 indicating a box number indicating a specific box area, such as "Number," are also registered in advance. As a result of the natural language processing and lexical analysis service 30 determining the natural language 301, in the example of FIG. 3, it is determined that additional information 2, "Box Number" (804), is missing from the "information for executing an action." Therefore, the natural language processing and lexical analysis service 30 sends a query to the multifunction device 10 to identify the "Box Number."

[0022] The dialogue agent of the multifunction device 10 outputs the received inquiry 302 by voice and / or display. In this embodiment, the additional information 803 and 804 is not necessarily required information, but is auxiliary information that is determined in conjunction with the action 802. As another example using this information, it is possible to operate a box function from multiple patterns of natural language data such as "Send the file saved x days ago" or "Print the file saved in box x."

[0023] The user 50 communicates the box number in which they wish to save in natural language through the dialogue agent on the multifunction device 10 at 303. The multifunction device 10 transmits the natural language data input to the natural language processing and lexical analysis service 30 to the natural language processing and lexical analysis service 30. The natural language processing and lexical analysis service 30 analyzes the received input. As a result, the information required to execute the pre-registered processing is complete, and the natural language processing and lexical analysis service 30 requests the multifunction device management service 20 to start the box function. Thereafter, the multifunction device management service 20 responds to the multifunction device 10 in natural language via the natural language processing and lexical analysis service 30 to start the processing (304), and executes the box saving process. The multifunction device 10 then provides audio guidance to the user informing them that the document is being scanned (305).

[0024] <Sequence of saving process> Next, the sequence of the saving process of this embodiment will be described with reference to Fig. 4. Here, the process flow for saving image data to any box desired by the user will be described as an example. The process described below is realized, for example, by the CPU of each device reading a control program stored in the ROM or HDD into the RAM and executing it.

[0025] When the dialogue agent on the multifunction peripheral 10 receives an operation from the user in natural language, in S401 the multifunction peripheral 10 transmits a user operation request expressed in natural language to the natural language processing and lexical analysis service 30. Next, in S402, the natural language processing and lexical analysis service 30 performs natural language processing and lexical analysis on the user operation request. After that, in S403, if the analysis results in the natural language processing and lexical analysis service 30 finding that the pre-registered information is not available, the natural language processing and lexical analysis service 30 transmits a user operation response expressed in natural language to the multifunction peripheral 10. The natural language processing and lexical analysis service 30 pre-registers the necessary information and the processing to be executed when the necessary information is available, and the multifunction peripheral 10 and the natural language processing and lexical analysis service 30 repeat the above-described exchange until the necessary information is available.

[0026] 3 and 8 as examples, the information required for the box function refers to information 800 for executing action 802 in Fig. 8. In Fig. 3, when user 50 tells multifunction device 10 that he wants to save to a box, information such as "which box number to save to" is missing. Therefore, natural language processing and lexical analysis service 30 queries user 50 for additional information 804 (box number) required to execute action 802, "save."

[0027] Next, in S404, when the natural language processing and lexical analysis service 30 has gathered the necessary information as a result of the above-mentioned exchange, it sends a request to the multifunction device management service 20 to start the box function of the multifunction device. In S405, when the multifunction device management service 20 receives a request to start the box function of the multifunction device from the natural language processing and lexical analysis service 30, it sends a box function start response to the natural language processing and lexical analysis service 30. In S406, when the natural language processing and lexical analysis service 30 receives the box function start response from the multifunction device management service 20, it sends the box function start response to the multifunction device 10. Also, in S407, the multifunction device management service 20 sends a box function start command to the multifunction device 10. The box function start response and the notification in the start command include the box number (e.g., 303) determined in dialogue with the user. Therefore, the multifunction device 10 executes box function processing for the specified box number. Note that the multifunction device 10 can also access boxes with box numbers other than the specified box number during box function processing.

[0028] Next, in S408, when the MFP 10 receives a command to start the box function, it executes a box saving process, which will be described later with reference to FIG. 12. The saving format may be any format, such as PDF, JPEG, or TIFF. Next, in S409, after the MFP 10 has finished the box saving process, it responds to the MFP management service 20 as a box function completion response, and ends the process. Here, the MFP 10 generates and sends file management information to the MFP management service 20. The MFP management service 20 stores the file management information received from the MFP 10 in a storage device, such as an HDD, provided in the MFP management service. The MFP management service 20 stores the file management information received from the MFP 10 in a file information management database so that the history of the file management information received in the past can be searched, and performs a database update process.

[0029] <File management information> An example of file management information 700 will now be described with reference to FIG. 7. Note that the file management information 700 described below is information managed by the multifunction device management service 20, but each multifunction device may also hold file management information 700 related to the interaction as a record of the interaction with the user. The file management information 700 is information generated for each piece of data (file) to be saved, and includes information related to the saved data and information related to the user who saved it. Note that there is no intention to limit the present invention to the file management information 700 described in FIG. 7, and it is not necessary to manage all information, or other information may be managed in addition.

[0030] The file management information 700 includes information such as a file name 701, a save date and time 702, a paper size 703, a number of pages 704, a box number 705, a user ID 706, and a voice ID 707. The user ID 706 may be identified by the user 50 performing personal authentication such as inputting an ID into the multifunction peripheral 10 or using an ID card. The voice ID 707 may be determined by the natural language processing and lexical analysis service 30 registering characteristic information of the user 50's voice in advance, identifying the user when a user operation is requested, and notifying the multifunction peripheral 10 of the identified user when responding to the user operation. The information included in the file management information 700 is not limited to the information described above, and may be other information. Furthermore, when searching for a relevant file in response to a user instruction, convenience can be further improved by searching only for files saved with the previously identified user ID 706 or voice ID 707.

[0031] <Processing Procedure> Next, a processing procedure of the multifunction peripheral 10 according to this embodiment will be described with reference to Fig. 11. The processing described below is realized, for example, by the CPU 101 of the multifunction peripheral 10 reading out a control program stored in advance in the ROM 103 and HDD 104 into the RAM 102 and executing the program. Note that the description here will be given of a case where a user interacts with a dialogue agent by voice.

[0032] In S1101, the CPU 101 receives a voice input from the user via the microphone 17 and recognizes the voice (identifies the speaker). The voice input from the user here is received at the time of voice input at the time of activation of the dialogue agent, as described with reference to FIG. 3, or at the time of input shown in 301. In speaker recognition, features of the received voice signal are extracted to generate feature information for each user. The information identified by speaker recognition is unique biometric information and is used as information for identifying the speaker. This process is performed using a known method, and a detailed description thereof will be omitted. Note that the speaker recognition may be performed by the multifunction peripheral 10 or by a server on the cloud. If performed by a server on the cloud, the received voice signal may be transmitted to the server, and a voice ID linked to the unique feature information may be obtained as a response.

[0033] Once the speaker is identified, in S1102, the CPU 101 determines whether the identified speaker is already registered. Here, a table of registered speakers may be referenced to determine whether information matching the identified speaker, such as a voice ID, is registered. This table may be a table held by the multifunction device 10, or file management information managed by the multifunction device management service 20. When using the file management information of the multifunction device management service 20, the CPU 101 may transmit the voice ID (feature information) of the identified speaker to the multifunction device management service 20 and, in response, obtain information on whether the identified speaker is already registered. Alternatively, the CPU 101 may first reference a table held by the multifunction device itself, and if no matching person is found, inquire of the multifunction device management service 20. This is effective when each multifunction device registers only related users in order to efficiently use the memory resources of each multifunction device. If the speaker is already registered, the process proceeds to S1104; otherwise, the process proceeds to S1103.

[0034] At S1103, CPU 101 registers the voice ID of the identified speaker in the table it holds or in the multifunction device management service 20, and proceeds to S1104. At S1104, CPU 101 determines whether the request by the user operation analyzed at S402 is a box storage process. If it is a box storage process, proceed to S1106, and if not, proceed to S1105. At S1105, CPU 101 accesses the box in response to the user request and ends the process. Meanwhile, at S1106, CPU 101 executes a process of storing data in a box, which is a predetermined area of ​​HDD 104, and ends the process. Details of the box storage process will be described later using FIG. 12.

[0035] <Save in Box process> Next, the box saving process of S1106 according to this embodiment will be described in detail with reference to Fig. 12. The process described below is realized, for example, by the CPU 101 of the multifunction peripheral 10 reading out a control program stored in advance in the ROM 103 and HDD 104 into the RAM 102 and executing the program.

[0036] In S1201, CPU 101 reads an original placed on the automatic document feeder or document tray of multifunction peripheral 10 using scanner unit 14 to generate image data. Subsequently, in S1202, CPU 101 saves the generated image data in the box having the box number specified in S406 or S407. Furthermore, in S1203, CPU 101 generates and saves file information for the saved image data. Here, the file information includes at least the file name, the date and time of saving, the file size (paper size), the number of pages, the box number of the box where the data is saved, and the user ID. Note that the voice ID (characteristic information) of the identified speaker may be included in addition to or instead of the user ID.

[0037] Next, in S1204, CPU 101 updates the box usage history of multifunction device 10 and performs learning. Here, the box usage history includes the content of operations performed on a box with a specific box number and the date and time. Next, in S1205, CPU 101 associates and saves the information of the identified speaker (user ID and voice ID), the generated file information, and the box usage history.

[0038] The information saved in S1203 to S1205 may be saved in the MFP 10, or may be sent to the MFP management service 20 and saved there, or both. This saved information is managed by the MFP 10 or the MFP management service 20 as file management information described with reference to FIG. 7, and can be used to estimate boxes that the user can access when the dialogue agent is launched. In other words, in the system according to this embodiment, speaker identification is performed on the voice received when the dialogue agent is launched or in the user input in 301 above, and boxes that the identified speaker has used in the past are presented. This can improve the convenience of the user interface.

[0039] In this embodiment, the flow for saving a scanned document in a box has been described, but the same can be done for printing or sending a file saved in a box to an external party. In this case, the user only needs to tell the interactive agent the destination (e.g., email address) of the file to be printed or sent, rather than the box number where the file is saved.

[0040] According to the present invention, management information for image data (files) saved by the box function is stored, and the next time the box function is used, the user can identify the image data they desire by referencing the management information for the image data via a dialogue agent. This makes it possible to improve the user's convenience when operating the box function using a dialogue agent. The method for using the management information will be described in more detail below.

[0041] As described above, the user 50 needs to communicate the box number in which the file is to be saved to the interactive agent of the multifunction device 10. However, when the user 50 wants to reuse a saved file (for printing, external transmission via a network, etc.), if the user 50 does not remember when, where, or what file was saved, searching for the file using only voice operations can be inconvenient. For this reason, the user 50 may consider using both voice and operation via the operation unit 15, but these operations can be cumbersome. Therefore, in this embodiment, an example of an operation that further improves the convenience when the user operates the box function by voice will be described.

[0042] <Box print processing> Next, an example of the operation of the dialogue agent during box print processing according to this embodiment will be described with reference to Fig. 5. First, the user 50 operates the operation unit 15 of the multifunction device 10, or speaks a predetermined word to the multifunction device 10, thereby activating the dialogue agent on the multifunction device 10 and causing the multifunction device 10 to transition to dialogue mode. Then, with the dialogue agent on the multifunction device 10 activated, the user 50 notifies the activation of the box function by voice in natural language at 501. Reference numerals 501 and 503 denote information input by the user 50's voice. Reference numerals 502 and 504 denote natural language data that is played back by voice via the speaker 16 or displayed on the operation unit 15 in response to transmission from the natural language processing and lexical analysis service 30.

[0043] In step 501, the user utters, "Print the file saved on ____ month ____ day." In response to this, as described above with reference to FIG. 11, the multifunction device 10 identifies the speaker from the user's input voice in accordance with the registration information (particularly, the voice ID) of the speaker when the user saved the data (file) and the file information linked thereto. Furthermore, the multifunction device 10 identifies the saved file linked to the identified speaker. Note that these identification processes may be performed by the multifunction device management service 20 or the natural language processing / lexical analysis service 30.

[0044] Thereafter, the multifunction device 10 responds to the user, "Is file name: abcdef OK?" in accordance with instructions from the natural language processing and lexical analysis service 30. If it is the desired file, the user 50 responds "Yes" at 503. The multifunction device 10 responds "Print file name: abcdef" at 504 and executes printing.

[0045] <Box print processing sequence> Next, the sequence of the box print process according to this embodiment will be described with reference to Fig. 6. The process described below is realized, for example, by the CPU of each device reading out a control program stored in the ROM or HDD into the RAM and executing it.

[0046] In S601, the multifunction peripheral 10 transmits the input natural language data (501) to the natural language processing and lexical analysis service 30. Here, the multifunction peripheral 10 identifies the speaker from the user-input voice, as described above with reference to FIG. 11 . In S602, the natural language processing and lexical analysis service 30 analyzes the input natural language data, divides it into phrases and words, extracts nouns from each word, and classifies these nouns into proper nouns and other nouns. As a result, the natural language processing and lexical analysis service 30 determines whether the classified nouns contain pre-registered "information for executing an action." If, as a result of the analysis, the natural language processing and lexical analysis service 30 determines that the pre-registered information is not present, in S603, the natural language processing and lexical analysis service 30 transmits a user operation response expressed in natural language to the multifunction peripheral 10. Necessary information and the processing to be executed when the information is collected are registered in the natural language processing and lexical analysis service 30 in advance, and the multifunction device 10 and the natural language processing and lexical analysis service 30 repeat this exchange until the necessary information is collected.

[0047] If the speaker can be identified, the natural language processing and lexical analysis service 30, in cooperation with the multifunction device management service 20 and the multifunction device 10, estimates the box number and file the user is trying to access. The estimated information is queried to the user in response to the user operation (502). By identifying managed information (boxes, files, etc.) from the user's voice ID and presenting it to the user in this way, the convenience of the user interface can be improved. For example, if the file management information includes a noun such as "month x day," which indicates the save date and time, the natural language processing and lexical analysis service 30 checks with the multifunction device management service 20 in S604 whether there is a file saved by the identified speaker on "month x day." Next, in S605, the multifunction device management service 20 searches the file management information (described later) for a file corresponding to the save date and time "month x day." If a matching file is found, the multifunction device management service 20 transmits the file management information check result to the natural language processing and lexical analysis service 30 in S606. Thereafter, in S607, the natural language processing and lexical analysis service 30 converts the collation result into a voice in natural language and transmits it to the multifunction device 10 (502).

[0048] Thereafter, in S608 to S610, similar to S601 to S603, the determination process is repeated until the pre-registered "information for executing an action" is obtained from the user's 50 voice (503). Next, in S611, when the natural language processing and lexical analysis service 30 has obtained the necessary information as a result of the above-mentioned exchange, it transmits a request to activate the box function of the multifunction device to the multifunction device management service 20. In S612, when the multifunction device management service 20 receives a request to start the box function of the multifunction device from the natural language processing and lexical analysis service 30, it transmits a box function start response to the natural language processing and lexical analysis service 30. In S613, when the natural language processing and lexical analysis service 30 receives the box function start response from the multifunction device management service 20, it transmits the box function start response to the multifunction device 10 (504). Also, in S614, the multifunction device management service 20 transmits a box function start command to the multifunction device 10. The start command includes at least the box number to be accessed, information indicating the specified file, and a box print instruction (print instruction).

[0049] Next, in S615, upon receiving the box function start command, the multifunction device 10 executes box print processing of a predetermined file stored in the box area of ​​the specified box number. In S616, after completing the box print processing, the multifunction device 10 transmits a response to the multifunction device management service 20 as a box function completion response. Next, in S617, the multifunction device 10 generates and transmits file management information to the multifunction device management service 20. The multifunction device management service 20 stores the file management information received from the multifunction device 10 in a storage device such as an HDD provided in the multifunction device management service.

[0050] In S618, the multifunction device management service 20 stores the previously received file management information in a file information management database so that the history of the information can be searched, and performs a database update process. In S619, the multifunction device management service 20 transmits a file management information update response to the multifunction device 10 indicating that the file management information update was successful, and ends the process.

[0051] As described above, the image processing device according to this embodiment is an image processing device having a dialogue agent capable of conversing with a user using natural language. The dialogue agent can be implemented in cooperation with natural language processing, lexical analysis services, and a multifunction device management service provided by various servers on the cloud. The image processing device receives the voice of a user who is conversing with the dialogue agent, identifies the user from characteristics of the received voice, estimates a storage area or stored data associated with the identified user, and presents the estimated storage area or stored data through the dialogue agent. As described above, according to this embodiment, when accessing information stored in the device through an operation via the dialogue agent, the information the user desires to access can be appropriately identified, improving user convenience.

[0052] Furthermore, if the image processing apparatus according to this embodiment cannot identify a user, it registers a new user based on the characteristics of the voice. At this time, identification information for identifying the user from the voice is linked to history information of the files and storage area used and registered. Furthermore, when a file is saved in a box area, file management information is managed in a database. The file management information includes at least one of the file name, the date and time of saving, the paper size of the document, the number of pages of the document, the box number indicating the box area where the file was saved, the user ID, and the voice ID. Thus, according to this embodiment, by storing file management information in a database, it is possible to improve the convenience of reusing files in a box.

[0053] <Second embodiment> The second embodiment of the present invention will be described below. In the first embodiment, file management information is compiled into a database, and an operation for printing a file in a box is described. In this embodiment, an example of an operation for sending a file in a box to an external device, such as an email address, via a network such as the LAN 12 will be described.

[0054] <Box sending process> An example of the operation of the dialogue agent according to this embodiment will be described with reference to Fig. 9. First, the dialogue agent is activated when the user 50 operates the operation unit 15 of the multifunction device 10 or when the user 50 speaks a predetermined word to the multifunction device 10. The multifunction device 10 transitions to dialogue mode.

[0055] Then, the user 50, with the dialogue agent on the multifunction peripheral 10 activated, issues a command 901 in natural language to activate the box function. The multifunction peripheral 10 then identifies the speaker in the same manner as described with reference to FIG. 5. 901, 903, and 905 are pieces of information input by the user 50 through their voice. 902, 904, and 906 are pieces of natural language data that are played back by voice via the speaker 16 or displayed on the operation unit 15 in response to transmission from the natural language processing and lexical analysis service 30.

[0056] Specifically, in 901, the user utters, "Send the file abcdef." In response, the natural language processing and lexical analysis service 30 interprets this as an instruction to execute a box transmission process, and acquires the information required for that, such as information on the transmission destination, through dialogue (902, 903). After that, once the required information is gathered, the natural language processing and lexical analysis service 30 confirms the final instruction (904, 905) and requests the multifunction device 10 to execute the process (906).

[0057] <Box sending process sequence> Next, the box transmission processing sequence according to this embodiment will be described with reference to Fig. 10. Here, the differences from Fig. 6 described in the first embodiment will be mainly described. That is, only the point in S1007 to S1009 where the natural language processing and lexical analysis service 30 checks the destination address for the multifunction device 10 will be described. The other processes of S1001 to S1006 and S1014 to S1022 are the same as those of S601 to S607 and S608 to S619 in Fig. 6, and therefore descriptions thereof will be omitted.

[0058] In this embodiment, the box of the multifunction device 10 has an address book function, and if a "name" and "email address" are registered in advance, the address book can be used to send a file when sending from the box. In S1007, the natural language processing and lexical analysis service 30 checks with the multifunction device 10 whether or not a corresponding name (here, "Sato") exists in the address book of the multifunction device 10 based on the analysis result of the voice input in 903.

[0059] In S1008, the multifunction device 10 checks the address book against the name included in the address information input from the natural language processing and lexical analysis service 30, and returns the check result to the natural language processing and lexical analysis service 30. If the name exists in the address book, in S1010 the natural language processing and lexical analysis service 30 confirms the final instruction (904).

[0060] Thereafter, in S1017, the multifunction device management service 20 adds at least destination information (name registered in the address book), box number, file, and box transmission processing instruction (transmission instruction) to the box function start command, and sends it to the multifunction device 10. In S1018, when executing the box transmission processing, the multifunction device 10 refers to the email address based on the destination information, determines the destination of the file, and executes the processing.

[0061] As described above, the image processing device according to this embodiment can obtain the same effects as those of the first embodiment, even when performing box transmission, which transmits a file stored in a box to an external device. More specifically, by identifying the user (speaker) performing the operation through a dialogue agent and presenting boxes and stored files related to the identified speaker, it is possible to improve the convenience when the user gives an instruction for box transmission.

[0062] <Modification> The present invention is not limited to the first and second embodiments and can be modified in various ways. For example, in the above embodiments, a control is described in which a user's voice ID is identified and a box number and files stored therein associated with the voice ID are identified. However, it is naturally expected that a user will not necessarily use only one box, but will use multiple boxes and files. In such cases, it is cumbersome for the user to check each box one by one to find the desired stored file. Therefore, in such cases, file management information may be associated with the voice ID, and control may be exercised to preferentially present specific boxes and files based on the frequency of use of boxes and files, etc., or the most recent usage status, based on the usage history. This can improve the convenience of the user interface even in the above cases.

[0063] <Other embodiments> The present invention can also be realized by supplying a program that realizes one or more functions of the above-described embodiments to a system or device via a network or a storage medium, and having one or more processors in the computer of the system or device read and execute the program.The present invention can also be realized by a circuit (e.g., ASIC) that realizes one or more functions. [Explanation of symbols]

[0064] 10: Multifunction printer management service, 30: Natural language processing, lexical analysis service, 40: Mobile terminal, 50: User

Claims

1. An image processing device capable of managing image data using a plurality of boxes in a box function, extraction means for extracting feature information obtained from voice information acquired as an input by a user; an identification means for identifying at least one box from the plurality of boxes based on the feature information in accordance with obtaining one or more words corresponding to an intention to execute the box function from the voice information; an output means for outputting response information including the box number of the box identified by the identification means; Equipped with When a plurality of boxes to be used by the user are identified based on the characteristic information, the output means outputs a predetermined box number to be preferentially presented to the user from among the plurality of boxes as the response information.

2. a storage unit for storing an image in one of the plurality of boxes based on the voice information including an instruction to store the image data; 2. The image processing apparatus according to claim 1, further comprising: a generating unit for generating management information indicating information about the image data.

3. 3. The image processing device according to claim 2, wherein the management information includes at least one of a file name, a save date and time, a paper size of the document, a number of pages of the document, a box number indicating the box area in which the document is saved, a user ID, and an audio ID indicating the characteristic information.

4. A control method for an image processing device capable of managing image data using a plurality of boxes in a box function, comprising: an extraction step of extracting feature information obtained from voice information acquired as an input by a user; an identifying step of identifying at least one box from the plurality of boxes based on the feature information in accordance with obtaining one or more words corresponding to an intention to execute the box function from the voice information; an output step of outputting response information including the box number of the box identified in the identification step; Including, In the output step, when a plurality of boxes to be used by the user are identified based on the characteristic information, a predetermined box number to be preferentially presented to the user from among the plurality of boxes is output as the response information.

Images