Information processing apparatus, information processing method, computer readable medium, and computer program product

Abstract

Information processing apparatus, information processing method, and computer-readable medium are provided. The information processing apparatus has a processor that receives an instruction to perform a service by audible signaling, the processor detects physical operations performed on the apparatus for performing the service, and the processor controls the execution of the received service in the apparatus at the moment closest to the moment the acceptance is performed.

Description

Technical Field

[0001] This invention relates to information processing apparatus, information processing method, and computer-readable medium. Background Technology

[0002] Japanese Patent Application Publication No. 2008-250236 discloses a speech recognition device comprising: a document generation unit that acquires information related to the state of a driver engaging in conversation with a passenger and generates document data based on the acquired information; a speech recipient determination unit that compares the driver's state with the document data during the driver's speech and determines whether the speech is a conversation with a passenger or a voice operation input; and a speech recognition unit that performs speech recognition on speech that is determined by the speech recipient determination unit to be a voice operation input.

[0003] Japanese Patent Application Publication No. 2019-095835 discloses a voice control system capable of controlling an output device via voice instructions. The voice control system comprises: a first voice controller that receives voice instructions; a second voice controller that receives voice instructions; a control unit that causes the output device to output based on a voice instruction received by at least one of the first and second voice controllers; and a determination unit that determines whether a first voice instruction received by the first voice controller and a second voice instruction received by the second voice controller are duplicate instructions. If the determination unit determines that the first and second voice instructions are duplicate instructions, the control unit restricts the output of the output device.

[0004] In the past, when multiple devices were present, the following problem occurred: a device that did not expect the speaker to perform the service corresponding to the instruction based on the speech was executed. Summary of the Invention

[0005] The object of the present invention is to provide an information processing apparatus, an information processing method, and a computer-readable medium, which prevents the following situation in which, in the presence of multiple devices, a device that does not want the speaker to perform a service corresponding to an instruction based on the speech is executed.

[0006] According to a first aspect of the present invention, an information processing apparatus is provided, wherein the information processing apparatus has a processor that receives an instruction to perform a service by audible signal, the processor detects a physical operation performed on the apparatus for performing the service, and the processor controls the execution of the received service in the apparatus at the moment closest to the moment when the operation is detected.

[0007] According to a second aspect of the invention, the processor controls the execution of the service in the device that detects the operation immediately before the acceptance is performed.

[0008] According to a third aspect of the invention, the processor controls the execution of the service in the device that detects the operation immediately after the acceptance is performed.

[0009] According to a fourth aspect of the invention, if no physical operation for performing the service is detected on the device within a predetermined time from the moment the acceptance is initiated, the processor controls the process to prompt the operation with information indicating an urgency to perform the operation.

[0010] According to a fifth aspect of the invention, the processor controls the execution of the accepted service in the device that has just performed the control of prompting the information.

[0011] In a sixth aspect of the present invention, if multiple devices are detected to have performed the operation within a predetermined time period closest to the time when the acceptance was performed, the processor further determines which device controls the execution of the service, and the processor controls the execution of the accepted service in the accepted device.

[0012] According to a seventh aspect of the invention, the processor further controls the prompting of determination information, which is information that can be determined by the means controlling the execution of the service.

[0013] According to the eighth aspect of the present invention, the processor further accepts a sound corresponding to the prompt of the determined information, and the processor performs control corresponding to the accepted sound.

[0014] According to a ninth aspect of the present invention, if the volume of the noise generated at the same time as the sound is greater than a predetermined level, the processor controls the prompting of the determination information, wherein the sound is a sound indicating the execution of the service.

[0015] According to a 10th aspect of the present invention, if multiple devices are detected to have performed the operation within a predetermined time period closest to the time when the acceptance was performed, the processor performs control to prompt the determination information.

[0016] According to the eleventh aspect of the present invention, the apparatus is an image forming apparatus.

[0017] According to a 12th aspect of the present invention, a computer-readable medium is provided, which stores a program that causes a computer to perform processing, wherein, by audibly receiving an instruction to perform a service, a physical operation performed on a device for performing the service is detected, and the device that detects the operation at the moment closest to the moment the service is performed controls the execution of the service.

[0018] According to a 13th aspect of the present invention, an information processing method is provided, wherein a physical operation performed on a device for performing the service is detected by audibly accepting an instruction to perform the service, and the device in which the operation is detected at the moment closest to the time when the acceptance is performed controls the execution of the service.

[0019] (Effect)

[0020] According to the first, 12, or 13th scheme, it is possible to prevent the following situation: in the presence of multiple devices, a device that does not want the speaker to perform the service corresponding to the instruction based on the speech is performed.

[0021] According to the second scheme, the desired device can be ensured.

[0022] According to the third scheme, it is not necessary to determine the device that performs the service before uttering a sound to instruct the performance of the service.

[0023] According to the fourth scheme, compared to the case where no control is provided with the following information, the time until the service is executed can be reduced, which indicates the situation that urges the physical operation for the execution of the service.

[0024] According to the fifth scheme, compared to the case where the operation is not performed in a device that has just been prompted with the information and the service is not executed, the time until the service is executed can be further reduced.

[0025] According to the sixth scheme, the device is able to determine the execution of the service based on the speaker's expectations.

[0026] According to the seventh scheme, the device enables the speaker to confirm the execution of the service in advance.

[0027] According to the eighth scheme, it is possible to accept changes made by the speaker to the device.

[0028] According to the ninth scheme, when the noise volume is greater than a predetermined level, it is possible to avoid performing useless services due to misidentification of the sound.

[0029] According to the 10th solution, when multiple devices are detected performing the same service, it is possible to avoid performing useless services.

[0030] According to the 11th solution, it is possible to prevent the following situation: in the presence of multiple image forming devices, a device that does not expect the speaker who made the sound to perform the service corresponding to the instruction based on the sound. Attached Figure Description

[0031] Figure 1 This is a block diagram illustrating an example of the structure of an image forming system according to various embodiments.

[0032] Figure 2 This is a block diagram illustrating an example of the hardware structure of a server according to various embodiments.

[0033] Figure 3 This is a block diagram illustrating an example of the functional structure of a server according to various embodiments.

[0034] Figure 4 This is a schematic diagram illustrating an example of the structure of the sound information database in each embodiment.

[0035] Figure 5 This is a schematic diagram illustrating an example of the structure of the physical operation information database in each implementation.

[0036] Figure 6 This is a schematic diagram illustrating an example of the structure of the service information database in each implementation.

[0037] Figure 7 This is a flowchart illustrating an example of the information processing flow of the first embodiment.

[0038] Figure 8 This is a front view showing an example of a physical operation prompt screen for each implementation.

[0039] Figure 9 This is a flowchart illustrating an example of the process for determining the object device according to the first embodiment.

[0040] Figure 10 This is a front view showing an example of the prompt screen of the detection device in each embodiment.

[0041] Figure 11 This is a front view showing an example of a confirmation information prompt screen for each implementation.

[0042] Figure 12 This is a flowchart illustrating an example of the information processing flow of the second embodiment.

[0043] Figure 13This is a flowchart illustrating an example of the process for determining the object device according to the second embodiment. Detailed Implementation

[0044] Hereinafter, embodiments for implementing the present invention will be described in detail with reference to the accompanying drawings. Furthermore, in this embodiment, the case of applying the present invention to an image forming system where services are performed by multiple image forming apparatuses will be described. Additionally, in this embodiment, the case of applying the information processing apparatus of the present invention to a server will be described.

[0045] [First Implementation]

[0046] First, refer to Figure 1 The structure of the image forming system 1 of this embodiment will be explained.

[0047] like Figure 1 As shown, the image forming system 1 of this embodiment includes a server 10 that plays a core role in the system, and multiple image forming apparatuses 90A, 90B, ... Hereinafter, without distinguishing between the image forming apparatuses 90A, 90B, ..., they will be collectively referred to as "image forming apparatus 90".

[0048] Server 10 is connected to multiple image forming apparatuses 90 via network N, and server 10 can communicate with each image forming apparatus 90 via network N. In this embodiment, network N uses enterprise communication lines such as LAN (Local Area Network) and WAN (Wide Area Network), but it is not limited to this method. For example, network N can use public communication lines such as the Internet and telephone lines, or a combination of these enterprise communication lines and public communication lines. Furthermore, in this embodiment, wired communication lines are used as network N, but it is not limited to this method; wireless communication lines can be used, or a combination of wired and wireless communication lines can be used.

[0049] In the image forming system 1 of this embodiment, the voice input / output device 80 is connected to a portion of the multiple image forming devices 90 (image forming device 90A in this embodiment). The voice input / output device 80 of this embodiment has the function of inputting the user's voice as voice data and the function of outputting voice based on predefined voice data.

[0050] Furthermore, in the image forming system 1 of this embodiment, the server 10 indirectly controls the voice input / output device 80 via the image forming apparatus 90A, but this is not a limitation. For example, it can be configured such that the voice input / output device 80 is connected to the server 10 without going through the image forming apparatus 90A, and the server 10 directly controls the voice input / output device 80. In addition, in this embodiment, a smart speaker is used as the voice input / output device 80, but this is not a limitation. For example, other devices capable of voice input and output, such as smartphones and portable information terminal devices, can also be used as the voice input / output device 80. Furthermore, in this embodiment, a digital multifunction printer with image printing, image reading, and image sending functions is used as the image forming apparatus 90, but this is not a limitation. Furthermore, in this embodiment, the image forming apparatus 90A and the voice input / output device 80 are described as separate components, but this is not a limitation; the voice input / output device 80 can also be integrally assembled with the image forming apparatus 90A.

[0051] Next, refer to Figure 2 and Figure 3 The structure of the server 10 in this embodiment will be described. Furthermore, examples of the server 10 include information processing devices such as personal computers and server computers.

[0052] like Figure 2 As shown, the server 10 of this embodiment includes a CPU (Central Processing Unit) 11 as a processor, a memory 12 as a temporary storage area, a non-volatile storage unit 13, an input unit 14 such as a keyboard and mouse, a display unit 15 such as a liquid crystal display, a media read / write device (R / W) 16, and a communication interface (I / F) unit 18. The CPU 11, memory 12, storage unit 13, input unit 14, display unit 15, media read / write device 16, and communication I / F unit 18 are interconnected via bus B. The media read / write device 16 reads information written to the recording medium 17 and writes information to the recording medium 17.

[0053] Storage unit 13 is implemented using HDD (Hard Disk Drive), SSD (Solid State Drive), flash memory, etc. Information processing program 13A is stored in storage unit 13, which serves as the storage medium. By placing a recording medium 17 on which the information processing program 13A is written, and by reading the information processing program 13A from the recording medium 17 by the media read / write device 16, the information processing program 13A is stored in storage unit 13. CPU 11 reads the information processing program 13A from storage unit 13 and loads it into memory 12, then executes the processing described in information processing program 13A sequentially.

[0054] In addition, the storage unit 13 stores a sound information database 13B, a physical operation information database 13C, and a service information database 13D. The sound information database 13B, the physical operation information database 13C, and the service information database 13D will then be described in detail.

[0055] Next, refer to Figure 3 This will illustrate the functional structure of the server 10 in this embodiment. For example... Figure 3 As shown, server 10 includes a receiving unit 11A, a detection unit 11B, and a control unit 11C. The CPU 11 of server 10 functions as the receiving unit 11A, the detection unit 11B, and the control unit 11C by executing information processing program 13A.

[0056] In this embodiment, the receiving unit 11A receives instructions to execute services using the image forming apparatus 90 by speaking through the voice input / output device 80. Furthermore, although, as described above, the receiving unit 11A obtains the service execution instructions from the image forming apparatus 90A via the communication I / F unit 18, this is not a limitation. Moreover, although an image reading service is used as the service in this embodiment, it is not limited to this method; for example, image printing services and image sending services could also be used. Furthermore, in this embodiment, the service includes both free and paid services.

[0057] Furthermore, the detection unit 11B in this embodiment detects physical operations performed by the image forming apparatus 90. In this embodiment, the physical operation described above includes placing the original document into the ADF (Auto Document Feeder) of the image forming apparatus 90, but it is not limited to this. For example, the physical operation described above could also include connecting the image forming apparatus 90 to a FAX (Facsimile) line or inserting a USB (Universal Serial Bus) memory into the image forming apparatus 90. That is, the physical operation described above includes carrying an external medium onto the apparatus. Furthermore, the physical operation described above also includes receiving electronic data from external sources, such as receiving electronic data via the FAX line or receiving electronic document data from an external device such as a personal computer. Furthermore, it is also possible to employ these individual operations or combinations of multiple operations. In this embodiment, the detection unit 11B uses a sensor provided in the ADF to detect the physical operation described above, but it is not limited to this method. For example, the physical operation described above can be detected by the detection unit 11B through the Vbus detection circuit in the image forming apparatus 90, or a combination of these individual operations or multiple operations can be used.

[0058] Furthermore, in this embodiment, the control unit 11C controls the image forming apparatus 90 that detects the operation at the moment closest to the moment the instruction was received to perform the operation to perform the service. Additionally, in this embodiment, the control unit 11C controls the image forming apparatus 90 that detects the operation immediately after the moment the instruction was received to perform the operation to perform the service.

[0059] Furthermore, if the detection unit 11B does not detect the operation within a predetermined time period from the moment the receiving unit 11A receives the aforementioned instruction, the control unit 11C of this embodiment controls the execution of a prompt message, which is a message urging the performance of the aforementioned operation. Specifically, in the control unit 11C of this embodiment, the prompt is provided via voice output through the voice output function of the voice input / output device 80. Additionally, in the control unit 11C of this embodiment, the prompt is also provided via display on the display unit of the image forming apparatus 90A.

[0060] Furthermore, in this embodiment, the control unit 11C controls the image forming apparatus 90 to perform the service immediately after it has triggered a prompt indicating an urging to perform the operation. However, this method is not limited to this. For example, it may be configured such that if the operation is not detected within a predetermined time from the moment the instruction is received, the control unit 11C controls the image forming apparatus 90 to perform the service in a manner pre-registered by the user or other user of the server 10.

[0061] Furthermore, if multiple image forming apparatuses 90 that detect the above-mentioned operation are present within a predetermined time immediately following the time when the receiving unit 11A of this embodiment receives the above-mentioned instruction, the receiving unit 11A further receives the control regarding which image forming apparatus 90 performs the above-mentioned service, and the control unit 11C performs the control of performing the above-mentioned service in the image forming apparatus 90 that received the instruction.

[0062] Furthermore, the control unit 11C in this embodiment also performs the following control: prompting the image forming apparatus 90 that can determine the information required to perform the aforementioned service. Additionally, in this embodiment, the control unit 11C prompts the aforementioned information when the volume of noise generated during the aforementioned sounding to instruct the implementation of the service received by the reception unit 11A is greater than a predetermined level, and when multiple image forming apparatuses 90 are detected performing the aforementioned operation within a predetermined time closest to the time of the reception unit 11A's instruction. However, this method is not limited to this. For example, the control unit 11C may also prompt the aforementioned information each time the service is executed.

[0063] Furthermore, in this embodiment, the aforementioned predetermined level is set to a situation where it is difficult to identify a sound occurring simultaneously with the noise when the volume of the noise is above this level. This is obtained through prior experiments using actual equipment or computer simulations, but it is not limited to this. For example, it is also possible to pre-set the aforementioned predetermined level by the user of server 10 or others based on the sound recognition accuracy required for information processing described later or the purpose of server 10.

[0064] In addition, the receiving unit 11A of this embodiment also receives the sound corresponding to the prompt of the above-mentioned confirmation information, and the control unit 11C of this embodiment performs control corresponding to the sound received by the receiving unit 11A.

[0065] Next, the sound information database 13B of this embodiment will be described. For example... Figure 4As shown, the voice information database 13B of this embodiment stores information such as service, date and time in association.

[0066] The aforementioned services refer to information about services received by the Receiving Department 11A, and the aforementioned dates and times refer to the dates and times on which instructions to execute the corresponding services were received. Figure 4 The example shown illustrates a case where an instruction to perform an image reading service was received at 15:10 on November 16, 2019.

[0067] Next, the physical operation information database 13C of this embodiment will be described. For example... Figure 5 As shown, the physical operation information database 13C of this embodiment stores information such as device ID (Identification), physical operation, date and time by establishing associations.

[0068] The aforementioned device ID represents information about the image forming apparatus that detects a physical operation by the detection unit 11B; the aforementioned physical operation represents information about a physical operation detected by the detection unit 11B; and the aforementioned date and time represent information about the date and time when the corresponding physical operation was detected. Figure 5 In the example shown, for instance, in an image forming apparatus 90 labeled "B" as the device ID, a USB memory was inserted at exactly 12:00 on November 16, 2019.

[0069] Next, the service information database 13D of this embodiment will be described. For example... Figure 6 As shown, the service information database 13D of this embodiment stores information related to services and physical operations.

[0070] The aforementioned services refer to information about services provided by the image forming apparatus 90, and the aforementioned physical operations refer to information about physical operations performed to execute the corresponding services. Figure 6 The example shown illustrates, for instance, the physical operation for performing the image reading service is placing the original document in the ADF.

[0071] Next, refer to Figures 7-11 The function of the server 10 in this embodiment will be explained below. When a user inputs an instruction to start executing the information processing program 13A via the input unit 14, the CPU 11 of the server 10 executes the information processing program 13A. Figure 7 The information processing is shown.

[0072] In step 200, the CPU 11 remains in standby mode until the user speaks to the voice input / output device 80 to instruct the execution of a service, and the corresponding voice data is input from the voice input / output device 80 via the image forming apparatus 90A. In step 202, the CPU 11 performs existing speech recognition processing on the input voice data to determine the service to be executed as instructed by the user. Hereinafter, the determined service is referred to as the "specified service". In step 204, the CPU 11 registers the specified service and the date and time of receiving the instruction to execute the specified service (hereinafter referred to as the "acceptance date and time") in the voice information database 13B.

[0073] In step 206, the CPU 11 reads the physical operation for performing the specified service from the service information database 13D, determines whether the read operation has been detected, and if the determination is positive, proceeds to step 218 (described later); if the determination is negative, proceeds to step 208. In step 208, the CPU 11 determines whether a predetermined time Ta has elapsed since the acceptance date and time. If the determination is negative, it returns to step 206; otherwise, if the determination is positive, it proceeds to step 210. Furthermore, in this embodiment, a time preset by users of the server 10 is used as the predetermined time Ta. However, it is also possible to set an automatically predetermined time as the predetermined time Ta based on the frequency of use of the image forming apparatus 90.

[0074] In step 210, in order to perform a specified service, the CPU 11 controls the display of a physical operation prompt screen on the display unit of the image forming apparatus 90A, which shows a prompt to perform a physical operation on a certain image forming apparatus 90.

[0075] like Figure 8 As shown, the physical operation prompt screen in this embodiment displays information indicating a prompt to perform a physical operation for a specified service on a certain image forming apparatus 90. Therefore, a user who verbally instructs the execution of a specified service can understand the physical operation for the specified service on the image forming apparatus 90 by referring to the physical operation prompt screen. For example, if the user instructs that the service to be performed is an image reading service, the physical operation prompt screen displays information indicating a prompt to place the original document on the ADF of a certain image forming apparatus 90.

[0076] In step 212, the CPU 11 controls the reproduction (output) of voice data via the voice input / output device 80. This voice data is data reproduced by voice (hereinafter referred to as "physical operation prompting voice") of the same content displayed on the physical operation prompting screen. Therefore, the user can also understand the physical operation performed on the image forming apparatus 90 to perform the service by listening to the voice reproduced by the voice input / output device 80.

[0077] In step 214, the CPU 11 determines whether a physical operation for performing the specified service has been detected. If the determination is positive, the process proceeds to step 218 (described later); if the determination is negative, the process proceeds to step 216. In step 216, the CPU 11 determines whether a predetermined time Tb has elapsed since the processing in step 212 began. If the determination is negative, the process returns to step 214; if the determination is positive, the process proceeds to step 218. Furthermore, although in this embodiment, the predetermined time Tb is a time preset by the user of the server 10, etc., it is also possible to use an automatically predetermined time based on the frequency of use of the image forming apparatus 90 as the predetermined time Tb.

[0078] In step 218, CPU 11 determines the physical operation detected by the processing in step 206 or step 214. Hereinafter, the determined physical operation will be referred to as a "prescribed operation". In step 220, CPU 11 registers the prescribed operation and the date and time of detection of the prescribed operation in the physical operation information database 13C.

[0079] In step 222, the CPU 11 executes an object device determination process, which is a process for determining an image forming apparatus (hereinafter referred to as "object device") that is controlled to perform the specified service determined by the process in step 202. Hereinafter, refer to... Figure 9 The process for determining the target device in this embodiment will be explained.

[0080] In step 400, the CPU 11 reads all information from the sound information database 13B and the physical operation information database 13C, and uses this information to determine whether there are multiple image forming apparatuses 90 that are detected performing the specified operation within a predetermined time T1 after the aforementioned acceptance date and time corresponding to the specified service. If the determination is negative, the process proceeds to step 402, where the CPU 11 designates the image forming apparatus 90 that was detected performing the specified operation at the time closest to the acceptance date and time after the acceptance date and time as the target apparatus, and ends the target apparatus determination process.

[0081] On the other hand, if the determination is affirmative in step 400, the process proceeds to step 404, where the CPU 11 controls the display of a detection device prompt screen on the display unit of the image forming apparatus 90A. This detection device prompt screen indicates the presence of multiple image forming apparatuses 90 determined in step 400. Furthermore, in this embodiment, a time preset by the user of the server 10, etc., is used as the predetermined time T1. However, it is also possible to use an automatically predetermined time based on the frequency of use of the image forming apparatuses 90 as the predetermined time T1.

[0082] like Figure 10 As shown, the detection device prompt screen in this embodiment displays information indicating the presence of multiple image forming apparatuses 90 determined to exist during the processing in step 400, and information urging the selection of a target device from among the multiple image forming apparatuses 90. Therefore, a user who performs a specified service by issuing an audio instruction can identify the image forming apparatus 90 that has been detected for the specified operation by referring to the detection device prompt screen.

[0083] In step 406, the CPU 11 controls the voice input / output device 80 to reproduce (output) voice data, which is the same content displayed on the detection device prompt screen, reproduced via voice (hereinafter referred to as "detection device prompt voice"). Therefore, the user can also understand the image forming apparatus 90 that has been detected to perform the prescribed operation by listening to the voice reproduced by the voice input / output device 80, thus indicating the image forming apparatus 90 that performs the instructed service. Therefore, in step 408, the CPU 11 remains in standby mode until the voice input / output device 80 emits a sound, which is a sound indicating the image forming apparatus 90 prompted by the processing in steps 404 and 406.

[0084] In step 410, the CPU 11 uses existing known speech recognition technology to determine the content of the speech received through the processing in step 408, designates the determined image forming apparatus 90 as the object apparatus, and ends the object apparatus determination process. When the object apparatus determination process ends, it transfers to information processing (see reference). Figure 7 Step 222.

[0085] Thus, in this embodiment, the following approach is applied: when multiple image forming apparatuses 90 that are detected performing the prescribed operation within a predetermined time from the acceptance date and time exist, the CPU 11 further controls which image forming apparatus 90 performs the aforementioned service and accepts the application, designating the accepted image forming apparatus 90 as the target apparatus. However, this approach is not limited to this method. For example, the following approach can also be used: the CPU 11 designates the image forming apparatus 90 whose prescribed operation is detected at the closest time immediately following the acceptance date and time as the target apparatus. As an example of this case, the following approach is illustrated: in Figure 9 In the object device determination process shown, CPU 11 does not execute the processes of steps 400 to 410, but only executes the process of step 402.

[0086] In step 224, the CPU 11 determines the volume of noise generated when the user speaks to instruct the execution of a specified service based on the voice data input during the processing of step 200, and determines whether the volume of the noise is above a predetermined threshold Tn. If the determination is affirmative, the process proceeds to step 228 described later; otherwise, if the determination is negative, the process proceeds to step 226.

[0087] In step 226, the CPU 11 determines whether there are multiple image forming apparatuses 90 that have been detected performing the prescribed operation within a predetermined time Tc closest to the aforementioned acceptance date and time. If the determination is negative, the process proceeds to step 238 (described later); otherwise, if the determination is positive, the process proceeds to step 228. Furthermore, in this embodiment, the predetermined time Tc is a time preset by the user of the server 10, etc. However, it is also possible to set an automatically predetermined time Tc based on the frequency of use of the image forming apparatus 90.

[0088] In step 228, the CPU 11 controls the display of a confirmation information prompt screen on the display unit of the image forming apparatus 90A. This confirmation information prompt screen displays confirmation information indicating that the target device can be determined. Alternatively, the target device can be made visible to the user by pre-installing a light-emitting part in the image forming apparatus 90 at a position visible from the surroundings and by controlling the CPU 11 to illuminate the light-emitting part of the target device.

[0089] like Figure 11 As shown, the confirmation information prompt screen in this embodiment displays the specified service and confirmation information. Therefore, users who instruct others to perform services by sound can understand the target device by referring to the confirmation information prompt screen.

[0090] In step 230, the CPU 11 causes the voice input / output device 80 to reproduce (output) voice data and proceeds to step 232. This voice data is data reproduced by voice (hereinafter referred to as "confirmation information prompt voice") of the same content displayed on the confirmation information prompt screen. Therefore, the user can also understand the target device by listening to the voice reproduced by the voice input / output device 80, and thus, make a sound to indicate whether control to enable the target device to perform the aforementioned service is possible. In addition, if control to enable the target device to perform the aforementioned service is not allowed, the user needs to make a sound to reset the device ID of the image forming apparatus 90 performing the service. Therefore, in step 232, the CPU 11 stands by until the voice input / output device 80 makes a sound that is a sound for the service and target device prompted by the processing in steps 228 and 230.

[0091] In step 234, the CPU 11 uses existing known speech recognition technology to determine the content of the speech received through the processing in step 232, and determines whether control can be performed by the target device prompted by the processing in steps 228 and 230 to execute the prescribed service. If the determination is negative, the process proceeds to step 236, where the CPU 11 sets the image forming apparatus 90, which was reset by the user's speech as determined by the processing in step 232, as the target device. Furthermore, when the CPU 11 repeatedly executes the processing in steps 228 to 234, in the processing of steps 228 and 230, the CPU 11 performs the following control: prompts the image forming apparatus 90, which was set as the target device through the processing in step 236, as confirmation information.

[0092] For example, although in the first steps 228 and 230, the CPU 11 provides control indicating that the image forming apparatus 90, marked with "B" as its device ID, is the determining information, in the first step 232, when the CPU 11 receives a call to have the image forming apparatus 90, marked with "C" as its device ID, perform the prescribed service, in the first step 236, the CPU 11 determines the target device as the image forming apparatus 90, marked with "C" as its device ID, and returns to step 228. In the second steps 228 and 230, the CPU 11 provides control indicating that the image forming apparatus 90, marked with "C" as its device ID, is the determining information.

[0093] On the other hand, if the determination is affirmative in step 234, the process proceeds to step 238, where the CPU 11 controls the image forming apparatus 90 to perform the prescribed service and ends the information processing. The image forming apparatus 90 is the image forming apparatus controlled by the CPU 11 and prompted with the confirmation information through the processing of steps 228 and 230.

[0094] [Second Implementation]

[0095] In the first embodiment described above, the method of controlling the execution of the accepted service in the image forming apparatus 90 when the physical operation for executing the service is detected immediately after the time the instruction to execute the service is received is explained. In contrast, in this second embodiment, an embodiment is described where control is performed in the image forming apparatus 90 when the operation is detected immediately before the time the instruction is received. Furthermore, regarding the structure of the image forming system 1 and server 10 in this second embodiment, since it differs from the structure of the first embodiment described above (see...),... Figure 1 , Figure 2 and Figure 3 The same applies, so the explanation here is omitted.

[0096] First, refer to Figure 12 The function of server 10 in this second embodiment will be explained, specifically its role in performing information processing. Furthermore, the function of server 10 during execution will be further explained. Figure 12 The information processing shown, and Figure 7 The information processing steps shown are the same as those in the labeling. Figure 7 The same step numbers are used, and their descriptions are omitted.

[0097] exist Figure 12 The information processing and Figure 7 The difference in the information processing shown is that the processing of steps 201 to 215 is applied instead of the processing of steps 200 to 222.

[0098] That is, in Figure 12 In step 201, the CPU 11 determines whether a physical operation for performing a service on the image forming apparatus 90 has been detected. If the determination is positive, the process proceeds to step 205 (described later); otherwise, if the determination is negative, the process proceeds to step 203. Furthermore, in step 201, the CPU 11 considers all physical operations stored in the service information database 13D as detection targets.

[0099] In step 203, the CPU 11 determines whether a predetermined time Td has elapsed since the execution of the information processing program 13A. If the determination is negative, it returns to step 201; otherwise, if the determination is positive, it proceeds to step 205. Furthermore, in this embodiment, the predetermined time Td is a time preset by the user of the server 10, but it is also possible to use an automatically predetermined time Td based on the frequency of use of the image forming apparatus 90.

[0100] In step 205, CPU 11 determines all the physical operations detected through the processing in step 201. Hereinafter, the determined physical operations will be referred to as "prescribed operations". In step 207, CPU 11 registers the prescribed operations and the date and time of detection of the prescribed operations in the physical operation information database 13C.

[0101] In step 209, the CPU 11 reads the service corresponding to the specified operation from the service information database 13D and remains in standby mode until the user speaks to instruct the execution of one of the read services. Voice data corresponding to the spoken instruction is input from the voice input / output device 80 via the image forming device 90A. In step 211, the CPU 11 determines the service instructed by the user by performing known speech recognition processing on the input voice data. Hereinafter, the determined service is referred to as the "specified service". Furthermore, if the CPU 11 determines multiple specified operations in step 205, in step 211, the CPU 11 determines one service as the specified service. In step 213, the CPU 11 registers the specified service and the date and time of receiving the instruction to execute the specified service (hereinafter referred to as the "acceptance date and time") in the voice information database 13B.

[0102] In the next step 215, the CPU 11 executes an object device determination process, which is a process for determining the image forming apparatus (hereinafter referred to as the "object device") to perform the specified service determined by the process in step 201. Hereinafter, refer to... Figure 13 The object device determination process of this embodiment will be described. Furthermore, the execution... Figure 13 The object device shown determines the process and Figure 9 The object device shown determines the same processing steps as labeled. Figure 9 The same step numbers are used, and their descriptions are omitted.

[0103] exist Figure 13 The information processing and Figure 9The difference in the information processing shown is that steps 401 and 403 are applied instead of steps 400 and 402.

[0104] That is, in Figure 13 In step 401, the CPU 11 reads all information from the sound information database 13B and the physical operation information database 13C, and uses this information to determine whether there are multiple image forming apparatuses 90 that are detected performing the specified operation within a predetermined time T2 immediately preceding the aforementioned acceptance date and time corresponding to the specified service. If the determination is negative, the process proceeds to step 403, where the CPU 11 designates the image forming apparatus 90 that was detected performing the specified operation at the closest time immediately preceding the acceptance date and time as the target apparatus, and ends the target apparatus determination process. Furthermore, in this embodiment, the predetermined time T2 is a time preset by the user of the server 10, but it is also possible to use an automatically predetermined time set based on the frequency of use of the image forming apparatus 90 as the predetermined time T2.

[0105] Furthermore, in the first embodiment described above, a method for controlling the execution of the accepted service was explained in which the image forming apparatus 90, whose physical operation for executing the service is detected immediately after the moment the instruction to execute the service is received. In addition, in the second embodiment described above, an embodiment was described in which the control is performed in which the image forming apparatus 90, whose operation is detected immediately before the moment the acceptance was performed; however, this method is not limited to this. These methods can also be combined to apply the following method: regardless of whether it is immediately before or immediately after the acceptance, the control is performed in the image forming apparatus 90 whose operation is detected at the moment closest to the moment the acceptance was performed.

[0106] The embodiments have been described above, but the technical scope of the present invention is not limited to the scope described in the above embodiments. Various changes or modifications can be made to the above embodiments without departing from the spirit of the disclosure, and the manner in which such changes or modifications are made is also included in the technical scope of the present invention.

[0107] Furthermore, the above embodiments are not limited to the disclosures covered by the claims. Additionally, not all combinations of features described in the embodiments are necessary for the disclosed solutions. The above embodiments include disclosures at various stages, and various disclosures can be extracted through combinations of multiple disclosed technical features. Even if several technical features are deleted from all the technical features shown in the embodiments, as long as the desired effect is achieved, the structure after deleting those structural elements can be extracted as an invention.

[0108] Furthermore, in the above embodiments, processor refers to processor in a broad sense, including general-purpose processors (e.g., CPU, etc.) and special-purpose processors (e.g., GPU: Graphics Processing Unit, ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, programmable logic device, etc.).

[0109] Furthermore, in the above embodiments, an image forming apparatus is used as an example of an apparatus for performing the service, but it is not limited to this. It may include stoves, washing machines, microwave ovens, etc., or it may be used in combination of these alone or in combination.

[0110] Furthermore, in the above embodiment, the case where the information processing program 13A is pre-installed in the storage unit 13 has been described; however, it is not limited to this. For example, the information processing program may be provided by storing it in a storage medium such as a CD-ROM (Compact Disc Read Only Memory) or by providing it via a network.

[0111] Furthermore, the above embodiments describe the use of a computer to perform information processing via a software structure by executing a program; however, the present invention is not limited thereto. For example, information processing can also be achieved using a hardware structure or a combination of hardware and software structures.

[0112] Furthermore, the structure of server 10 described in the above embodiments (refer to...) Figures 1-6 This is just one example; of course, unnecessary parts can be deleted or new parts can be added without departing from the spirit of the invention.

[0113] Furthermore, the information processing flow described in the above embodiments (refer to...) Figure 7 and Figure 12 ) and setting up a defined processing flow (refer to Figure 9 and Figure 13 This is just one example; of course, unnecessary steps can be deleted, new steps can be added, or the processing order can be changed without departing from the spirit of this invention.

Claims

1. An information processing apparatus, wherein the information processing apparatus has a processor, the processor accepts an instruction to cause a service to be executed by making a sound, the processor detects whether a physical operation for causing the service to be executed has been performed on a plurality of apparatuses, the processor controls the service to be executed in the apparatus that detected the physical operation at a time closest to a time when the instruction was accepted among the plurality of apparatuses.

2. The information processing apparatus according to claim 1, wherein the processor controls the service to be executed in the apparatus that detected the physical operation immediately before the time when the instruction was accepted.

3. The information processing apparatus according to claim 1, wherein the processor controls the service to be executed in the apparatus that detected the physical operation immediately after the time when the instruction was accepted.

4. The information processing apparatus according to claim 3, wherein in a case where no physical operation for causing the service to be executed has been detected on the apparatus within a prescribed time from the time when the instruction was accepted, the processor controls to prompt information indicating that the physical operation is urged to be performed.

5. The information processing apparatus according to claim 4, wherein the processor controls the service to be executed in the apparatus that detected the physical operation immediately after the control to prompt the information was performed.

6. The information processing apparatus according to claim 1, wherein in a case where there are a plurality of apparatuses that detected the physical operation within a prescribed time closest to the time when the instruction was accepted, the processor further accepts an instruction to control the service to be executed in which apparatus, the processor controls the service to be executed in the accepted apparatus.

7. The information processing apparatus according to claim 1, wherein the processor further controls to prompt determination information that can be determined by the apparatus that controls the service to be executed.

8. The information processing apparatus according to claim 7, wherein the processor further accepts a sound corresponding to the prompting of the determination information, the processor controls corresponding to the accepted sound.

9. The information processing apparatus according to claim 7 or 8, wherein in a case where a volume of noise generated in parallel with the sound is greater than a prescribed level, the sound being a sound indicating the service to be executed, the processor controls to prompt the determination information.

10. The information processing apparatus according to claim 7 or 8, wherein in a case where there are a plurality of apparatuses that detected the operation within a prescribed time closest to the time when the instruction was accepted, the processor controls to prompt the determination information.

11. The information processing apparatus according to any one of claims 1 to 7, wherein the apparatus is an image forming apparatus.

12. A computer-readable medium storing a program that causes a computer to execute a process in which, the computer accepts an instruction to cause a service to be executed by making a sound, the computer detects whether a physical operation for causing the service to be executed has been performed on a plurality of apparatuses, the computer controls the service to be executed in the apparatus that detected the physical operation at a time closest to a time when the instruction was accepted among the plurality of apparatuses. receiving an instruction to execute a service by voice, detecting whether a physical operation for causing the service to execute has been performed on a plurality of devices, controlling execution of the service received in a device of the plurality of devices in which the physical operation was detected at a time closest to a time at which the reception was performed.

13. An information processing method, wherein receiving an instruction to execute a service by voice, detecting whether a physical operation for causing the service to execute has been performed on a plurality of devices, controlling execution of the service received in a device of the plurality of devices in which the physical operation was detected at a time closest to a time at which the reception was performed.

14. A computer program product including a program that causes a computer to execute processing, receiving an instruction to execute a service by voice, detecting whether a physical operation for causing the service to execute has been performed on a plurality of devices, controlling execution of the service received in a device of the plurality of devices in which the physical operation was detected at a time closest to a time at which the reception was performed.

Images