Information processing system and information processing method
The system addresses the challenge of identifying multiple image forming apparatuses by associating captured images with address information, enabling easy and accurate device selection and operation through image processing and registration.
Patent Information
- Application Number
- JP2023085343
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-12-05
AI Technical Summary
Users find it difficult to identify the correspondence between image forming apparatuses and their drivers or functional information when multiple devices are present in the same space, making it challenging to select the correct device for operations.
An information processing system and method that utilizes image acquisition, identification, and registration processes to associate captured images of the device layout with address information, enabling easy identification and operation of image forming apparatuses using a user terminal.
Facilitates easy identification and operation of image forming apparatuses by superimposing identification information on captured images, allowing users to intuitively select and instruct devices based on their functions and status, reducing errors in device selection.
Smart Images

Figure 2024168558000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an information processing system and an information processing method. [Background technology]
[0002] In general, image forming devices such as network printers and multifunction devices communicate data with an operation terminal (user terminal) to execute image processing such as printing. In this case, a driver program corresponding to the model of the image forming device is installed in advance in a storage unit of the operation terminal (see, for example, Patent Document 1). [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2018-112773 A Summary of the Invention [Problem to be solved by the invention]
[0004] When a user wants an image forming device to execute a job, the user selects a driver for the image forming device on the operation screen of the operation terminal and issues an execution instruction (e.g., a print instruction). Here, when multiple image forming devices exist in the same space, it becomes difficult for the user to grasp the correspondence between the image forming device and the driver, and the correspondence between the image forming device and the function information. Thus, with conventional technology, it is difficult for the user to identify the image forming device on the operation terminal.
[0005] An object of the present invention is to provide an information processing system and an information processing method that allow a user to easily identify an image forming apparatus on an operation terminal. [Means for solving the problem]
[0006] An information processing system according to one aspect of the present invention includes an image acquisition processing unit, an identification processing unit, an address acquisition processing unit, and a registration processing unit. The image acquisition processing unit acquires a captured image of a predetermined space in which an image forming device is placed. The identification processing unit identifies the image forming device from the captured image acquired by the image acquisition processing unit. The address acquisition processing unit acquires address information of the image forming device identified by the identification processing unit. The registration processing unit associates the address information acquired by the address acquisition processing unit with the image forming device in the captured image and registers it.
[0007] An information processing method according to another aspect of the present invention includes an image acquisition step, an identification step, an address acquisition step, and a registration step. In the image acquisition step, a captured image of a predetermined space in which an image forming device is placed is acquired. In the identification step, the image forming device is identified from the captured image acquired in the image acquisition step. In the address acquisition step, address information of the image forming device identified in the identification step is acquired. In the registration step, the address information acquired in the address acquisition step is associated with the image forming device in the captured image and registered. Effect of the Invention
[0008] According to the present invention, it is possible to provide an information processing system and an information processing method that allow a user to easily identify an image forming apparatus on an operation terminal. [Brief description of the drawings]
[0009] [Figure 1] FIG. 1 is a schematic diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 2] FIG. 2 is a block diagram showing a configuration of an information processing system according to an embodiment of the present invention. [Diagram 3] FIG. 3 is a diagram showing an example of device information used in the server device according to the embodiment of the present invention. [Figure 4]FIG. 4 is a diagram showing an example of a registration screen according to an embodiment of the present invention. [Diagram 5] FIG. 5 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 6] FIG. 6 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 7] FIG. 7 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 8] FIG. 8 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 9] FIG. 9 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 10] FIG. 10 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 11] FIG. 11 is a diagram showing an example of a setting screen according to an embodiment of the present invention. [Figure 12] FIG. 12 is a flowchart showing an example of a procedure of a registration process executed by the server device according to the embodiment of the present invention. [Figure 13] FIG. 13 is a flowchart showing an example of a procedure of a job execution process executed by the server device according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings to facilitate understanding of the present invention. Note that the following embodiment is an example of the present invention and does not limit the technical scope of the present invention.
[0011] [Information Processing System 10] As shown in FIG. 1, an information processing system 10 according to an embodiment of the present invention includes a server device 1, an operation terminal 2, and an image forming device 3. The information processing system 10 is an example of the information processing system of the present invention. The server device 1, the operation terminal 2, and the image forming device 3 can communicate with each other via a network N1 such as a telephone line, the Internet, or a LAN. The image forming device 3 may be one or more. In this embodiment, an example will be described in which there are multiple image forming devices 3. The operation terminal 2 is, for example, a portable information processing device such as a smartphone or a mobile terminal. The operation terminal 2 is an example of a user terminal.
[0012] In the information processing system 10, for example, when a user photographs a space (room) in which the image forming device 3 is placed using the camera 25 (see FIG. 2) of the operation terminal 2, the server device 1 executes a process (registration process) of recognizing and registering the image forming device 3 based on the captured image, and a process (job execution process) of executing a job (e.g., a print job) based on user operations on the captured image.
[0013] However, when multiple image forming devices exist in the same space, it is difficult for the user to grasp the correspondence between the image forming devices and the drivers (printer drivers) and the correspondence between the image forming devices and the functional information. Thus, with the conventional technology, it is difficult for the user to identify the image forming device on the operation terminal. In contrast, with the information processing system 10 according to this embodiment, the user can easily identify the image forming device 3 on the operation terminal 2 as shown below.
[0014] [Image forming device 3] The image forming device 3 includes an operation display unit, an image forming unit, a communication I / F, a storage unit, and a control unit. Illustration of each component of the image forming device 3 is omitted. The image forming device 3 may be a multifunction machine having a printer function, a scanner function, a copy function, a facsimile function, etc., or may be a printer having only a printer function. Well-known technologies can be applied to the image forming device of the present invention.
[0015] The operation display unit includes a display unit such as a liquid crystal display for displaying information, and an operation unit such as a touch panel and operation buttons for accepting user operations.
[0016] The image forming unit is capable of performing a printing process based on image data by an electrophotographic method or an inkjet method, and forms an image on a sheet based on the image data. For example, when the image forming unit is an electrophotographic image forming unit, the image forming unit includes a photosensitive drum, a charging device, an exposure device, a developing device, a transfer device, and a fixing device.
[0017] The communication I / F is a communication interface capable of performing wired or wireless data communication with an external communication device.
[0018] The storage unit is a non-volatile storage unit such as a hard disk or an EEPROM (registered trademark).
[0019] The control unit includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile or non-volatile storage unit used as a temporary storage memory (work area) for various processes executed by the CPU.
[0020] [Server device 1] 2, the server device 1 includes a control unit 11, a storage unit 12, an operation display unit 13, a communication I / F 14, etc. The server device 1 may be configured as, for example, a cloud server. The server device 1 is not limited to a single computer, and may be a computer system in which multiple computers work in cooperation. Various processes executed by the server device 1 may be distributed and executed by one or multiple processors.
[0021] The operation display unit 13 includes a display unit such as a liquid crystal display for displaying information, and an operation unit such as a touch panel and operation buttons for accepting user operations.
[0022] The communication I / F 14 is a communication interface capable of performing wired or wireless data communication with an external device. For example, the communication I / F 14 performs data communication with the operation terminal 2 and the image forming device 3 via the network N1.
[0023] The storage unit 12 is a non-volatile storage unit such as an HDD (Hard Disk Drive), an SSD (Solid State Drive), or a flash memory that stores various information. The storage unit 12 stores various control programs executed by the control unit 11, various data, and the like. The control programs executed by the control unit 11 include a registration program for causing the control unit 11 to execute a registration process (see FIG. 12) described below, and a job execution program for causing the control unit 11 to execute a job execution process (see FIG. 13). The control programs are non-temporarily recorded on a computer-readable recording medium such as a CD or a DVD, and are read by a reading device (not shown) such as a CD drive or a DVD drive provided in the server device 1 and stored in the storage unit 12.
[0024] Furthermore, data such as device information D1 is stored in the storage unit 12. Fig. 3 is a diagram showing an example of the device information D1.
[0025] As shown in FIG. 3, the device information D1 includes location information, model name, IP address, function information, status information, etc. for each image forming device 3. The location information is information indicating the position (coordinates) of the image forming device 3 in a captured image of the image forming device 3 arranged in a specific space (room) captured by a camera. That is, the location information is the position coordinates of the image forming device 3 when the captured image is expressed as planar coordinates. The model name is the model name of the image forming device 3. The IP address is the IP address of the image forming device 3 (an example of address information of the present invention). The function information is information regarding functions possessed by the image forming device 3. The functions include, for example, a color selection function for selecting a color (color, monochrome, etc.) and a paper selection function for selecting a paper size (A4, A5, etc.). The status information is information indicating the current status of the image forming device 3. The status information is, for example, information indicating whether the image forming device 3 is in a state in which printing can be performed (normal state or abnormal state). In the case of an abnormal state, information on the cause of the abnormality (insufficient paper, insufficient toner, paper jam, etc.) is registered as the state information.
[0026] The location information, the function information, and the status information are automatically registered by the control unit 11. The model name and the IP address are registered by a registration operation performed by the user.
[0027] In another embodiment, a part or all of the device information D1 may be stored in another server accessible from the server device 1 via the network N1. In this case, the control unit 11 of the server device 1 may acquire the information from the other server and execute various processes such as a registration process and a job execution process described below.
[0028] The storage unit 12 also stores layout data and image data for generating various web pages included in an image forming site (image forming application) displayed on the operation terminal 2. In this embodiment, the control unit 11 of the server device 1 can generate the various web pages and transmit information of the web pages to the operation terminal 2, thereby causing the operation terminal 2 to display the various web pages. In another embodiment, the control unit 11 may cause the control unit 21 of the operation terminal 2 to display the various web pages by transmitting data required for displaying the various web pages to the operation terminal 2.
[0029] The control unit 11 includes control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic operations. The ROM is a non-volatile storage unit in which information such as a control program for causing the CPU to execute various processes is stored in advance. The RAM is a volatile or non-volatile storage unit used as a temporary storage memory (work area) for various processes executed by the CPU.
[0030] Specifically, the control unit 11 includes an image acquisition processing unit 111, a specific processing unit 112, an address acquisition processing unit 113, a registration processing unit 114, a display processing unit 115, an execution processing unit 116, etc. The control unit 11 functions as each of these processing units by executing various processes according to the control program. The control unit 11 may also include an electronic circuit that realizes a part or a plurality of processing functions of each of these processing units.
[0031] The image acquisition processing unit 111 acquires a captured image of a predetermined space in which the image forming devices 3 are placed. For example, when a user captures an image of a room in which three image forming devices 3 are placed with the camera 25 (see FIG. 2) of the operation terminal 2, the image acquisition processing unit 111 acquires a captured image P1 (image data) of the room from the operation terminal 2. The image acquisition processing unit 111 may acquire a plurality of captured images P1 of the room from the operation terminal 2. As another embodiment, the image acquisition processing unit 111 may acquire the captured image P1 from a camera (e.g., a network camera) installed in the room.
[0032] The identification processing unit 112 identifies the image forming device 3 from the captured image P1 acquired by the image acquisition processing unit 111. Specifically, the identification processing unit 112 performs image analysis on the captured image P1 to recognize the image forming device 3. For each recognized image forming device 3, the identification processing unit 112 identifies the position (coordinates) in the captured image P1 and registers the position information in the device information D1 (see FIG. 3).
[0033] The address acquisition processing unit 113 acquires address information of the image forming device 3 identified by the identification processing unit 112. Specifically, the display processing unit 115 displays the captured image P1 and an IP address list L1 of pre-registered address information (IP addresses) on a registration screen P10 of the operation display unit 13 of the operation terminal 2 (see FIG. 4). The IP address list L1 includes the model names and IP addresses of multiple image forming devices 3. The control unit 11 may acquire the IP addresses and model names from each image forming device 3 in advance and register them in the IP address list L1. On the registration screen P10, the user selects an image forming device 3 included in the captured image P1, and selects an IP address and model name corresponding to the selected image forming device 3 from the IP address list L1. The address acquisition processing unit 113 acquires the IP address and model name selected by the user from the IP address list L1.
[0034] As another embodiment, the address acquisition processing unit 113 may acquire an IP address input by the user on the registration screen P10. That is, the user may select an IP address from the IP address list L1 on the registration screen P10, or may manually input an IP address. Also, the user may select a model name from the IP address list L1 on the registration screen P10, or may manually input a model name.
[0035] The registration processing unit 114 associates the address information (IP address) acquired by the address acquisition processing unit 113 with the image forming device 3 in the captured image P1 and registers them. Specifically, the registration processing unit 114 registers the image forming device 3 and IP address (see FIG. 4) selected by the user on the operation terminal 2 in the device information D1 (see FIG. 3) in association with the location information of the image forming device 3. FIG. 4 shows an example in which the user selects the image forming device 3 located at the center of the captured image P1 and selects "192.168.001.XXX" as the IP address corresponding to the image forming device 3. The registration processing unit 114 associates the model name ("MFP0001") and IP address ("192.168.001.XXX") with the location information ("A1") of the image forming device 3 selected by the user in the captured image P1, and registers them in the device information D1.
[0036] Similarly, the registration processing unit 114 associates the model name ("MFP0002") and IP address ("192.168.002.YYY") with the location information ("A2") of the image forming device 3 selected by the user in the captured image P1, and registers them in the device information D1. The registration processing unit 114 also associates the model name ("MFP0003") and IP address ("192.168.003.ZZZ") with the location information ("A3") of the image forming device 3 selected by the user in the captured image P1, and registers them in the device information D1.
[0037] Furthermore, the registration processing unit 114 registers in the device information D1 function information related to functions of the image forming device 3. The function information of each image forming device 3 may be stored in advance in the storage unit 12, or may be acquired from an external server via the network N1.
[0038] As described above, the control unit 11 executes a registration process to analyze the captured image P1 to identify the image forming device 3, and to register the address information (IP address) of the identified image forming device 3 in association with the image forming device 3. The control unit 11 executes the registration process every time the captured image P1 is acquired.
[0039] After performing the registration process in advance, the control unit 11 performs a process of executing a job (job execution process) based on a user operation on a captured image (hereinafter referred to as a captured image P2) taken by the user. Note that the captured image P2 may be a moving image or a still image.
[0040] Specifically, the display processing unit 115 displays the captured image P2 on the operation terminal 2 that outputs an image formation instruction to the image forming device 3, and displays identification information capable of identifying the image forming device 3 specified by the identification processing unit 112 in the captured image P2 in association with the image forming device 3. Specifically, when a user photographs a room with the camera 25 of the operation terminal 2, the display processing unit 115 displays a rectangular image F1 surrounding the image forming device 3 superimposed on the captured image P2. For example, the identification processing unit 112 identifies each image forming device 3 based on the scenery of the captured image P1 and the captured image P2 used in the registration process, and the display processing unit 115 displays the rectangular image F1 in association with each identified image forming device 3. The rectangular image F1 is an example of the identification information. As shown in the setting screen P20 of Figure 5, when the identification processing unit 112 identifies three image forming devices 3 in the captured image P2, the display processing unit 115 displays three rectangular images F1 surrounding each image forming device 3 superimposed on each image forming device 3 in the captured image P2.
[0041] As another embodiment, the display processing unit 115 may superimpose an image of a predetermined color, pattern, or contrast on the image forming device 3 in the captured image P2. Furthermore, the display processing unit 115 may change the contrast of a portion (area) of the image forming device 3 in the captured image P2. Furthermore, as another embodiment, the display processing unit 115 may display text information (e.g., model name) in association with the image forming device 3 in the captured image P2.
[0042] According to the above configuration, the user can easily recognize the image forming device 3 included in the captured image P2.
[0043] Furthermore, the display processing unit 115 displays the captured image P2 on the operation terminal 2 that outputs an image formation instruction to the image forming apparatus 3, and also displays, in the captured image P2, function information F2 related to the functions of the image forming apparatus 3 identified by the identification processing unit 112 in association with the image forming apparatus 3. For example, as shown in FIG. 6, the display processing unit 115 displays information (function information F2) on the functions possessed by the image forming apparatus 3, here "color machine" and "screen option available", in association with the image forming apparatus 3 in the captured image P2. The display processing unit 115 displays the function information F2 registered in advance in the device information D1 (see FIG. 3) in association with the image forming apparatus 3.
[0044] In addition, when the user selects a desired image forming device 3 in the captured image P2, the display processing unit 115 may display function information F2 of the selected image forming device 3, or may display function information F2 of all image forming devices 3 included in the captured image P2.
[0045] Furthermore, for example, in a case where the first function information F2 corresponding to the first image forming apparatus 3 is displayed in the captured image P2, when an operation to select the second image forming apparatus 3 is received from the user, the display processing unit 115 may delete the first function information F2 and display the second function information F2 corresponding to the second image forming apparatus 3. In this manner, the display processing unit 115 may update the function information F2 to be displayed in the captured image P2 every time an operation to select the image forming apparatus 3 is received from the user in the captured image P2.
[0046] Furthermore, the display processing unit 115 displays the captured image P2 on the operation terminal 2 that outputs an image formation instruction to the image forming apparatus 3, and also displays, in the captured image P2, status information F3 regarding the status of the image forming apparatus 3 identified by the identification processing unit 112 in association with the image forming apparatus 3. For example, as shown in FIG. 6, when the image forming apparatus 3 is in a "paper shortage" status (abnormal status), the display processing unit 115 displays information on "paper shortage" (status information F3) in association with the image forming apparatus 3 in the captured image P2. The display processing unit 115 acquires the current status (normal status or abnormal status (paper shortage, toner shortage, paper jam, etc.)) from the image forming apparatus 3, registers it in the device information D1 (see FIG. 3), and also displays it in association with the image forming apparatus 3 in the captured image P2. In addition, when the image forming device 3 is in a state (abnormal state) in which it cannot execute an image formation process (e.g., a print job), the display processing unit 115 may be configured to display status information F3 indicating the cause of the inability to execute the image formation process, and not to display function information F2 regarding the functions of the image forming device 3 (see Figure 6).
[0047] Furthermore, when a user selects a desired image forming device 3 in the captured image P2, the display processing unit 115 may display status information F3 of the selected image forming device 3, or may display status information F3 of all image forming devices 3 included in the captured image P2. Furthermore, when an abnormality occurs in an image forming device 3, the display processing unit 115 may display status information F3 (abnormality information) at the time the abnormality occurs.
[0048] Here, the display processing unit 115 can display thumbnail images of print targets for which print processing is executed in the image forming apparatus 3. For example, in Fig. 5 and Fig. 6, the display processing unit 115 displays a thumbnail image list L2 alongside the captured image P2 on the setting screen P20 of the operation terminal 2. The thumbnail image list L2 includes thumbnails of multiple images to be printed that are stored in the storage unit 12 of the operation terminal 2 or the storage unit 12 of the server device 1. The user can select a desired thumbnail image from the thumbnail image list L2.
[0049] The execution processing unit 116 causes the image forming device 3 to execute a job. Specifically, when the execution processing unit 116 receives an operation from the user to move a thumbnail image onto the image forming device 3 in the captured image P2 on the setting screen P20, the execution processing unit 116 causes the image forming device 3 to execute a print process of the thumbnail image. For example, as shown in FIG. 7, when the user selects the thumbnail image of "Image 1" on the setting screen P20 of the operation terminal 2 and moves (drag-and-drop) it to a position in the captured image P2 where it overlaps with the desired image forming device 3, the execution processing unit 116 acquires a print instruction to cause the image forming device 3 to execute a print process of "Image 1". Note that in order to prevent an erroneous operation due to a user's shaking on the operation terminal 2, the execution processing unit 116 may permit the acceptance of the drag-and-drop operation when the captured image P2 is set to a still image.
[0050] When the execution processing unit 116 acquires the print instruction for the image forming device 3, if the function information F2 is registered in the image forming device 3, the execution processing unit 116 accepts a setting operation from the user corresponding to the function information F2. For example, if the image forming device 3 has a color selection function, the execution processing unit 116 displays a color setting screen M1 on the captured image P2 as shown in Fig. 8, and accepts an operation from the user to select "color" or "monochrome". The execution processing unit 116 sets the color selected by the user ("color" or "monochrome").
[0051] Furthermore, for example, if the image forming device 3 has a paper selection function, the execution processing unit 116 displays a paper setting screen M2 on the captured image P2 as shown in Fig. 9, and accepts an operation from the user to select "A4" or "A5". The execution processing unit 116 sets the paper selected by the user ("A4" or "A5").
[0052] If function information F2 is not registered in image forming apparatus 3, or if the acceptance of the setting operation corresponding to function information F2 (see FIGS. 8 and 9) has been completed, execution processing unit 116 displays execution screen M3 shown in FIG. 10. When the user selects "Yes" on execution screen M3, execution processing unit 116 outputs a print instruction for "Image 1" to image forming apparatus 3. Image forming apparatus 3 executes print processing for "Image 1" based on the print instruction acquired from server apparatus 1. For example, image forming apparatus 3 with model name "MFP0001" prints "Image 1" in color on A4 paper. Execution processing unit 116 outputs a print completion notification to operation terminal 2.
[0053] When the print process is completed in the image forming device 3, the execution processing unit 116 displays an inquiry screen M4 on the setting screen P20, inquiring whether or not to continue the print process (see FIG. 11). If the user selects "Yes" on the inquiry screen M4, the control unit 11 displays the thumbnail image list L2 again on the setting screen P20 (see FIGS. 5 and 6), and accepts an image selection operation from the user. If the user selects "Cancel" on the inquiry screen M4, the control unit 11 ends the image forming application.
[0054] Here, when the user performs an operation to move the thumbnail image in the captured image P2 on the setting screen P20 to an image forming device 3 (see FIG. 6) that is in an abnormal state (e.g., "out of paper"), the control unit 11 may be configured not to display the setting screen (e.g., color setting screen M1 (see FIG. 8), paper setting screen M2 (see FIG. 9)) and execution screen M3 (see FIG. 10). In other words, the control unit 11 may be configured not to display the setting screen and execution screen M3 when the image forming device 3 is in a state in which it cannot execute the print process. In this case, the control unit 11 may notify the operation terminal 2 of a message to the effect that it is not possible to execute the print process.
[0055] [Operation terminal 2] 2, the operation terminal 2 includes a control unit 21, a storage unit 22, an operation display unit 23, a communication I / F 24, and a camera 25. The operation terminal 2 is an information processing device having a communication function, such as a smartphone, a mobile terminal, or a notebook computer.
[0056] Camera 25 is a digital camera that captures an image of a space (room) that is a subject and outputs it as digital image data. For example, a user activates camera 25, adjusts its position so that the entire room in which image forming device 3 is placed is included in the shooting range, and shoots the image. The user may change the shooting range and shoot multiple images of the room. Image data of the captured image captured by camera 25 is transmitted to server device 1 by control unit 21.
[0057] The operation display unit 23 includes a display unit such as a liquid crystal display that displays information, and an operation unit such as a touch panel and operation buttons that accept user operations.
[0058] The communication I / F 24 is a communication interface capable of performing wired or wireless data communication with an external device. For example, the communication I / F 24 performs data communication with the server device 1 and the image forming device 3 via the network N1.
[0059] The storage unit 22 is a non-volatile storage unit such as an HDD, SSD, or flash memory that stores various information. For example, a control program such as a browser program is stored in the storage unit 22. Specifically, the browser program is a control program for causing the control unit 21 to execute communication processing with an external device such as the server device 1 according to a communication protocol such as HTTP (Hypertext Transfer Protocol). The browser program may also be a dedicated application for executing communication processing with the server device 1 according to a predetermined communication protocol.
[0060] The control unit 21 has control devices such as a CPU, a ROM, and a RAM. The CPU is a processor that executes various arithmetic processing. The ROM is a non-volatile storage unit in which control programs such as a BIOS and an OS for causing the CPU to execute various processes are stored in advance. The RAM is a volatile or non-volatile storage unit that stores various information, and is used as a temporary storage memory (work area) for various processes executed by the CPU. The control unit 21 controls the operation terminal 2 by executing, by the CPU, various control programs stored in advance in the ROM or the storage unit 22.
[0061] Specifically, the control unit 21 functions as a browser processing unit 211 by executing various processes in accordance with the browser program stored in the storage unit 22. The browser processing unit 211 is capable of executing browser processing for displaying a web page provided from the server device 1 via the network N1 on the operation display unit 23 and inputting operations on the operation display unit 23 to the server device 1. That is, the operation terminal 2 can function as an operation terminal for the server device 1 by the control unit 21 executing the browser program. Note that some or all of the processing units included in the control unit 21 may be configured with electronic circuits.
[0062] For example, on a registration screen P10 (see FIG. 4) displayed on the operation terminal 2, the user performs a registration operation to select and register the image forming device 3 included in the captured image P1 and the IP address and model name included in the IP address list L1. The control unit 11 of the server device 1 executes the registration process (see FIG. 3) in response to the registration operation of the user on the operation terminal 2.
[0063] Also, for example, on the setting screen P20 (see FIG. 6) displayed on the operation terminal 2, the user performs an operation to select the image forming device 3 included in the captured image P1 and check the function information F2.
[0064] Also, for example, on the setting screen P20 (see FIG. 7) displayed on the operation terminal 2, the user performs an operation (drag-and-drop operation) to move the image to be printed to the image forming device 3 included in the captured image P1. Also, for example, on the color setting screen M1 (see FIG. 8) and the paper setting screen M2 (see FIG. 9) displayed on the operation terminal 2, the user performs an operation to select a print color and a print paper.
[0065] Also, for example, the user issues a print instruction on the execution screen M3 (see FIG. 10) displayed on the operation terminal 2. The control unit 11 of the server device 1 executes the job execution process (see FIGS. 7 to 10) in response to the print instruction from the user on the operation terminal 2.
[0066] [Registration process] Next, an example of the procedure of the registration process executed by the control unit 11 of the server device 1 will be described with reference to Fig. 12. The control unit 11 starts execution of the registration process by starting execution of the registration program in response to a user operation, for example, a registration operation on an operation screen displayed on the operation terminal 2. Here, steps S11, S12, ... represent the numbers of the processing procedures (steps) executed by the control unit 11.
[0067] The present invention can be understood as an invention of an information processing method for executing one or more steps included in the registration process.
[0068] First, in step S11, the control unit 11 determines whether or not a captured image P1 has been acquired from the operation terminal 2. For example, when a user captures an image of a room in which three image forming devices 3 are placed with the camera 25 (see FIG. 2) of the operation terminal 2, the control unit 11 acquires one or more captured images P1 (e.g., still image data) of the room from the operation terminal 2. When the control unit 11 acquires the captured image P1 (S11: Yes), it shifts the process to step S12. The control unit 11 waits until the captured image P1 is acquired (S11: No). Step S11 is an example of an image acquisition step of the present invention.
[0069] In step S12, the control unit 11 identifies the image forming devices 3 from the acquired captured image P1. Specifically, the control unit 11 performs image analysis on the captured image P1 to recognize the image forming devices 3. For each recognized image forming device 3, the control unit 11 identifies the position (coordinates) in the captured image P1 and registers the position information in the device information D1 (see FIG. 3). Step S12 is an example of a identifying step of the present invention.
[0070] Next, in step S13, the control unit 11 determines whether or not a selection operation of an image forming device and an IP address has been accepted. For example, the user selects the image forming device 3 included in the captured image P1 on the registration screen P10 (see FIG. 4), and selects the IP address and model name corresponding to the selected image forming device 3 from the IP address list L1. When the control unit 11 accepts the selection operation from the user (S13: Yes), it shifts the process to step S14. The control unit 11 waits until the selection operation is accepted from the user (S13: No). Step S13 is an example of an address acquisition step of the present invention.
[0071] In step S14, the control unit 11 registers the acquired image forming device 3, IP address, and model name in association with each other. Specifically, the control unit 11 registers the image forming device 3, IP address, and model name selected by the user on the operation terminal 2, in the device information D1 (see FIG. 3) in association with the location information of the image forming device 3. Step S14 is an example of a registration step of the present invention.
[0072] As described above, the control unit 11 executes the process of registering the IP address of each image forming device 3 included in the captured image P1 in response to a user operation on the captured image P1.
[0073] [Job execution processing] Next, an example of the procedure of the job execution process executed by the control unit 11 of the server device 1 will be described with reference to Fig. 13. The control unit 11 starts execution of the job execution process by starting execution of the job execution program in response to a user operation, for example, a job execution instruction (for example, a print instruction) on an operation screen displayed on the operation terminal 2. Here, steps S21, S22, ... represent the numbers of the processing procedures (steps) executed by the control unit 11.
[0074] The present invention can be understood as an invention of an information processing method for executing one or more steps included in the job execution process.
[0075] First, in step S21, the control unit 11 determines whether or not a captured image P2 has been acquired from the operation terminal 2. For example, when a user captures an image of a room in which three image forming devices 3 are placed with the camera 25 (see FIG. 2) of the operation terminal 2, the control unit 11 acquires the captured image P2 (e.g., video image data) of the room from the operation terminal 2. When the control unit 11 acquires the captured image P2 (S21: Yes), it shifts the process to step S22. The control unit 11 waits until the captured image P2 is acquired (S21: No).
[0076] In step S22, the control unit 11 identifies the image forming device 3 from the acquired captured image P2. Specifically, the control unit 11 compares the room scenery, etc., between the captured image P1 used in the registration process and the captured image P2 acquired in step S21, and identifies the position, model name, IP address, etc. of the image forming device 3 included in the captured image P2 with reference to the device information D1 (see FIG. 3).
[0077] Next, in step S23, the control unit 11 displays identification information for identifying the image forming device 3 in the captured image P2. For example, as shown in Fig. 5, the control unit 11 displays a rectangular image F1 surrounding each image forming device 3 so as to be superimposed on each image forming device 3. The control unit 11 may execute a display process using an AR (Augmented Reality) technique in the captured image P2. The control unit 11 may display a moving image (animation image) corresponding to the image forming device 3 instead of the rectangular image F1.
[0078] Next, in step S24, the control unit 11 determines whether or not an image to be printed has been acquired. For example, when the user selects a thumbnail image of "Image 1" on the setting screen P20 (see FIG. 7) of the operation terminal 2 and moves (drag-and-drop) it to a position in the captured image P2 where it overlaps with the image forming device 3, the control unit 11 acquires the image to be printed ("Image 1"). If the control unit 11 acquires the image to be printed (S24: Yes), it transitions the process to step S25. On the other hand, if the control unit 11 does not acquire an image to be printed (S24: No), it ends the job execution process.
[0079] In step S25, the control unit 11 causes the operation terminal 2 to display a setting screen for accepting operations related to job (here, a print job) settings. For example, as shown in Fig. 8, the control unit 11 causes the operation terminal 2 to display a color setting screen M1 and accepts an operation from the user to select "color" or "monochrome." Also, for example, as shown in Fig. 9, the control unit 11 causes the operation terminal 2 to display a paper setting screen M2 and accepts an operation from the user to select "A4" or "A5."
[0080] Next, in step S26, the control unit 11 determines whether or not a print instruction has been received from the operation terminal 2. Specifically, when the control unit 11 has completed the acceptance of the setting operation (see Figs. 8 and 9), it displays the execution screen M3 shown in Fig. 10 and accepts the print instruction from the user. When the user selects "Yes" on the execution screen M3, the control unit 11 acquires the print instruction. When the user selects "Cancel" on the execution screen M3, the control unit 11 transitions the process to step S23 and displays the setting screen P20 shown in Fig. 5.
[0081] Next, in step S27, the control unit 11 executes the print process. Specifically, the control unit 11 outputs a print instruction for "Image 1" to the image forming device 3. The image forming device 3 executes the print process for "Image 1" based on the print instruction acquired from the server device 1. For example, the image forming device 3 with the model name "MFP0001" prints "Image 1" in color on A4 paper. The control unit 11 outputs a print completion notification to the operation terminal 2.
[0082] As described above, the control unit 11 causes identification information capable of identifying the image forming apparatus 3 to be superimposed on the captured image P2 and causes the image forming apparatus 3 to execute a job based on a user operation on the captured image P2.
[0083] The information processing system 10 according to the present embodiment acquires a captured image P1 capturing an image of a predetermined space in which an image forming device 3 is placed, and identifies the image forming device 3 from the acquired captured image P1. The information processing system 10 also acquires address information (IP address) of the identified image forming device 3, and registers the acquired address information in association with the image forming device 3 in the captured image P1.
[0084] In addition, the information processing system 10 displays the captured image P2 on the operation terminal 2 that outputs an image formation instruction to the image forming device 3, and displays identification information (rectangular image F1) that can identify the image forming device 3 in the captured image P2, in association with the image forming device 3.
[0085] According to the above configuration, the user can easily identify the image forming device 3 in the captured image P2 displayed on the operation terminal 2.
[0086] Also, the user can manually select the image forming device 3 present in the space photographed by the camera 25 and set the IP address of the selected image forming device 3. Also, no matter where the user is in the office, for example, the image forming device 3 (MFP, printer, etc.) present in the space can be recognized simply by photographing the space with the camera 25, and it becomes possible to check the function information, status information, etc. by display processing using AR technology in the photographed image P2 without moving close to the image forming device 3. Also, since the user can select the image forming device 3 in the photographed image (photographed image P2) and issue a print instruction, the user can intuitively grasp the intended image forming device 3 and prevent the user from erroneously executing the print process on another image forming device 3.
[0087] The information processing system of the present invention may be composed of a server device 1, an operation terminal 2, and an image forming device 3, or may be composed of the server device 1 alone, or may be composed of the operation terminal 2 alone. For example, the operation terminal 2 may have the functions of the server device 1.
[0088] [Disclosure Note] The following will provide an outline of the disclosure extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined in any desired manner.
[0089] <Appendix 1> an image acquisition processing unit that acquires an image of a predetermined space in which the image forming apparatus is disposed; an identification processing unit that identifies the image forming apparatus from the captured image acquired by the image acquisition processing unit; an address acquisition processing unit that acquires address information of the image forming apparatus identified by the identification processing unit; a registration processing unit that associates the address information acquired by the address acquisition processing unit with the image forming device in the captured image and registers the associated address information; An information processing system comprising:
[0090] <Appendix 2> a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and displays, in the captured image, identification information that can identify the image forming apparatus identified by the identification processing unit in association with the image forming apparatus. 2. The information processing system according to claim 1.
[0091] <Appendix 3> a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and that displays, in the captured image, function information related to a function of the image forming apparatus identified by the identification processing unit in association with the image forming apparatus, 3. An information processing system according to claim 1 or 2.
[0092] <Appendix 4> a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and that displays, in the captured image, status information regarding the status of the image forming apparatus identified by the identification processing unit in association with the image forming apparatus. 4. An information processing system according to claim 1.
[0093] <Appendix 5> When the image forming apparatus is in a state where the image forming process cannot be executed, the display processing unit displays the status information indicating a cause why the image forming process cannot be executed, and does not display function information related to functions of the image forming apparatus. 5. An information processing system according to claim 4.
[0094] <Appendix 6> when an operation for selecting a second image forming device is received from a user in the captured image in which first function information corresponding to a first image forming device is displayed, the display processing unit deletes the first function information and displays second function information corresponding to the second image forming device. 6. An information processing system according to claim 3.
[0095] <Appendix 7> displaying thumbnail images of print targets for which printing processing is to be performed in the image forming apparatus; 7. An information processing system according to claim 1.
[0096] <Appendix 8> when an operation of moving the thumbnail image of the captured image onto the image forming device is received, causing the image forming device to execute a print process of the thumbnail image. 8. An information processing system according to claim 7.
[0097] <Appendix 9> When the captured image is a still image, acceptance of the operation is permitted. 9. An information processing system according to claim 8.
[0098] <Appendix 10> when the operation is accepted, a setting screen relating to print settings is displayed and a setting operation is accepted from the user; 10. An information processing system according to claim 8 or 9.
[0099] <Appendix 11> When the operation is accepted and the image forming apparatus is in a state in which the image forming apparatus cannot execute a print process, the setting screen is not displayed. 11. The information processing system according to claim 10. [Explanation of symbols]
[0100] 1: Server device 2: Operation terminal 3: Image forming device 10: Information processing systems 11: Control section 12: Storage section 13: Operation display section 14: Communication I / F 21: Control section 22: Storage section 23: Operation display section 24: Communication I / F 25: Camera 111: Image acquisition processing unit 112: Specific processing unit 113: Address acquisition processing unit 114: Registration processing unit 115: Display processing unit 116: Execution processing unit 211: Browser processing unit D1:Device information F1: Rectangular image F2 :Function Information F3 :Status Information L1: IP address list L2: Thumbnail image list M1: Color setting screen M2: Paper setting screen M3: Execution screen M4: Inquiry screen N1: Network P1: Captured image P10: Registration screen P2: Captured image P20: Setting screen
Claims
1. an image acquisition processing unit that acquires an image of a predetermined space in which the image forming apparatus is disposed; an identification processing unit that identifies the image forming apparatus from the captured image acquired by the image acquisition processing unit; an address acquisition processing unit that acquires address information of the image forming apparatus identified by the identification processing unit; a registration processing unit that associates the address information acquired by the address acquisition processing unit with the image forming device in the captured image and registers the associated address information; An information processing system comprising:
2. a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and displays, in the captured image, identification information that can identify the image forming apparatus identified by the identification processing unit in association with the image forming apparatus. The information processing system according to claim 1 .
3. a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and that displays, in the captured image, function information related to a function of the image forming apparatus identified by the identification processing unit in association with the image forming apparatus, The information processing system according to claim 1 .
4. a display processing unit that displays the captured image on an operation terminal that outputs an image formation instruction to the image forming apparatus, and that displays, in the captured image, status information regarding the status of the image forming apparatus identified by the identification processing unit in association with the image forming apparatus. The information processing system according to claim 1 .
5. When the image forming apparatus is in a state where the image forming process cannot be executed, the display processing unit displays the status information indicating a cause why the image forming process cannot be executed, and does not display function information related to functions of the image forming apparatus.
5. The information processing system according to claim 4.
6. when an operation for selecting a second image forming device is received from a user in the captured image in which first function information corresponding to a first image forming device is displayed, the display processing unit deletes the first function information and displays second function information corresponding to the second image forming device. The information processing system according to claim 3 .
7. displaying thumbnail images of print targets for which printing processing is to be performed in the image forming apparatus; 7. The information processing system according to claim 1.
8. when an operation of moving the thumbnail image of the captured image onto the image forming device is received, causing the image forming device to execute a print process of the thumbnail image. The information processing system according to claim 7.
9. When the captured image is a still image, acceptance of the operation is permitted. The information processing system according to claim 8.
10. when the operation is accepted, a setting screen relating to print settings is displayed and a setting operation is accepted from the user; The information processing system according to claim 8.
11. When the operation is accepted and the image forming apparatus is in a state in which the image forming apparatus cannot execute a print process, the setting screen is not displayed. The information processing system according to claim 10.
12. an image acquiring step of acquiring a captured image of a predetermined space in which the image forming apparatus is disposed; a step of identifying the image forming apparatus from the captured image acquired in the image acquiring step; an address acquisition step of acquiring address information of the image forming apparatus identified in the identification step; a registration step of registering the address information acquired in the address acquisition step in association with the image forming device in the captured image; An information processing method comprising:
Citation Information
Patent Citations
Install method of printer driver, install system, program and image formation apparatus
JP2018112773A
