Operation guide system of device
The operation guidance system uses a mobile terminal device to emit light on the actual operation parts of an image forming apparatus, addressing the complexity of error detection in existing systems by providing clear, actionable guidance for users.
Patent Information
- Application Number
- JP2023072738
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-04-26
- Publication Date
- 2024-11-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing image forming apparatuses require complex configurations with multiple cameras for error detection and display broken-line images, making it difficult for users without prior knowledge to identify error locations accurately.
An operation guidance system with a detachable mobile terminal device that emits light onto the actual operation parts of the apparatus, guided by a mobile terminal display, using pattern matching and laser beams to illuminate the correct parts for error resolution.
Users can quickly and accurately identify and resolve errors by aligning the mobile terminal display with the apparatus, enabling immediate understanding of error locations and following operation guidance.
Smart Images

Figure 2024157995000001_ABST
Abstract
Description
[Technical field]
[0001] The present invention relates to an operation guidance system including a main unit and a portable terminal device that is detachably attached to the main unit, and in particular to a technology for providing guidance on operation sections that require operation on the main unit by displaying the portable terminal device. [Background technology]
[0002] Since an image forming apparatus has a complex structure and many components, when a user performs operations such as maintenance or troubleshooting, accurate operation guidance is required for the user. In the image forming apparatus described in Patent Document 1, when a paper jam occurs, the location of the jam is photographed by cameras installed at various locations, and operation guidance is displayed on the display unit together with the image of the jammed location. The user views the image and performs the jam removal work. In addition, in the image forming apparatus described in Patent Document 2, an imaging means is provided on an operation panel that is detachable from the apparatus body and capable of wireless communication, and when the location of an error is photographed by the imaging means while the operation panel is not attached, the location of the error is displayed by a broken line or the like in the image photographed and displayed by the imaging means, and an image showing operation guidance is synthesized and displayed on the displayed image, thereby further improving the operability of the operator. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] JP 2009-143141 A [Patent Document 2] JP 2012-096448 A Summary of the Invention [Problem to be solved by the invention]
[0004] However, in Patent Document 1, when a jam is detected at each location by multiple sheet detection sensors, it is necessary to provide a camera for photographing each location, which complicates the configuration. In addition, in Patent Document 2, the location of the error is displayed by a dashed line or the like in the image photographed by the imaging means and displayed on the display unit, but the user may not immediately understand the location of the error in the actual device even if he or she looks at the image displayed on the display unit. Since many parts are provided inside the image forming device, a user who has no prior knowledge or no experience in dealing with errors must repeatedly compare the location of the error in the image with the actual location of the error to confirm.
[0005] The present invention has been made in consideration of the above circumstances, and has an object to enable a user to immediately understand where an error has actually occurred in a device by shining light onto an operating part of the device that is to be operated and promptly informing the user of this operating part. [Means for solving the problem]
[0006] An operation guidance system for a device according to one aspect of the present invention includes a main device and a portable terminal device detachably attached to the main device, the main device including a first communication unit that performs data communication with the portable terminal device, and a first control unit that determines an operation part of the main device and causes an image of the operation part of the main device to be transmitted from the first communication unit to the portable terminal device, and the portable terminal device includes the second communication unit that performs data communication with the main device, an imaging unit, a display unit that displays an image captured by the imaging unit, a light emission unit that emits light into an imaging range of the imaging unit, and a second control unit that, when an image of the operation part of the main device is received by the second communication unit, determines whether the image of the operation part of the main device is included in the image captured by the imaging unit, and causes the light emission unit to emit light to the operation part of the main device when it is determined that the image of the operation part of the main device is included in the image captured by the imaging unit.
[0007] According to another aspect of the present invention, an operation guidance system for a device includes a main device and a portable terminal device detachably attached to the main device, the main device including a first communication unit that performs data communication with the portable terminal device, and a first control unit that determines an operation part of the main device and causes an image of the operation part of the main device to be transmitted from the first communication unit to the portable terminal device, the portable terminal device including the second communication unit that performs data communication with the main device, an imaging unit, a display unit that displays an image captured by the imaging unit, and a light emission unit that emits light into an imaging range of the imaging unit, the first control unit of the main device determining whether an image of the operation part of the main device is included in the image captured by the imaging unit, and causing the light emission unit to emit light to the operation part of the main device when it is determined that the image of the operation part of the main device is included in the image captured by the imaging unit. Effect of the Invention
[0008] According to the present invention, by shining light onto the actual operating portion of the device that is to be operated, the user is immediately informed of the portion that should be operated, thereby enabling the user to immediately understand where the actual error has occurred in the device. [Brief description of the drawings]
[0009] [Figure 1] 1 is a diagram showing an image forming apparatus and a mobile terminal device in an apparatus operation guidance system according to an embodiment of the present invention, in which a cross section of the image forming apparatus is shown. [Diagram 2] 1 is a perspective view showing an external appearance of an image forming apparatus according to an embodiment of the present invention; [Diagram 3] FIG. 2A is a perspective view showing the mobile terminal device as viewed from the front side, and FIG. 2B is a perspective view showing the mobile terminal device as viewed from the back side. [Figure 4] FIG. 2 is a block diagram showing the internal configuration of the image forming apparatus and the portable terminal apparatus. [Diagram 5]13 is a diagram conceptually showing a data table in which, for each type of error, an image of one or more operation parts to be operated to resolve the error, and an operation sequence of the one or more operation parts are associated with each other. FIG. [Figure 6] 13 is a diagram conceptually showing a data table in which each operation part is associated with each operation guide information indicating an operation procedure for each operation part. FIG. [Figure 7] 10 is a flowchart showing a control procedure for irradiating light from a light emission section of a portable terminal device onto an operating portion of an image forming device that should be operated to resolve an error, and displaying operation guide information for resolving the error on a display section of the portable terminal device. [Figure 8] 1A and 1B are diagrams showing captured images captured by an imaging camera of the mobile terminal device and displayed on a display unit. [Figure 9] 11 is a diagram showing an image of an operation part captured by an imaging camera of the mobile terminal device and displayed on a display unit. FIG. [Figure 10] FIG. 2 is a diagram showing a state in which a laser beam is irradiated onto an operating portion of an image forming apparatus that actually exists; [Figure 11] FIG. 13 is a diagram showing a message displayed on a display unit of the portable terminal device. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0010] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. Fig. 1 is a diagram showing an image forming apparatus and a mobile terminal device in an apparatus operation guidance system according to an embodiment of the present invention, and shows a cross section of the image forming apparatus. Fig. 2 is a perspective view showing the appearance of the image forming apparatus of this embodiment.
[0011] 1 and 2, the operation guidance system Sy of the present embodiment includes an image forming apparatus 10 and a mobile terminal device 30. In the operation guidance system Sy, the mobile terminal device 30 is detachably attached to the image forming apparatus 10, and the image forming apparatus 10 and the mobile terminal device 30 are connected via wireless communication. Note that the image forming apparatus 10 corresponds to the main body device in the claims, and the mobile terminal device 30 corresponds to the mobile terminal device in the claims.
[0012] The image forming apparatus 10 is provided with an image reading unit 11 and an image forming unit 12. The image reading unit 11 has an image pickup element that optically reads an image of a document, and the output of the image pickup element is converted into image data representing the image of the document.
[0013] The image forming unit 12 prints an image represented by the image data on a recording sheet, and includes an image forming unit 3M for magenta, an image forming unit 3C for cyan, an image forming unit 3Y for yellow, and an image forming unit 3Bk for black. In each of the image forming units 3M, 3C, 3Y, and 3Bk, the surface of the photosensitive drum 4 is uniformly charged, the surface of the photosensitive drum 4 is exposed to light, an electrostatic latent image is formed on the surface of the photosensitive drum 4, the electrostatic latent image on the surface of the photosensitive drum 4 is developed into a toner image, and the toner image on the surface of the photosensitive drum 4 is transferred to the intermediate transfer belt 5. As a result, a color toner image is formed on the intermediate transfer belt 5. This color toner image is secondarily transferred to the recording sheet P conveyed from the paper feed unit 14 through the conveying path 8 in the nip area N between the intermediate transfer belt 5 and the secondary transfer roller 6.
[0014] Thereafter, the recording paper P is heated and pressed in the fixing device 15 so that the toner image on the recording paper P is fixed by thermocompression, and the recording paper P is then discharged onto a discharge tray 17 via discharge rollers 16 .
[0015] Moreover, the mobile terminal device 30 has a flat main body housing 32 as shown in Figures 3(A) and (B), with a display unit 31 provided on the front surface of the main body housing 32, and an imaging camera 35 and a light emission unit 36 provided on the back surface of the main body housing 32.
[0016] FIG. 4 is a block diagram showing the internal configuration of the image forming apparatus 10 and the mobile terminal device 30 in the device operation guidance system Sy.
[0017] The image forming device 10 includes an image reading unit 11, an image forming unit 12, a display unit 21, an operation unit 22, a touch panel 23, a short-range communication unit 24, an image memory 25, a plurality of sensors 26, a storage unit 27, and a control unit 28. These components are capable of transmitting and receiving data or signals to and from each other via a bus.
[0018] The operation unit 22 includes physical keys such as a numeric keypad, a decision key, a start key, etc. The operation unit 22 accepts various instructions in response to the operation of each key.
[0019] The display unit 21 is configured with a liquid crystal display (LCD) or an organic light-emitting diode (OLED) display.
[0020] A touch panel 23 is disposed on the screen of the display section 21. The touch panel 23 is a so-called resistive film type or capacitive type touch panel, which detects a contact (touch) made by a user's finger or the like on the touch panel 23 together with the contact position, and outputs a detection signal indicating the coordinates of the contact position to a control section 29 of the control unit 28, which will be described later.
[0021] The short-range communication unit 24 performs data communication with the mobile terminal device 30 by short-range communication based on, for example, Bluetooth Low Energy (BLE, Bluetooth: registered trademark).
[0022] The image memory 25 temporarily stores image data representing the image of the document read by the image reading unit 11 .
[0023] The storage unit 27 is a large-capacity storage device such as a solid state drive (SSD) or a hard disk drive (HDD), and stores various application programs and various data.
[0024] The multiple sensors 26 are provided at various locations in the image forming apparatus 10, detect preset errors at each location, and output the respective detection results to a control unit 29, which will be described later. For example, one of the sensors 26 is an optical sensor that detects whether the paper feed cassette in the paper feed unit 14 is empty. Another of the sensors 26 is an optical sensor that detects a jam of the recording paper P that has occurred in the transport path 8.
[0025] The control unit 28 is composed of a processor, a RAM (Random Access Memory), a ROM (Read Only Memory), etc. The processor is, for example, a CPU (Central Processing Unit), an ASIC (Application Specific Integrated Circuit), or an MPU (Micro Processing Unit), etc. The control unit 28 functions as a control unit 29 by the processor executing a control program stored in the ROM or the storage unit 27.
[0026] The control unit 29 comprehensively controls the image forming apparatus 10. The control unit 28 is connected to the image reading unit 11, the image forming unit 12, the display unit 21, the operation unit 22, the touch panel 23, the short-range communication unit 24, the image memory 25, the multiple sensors 26, and the storage unit 27. The control unit 29 controls the operation of these components and transmits and receives signals or data between these components.
[0027] The control unit 29 serves as a processing unit that executes various processes necessary for image formation by the image forming apparatus 10. The control unit 29 also accepts operation instructions input by the user based on a detection signal output from the touch panel 23 or an operation of a physical key of the operation unit 22. Furthermore, the control unit 29 has a function of controlling the display operation of the display unit 21 and the communication operation of the near-field communication unit 24.
[0028] Furthermore, the control unit 29 judges whether the paper feed cassette of the paper feed unit 14 is empty, whether a jam has occurred in the conveying path 8, etc., based on the detection outputs of the multiple sensors 26.
[0029] The mobile terminal device 30 also includes a display unit 31, a touch panel 33, a short-range communication unit 34, an imaging camera 35, a light emitting unit 36, a storage unit 37, and a control unit 38. These components are capable of transmitting and receiving data or signals to and from each other via a bus.
[0030] The display unit 31 is configured with a known flexible liquid crystal display (LCD) or an organic light emitting diode (OLED) display, and is provided on the surface of a main body housing 32 as shown in FIG. 3(A).
[0031] The touch panel 33 is a known flexible touch panel that is bendable. The touch panel 33 is disposed on the screen of the display unit 31, and detects a contact (touch) of a user's fingertip or the like on the touch panel 33 together with the contact position, and inputs an instruction from the user to the screen of the display unit 31.
[0032] The short-range communication unit 34 is, for example, a communication device based on Bluetooth Low Energy, and performs data communication with the image forming apparatus 10 via short-range communication.
[0033] The imaging camera 35 includes, for example, a CCD, and is provided on the back surface of the main body housing 32 as shown in FIG. 3(B), and captures an image of an object facing the back surface of the main body housing 32 from the back surface side.
[0034] The light emitting section includes a laser diode or the like, and is provided on the rear surface side of the main body housing 32 as shown in FIG. 3(B).
[0035] The storage unit 37 is a memory storage device such as a RAM or a ROM, and stores various application programs, various information, and the like.
[0036] The control unit 38 is composed of a processor, a RAM, a ROM, etc. The processor is, for example, a CPU, an ASIC, an MPU, etc. The control unit 38 functions as a control unit 39 by the processor executing a control program stored in the ROM or the storage unit 37.
[0037] The control unit 38 is connected to the display unit 31, the touch panel 33, the short-range communication unit 34, the imaging camera 35, the light emitting unit 36, and the storage unit 37. The control unit 39 controls the operation of these components and transmits and receives signals or data between these components.
[0038] The control unit 39 serves as a processing unit that executes various processes. The control unit 39 also has a function of controlling the display operation of the display unit 31 and a function of controlling the communication operation of the near-field communication unit 34. The control unit 39 also determines a touch operation on the screen of the display unit 31 based on the detection output of the touch panel 33, and determines a position on the screen of the display unit 31 designated by the touch operation.
[0039] Here, if any error occurs in the image forming apparatus 10, the user must deal with the error. However, since the image forming apparatus 10 has a complex structure and is composed of many parts, a user who has no prior knowledge or experience in dealing with errors is forced to carry out difficult tasks.
[0040] Therefore, in the device operation guidance system Sy of this embodiment, when an error occurs in the image forming device 10, the control unit 29 of the image forming device 10 identifies an operation part of the image forming device 10 that is operated to resolve the error, selects an image of the operation part, and transmits the image of the operation part of the image forming device 10 from the short-range communication unit 24 to the mobile terminal device 30.
[0041] The control unit 39 of the mobile terminal device 30 acquires the image of the operation part received by the short-range communication unit 34, displays the captured image captured by the imaging camera 35 on the screen of the display unit 31, and judges by pattern matching technology or the like whether or not the captured image captured by the imaging camera 35 contains an image having a predetermined identity (e.g., 80%) with the image of the operation part received by the short-range communication unit 34, that is, whether or not the captured image contains the image of the operation part. When the control unit 39 judges that the captured image contains the image of the operation part, it judges the movement direction of the imaging camera 35 that needs to be moved in order to display the image of the operation part in the center of the screen of the display unit 31, and displays an image indicating this movement direction on the screen of the display unit 31.
[0042] Here, the imaging camera 35 is set to capture an image of a subject present directly in front of the imaging lens arranged in an imaging aperture provided on the back side of the main body housing 32 so that the subject is positioned at the center of an imaging area (here, the imaging area is set to be the same as the display area of the display screen of the display unit 21). The control unit 39 determines the above-mentioned direction of movement by calculating a movement amount and a movement direction (amount and direction of deviation of the first coordinate position from the first coordinate position) required to match the second coordinate position with the first coordinate position based on a difference between a first coordinate position indicating the center position of the imaging area of the imaging camera 35 and a second coordinate position (preferably a predetermined center position of the operation part) in the imaging area (display area) where the operation part is imaged and displayed.
[0043] The user refers to the captured image displayed on the screen of the display unit 31 and its movement direction to change the position and orientation of the portable terminal device 30 relative to the image forming apparatus 10 to change the imaging area captured by the imaging camera 35, causes the imaging camera 35 to capture an image of the operation part of the image forming apparatus 10, and causes the captured image of the operation part to be displayed in the center of the display screen of the display unit 31.
[0044] Here, the light emitting unit 36 emits the laser light L linearly along the center of the imaging range of the imaging camera 35, and the position and emission direction of the light emitting unit 36 in the main body housing 32 are fixed so that the light is irradiated onto a subject present directly in front of the photographing lens of the imaging camera 35 in the horizontal direction. When the range irradiated by the light emitting unit 36 is to be expanded, a plurality of laser diodes are provided to expand the irradiated area.
[0045] The control unit 39 of the mobile terminal device 30 causes the light emitting unit 36 to emit laser light L to the operation site, and also causes the display unit 31 to display an image showing operation guide information for solving the error.
[0046] This allows the user to reliably and quickly know the part of the image forming apparatus 10 irradiated with the laser light L, i.e., the operation part, and to perform an appropriate operation to eliminate the error by viewing the operation guide information displayed on the display unit 31. Even if the above operation part is displayed as an image on the display unit 21, the user can only confirm the operation part in the image displayed on the display screen of the display unit 21, making it difficult to confirm the image forming apparatus 10 that actually exists physically and the mechanical parts therein. This is because, after confirming the operation part in the image displayed on the display screen of the display unit 21, the user must shift his / her line of sight from the display screen and identify again on the actual image forming apparatus 10 itself where the operation part previously confirmed on the display screen is located. In this regard, in the present embodiment, the laser light L irradiates the operation part on the actual image forming apparatus 10, so that the user can immediately know the location of the operation part after shifting his / her line of sight from the display screen and directing it toward the actual image forming apparatus 10.
[0047] The configurations of the light emitting unit 36 and the imaging camera 35 are not limited to the above configurations, and may be, for example, the following configurations. For example, the light emitting unit 36 is provided on the main body housing 32 by a movable mechanism that changes the orientation of the light emitting unit 36 relative to the main body housing 32 in all directions, up, down, left, and right, so that the laser emission direction can be freely changed. Then, the control unit 39 of the mobile terminal device 30 specifies a coordinate position where an image showing an operation part exists in the imaging area (image) captured by the imaging camera 35, and determines the laser emission direction of the light emitting unit 36 with which the light emitting unit 36 irradiates the position of the operation part in the actual image forming device 10 when the mobile terminal device 30 and the imaging camera 35 are in this position and orientation. The control unit 39 drives and controls the movable mechanism to change the orientation of the light emitting unit 36 so that the laser emission direction becomes the determined laser emission direction, and then causes the light emitting unit 36 to emit a laser. That is, the control unit 39 stores the coordinate position where an image showing the operation part is located in the imaging area (captured image) captured by the imaging camera 35, and the orientation of the light emitting unit 36 which is the laser emission direction capable of irradiating the laser to the operation part located at the coordinate position, and changes the orientation of the light emitting unit 36 to the orientation associated with the coordinate position where the image showing the operation part is located in the imaging area.
[0048] In order to support the user's operations as described above, a data table DT1 is stored in advance in the storage unit 27 of the image forming apparatus 10, which associates, for each of various errors detected by the multiple sensors 26, an image of one or more operation parts to be operated to resolve the error with an operation sequence of the one or more operation parts, as shown in an example in Fig. 5. For example, the images of the operation parts are an image of a paper feed cassette of the image forming apparatus 10, an image of a door of the image forming apparatus 10 that is opened and closed, and an image of an operation lever inside the image forming apparatus 10. In addition, since multiple operation parts may be operated in sequence to resolve one error, the operation sequence of each operation part is stored.
[0049] Further, the storage unit 37 of the mobile terminal device 30 stores in advance a data table DT2 in which each operation part is associated with operation guide information indicating the operation procedure of each operation part, as shown in Fig. 6. For example, the operation guide information is text, illustrations, and video showing the operation procedure and operation method of a paper feed cassette, an operation lever, etc.
[0050] Next, a control procedure for irradiating the laser light L from the light emission unit 36 of the portable terminal device 30 to the operation part of the image forming device 10 that should be operated to resolve the error and displaying operation guide information for resolving the error on the display unit 31 of the portable terminal device 30 in the device operation guidance system Sy will be described with reference to the flowchart shown in FIG.
[0051] When an error occurs in the image forming apparatus 10, the control unit 29 of the image forming apparatus 10 determines the occurrence of the error and the content of the error based on the detection output of any one of the sensors 26 (S201), and causes an icon indicating a function of operation guidance by the mobile terminal device 30 to be displayed on the screen of the display unit 21 (S202). When the user touches the icon on the screen of the display unit 21, the control unit 29 receives a guidance display instruction corresponding to the touch operation through the touch panel 23, and causes the screen of the display unit 21 to display a method for using the function of operation guidance by the mobile terminal device 30 in accordance with this instruction (S203). This method is displayed by text, illustrations, video, etc.
[0052] The user views a method for utilizing the functions of the operation guide of the portable terminal device 30 displayed on the screen of the display unit 21 of the image forming apparatus 10, removes the portable terminal device 30 from the image forming apparatus 10, and starts capturing an image of the image forming apparatus 10 using the imaging camera 35 of the portable terminal device 30 (S101). The control unit 39 of the portable terminal device 30 controls the imaging camera 35 to display an image Qb captured by the imaging camera 35 on the display unit 31 (S103).
[0053] In addition, the control unit 29 of the image forming apparatus 10 reads out an image Qs of one or more operation parts to be operated to resolve the error determined in S201 and the operation sequence of the one or more operation parts from the data table DT1 shown in FIG. 5 stored in the memory unit 27, and transmits the read out image Qs of the one or more operation parts and a control signal indicating the operation sequence of the one or more operation parts from the short-range communication unit 24 to the mobile terminal device 30 (S204).
[0054] In the mobile terminal device 30, when an image Qs of one or more operation parts to be operated to resolve the error and a control signal indicating the operation sequence of the one or more operation parts are received by the short-range communication unit 34 (S104), the control unit 39 selects the image Qs of the operation part that is first in the operation sequence according to the operation sequence of the one or more operation parts (S105).
[0055] Then, the control unit 39 of the mobile terminal device 30 determines, by a known pattern matching technique, whether or not the image Qs of the operation part selected in S105 is included in the captured image Qb displayed on the screen of the display unit 31 (S106).
[0056] When the control unit 39 of the mobile terminal device 30 determines that the image Qs of the operation part selected in S105 is not included in the captured image Qb displayed on the screen of the display unit 31 (S106 "No"), it causes the display unit 31 to display an error message indicating that the imaging range of the imaging camera 35 is not appropriate since the operation part to be operated is not displayed on the screen of the display unit 31 (S107). Then, the process returns to S106. In this case, the user moves around the mobile terminal device 30 to appropriately change the imaging range of the imaging camera 35.
[0057] Furthermore, when the control unit 39 determines that the image Qs of the operation part selected in S105 is included in the captured image Qb displayed on the screen of the display unit 31 (S106 "Yes"), it determines the amount and direction of deviation of the center position q (second coordinate position) of the image Qs of the operation part relative to the center position p (first coordinate position) of the screen of the display unit 31 (S108). Note that the center position p of the screen of the display unit 31 is, for example, the intersection of a horizontal line passing through a position half the vertical width of the screen and a vertical line passing through a position half the horizontal width. Also, the center position q of the image Qs of the operation part is, for example, the intersection of a horizontal line passing through a position half the vertical width of the image Qs and a vertical line passing through a position half the horizontal width.
[0058] The control unit 39 of the mobile terminal device 30 determines whether or not the center position q of the image Qs of the operation part falls within a preset target range M around the center position p of the screen of the display unit 31, based on the amount of deviation of the center position q of the image Qs of the operation part from the center position p of the screen of the display unit 31 determined in S108 (S109). Note that "the center position q falls within the target range M" is an example of "the second coordinate position coincides with the first coordinate position" described above.
[0059] When the control unit 39 determines that the center position q of the image Qs of the operation part is not within the target range M (S109 "No"), it determines the direction of deviation of the center position q of the image Qs of the operation part from the center position p of the screen of the display unit 31 determined in S108 as the direction of movement of the displayed image Qs of the operation part, displays this direction of movement on the display unit 31, and also displays a message on the display unit 31 prompting the user to change the imaging range of the imaging camera 35 in the direction of movement (S110). After this, the process returns to S106, and the control unit 39 repeats the processes of S108 to S110 again until it determines that the image Qs of the operation part selected in S105 is included in the captured image Qb displayed on the screen of the display unit 31 (S106 "NO").
[0060] For example, when an error A in the data table DT1 occurs, the paper feed cassette 18 shown in Fig. 2 is the only operation part, and the operation order of the paper feed cassette 18 is first, the paper feed cassette 18 is imaged by the imaging camera 35, and in a state in which the image Qs of the paper feed cassette 18 is captured by the imaging camera 35 and included in the captured image Qb displayed on the display unit 31 as shown in Fig. 8, the direction of deviation from the center position p of the screen of the display unit 31 to the center position q of the image Qs of the paper feed cassette 18 is determined, an arrow Y indicating the determined direction of deviation as a movement direction is displayed on the display unit 31, and a message E1 prompting the user to change the imaging range of the imaging camera 35 in the movement direction is displayed on the display unit 31. The user sees the message E1, and changes the position of the mobile terminal device 30 relative to the image forming apparatus 10 to change the imaging range of the imaging camera 35 of the mobile terminal device 30 in the movement direction of the arrow Y.
[0061] This allows the user to place the operation part selected in S105 within the range of photography by the imaging camera 35, and move the center position q of the image Qs of the operation part that has been photographed by the imaging camera 35 and is displayed on the display unit 31, into the target range M around the center position p of the screen of the display unit 31.
[0062] For example, as shown in FIG. 9, the center position q of an image Qs of the paper feed cassette 18 captured by the imaging camera 35 moves into the target range M around the center position p of the screen of the display unit 31.
[0063] At this time, the control unit 39 of the portable terminal device 30 determines that the center position q of the image Qs of the paper feed cassette 18 captured by the imaging camera 35 falls within the target range M around the center position p of the screen of the display unit 31 (S109 "Yes"), and causes the light emission unit 36 to emit laser light L. Also, it reads out operation guide information associated with that operation part from the data table DT2 shown in FIG. 6 stored in the memory unit 37, and causes this operation guide information to be displayed on the screen of the display unit 31 (S111).
[0064] For example, operation guide information associated with the paper feed cassette 18 is read out from the data table DT2, and operation guide information as shown in Fig. 9 is displayed on the screen of the display unit 31. This operation guide information indicates the operating procedure and method of the paper feed cassette 18 to be operated to resolve the error determined in S201. An OK key K11 is also displayed on the screen of the display unit 31.
[0065] As described above, the light emitting unit 36 emits the laser light L toward the center of the imaging range of the imaging camera 35. Therefore, in a state in which the central position q of the image Qs of the paper feed cassette 18 captured by the imaging camera 35 falls within the target range M around the central position p of the screen of the display unit 31, the laser light L is irradiated onto an area TP that is an operating portion of the image forming apparatus 10 that actually exists and that includes a position q1 that corresponds to the central position q of the image Qs of the paper feed cassette 18, as shown in Fig. 10 .
[0066] This allows the user to shift his / her gaze away from the display screen and then directly confirm the operation portion of the image forming apparatus 10 irradiated with the laser light L through the area TP, thereby reliably and quickly learning about it. Furthermore, the user can look at the operation guide information displayed on the display unit 31 and perform an appropriate operation to resolve the error. When the error is resolved, the user touches the OK key K11 displayed on the screen of the display unit 31. The control unit 39 accepts the input of a completion instruction associated with this touch operation.
[0067] The control unit 39 of the mobile terminal device 30 waits for the completion instruction to be input (S112 "No"), and when the completion instruction is received (S112 "Yes"), it determines whether or not all of the one or more operation parts received in S101 have been operated (S113). When the control unit 39 determines that all of the one or more operation parts have been operated (S113 "Yes"), it causes the short-range communication unit 34 to transmit a control signal indicating the completion of the operation (S115). After that, the control procedure shown in FIG. 7 ends.
[0068] In the image forming apparatus 10, when the control signal indicating the completion of the operation is received by the short-range communication unit 24, the control unit 29 displays a message E2 on the screen of the display unit 21, as shown in Fig. 11, to prompt the user to return the portable terminal device 30 to the image forming apparatus 10 and attach it thereto (S205). After that, the control procedure shown in Fig. 7 ends.
[0069] On the other hand, in S113, when the control unit 39 of the mobile terminal device 30 determines that one or all of the operation parts have not been operated (S113 "No"), it selects an image of the operation part in the next operation sequence (S114) and repeats the process from S106 until one or all of the operation parts have been operated. Then, when the control unit 39 of the mobile terminal device 30 determines that one or all of the operation parts have been operated (S113 "Yes"), it transmits a control signal indicating completion of the operation from the short-range communication unit 34 (S115). After this, the control procedure shown in FIG. 7 ends.
[0070] For example, for error B in the data table DT1, the operation parts of the door 19, the first lever (not shown), and the second lever (not shown) shown in FIG. 2 are set, and the operation sequence of the door 19, the first lever, and the second lever is set. The door 19 is opened, the first lever inside the device is operated, and finally the second lever inside the device is operated. In this case, the image Qs of the door 19, the image Qs of the first lever, and the image Qs of the second lever, as well as the operation sequence of the door 19, the first lever, and the second lever are transmitted from the image forming device 10 to the mobile terminal device 30. In the mobile terminal device 30, the process of S106 to S114 is performed for the image Qs of the door 19 in the first operation sequence, the process of S106 to S114 is performed for the image Qs of the first lever in the next operation sequence, and the process of S106 to S113 and S115 is performed for the image Qs of the second lever in the last operation sequence, thereby eliminating error B, and a control signal indicating completion of the operation is transmitted from the mobile terminal device 30 to the image forming device 10.
[0071] The user moves around the mobile terminal device 30 for each operation part that needs to be operated, captures an image of the operation part with the imaging camera 35, and visually checks the laser light L irradiated to each operation part, thereby grasping the position of each operation part on the actual image forming device 10.
[0072] In this embodiment, the captured image Qb captured by the imaging camera 35 is displayed on the screen of the display unit 31. When the captured image Qb includes the image Qs of the operation part, the moving direction from the center position p of the screen of the display unit 31 to the center position q of the image Qs of the operation part is determined and displayed on the screen of the display unit 31. Then, when the user changes the orientation of the imaging camera 35 to the moving direction and the center position q of the image Qs of the operation part captured by the imaging camera 35 enters the target range M around the center position p of the screen of the display unit 31, the laser light L of the light emitting unit 36 is emitted to the operation part, and operation guide information for solving the error is displayed on the display unit 31. This allows the user to reliably and quickly know the operation part of the image forming apparatus 10 irradiated with the laser light L, and to properly perform the operation for solving the error by looking at the operation guide information displayed on the display unit 31.
[0073] The mobile terminal device 30 may further be provided with a notification unit consisting of a speaker or a vibrator mechanism that generates sound or vibration. When the control unit 39 of the mobile terminal device 30 causes the light emitting unit 36 to emit the laser light L to the operation site, the control unit 39 activates the notification unit to generate sound or vibration. The user senses the sound or vibration generated by the notification unit and knows that the laser light L has been irradiated to the operation site.
[0074] The mobile terminal device 30 may be provided with a projection device that projects operation guide information consisting of text, illustrations, and video around the area TP of the image forming device 10. When the control unit 39 of the mobile terminal device 30 causes the light emitting unit 36 to emit the laser light L to the operation part, the control unit 39 reads out the operation guide information associated with the operation part from the data table DT2 shown in FIG. 6, displays the operation guide information on the screen of the display unit 31, and projects the operation guide information from the projection device to the periphery of the operation part of the image forming device 10. The user can check the operation part to which the laser light L is irradiated, and can view the projected operation guide information directly around the operation part. The control unit 39 may also cause the projection device to project an image showing an arrow mark or the like instead of the operation guide information.
[0075] 7 to irradiate the laser light L from the light emitting unit 36 to the operation portion of the image forming apparatus 10 and to display operation guide information for eliminating an error on the display unit 31, the control unit 29 of the image forming apparatus 10 may perform the process that is handled by the control unit 39 of the mobile terminal device 30. In this case, the control unit 39 is not provided in the mobile terminal device 30, or the control unit 39 performs the process of operating each mechanism according to an instruction from the control unit 29 of the image forming apparatus 10.
[0076] When this configuration is adopted, the display unit 21 of the image forming device 10 can be configured to be easily removed from the main body of the image forming device 10, and the display unit 21 can be equipped with a touch panel 33 (which may be touch panel 23), a short-range communication unit 34, an imaging camera 35, and a light emission unit 36, and if necessary, a memory unit 37 and a control unit 38, so that the display unit 21 can function in the same way as the mobile terminal device 30.
[0077] Moreover, the configuration of the above embodiment described with reference to FIGS. 1 to 11 is merely one example of the present invention, and is not intended to limit the present invention to this configuration. [Explanation of symbols]
[0078] 10 Image forming device 12 Image reading unit 11 Image forming section 21, 31 Display section 22 Control section 23, 33 Touch panel 24, 34 Near field communication department 25 Image Memory 26 Multiple Sensors 27, 37 Memory section 28, 38 Control unit 29, 39 Control section 30 Portable terminal device 35 Imaging camera 36 Light output section
Claims
1. A main unit; A portable terminal device detachably provided on the main body device, The main body device includes: A first communication unit that performs data communication with the mobile terminal device; a first control unit that determines an operation part of the main body device and causes the first communication unit to transmit an image of the operation part of the main body device to the mobile terminal device; The mobile terminal device the second communication unit performing data communication with the main body device; An imaging unit; a display unit that displays an image captured by the imaging unit; a light emitting unit that emits light to an imaging range of the imaging unit; and a second control unit that, when an image of the operation part of the main body device is received by the second communication unit, determines whether or not the image of the operation part of the main body device is included in the captured image captured by the imaging unit, and, if it determines that the image of the operation part of the main body device is included in the captured image captured by the imaging unit, emits light from the light emitting unit to the operation part of the main body device.
2. The second control unit is 2. The operation guidance system according to claim 1, wherein, when an image of the operation part of the main body device is received by the second communication unit, it is determined whether or not the image of the operation part of the main body device is included in the captured image captured by the imaging unit, and if the image of the operation part of the main body device is included in the captured image captured by the imaging unit, a moving direction from a center of the screen of the display unit to the image of the operation part of the main body device is determined and the determined moving direction is displayed on the display unit, and when the image of the operation part of the main body device is moved to the center of the screen of the display unit by changing the imaging range of the imaging unit in the moving direction, light is emitted from the light emitting unit to the operation part of the main body device.
3. a first control unit of the main body device sets a plurality of operation parts of the main body device and a display order of the operation parts, and transmits the operation parts of the main body device and the display order of the operation parts from the first communication unit to the mobile terminal device; 3. The operation guidance system for a device according to claim 2, wherein, when each operation part of the main body device and a display order of the each operation part are received by the second communication unit, the second control unit of the portable terminal device sequentially selects each operation part of the main body device in the display order of the each operation part, and each time an operation part of the main body device is selected, determines whether or not an image of the selected operation part of the main body device is included in the captured image captured by the imaging unit, and if the image of the operation part of the main body device is included in the captured image captured by the imaging unit, determines a moving direction from the center of the screen of the display unit to the image of the operation part of the main body device and displays the determined moving direction on the display unit, and when the image of the operation part of the main body device is moved to the center of the screen of the display unit by changing the imaging range of the imaging unit in the moving direction, emits light from the light emission unit to the operation part of the main body device.
4. a storage unit of the mobile terminal device stores operation guide information indicating operations that are preset in association with operation parts of the main body device; The second control unit of the mobile terminal device, when an image of the operation part of the main body device is moved to the center of the screen of the display unit, emits light from the light emitting unit to the operation part of the main body device and displays operation guidance information corresponding to the operation part stored in the memory unit on the display unit.
5. the mobile terminal device includes a projection unit that projects the operation guide information, a storage unit of the mobile terminal device stores operation guide information indicating operations that are preset in association with operation parts of the main body device; The second control unit of the mobile terminal device, when an image of the operation part of the main body device is moved to the center of the screen of the display unit, emits light from the light emitting unit to the operation part of the main body device and projects operation guide information corresponding to the operation part stored in the memory unit by the projection unit.
6. A notification unit that generates sound or vibration is provided, The second control unit of the mobile terminal device emits light from the light emitting unit to the operation part of the main body device and generates sound or vibration by the alarm unit when the image of the operation part of the main body device is moved to the center of the screen of the display unit.
7. A main unit; A portable terminal device detachably provided on the main body device, The main body device includes: A first communication unit that performs data communication with the mobile terminal device; a first control unit that determines an operation part of the main body device and causes the first communication unit to transmit an image of the operation part of the main body device to the mobile terminal device; The mobile terminal device the second communication unit performing data communication with the main body device; An imaging unit; a display unit that displays an image captured by the imaging unit; a light output unit that outputs light to an imaging range of the imaging unit, The first control unit of the main body device determines whether an image of the operation part of the main body device is included in an image captured by the imaging unit, and when it determines that the image of the operation part of the main body device is included in the image captured by the imaging unit, emits light from the light emitting unit to the operation part of the main body device.
8. the main body device is an image forming device including an image reading unit that reads an image of an original document, and an image forming unit that forms the image of the original document read by the image reading unit on a recording sheet; 8. The apparatus operation guidance system according to claim 1, wherein the portable terminal device is detachably attached to the image forming apparatus.
Citation Information
Patent Citations
Image forming device
JP2009143141A
Image forming apparatus, method for controlling the same, and image display apparatus
JP2012096448A
