Work auxiliary system and work auxiliary method

Abstract

To provide a work auxiliary system and a work auxiliary method, capable of guiding a hand of a worker to a work object of an electronic apparatus.SOLUTION: A work auxiliary system 100 comprises: a camera 2 for imaging a picture processing device 1 and a worker performing work to the picture processing device 1; an output processing section for outputting guidance information for guiding a work content for part of the picture processing device 1; a determination processing section that when the guidance information is output by the output processing section, determines a movement destination of a hand of the worker in the picture processing device 1 on the basis of an imaged picture imaged by the camera 2; and a notification processing section that when the movement destination of the hand determined by the determination processing section is not a work object corresponding to the guidance information output by the output processing section, performs notification of the situation using voice.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present invention relates to a work assistance system and a work assistance method. [Background technology]

[0002] In electronic devices such as multifunction peripherals, if an abnormality such as a paper jam occurs while the electronic device is operating, multiple guide screens corresponding to multiple tasks to resolve the abnormality may be displayed sequentially (see, for example, Patent Document 1). [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2019-186850 Summary of the Invention [Problem to be solved by the invention]

[0004] However, a worker who is unfamiliar with using the electronic device may not be able to identify which part of the electronic device is the target of the work even when looking at the guidance screen. However, there has not been a conventional configuration that allows the worker's hand to be guided to the target of the work on the electronic device.

[0005] An object of the present invention is to provide a work assistance system and a work assistance method that can guide a worker's hand to a work target in an electronic device. [Means for solving the problem]

[0006] According to one aspect of the present invention, a work assistance system includes a camera, an output processing unit, a determination processing unit, and a notification processing unit. The camera captures an image of an electronic device and a worker performing work on the electronic device. The output processing unit outputs guidance information that provides guidance on work content for a part of the electronic device. When the guidance information is output by the output processing unit, the determination processing unit determines a destination of the worker's hand on the electronic device based on an image captured by the camera. When the destination of the hand determined by the determination processing unit is not the work target corresponding to the guidance information output by the output processing unit, the notification processing unit notifies the worker by voice.

[0007] A work assistance method according to another aspect of the present invention is executed by a work assistance system including an electronic device and a camera that captures an image of a worker working on the electronic device, and includes an output step, a determination step, and a notification step. In the output step, guidance information that provides guidance on work content for a part of the electronic device is output. In the determination step, when the guidance information is output in the output step, a destination of the worker's hand on the electronic device is determined based on an image captured by the camera. In the notification step, if the destination of the hand determined in the determination step is not the work target corresponding to the guidance information output in the output step, this fact is notified by voice. [Effects of the Invention]

[0008] According to the present invention, it is possible to guide the worker's hand to a work target in an electronic device. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 1 is a block diagram showing the configuration of a work assistance system according to an embodiment of the present invention. [Figure 2] FIG. 2 is a block diagram showing the configuration of an image processing device of the work assistance system according to the embodiment of the present invention. [Figure 3]FIG. 3 is a perspective view showing the external configuration of the image processing device of the work assistance system according to the embodiment of the present invention. [Figure 4] FIG. 4 is a flowchart showing an example of work assistance processing executed by the work assistance system according to the embodiment of the present invention. [Figure 5] FIG. 5 is a flowchart showing an example of a reach assistance process executed by the work assistance system according to the embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, an embodiment of the present invention will be described with reference to the accompanying drawings. Note that the following embodiment is an example of a specific embodiment of the present invention and does not limit the technical scope of the present invention.

[0011] [Configuration of work assistance system 100] First, with reference to FIG. 1, the configuration of a work assistance system 100 according to an embodiment of the present invention will be described.

[0012] 1, the work assistance system 100 includes an image processing device 1 and multiple cameras 2 (2A, 2B). In the work assistance system 100, the image processing device 1 and the cameras 2 are connected to each other so as to be able to communicate with each other via a communication network 3. For example, the communication network 3 is the Internet or a LAN (local area network).

[0013] [Configuration of image processing device 1] Next, the configuration of the image processing device 1 will be described with reference to FIGS.

[0014] For ease of explanation, the vertical direction in the installation state where the image processing device 1 is usable (the state shown in FIG. 3) is defined as the up-down direction D1. Also, the front-to-back direction D2 is defined with the face on the left front side of the paper of the image processing device 1 shown in FIG. 3 as the front (front face). Also, the left-to-right direction D3 is defined with the front face of the image processing device 1 in the installation state as the reference point.

[0015] The image processing device 1 is a multifunction device that has multiple functions, such as a scanning function for reading an image from a document, a printing function for forming an image on a sheet based on image data, a fax function, and a copy function. The image processing device 1 is an example of an electronic device of the present invention. The electronic device of the present invention may also be a printer, a copy machine, a washing machine, a refrigerator, a vending machine, an ATM (automated teller machine), or the like.

[0016] As shown in FIG. 2, the image processing device 1 includes a control unit 11, an ADF (automatic document feeder) 12, an image reading unit 13, an image forming unit 14, a paper feeding unit 15, an operation display unit 16, a communication unit 17, a memory unit 18, and an audio input / output unit 19.

[0017] The control unit 11 comprehensively controls the image processing device 1. As shown in FIG. 2, the control unit 11 includes a CPU 21, a ROM 22, and a RAM 23. The CPU 21 is a processor that executes various types of arithmetic processing. The ROM 22 is a non-volatile storage device that stores in advance information such as control programs for causing the CPU 21 to execute various types of processing. The RAM 23 is a volatile or non-volatile storage device that is used as a temporary storage memory (work area) for the various types of processing executed by the CPU 21. The CPU 21 executes the various control programs pre-stored in the ROM 22. In this way, the CPU 21 comprehensively controls the image processing device 1.

[0018] The ADF 12 transports a document whose image is to be read by the image reading unit 13. The ADF 12 includes a document setting unit, a plurality of transport rollers, a document holder, and a paper discharge unit.

[0019] The image reading unit 13 realizes the scanning function and includes a document table, a light source, a plurality of mirrors, an optical lens, and a CCD.

[0020] The image forming unit 14 realizes the printing function. For example, the image forming unit 14 forms a color or monochrome image by electrophotography on a sheet supplied by the paper feed unit 15. For example, the image forming unit 14 includes a plurality of image forming units, an optical scanning device, an intermediate transfer device, a fixing device, and a plurality of toner containers.

[0021] The image forming unit 14 also includes a front cover 31 and a right side cover 32 shown in FIG.

[0022] The front cover 31 is provided so as to be able to open and close a first opening 31A (see FIG. 3) formed in the front of the housing of the image processing device 1. The components of the image forming unit 14 are housed behind the first opening 31A in the housing of the image processing device 1. When the front cover 31 is opened, the components of the image forming unit 14 housed in the housing of the image processing device 1 are exposed to the outside through the first opening 31A. For example, the front cover 31 is opened and closed when the toner container is replaced.

[0023] The right side cover 32 is provided so as to be able to open and close a second opening 32A (see FIG. 3) formed on the right side of the housing of the image processing device 1. A first transport guide is disposed at the back of the second opening 32A in the housing of the image processing device 1, guiding the transport of a sheet on which an image is formed by the image forming unit 14. When the right side cover 32 is disposed in a closed position that closes the second opening 32A, a sheet transport path is formed between the right side cover 32 and the first transport guide. In other words, the right side cover 32 functions as a second transport guide that guides the transport of a sheet. When the right side cover 32 is opened, the first transport guide disposed inside the housing of the image processing device 1 is exposed to the outside through the second opening 32A. The right side cover 32 is opened and closed when a sheet jammed inside the housing of the image processing device 1 is to be removed.

[0024] The paper feed unit 15 supplies sheets to the image forming unit 14. The paper feed unit 15 includes a paper feed cassette 33 shown in Fig. 3. The paper feed unit 15 also includes a plurality of transport rollers used to transport the sheets.

[0025] 3, the paper feed cassette 33 is provided at the bottom of the housing of the image processing device 1. The paper feed cassette 33 is provided so as to be removable in the front-rear direction D2 relative to the housing of the image processing device 1. The paper feed cassette 33 stores sheets on which images are to be formed by the image forming unit 14.

[0026] The operation display unit 16 is a user interface of the image processing device 1. The operation display unit 16 includes a display unit and an operation unit. The display unit displays various information in response to control instructions from the control unit 11. For example, the display unit is a flat panel display such as a liquid crystal display. The operation unit inputs various information to the control unit 11 in response to user operations. For example, the operation unit includes operation keys and a touch panel.

[0027] The communication unit 17 is a communication interface that executes wired or wireless data communication via the communication network 3 with an external communication device such as the camera 2.

[0028] The storage unit 18 is a nonvolatile storage device, such as a nonvolatile memory such as a flash memory, a solid state drive (SSD), or a hard disk drive (HDD).

[0029] The audio input / output unit 19 includes a microphone used to input audio to the image processing device 1 and a speaker used to output audio from the image processing device 1.

[0030] [Configuration of cameras 2A and 2B] Next, the configuration of cameras 2A and 2B will be described.

[0031] Each camera 2 is used to capture an image of the image processing device 1 and a worker performing work on the image processing device 1.

[0032] For example, each of the cameras 2 is a network camera. Each of the cameras 2 captures an image of a subject at a predetermined interval and transmits the captured images to the image processing device 1 in sequence.

[0033] Specifically, camera 2A is provided so as to be able to capture images of the front of image processing device 1 and the worker performing work on image processing device 1 in front of image processing device 1. Furthermore, camera 2A is provided so as to be able to capture images of each part of image forming unit 14 exposed to the outside of image processing device 1 through first opening 31A when front cover 31 is open. For example, camera 2A is provided on the ceiling of a room in which image processing device 1 is installed. Furthermore, camera 2A is provided diagonally above the right side of image processing device 1, so as to be able to capture images of front cover 31 and the worker's hands from a position in front of image processing device 1.

[0034] Furthermore, camera 2B is provided so as to be able to capture an image of the right side surface of image processing device 1 and the worker working on image processing device 1 on the right side of image processing device 1. Furthermore, camera 2B is provided so as to be able to capture an image of the first conveying guide exposed to the outside of image processing device 1 through second opening 32A when right side cover 32 is opened. For example, camera 2B is provided on the ceiling of a room in which image processing device 1 is installed. Furthermore, camera 2B is provided so as to be able to capture an image of right side cover 32 and the worker's hands from a position diagonally above and in front of image processing device 1 and to the right of image processing device 1.

[0035] The work assistance system 100 may include three or more cameras 2.

[0036] [Configuration of control unit 11] Next, the configuration of the control unit 11 will be described with reference to FIG.

[0037] As shown in FIG. 2, the control unit 11 includes a detection processing unit 41, an output processing unit 42, a determination processing unit 43, a notification processing unit 44, a recording processing unit 45, and an evaluation processing unit 46.

[0038] Specifically, a work assistance program for causing the control unit 11 to function as each of the above-mentioned functional units is stored in advance in the storage unit 18 of the image processing device 1. The CPU 21 of the control unit 11 executes the work assistance program to function as each of the above-mentioned functional units.

[0039] Note that some or all of the functional units included in the control unit 11 may be configured with electronic circuits. Also, the work assistance program may be a program for causing a plurality of processors to function as the functional units shown in FIG.

[0040] The detection processing unit 41 detects abnormalities in the image processing device 1.

[0041] For example, the abnormalities include a toner shortage in which the amount of toner contained in any of the toner containers falls below a predetermined reference value, a paper jam in which a sheet gets stuck in the sheet transport path, and a sheet shortage in which there are no sheets contained in the paper feed cassette 33.

[0042] The detection processing unit 41 may detect the toner shortage, paper jam, and sheet shortage using a conventionally known method.

[0043] The output processing unit 42 outputs guidance information that provides guidance on the work content for a part of the image processing device 1. The guidance information is information that includes a message indicating the work content for a part (work target) of the image processing device 1, such as "Please open the front cover 31."

[0044] For example, in the image processing device 1, work procedure data corresponding to each of the abnormalities is stored in advance in the storage unit 18. The work procedure data includes one or more pieces of the guidance information. Each piece of the guidance information included in the work procedure data is assigned a sequence number indicating the work sequence.

[0045] When the abnormality is detected by the detection processing unit 41, the output processing unit 42 sequentially outputs one or more pieces of guidance information included in the work procedure data corresponding to the abnormality. Specifically, when the abnormality is detected by the detection processing unit 41, the output processing unit 42 reads out the work procedure data corresponding to the detected abnormality from the storage unit 18. Then, the output processing unit 42 sequentially outputs each piece of guidance information included in the read work procedure data in accordance with the sequence number assigned to the guidance information.

[0046] For example, when outputting the first guidance information, the output processing unit 42 displays a guidance screen including the guidance information on the operation display unit 16. For example, the guidance screen includes an image showing the position of the work object corresponding to the guidance information in the image processing device 1. Note that the guidance screen may include a video playback field in which a video showing the work content is played.

[0047] After outputting the first piece of guidance information, the output processing unit 42 outputs the next piece of guidance information in response to reception of a predetermined specific operation.

[0048] For example, the specific operation includes a first operation on the operation display unit 16 and a second operation by voice. For example, the first operation is an operation on a predetermined operation key included in the guidance screen. Also, the second operation is an operation of inputting a voice including a predetermined word into the voice input / output unit 19.

[0049] Then, when the first operation is accepted, the output processing unit 42 displays the next guidance information on the operation display unit 16, and when the second operation is accepted, outputs the next guidance information by voice. Specifically, when the first operation is accepted, the output processing unit 42 displays the guidance screen including the next guidance information on the operation display unit 16. Furthermore, when the second operation is accepted, the output processing unit 42 outputs first voice data including the next guidance information using the voice input / output unit 19. This allows the worker to select the output method of the guidance information.

[0050] In addition, the output processing unit 42 may determine whether the task corresponding to the guidance information last output has been completed based on the captured image captured by the camera 2, and output the next guidance information if it is determined that the task has been completed.

[0051] However, the worker who is unfamiliar with using the image processing device 1 may not be able to identify which part of the image processing device 1 is the target of the work even when looking at the guidance screen. However, conventionally, there has been no configuration that allows the worker's hand to be guided to the target of the work on the image processing device 1.

[0052] In contrast to this, the work assistance system 100 according to the embodiment of the present invention can guide the worker's hand to the work target in the image processing device 1, as will be described below.

[0053] When the guidance information is output by the output processing unit 42, the determination processing unit 43 determines the destination of the worker's hand in the image processing device 1 based on the captured image captured by the camera 2.

[0054] For example, when the detection processing unit 41 detects that the toner is out, the judgment processing unit 43 determines the destination of the worker's hand in the image processing device 1 based on the captured image taken by the camera 2A that can capture the front cover 31 (see Figure 3).

[0055] In addition, when the detection processing unit 41 detects the paper jam, the judgment processing unit 43 determines the destination of the operator's hand in the image processing device 1 based on the captured image taken by the camera 2B capable of capturing an image of the right side cover 32 (see Figure 3).

[0056] In addition, when the detection processing unit 41 detects that the sheet is out, the judgment processing unit 43 determines the destination of the operator's hand in the image processing device 1 based on the captured image taken by the camera 2A capable of capturing an image of the paper feed cassette 33 (see Figure 3).

[0057] For example, the determination processing unit 43 detects a human hand included in the captured image transmitted from the camera 2. When a human hand is detected in the captured image transmitted from the camera 2, the determination processing unit 43 determines the movement direction of the detected hand based on the plurality of captured images transmitted consecutively from the camera 2. Then, based on the determination result of the movement direction of the detected hand, the determination processing unit 43 determines to which of the components of the image processing device 1 included in the captured image the detected hand has moved. Note that the determination processing unit 43 may detect the human hand included in the captured image using a first sample image showing a human hand stored in advance in the storage unit 18. Furthermore, the determination processing unit 43 may identify the components of the image processing device 1 included in the captured image using second sample images showing each of the components of the image processing device 1 stored in advance in the storage unit 18.

[0058] Each time the output processing unit 42 outputs the guidance information, the determination processing unit 43 determines the destination of the worker's hand in the image processing device 1 based on the captured image.

[0059] If the destination of the worker's hand determined by the judgment processing unit 43 is not the work object corresponding to the guidance information output by the output processing unit 42, the notification processing unit 44 notifies this fact by voice.

[0060] For example, the notification processing unit 44 uses the voice input / output unit 19 to output second voice data including a message indicating that the worker's hand is moving toward a component different from the work target.

[0061] Furthermore, when the destination of the worker's hand determined by the determination processing unit 43 is a predetermined contact prohibited portion in the image processing device 1, the notification processing unit 44 notifies the fact by voice.

[0062] For example, the contact prevention unit is the fixing device of the image forming unit 14.

[0063] For example, the notification processing unit 44 uses the voice input / output unit 19 to output third voice data including a message that the worker's hand is moving toward the contact-prohibited portion.

[0064] The contact prevention portion is not limited to the fixing device, and may be any portion that may cause injury, burns, or other damage to the worker if the worker comes into contact with it. The contact prevention portion may also be any portion that may cause a malfunction or failure if the worker comes into contact with it.

[0065] Furthermore, when the destination of the worker's hand determined by the determination processing unit 43 is the work object corresponding to the guidance information output by the output processing unit 42, and the worker's hand is one of the left and right hands and the hand associated with the work object is the other of the left and right hands, the notification processing unit 44 may notify this by voice. Furthermore, when the destination of the worker's hand determined by the determination processing unit 43 is the work object corresponding to the guidance information output by the output processing unit 42, and the orientation of the worker's hand (upward, downward, leftward, rightward) is different from the orientation associated with the work object, the notification processing unit 44 may notify this by voice.

[0066] The record processing unit 45 records the progress of the work performed by the worker.

[0067] For example, when the specific operation is accepted, the recording processing unit 45 stores in the storage unit 18 first work log data indicating the completion of the work corresponding to the guidance information that was last output. In addition, when a predetermined failed operation on the operation display unit 16 is accepted, the recording processing unit 45 stores in the storage unit 18 second work log data indicating a failure of the work corresponding to the guidance information that was last output.

[0068] The control unit 11 causes the operation display unit 16 to display the first work log data and the second work log data stored in the memory unit 18 in response to a predetermined operation on the operation display unit 16. This allows the successor worker to understand the progress of the work performed by the previous worker when the worker is replaced.

[0069] The evaluation processing unit 46 evaluates the work performed by the worker.

[0070] For example, when the specific operation is accepted, the evaluation processing unit 46 evaluates the operation corresponding to the guidance information that was last output.

[0071] For example, when the specific operation is accepted, the evaluation processing unit 46 acquires the elapsed time since the last output of the guidance information. Furthermore, when the specific operation is accepted, the evaluation processing unit 46 acquires the number of times the second voice data has been output since the last output of the guidance information. Furthermore, when the specific operation is accepted, the evaluation processing unit 46 acquires the number of times the third voice data has been output since the last output of the guidance information.

[0072] Then, the evaluation processing unit 46 acquires an evaluation score for the task corresponding to the guidance information based on the acquired elapsed time, the number of times the second voice data is output, and the number of times the third voice data is output. For example, the evaluation processing unit 46 calculates the evaluation score using a predetermined calculation formula. Here, the calculation formula may be set so that the shorter the acquired elapsed time, the higher the evaluation score. Furthermore, the calculation formula may be set so that the fewer times the acquired second voice data is output, the higher the evaluation score. Furthermore, the calculation formula may be set so that the fewer times the acquired third voice data is output, the higher the evaluation score.

[0073] When a series of tasks corresponding to the plurality of pieces of guidance information outputted sequentially is completed, the evaluation processing unit 46 adds up the evaluation points for each task to obtain a total evaluation point corresponding to the series of tasks. Then, the evaluation processing unit 46 displays the obtained total evaluation point on the operation display unit 16. This allows the worker to check the evaluation of the tasks he or she has performed.

[0074] The evaluation processing unit 46 may store the acquired overall evaluation score in a memory area allocated to a user logged in to the image processing device 1. Furthermore, the control unit 11 may cause the operation display unit 16 to display the overall evaluation score stored in the memory area corresponding to the user logged in to the image processing device 1 in response to a predetermined operation by the user. Furthermore, the control unit 11 may consume a portion of the points acquired by the user logged in to the image processing device 1 (the total value of the overall evaluation scores stored in the memory area corresponding to the user) to issue a voucher to be used in exchange for a prize corresponding to the consumed points. For example, the control unit 11 may print the voucher using the image forming unit 14.

[0075] [Work support processing] 4, the work assistance method of the present invention will be described below along with an example of the procedure of the work assistance process executed by the control unit 11 of the image processing device 1 in the work assistance system 100. Here, steps S11, S12, etc. represent the numbers of the processing procedures (steps) executed by the control unit 11. For example, the work assistance process is executed when the detection processing unit 41 detects the abnormality.

[0076] <Step S11> First, in step S11, the control unit 11 activates one of the two cameras 2.

[0077] Specifically, the control unit 11 activates the camera 2A when the detection processing unit 41 detects that the toner has run out. In addition, the control unit 11 activates the camera 2B when the detection processing unit 41 detects that the paper jam has occurred. In addition, the control unit 11 activates the camera 2A when the detection processing unit 41 detects that the sheet has run out.

[0078] <Step S12> In step S12, the control unit 11 outputs the guidance information to which the sequence number corresponding to the number of times the processing of step S12 has been executed has been assigned, from among one or more pieces of guidance information included in the work procedure data corresponding to the abnormality detected by the detection processing unit 41. The processing of step S12 is an example of an output step of the present invention, and is executed by the output processing unit 42 of the control unit 11.

[0079] Specifically, when outputting the first piece of guidance information, the control unit 11 causes the operation display unit 16 to display the guidance screen including the guidance information.

[0080] In addition, when outputting the second or subsequent guidance information, if the specific operation last performed is the first operation, the control unit 11 causes the operation display unit 16 to display the guidance screen including the guidance information.

[0081] In addition, when outputting the second or subsequent guidance information, if the last specific operation performed is the second operation, the control unit 11 outputs the first voice data including the guidance information using the voice input / output unit 19.

[0082] <Step S13> In step S13, the control unit 11 executes the arrival assistance process described below.

[0083] [Reaching assistance processing] Here, an example of the procedure of the arrival assistance process executed in step S13 of the task assistance process will be described with reference to FIG.

[0084] <Step S21> First, in step S21, the control unit 11 determines whether or not a human hand has been detected from the captured image transmitted from the camera 2 that is running.

[0085] Here, when the control unit 11 determines that a human hand has been detected in the captured image transmitted from the camera 2 (Yes in S21), it shifts the processing to step S22. On the other hand, when a human hand has not been detected in the captured image transmitted from the camera 2 (No in S21), the control unit 11 waits for a human hand to be detected in the captured image transmitted from the camera 2 in step S21.

[0086] <Step S22> In step S22, the control unit 11 determines the destination of the person's hand detected from the captured image. The processes of steps S21 and S22 are an example of a determination step of the present invention, and are executed by the determination processing unit 43 of the control unit 11.

[0087] Specifically, the control unit 11 determines the movement direction of the hand detected from the captured images based on the plurality of captured images continuously transmitted from the activated camera 2. Then, based on the determination result of the movement direction of the detected hand, the control unit 11 determines to which of the components of the image processing device 1 included in the captured images the detected hand has moved.

[0088] <Step S23> In step S23, the control unit 11 branches the process depending on whether the determination result of the process in step S22 is the contact prohibition unit. Specifically, if the determination result of the process in step S22 is the contact prohibition unit (Yes in S23), the control unit 11 shifts the process to step S24. On the other hand, if the determination result of the process in step S22 is not the contact prohibition unit (No in S23), the control unit 11 shifts the process to step S25.

[0089] <Step S24> In step S24, the control unit 11 notifies by voice that the determined destination of the worker's hand is the contact prohibited portion. The process of step S24 is executed by the notification processing unit 44 of the control unit 11.

[0090] Specifically, the control unit 11 uses the voice input / output unit 19 to output the third voice data including a message that the worker's hand is moving toward the contact-prohibited portion.

[0091] <Step S25> In step S25, the control unit 11 branches the process depending on whether the determination result of the process in step S22 is the work target. Specifically, if the determination result of the process in step S22 is the work target (Yes in S25), the control unit 11 shifts the process to step S27. On the other hand, if the determination result of the process in step S22 is not the work target (No in S25), the control unit 11 shifts the process to step S26.

[0092] <Step S26> In step S26, the control unit 11 notifies by voice that the determined destination of the worker's hand is not the work target. The process of step S26 is an example of a notification step of the present invention, and is executed by the notification processing unit 44 of the control unit 11.

[0093] Specifically, the control unit 11 uses the voice input / output unit 19 to output the second voice data including a message that the worker's hand is moving toward a component different from the work target.

[0094] <Step S27> In step S27, the control unit 11 determines, based on the captured image transmitted from the camera 2 that is running, whether or not the worker's hand has reached the work object.

[0095] Here, if the control unit 11 determines that the worker's hand has reached the work object (Yes in S27), it ends the reach assistance process. On the other hand, if the worker's hand has not reached the work object (No in S27), the control unit 11 shifts the process to step S22.

[0096] This concludes the description of the arrival assistance process, and the description of the work assistance process from step S14 onwards will be resumed.

[0097] <Step S14> In step S14, the control unit 11 determines whether or not the specific operation has been accepted.

[0098] Here, if the control unit 11 determines that the specific operation has been accepted (Yes in S14), it shifts the process to step S15. On the other hand, if the specific operation has not been accepted (No in S14), the control unit 11 waits for acceptance of the specific operation in step S14.

[0099] <Step S15> In step S15, the control unit 11 records the completion of the task corresponding to the guidance information that was last output. The process of step S15 is executed by the record processing unit 45 of the control unit 11.

[0100] Specifically, the control unit 11 stores the first work log data indicating the completion of the work corresponding to the guidance information that was last output in the storage unit 18. Note that, when the failed operation is accepted instead of the specific operation, the control unit 11 may store the second work log data indicating the failure of the work corresponding to the guidance information that was last output in the storage unit 18.

[0101] <Step S16> In step S16, the control unit 11 evaluates the task corresponding to the guidance information that was last output. The process of step S16 is executed by the evaluation processing unit 46 of the control unit 11.

[0102] Specifically, the control unit 11 acquires the elapsed time since the last output of the guidance information. The control unit 11 also acquires the number of times the second voice data has been output since the last output of the guidance information. The control unit 11 also acquires the number of times the third voice data has been output since the last output of the guidance information. The control unit 11 then substitutes the acquired elapsed time, the number of times the second voice data has been output, and the number of times the third voice data has been output into the calculation formula to calculate the evaluation score for the task corresponding to the last output of the guidance information.

[0103] <Step S17> In step S17, the control unit 11 determines whether all of the guidance information included in the work procedure data corresponding to the abnormality detected by the detection processing unit 41 has been output.

[0104] Here, if the control unit 11 determines that all of the guidance information has been output (Yes in S17), it shifts the process to step S18. On the other hand, if all of the guidance information has not been output (No in S17), the control unit 11 shifts the process to step S12.

[0105] <Step S18> In step S18, the control unit 11 stops the camera 2 that is running.

[0106] <Step S19> In step S19, the control unit 11 adds up the evaluation points obtained in the processing of step S16 to obtain the overall evaluation point corresponding to the series of tasks, and displays the obtained overall evaluation point on the operation display unit 16. The processing of step S19 is executed by the evaluation processing unit 46 of the control unit 11.

[0107] In this way, in the work assistance system 100, when the guidance information is output, the destination of the worker's hand is determined, and if the determined destination of the worker's hand is not the work object corresponding to the output guidance information, a voice notification to that effect is given. This makes it possible to guide the worker's hand to the work object in the image processing device 1.

[0108] The image processing device 1 may include one or more cameras 2. In this case, the image processing device 1 is an example of the task assistance system of the present invention.

[0109] [Notes on the Invention] The following is a summary of the invention extracted from the above-described embodiment. Note that the configurations and processing functions described in the following supplementary notes can be selected and combined as desired.

[0110] <Appendix 1> a determination processing unit that, when the guidance information is output by the output processing unit, determines a destination of the worker's hand on the electronic device based on an image captured by the camera; and a notification processing unit that, when the destination of the hand determined by the determination processing unit is not a work target corresponding to the guidance information output by the output processing unit, notifies the worker by voice.

[0111] <Appendix 2> 2. The work assistance system according to claim 1, wherein the notification processing unit notifies the user by voice when the destination of the hand determined by the determination processing unit is a predetermined contact-prohibited portion of the electronic device.

[0112] <Appendix 3> the output processing unit sequentially outputs one or more pieces of guidance information corresponding to the abnormality when the abnormality is detected by the detection processing unit; and the determination processing unit determines a destination of the hand in the electronic device based on the captured image every time the output processing unit outputs the guidance information.

[0113] <Appendix 4> the electronic device includes an operation display unit, and the output processing unit outputs the next guidance information in response to acceptance of a predetermined specific operation, the specific operation including a first operation on the operation display unit and a second operation by voice, and the output processing unit displays the next guidance information on the operation display unit when the first operation is accepted, and outputs the next guidance information by voice when the second operation is accepted.

[0114] <Appendix 5> A work assistance method executed by a work assistance system including an electronic device and a camera that captures an image of a worker working on the electronic device, the work assistance method including: an output step of outputting guidance information that provides guidance on work content for a part of the electronic device; a determination step of determining a destination of the worker's hands on the electronic device based on an image captured by the camera when the guidance information is output by the output step; and a notification step of notifying by voice that the destination of the hands determined by the determination step is not a work target corresponding to the guidance information output by the output step. [Explanation of symbols]

[0115] 1. Image processing device 2 Cameras 3. Communication Network 11 Control section 12 ADF 13 Image reading unit 14 Image forming unit 15 Paper feed section 16 Operation display section 17 Communications Department 18 Memory section 19 Audio input / output section 31 Front cover 32 Right side cover 33 Paper cassette 41 Detection processing section 42 Output processing section 43 Judgment processing unit 44 Notification processing section 45 Recording processing section 46 Evaluation processing section 100 Work assistance system

Claims

1. a camera that captures an image of an electronic device and a worker performing work on the electronic device; an output processing unit that outputs guidance information that provides guidance on work content for a part of the electronic device; a determination processing unit that determines a destination of the worker's hand in the electronic device based on an image captured by the camera when the guidance information is output by the output processing unit; a notification processing unit that, when the destination of the hand determined by the determination processing unit is not the task target corresponding to the guidance information output by the output processing unit, notifies the user of this fact by voice; A work assistance system comprising:

2. the notification processing unit, when the destination of the hand determined by the determination processing unit is a predetermined contact prohibited portion of the electronic device, notifies the user of this by voice. The work assistance system according to claim 1 .

3. a detection processing unit that detects an abnormality in the electronic device; the output processing unit sequentially outputs, when the abnormality is detected by the detection processing unit, one or more pieces of guidance information corresponding to the abnormality; the determination processing unit determines a destination of the hand in the electronic device based on the captured image each time the guidance information is output by the output processing unit. The work assistance system according to claim 1 or 2.

4. the electronic device includes an operation display unit, the output processing unit outputs the next guidance information in response to reception of a predetermined specific operation; the specific operation includes a first operation on the operation display unit and a second operation by voice, the output processing unit displays the next guidance information on the operation display unit when the first operation is accepted, and outputs the next guidance information by voice when the second operation is accepted. The work assistance system according to claim 3 .

5. A work assistance method executed by a work assistance system including an electronic device and a camera that captures an image of a worker performing work on the electronic device, an output step of outputting guidance information that provides guidance on work content for a part of the electronic device; a determination step of determining a destination of the worker's hand in the electronic device based on an image captured by the camera when the guidance information is output by the output step; a notification step of notifying by voice, when the destination of the hand determined by the determination step is not the task object corresponding to the guidance information output by the output step; A work assistance method including:

Images