Operation training device, operation training method, and computer-readable storage medium

Abstract

This operation training device comprises: a first storage unit that stores a plurality of operation images; a second storage unit that stores association information associating one operation region included in a first operation image among the operation images stored in the first storage unit with a second operation image among the operation images; a speech recognition unit that recognizes speech information of speech being uttered; and a display unit that displays, when the speech recognition unit recognizes that the speech information specifies the second operation image, the first operation image while highlighting the one operation region, and then displays the second operation image associated with the one operation image.

Description

Technical Field

[0001] The present disclosure relates to an operation training device, an operation training method, and a computer-readable storage medium.

Background Art

[0002] Conventionally, a control device having a guide function for guiding an operation method of an industrial machine has been known (Patent Document 1). By using such a guide function, even an operator with little experience can appropriately operate the industrial machine.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] However, when using the guide function, the operator needs to perform the operation while referring to the guide information. Therefore, the operator cannot quickly perform the operation on the industrial machine. Under such circumstances, it is desired to train the operator so that the operator can quickly perform the operation on the industrial machine without using the guide function.

[0005] An object of the present disclosure is to provide an operation training device, an operation training method, and a computer-readable storage medium capable of effectively training an operation on an industrial machine.

Means for Solving the Problems

[0006] The operation training device includes a first storage unit that stores a plurality of operation images, a second storage unit that stores association information associating one operation area included in a first operation image among the plurality of operation images stored in the first storage unit with a second operation image among the plurality of operation images, a voice recognition unit that recognizes voice information of the spoken voice, and when the voice recognition unit recognizes that the voice information designates the second operation image, A specifying unit that specifies the order of image transitions from the operation image displayed on the display screen to the second operation image at the time of recognition by the voice recognition unit, and based on the order of screen transitions specified by the specifying unit, after the operation image displayed on the display screen, displays the first operation image while emphasizing one operation area causes the first operation image to be displayed after the first operation image is displayed, and a display unit that displays a second operation image associated with the first operation image.

[0007] The operation training method includes An operation training method performed by the device, storing a plurality of operation images, storing association information associating one operation area included in a first operation image among the plurality of stored operation images with a second operation image among the plurality of operation images, recognizing voice information of the spoken voice, and when the voice information is recognized as designating the second operation image, specifying the order of image transitions from the operation image to the second operation image at the time of recognition, and based on the specified order of screen transitions, after the displayed operation image, displaying the first operation image while emphasizing one operation area causes the first operation image to be displayed after the first operation image is displayed, and displaying a second operation image associated with the first operation image. It includes.

[0008] The computer-readable storage medium stores instructions for causing a computer to store a plurality of operation images, store association information associating one operation area included in a first operation image among the plurality of stored operation images with a second operation image among the plurality of operation images, recognize voice information of the spoken voice, and when the voice information is recognized as designating the second operation image, specifying the order of image transitions from the operation image to the second operation image at the time of recognition, and based on the specified order of screen transitions, after the displayed operation image, display the first operation image while emphasizing one operation area causes the first operation image to be displayed after the first operation image is displayed, and display a second operation image associated with the first operation image.

Effect of the Invention

[0009] According to one aspect of the present disclosure, it becomes possible to effectively perform training for operations on industrial machines.

Brief Description of the Drawings

[0010]

Figure 1

Figure 2

Figure 3

Figure 4A

Figure 4B

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Modes for Carrying Out the Invention

[0011] Hereinafter, an operation training device according to an embodiment of the present disclosure will be described with reference to the drawings. Note that not all combinations of the features described in the following embodiments are necessarily required to solve the problems. Also, there may be cases where more detailed explanations than necessary are omitted. Further, the following descriptions of the embodiments and the drawings are provided for those skilled in the art to fully understand the present disclosure and are not intended to limit the scope of the claims.

[0012] The operation training device is a device used for training an operator to learn the operation of industrial machines. The industrial machine is a concept including a numerical control device and a teaching pendant. That is, the operation training device can also be said to be a device used for training to learn the operation of a numerical control device and a teaching pendant. Here, the numerical control device is a device that executes numerical control of a machine tool. The teaching pendant is a device for teaching the operation of an industrial robot such as a manipulator.

[0013] FIG. 1 is a block diagram showing an example of the hardware configuration of the operation training device. The operation training device 1 is implemented, for example, in a PC (Personal Computer) and a portable terminal. The portable terminal is, for example, a tablet and a smartphone. The operation training device 1 may be implemented in a numerical control device and a teaching pendant.

[0014] The operation training device 1 includes, for example, a hardware processor 11, a bus 12, a ROM (Read Only Memory) 13, a RAM (Random Access Memory) 14, a non-volatile memory 15, an interface 16, and an input / output device 17.

[0015] The hardware processor 11 is a processor that controls the entire operation training device 1 according to a system program. The hardware processor 11 reads out a system program stored in the ROM 13 via the bus 12 and performs various processes based on the system program. The hardware processor 11 is, for example, a CPU (Central Processing Unit).

[0016] The bus 12 is a communication path that connects each piece of hardware in the operation training device 1 to each other. Each piece of hardware in the operation training device 1 exchanges data via the bus 12.

[0017] The ROM 13 is a storage device that stores a system program for controlling the entire operation training device 1 and the like. The ROM 13 is a computer-readable storage medium.

[0018] The RAM 14 is a storage device that temporarily stores various data. The RAM 14 functions as a work area for the hardware processor 11 to process various data.

[0019] The non-volatile memory 15 is a storage device that retains data even when the power of the operation training device 1 is turned off and no power is supplied to the operation training device 1. The non-volatile memory 15 stores, for example, image data of a plurality of operation images. The non-volatile memory 15 is composed of, for example, an SSD (Solid State Drive). The non-volatile memory 15 is a computer-readable storage medium.

[0020] The interface 16 connects the bus 12 and the input / output device 17. The interface 16 sends various data processed by the hardware processor 11 to the input / output device 17, for example.

[0021] The input / output device 17 is a device that receives various data via the interface 16 and displays various data. Further, the input / output device 17 receives the input of various data and sends the various data to the hardware processor 11 via the interface 16. The input / output device 17 is, for example, a touch panel. When the input / output device 17 is a touch panel, the touch panel is, for example, a capacitive touch panel. Note that the touch panel is not limited to the capacitive method and may be a touch panel of another method.

[0022] Next, the functions of the operation training device 1 will be described. As described above, the operation training device 1 is a device used for training an operator to learn the operation of industrial machinery. Learning the operation of industrial machinery means, for example, learning which operation image will be displayed next when an operation is performed on any button or icon displayed on the display device of a numerical control device or a teaching pendant. Further, it is to learn how to transition the operation image to reach the target operation image. Here, the operation image is an image displayed on the entire screen of the display device of the numerical control device or an image displayed on the pop-up screen of the display device. The operation image includes an operation area where soft keys and the like are displayed.

[0023] FIG. 2 is a block diagram showing an example of the functions of the operation training device 1. The operation training device 1 includes, for example, a first storage unit 101, a second storage unit 102, a voice recognition unit 103, a specifying unit 104, and a display unit 105. The first storage unit 101 and the second storage unit 102 are realized, for example, by storing the information received via a network (not shown) in the RAM 14 or the non-volatile memory 15. The voice recognition unit 103, the specifying unit 104, and the display unit 105 are realized, for example, by the hardware processor 11 performing arithmetic processing using the system program stored in the ROM 13 and various data stored in the non-volatile memory 15.

[0024] The first storage unit 101 stores a plurality of operation images. The plurality of operation images are, for example, the same images as the operation images displayed on the display device of the numerical control device or the operation images displayed on the display device of the teaching pendant. The term "the same" does not necessarily mean exactly the same. The plurality of operation images stored in the first storage unit 101 only need to be similar to the extent that they are recognized as corresponding to the plurality of operation images displayed on the display device of the numerical control device or the teaching pendant, respectively.

[0025] The plurality of operation images include, for example, an operation image showing a home screen, an operation image showing a setup screen, an operation image showing an external interface screen, and an operation image showing a machine state output setting screen. The operation image showing the setup screen includes, for example, operation images respectively showing a first setup screen, a second setup screen, and a third setup screen.

[0026] The plurality of operation images further include an operation image showing a PC operation screen, an operation image showing a tool information management screen, an operation image showing a network screen, an operation image showing an NC operation screen, an operation image showing a production management screen, an operation image showing an accuracy adjustment screen, an operation image showing a maintenance information management screen, an operation image showing a maintenance inspection screen, an operation image showing a servo viewer screen, an operation image showing a setting screen, an operation image showing a file manager screen, an operation image showing a manual screen, an operation image showing a memo screen, and an operation image showing a browser screen.

[0027] The second storage unit 102 stores association information associating one operation area included in a first operation image among the plurality of operation images stored in the first storage unit 101 with a second operation image among the plurality of operation images.

[0028] The operation area is the area where soft keys included in the operation image are displayed. The soft keys are, for example, operation buttons and icons. The second operation image is an operation image specified based on voice information, as will be described later. In other words, it is an operation image specified based on the voice uttered by the operator, and is the image that the operator wants to display on the display screen.

[0029] In other words, the related information is information that defines the next operation image to be displayed according to the operation of each operation area included in a plurality of operation images. That is, the related information stores the association between the operation area and the operation image to be displayed next when the operation area is operated. In yet another way of putting it, the related information is information indicating the order of transition of a plurality of operation images.

[0030] Figure 3 is a diagram for explaining related information. Each block in Figure 3 shows an operation image. The blank block shows an arbitrary operation image. The lines connecting the blocks indicate that the blocks are associated with each other.

[0031] For example, the operation image showing the home screen A1 is associated with the operation image showing the first setup screen A2, the operation image showing the PC operation screen A3, and the operation image showing the tool information management screen A4. In this case, on the home screen A1, a first setup key for transitioning to the first setup screen A2, a PC operation key for transitioning to the PC operation screen A3, and a tool information management screen key for transitioning to the tool information management screen A4 are displayed.

[0032] Although not shown in the drawings, the operation image showing the home screen A1 may be associated with the operation image showing the network screen, the operation image showing the NC operation screen, the operation image showing the production management screen, the operation image showing the accuracy adjustment screen, the operation image showing the maintenance information management screen, the operation image showing the maintenance inspection screen, the operation image showing the servo viewer screen, the operation image showing the setting screen, the operation image showing the file manager screen, the operation image showing the manual screen, the operation image showing the memo screen, and the operation image showing the browser screen.

[0033] In this case, on the home screen, there are further displayed a network key for transitioning to the network screen, an NC operation key for transitioning to the NC operation screen, a production management key for transitioning to the production management screen, an accuracy adjustment key for transitioning to the accuracy adjustment screen, a maintenance information management key for transitioning to the maintenance information management screen, a maintenance inspection key for transitioning to the maintenance inspection screen, a servo viewer key for transitioning to the servo viewer screen, a setting key for transitioning to the setting screen, a file manager key for transitioning to the file manager screen, a manual key for transitioning to the manual screen, a memo key for transitioning to the memo screen, and a browser key for transitioning to the browser screen.

[0034] The operation image showing the first setup screen A2 is associated with the operation image showing the second setup screen A5 and the operation image showing the third setup screen A6. In this case, on the first setup screen A2, there are displayed a second setup key for transitioning to the second setup screen A5 and a third setup key for transitioning to the third setup screen A6.

[0035] The operation image showing the second setup screen A5 is associated with the operation images showing a plurality of operation screens, although detailed description is omitted. In this case, on the second setup screen A5, soft keys for transitioning to each operation image are displayed.

[0036] The operation image showing the third setup screen A6 is associated with, for example, the operation image showing the external interface screen A7, the operation image showing the custom control panel screen A8, and the operation image showing the I / O setting screen A9. In this case, on the third setup screen A6, an external interface key for transitioning to the external interface screen A7, a custom control panel key for transitioning to the custom control panel screen A8, and an I / O setting key for transitioning to the I / O setting screen A9 are displayed.

[0037] The operation image showing the external interface screen A7 is associated with, for example, the operation image showing the machine state output setting screen A10, the operation image showing the M code setting screen A11, and the operation image showing the output signal screen A12. In this case, on the external interface screen A7, a machine state output setting key for transitioning to the machine state output setting screen A10, an M code setting key for transitioning to the M code setting screen A11, and an output signal key for transitioning to the output signal screen A12 are displayed.

[0038] Similarly, the operation image showing the custom control panel screen A8 and the operation image showing the I / O setting screen A9 are each associated with operation images showing a plurality of operation screens. In this case, on the custom control panel screen A8 and the I / O setting screen A9, a plurality of soft keys for transitioning to other operation screens are respectively displayed.

[0039] Similarly, the operation image showing the PC operation screen A3 and the operation image showing the tool information management screen A4 are each associated with operation images showing a plurality of operation screens. In this case, on the PC operation screen A3 and the tool information management screen A4, soft keys for transitioning to a plurality of operation screens are respectively displayed. Note that only a part of the related information is depicted in FIG. 3. Now, let's return to the description of FIG. 2.

[0040] The voice recognition unit 103 recognizes the voice information of the uttered voice. That is, the voice recognition unit 103 recognizes what kind of words the uttered voice represents. The voice recognition unit 103 acquires the voice information of the voice uttered by the operator from a microphone (not shown) connected to the operation training device 1. The voice information is an electrical signal generated by the microphone.

[0041] The voice recognition unit 103 recognizes the voice information using a voice recognition model. The voice recognition model is used to discriminate the phonemes included in the voice information. The voice recognition model is generated, for example, by performing machine learning using the voice information of the voice uttered by the speaker as teacher data.

[0042] The uttered voice is, for example, the voice uttered by the operator. When the operator wants to display a desired operation image on the display screen, the operator utters the name of the operation image to be displayed. For example, when the operator wants to display the machine state output setting screen on the display screen, the operator utters "machine state output setting screen".

[0043] Alternatively, the operator may utter something like "I want to set the machine state output." about the desired operation. In this case, the voice recognition device may convert the uttered voice into "machine state output setting screen" and recognize it according to a predetermined voice recognition program.

[0044] When the specifying unit 104 recognizes that the voice information designates the second operation image by the voice recognition unit 103, based on the related information stored in the second storage unit 102, the specifying unit 104 specifies the order of the image transition from the operation image displayed on the display screen at the time when the voice recognition is performed to the second operation image. Further, the specifying unit 104 specifies the operation area to be highlighted when sequentially displaying from the operation image displayed on the display screen to the second operation image based on the related information.

[0045] When the display unit 105 recognizes that the voice information designates the second operation image by the voice recognition unit 103, the display unit 105 sequentially displays on the display screen the operation images from the operation image currently displayed on the display screen to the second operation image in the order of the image transitions specified by the specifying unit 104. Here, the display screen refers to the display screen of the input / output device 17.

[0046] Here, the operation image immediately preceding the second operation image is referred to as the first operation image. In this case, after highlighting one operation area and displaying the first operation image, the display unit 105 displays the second operation image associated with the one operation image. In other words, the display unit 105 makes a transition from the state of displaying the first operation image to the state of displaying the second operation image.

[0047] The display unit 105 makes a transition from the first operation image to the second operation image in response to having displayed the first operation image for a predetermined period. That is, in the first operation image, one operation area is maintained in a state of being highlighted for a predetermined period.

[0048] To highlight and display one operation area means to perform a display that prompts an operation on the one operation area. That is, the one operation area is displayed so as to be more prominent than other areas. The highlighting display is, for example, to display the brightness of the operation area where the soft key is displayed to be higher than the brightness of other areas. The highlighting display may be to blink only the one operation area. Also, the highlighting display may be to display an arrow indicating the one operation area.

[0049] FIGS. 4A and 4B are diagrams for explaining an example of the transition of the operation images displayed by the display unit 105. When the voice recognition unit 103 recognizes that the voice information designates the setup screen in a state where the operation image showing the home screen is being displayed, the display unit 105 highlights the setup key included in the home screen (see FIG. 4A).

[0050] Thereafter, the display unit 105 transitions from the state in which the setup key is highlighted to the state in which an operation image showing the first setup screen is displayed (see FIG. 4B). For example, after maintaining the state in which the setup key is highlighted on the home screen for a predetermined period, the display unit 105 displays an operation image showing the first setup screen. The predetermined period is, for example, 2 seconds. In the examples shown in FIGS. 4A and 4B, the image showing the home screen corresponds to the first operation image, and the operation image showing the first setup screen corresponds to the second operation image.

[0051] FIGS. 5A to 5D are diagrams for explaining another example of the transition of the operation image displayed by the display unit 105. When the voice recognition unit 103 recognizes that the voice information designates the machine state output setting screen in the state in which the operation image showing the first setup screen is displayed, the display unit 105 highlights the external device key included in the first setup screen (see FIG. 5A).

[0052] Thereafter, the display unit 105 transitions from the state in which the external device key is highlighted to the state in which an image showing the third setup screen is displayed (see FIG. 5B). When the display unit 105 displays the operation image showing the third setup screen, it highlights the external interface key. After maintaining the state in which the external interface key is highlighted for a predetermined period, the display unit 105 displays an operation image showing the external interface screen (see FIG. 5C).

[0053] When the display unit 105 displays the external interface screen, it highlights the machine state output setting key. After maintaining the state in which the machine state output setting key is highlighted for a predetermined period, the display unit 105 displays an operation image showing the machine state output setting screen on a pop-up screen (see FIG. 5D). In the examples shown in FIGS. 5A to 5D, the operation image showing the external interface screen corresponds to the first operation image, and the operation image showing the machine state output setting screen corresponds to the second operation image.

[0054] Next, an example of the processing flow executed by the operation training device 1 will be described.

[0055] FIG. 6 is a flowchart showing an example of the flow of processing executed by the operation training device 1.

[0056] First, the first storage unit 101 stores, for example, a plurality of operation images received from an external server (step SA1).

[0057] Next, the second storage unit 102 stores, for example, related information received from an external server (step SA2).

[0058] Next, the voice recognition unit 103 recognizes the voice information of the voice acquired by the microphone (step SA3).

[0059] Next, the specifying unit 104 specifies the order of image transitions from the operation image displayed on the display screen to the second operation image at the time when the voice recognition of the voice information is performed based on the related information (step SA4). Further, the specifying unit 104 specifies an operation area to be highlighted when sequentially displaying from the operation image displayed on the display screen to the second operation image.

[0060] Next, the display unit 105 highlights one operation area included in the operation image displayed on the display screen based on the information indicating the operation area specified by the specifying unit 104 (step SA5).

[0061] When a predetermined period has elapsed after highlighting one operation area, the display unit 105 displays the operation image to be displayed next to the operation image displayed on the display screen (step SA6).

[0062] When the operation image displayed on the display screen is the second operation image (Yes in step SA7), the operation training device 1 ends the process. Alternatively, the operation training device 1 may wait for the voice recognition unit 103 to perform voice recognition again.

[0063] When the operation image displayed on the display screen is not the second operation image (if No in step SA7), the process returns to step SA5.

[0064] As described above, the operation training device 1 includes a first storage unit 101 that stores a plurality of operation images, a second storage unit 102 that stores association information associating one operation area included in the first operation image among the plurality of operation images stored in the first storage unit 101 with the second operation image among the plurality of operation images, a voice recognition unit 103 that recognizes voice information of the spoken voice, and a display unit 105 that, when the voice recognition unit 103 recognizes that the voice information designates the second operation image, displays the first operation image with one operation area emphasized and then displays the second operation image associated with the first operation image.

[0065] Therefore, the operation training device 1 can sequentially display the operation images from the operation image displayed on the display screen when the voice recognition unit 103 recognizes the voice information to the operation image designated by the voice information. Also, one operation area included in the sequentially displayed operation images is displayed in an emphasized manner. Therefore, the operation training device 1 can effectively perform the training of the operation on the industrial machine. That is, the operator can effectively learn the procedure for displaying the desired operation image on the display screen.

[0066] In the above-described embodiment, the operation image is an operation image displayed on the display device of the numerical control device or the display device of the teaching pendant. However, not limited to this example, the operation image may be an image of part or all of the operation panel of the numerical control device, or an image of part or all of the teaching pendant. Among the operation images, the area where the hard keys of the operation panel or the teaching pendant are displayed becomes the operation area. In this case, the operation training device 1 can perform training not only on operations on the operation image displayed on the display device of the numerical control device, but also on operations on the buttons or switches of the operation panel. Further, the operation training device 1 can perform training not only on operations on the operation image displayed on the display device of the teaching pendant, but also on operations on the buttons or switches of the teaching pendant.

[0067] In the above-described embodiment, an example of transitioning the operation image from the left side to the right side of FIG. 3 has been described. However, the transition order of the operation images is not limited to this.

[0068] For example, when it is recognized by the voice recognition unit 103 that the voice information designates the machine state output setting screen A10 in a state where an image showing the I / O setting screen A9 is being displayed, the specifying unit 104 specifies the order of image transition, for example, in the order of the operation image showing the I / O setting screen A9, the operation image showing the third setup screen A6, the operation image showing the external interface screen A7, and the operation image showing the machine state output setting screen A10.

[0069] Also, the operation image showing the I / O setting screen A9 is made to include, for example, a "return button" for transitioning from the block on the left side to the block on the right side of FIG. 3. In this case, when transitioning from the operation image showing the I / O setting screen A9 to the operation image showing the third setup screen A6, the display unit 105 may highlight the "return button".

[0070] In the above-described embodiment, when a predetermined period elapses, the display unit 105 automatically transitions the displayed operation image to the next operation image. However, the operation training device 1 may be configured to transition to the next operation image in response to receiving an operation on the highlighted operation area. In this case, the operation training device 1 further includes a first reception unit, and the first reception unit receives an operation on the operation area.

[0071] FIG. 7 is a block diagram showing an example of the functions of the operation training device 1 including the first reception unit. The operation training device 1 shown in FIG. 7 is different from the operation training device 1 shown in FIG. 2 in that it includes a first reception unit 106. Therefore, descriptions of functions other than the first reception unit 106 will be omitted as appropriate.

[0072] The first reception unit 106 is realized, for example, by the hardware processor 11 performing arithmetic processing using the system program stored in the ROM 13 and various data stored in the non-volatile memory 15.

[0073] The first reception unit 106 receives an operation on one operation area displayed by the display unit 105. The first reception unit 106 receives an operation on one operation area included in the operation image displayed on the display screen. When the input / output device 17 is a touch panel, the reception unit receives a touch operation on one operation area displayed on the display screen of the touch panel.

[0074] In response to the first reception unit 106 receiving an operation on one operation area, the display unit 105 displays a second operation image.

[0075] For example, in a state where an operation image showing the home screen A1 is displayed on the display screen of the input / output device 17, when the voice recognition unit 103 recognizes that the voice information designates the setup screen, the display unit 105 highlights the setup key included in the home screen A1 (see FIG. 4A).

[0076] Next, when the first reception unit 106 receives an operation on the setup key included in the home screen A1, the display unit 105 displays an operation image showing the first setup screen A2 (see FIG. 4B). When the first reception unit 106 does not receive an operation on the setup key, the display unit 105 maintains the display of the operation image showing the home screen with the setup key highlighted.

[0077] Next, the flow of processing executed by the operation training device 1 having the first reception unit 106 will be described.

[0078] FIG. 8 is a flowchart showing an example of the flow of processing executed by the operation training device 1.

[0079] First, the first storage unit 101 stores, for example, a plurality of operation images received from an external server (step SB1).

[0080] Next, the second storage unit 102 stores, for example, related information received from an external server (step SB2).

[0081] Next, the voice recognition unit 103 recognizes the voice information of the voice acquired by the microphone (step SB3).

[0082] Next, the specifying unit 104 specifies the order of image transitions from the operation image displayed on the display screen to the second operation image based on the related information at the time when the voice information is recognized (step SB4). Further, the specifying unit 104 specifies the operation area to be highlighted when sequentially displaying from the operation image displayed on the display screen to the second operation image.

[0083] Next, the display unit 105 highlights one operation area included in the operation image displayed on the display screen based on the information indicating the operation area specified by the specifying unit 104 (step SB5).

[0084] In a state where one operation area is highlighted by the display unit 105, when the first reception unit 106 receives an operation on the one operation area (when Yes in step SB6), the display unit 105 causes the operation image to be displayed next to the operation image displayed on the display screen (step SB7). On the other hand, when the first reception unit 106 does not receive an operation on the one operation area (when No in step SB6), the display unit 105 maintains the highlighting of the one operation area.

[0085] When the operation image displayed on the display screen is the second operation image (step SB8), the operation training device 1 ends the process.

[0086] On the other hand, when the operation image displayed on the display screen is not the second operation image (when No in step SB8), the process returns to step SB5.

[0087] As described above, the operation training device 1 further includes a first reception unit 106 that receives an operation on one operation area displayed by the display unit 105, and the display unit 105 causes the second operation image to be displayed in response to the first reception unit 106 receiving an operation on the one operation area.

[0088] Therefore, the operation training device 1 can cause the operator to actually perform an operation on the operation image. As a result, the operation training device 1 can effectively perform training on operations on industrial machines.

[0089] Note that the voice recognition unit 103 may analyze the voice information to calculate recognition candidates before recognizing the voice information, and receive an operation on the calculated recognition candidates. In this case, the operation training device 1 further includes a second reception unit that receives an operation on the calculated recognition candidates.

[0090] FIG. 9 is a block diagram showing an example of the functions of the operation training device 1 including the second reception unit. Note that, among the functions of the operation training device 1 shown in FIG. 9, the description of the functions that are the same as those of the operation training device 1 shown in FIG. 2 will be omitted as appropriate.

[0091] The second reception unit 107 is realized, for example, by the hardware processor 11 performing arithmetic processing using the system program stored in the ROM 13 and various data stored in the non-volatile memory 15.

[0092] The voice recognition unit 103 analyzes voice information to calculate recognition candidates. For example, when the voice spoken by the operator is "I want to set the machine state output.", the voice recognition unit 103 analyzes the voice information using a predetermined voice recognition program to calculate the analysis result. Here, the analysis result is, for example, "machine state output setting".

[0093] The display unit 105 displays a candidate image indicating the calculated recognition candidates on the display screen. The display unit 105 displays the candidate image, for example, on a pop-up screen.

[0094] FIG. 10 is a diagram showing an example of the candidate image displayed on the display screen. The display unit 105 displays a candidate image including characters "machine state output setting", "yes", and "no" on the pop-up screen. The areas where "yes" and "no" are drawn are soft keys, respectively.

[0095] The second reception unit 107 receives an operation on the candidate image. When soft keys with characters "yes" and "no" are displayed on the candidate image displayed on the display screen, the second reception unit 107 receives an operation on any of the soft keys.

[0096] The voice recognition unit 103 recognizes the voice information based on the operation on the candidate image received by the second reception unit 107. When the second reception unit 107 receives an operation on the soft key with the character "yes" drawn, the voice recognition unit 103 recognizes that the voice information indicates "machine state output setting". Also, the subsequent processing is the same as that of the above-described embodiment.

[0097] When the second reception unit 107 receives an operation on the soft key with the character "No", the voice recognition unit 103 recognizes that the voice information does not indicate "Machine state output setting". In this case, the display unit 105 may display, for example, a message prompting the operator to input voice again.

[0098] FIG. 11 is a diagram showing an example of a candidate image displayed on the display screen. The display unit 105 displays a candidate image including the characters "Machine state output setting?" and "External interface?" on the pop-up screen. The areas where these characters are displayed are soft keys respectively.

[0099] The second reception unit 107 receives an operation on the candidate image. When soft keys with the characters "Machine state output setting?" and "External interface?" are displayed on the candidate image displayed on the display screen, the second reception unit 107 receives an operation on any of the soft keys.

[0100] The voice recognition unit 103 recognizes voice information based on the operation on the candidate image received by the second reception unit 107.

[0101] When the second reception unit 107 receives an operation on the soft key with the character "Machine state output setting?", the voice recognition unit 103 recognizes that the voice information indicates "Machine state output setting".

[0102] When the second reception unit 107 receives an operation on the soft key with the character "External interface?", the voice recognition unit 103 recognizes that the voice information indicates "External interface".

[0103] Note that the display unit 105 may, for example, display a soft key with the character "Other" on the display screen. When the second reception unit 107 receives an operation on the soft key with the character "Other", the voice recognition unit 103 recognizes that the voice information does not indicate "Machine state output setting" and "External interface". In this case, the display unit 105 may, for example, display a message prompting the operator to input voice again.

[0104] As described above, the operation training device 1 further includes a second reception unit 107, and the voice recognition unit 103 analyzes voice information to calculate recognition candidates. The display unit 105 displays a candidate image indicating the calculated recognition candidates. The second reception unit 107 receives an operation on the candidate image. The voice recognition unit 103 recognizes voice information based on the operation on the candidate image received by the second reception unit 107.

[0105] Therefore, the voice recognition device can surely recognize the voice spoken by the operator.

[0106] Note that the present disclosure is not limited to the above-described embodiments, and can be appropriately modified without departing from the gist. In the present disclosure, combinations of embodiments, modifications of any component, or omission of any component are possible.

Explanation of reference numerals

[0107] 1 Operation training device 11 Hardware processor 12 Bus 13 ROM 14 RAM 15 Non-volatile memory 16 Interface 17 Input / output device 101 First storage unit 102 Second storage unit 103 Voice recognition unit 104 Specifying unit 105 Display unit 106 First reception unit 107 Second receiving unit

Claims

1. A first storage unit that stores a plurality of operation images; A second storage unit that stores association information associating one operation area included in a first operation image among the plurality of operation images stored in the first storage unit with a second operation image among the plurality of operation images; A voice recognition unit that recognizes voice information of spoken voice; When the voice information is recognized by the voice recognition unit as designating the second operation image, a specifying unit that specifies the order of image transition from the operation image displayed on the display screen at the time of recognition by the voice recognition unit to the second operation image; Based on the order of the screen transition specified by the specifying unit, after the operation image displayed on the display screen, the first operation image is displayed with the one operation area emphasized, and after the first operation image is displayed, the second operation image associated with the one operation image is displayed by a display unit; An operation training device comprising:

2. Further comprising a first reception unit that receives an operation on the one operation area displayed by the display unit; The operation training device according to claim 1, wherein the display unit displays the second operation image in response to the first reception unit receiving an operation on the one operation area.

3. Further comprising a second reception unit; The voice recognition unit analyzes the voice information to calculate recognition candidates; The display unit displays a candidate image indicating the calculated recognition candidates; The second reception unit receives an operation on the candidate image; The operation training device according to claim 1 or 2, wherein the voice recognition unit recognizes the voice information based on the operation on the candidate image received by the second reception unit.

4. An operation training method performed by a device, comprising: Storing a plurality of operation images; Storing association information associating one operation area included in a first operation image among the stored plurality of operation images with a second operation image among the plurality of operation images; Recognizing voice information of the spoken voice; When the voice information is recognized as specifying the second operation image, specifying the order of image transition from the operation image to the second operation image at the time of recognition; Based on the specified order of the screen transition, after the displayed operation image, displaying the first operation image with the one operation area emphasized, and after displaying the first operation image, displaying the second operation image associated with the one operation image; An operation training method including the above.

5. Storing a plurality of operation images; Storing association information associating one operation area included in a first operation image among the stored plurality of operation images with a second operation image among the plurality of operation images; Recognizing voice information of the spoken voice; When the voice information is recognized as specifying the second operation image, specifying the order of image transition from the operation image to the second operation image at the time of recognition; Based on the specified order of the screen transition, after the displayed operation image, displaying the first operation image with the one operation area emphasized, and after displaying the first operation image, displaying the second operation image associated with the one operation image; A computer-readable storage medium storing instructions for causing a computer to execute the above.

Images