Work support system, work support device, and work support method

Abstract

To properly set an important work showing work details.SOLUTION: A work support system 30 has: a past processing part 10 which learns an evaluation model 17, which evaluates an important word included in manual data 36 from a sentence structure extracted from the manual data 36, so as to output a word included in a past manual 11 as an important word included in the past manual 11 to the evaluation model 17; and a current processing part 20 which inputs the sentence structure extracted from the current manual data 36 to the evaluation model 17 to register the important word output from the evaluation model 17 in an important word dictionary 34 as a keyword for confirming whether the important word is uttered in inspection work based upon the current manual data 36.SELECTED DRAWING: Figure 8

Description

Technical Field

[0001] The present invention relates to a work support system, a work support device, and a work support method.

Background Art

[0002] Voice recognition systems have become widespread as a means of inputting character data. Since workers at work sites have their hands occupied with work tools and the like, it is convenient to use voice input for the input work of character data. For example, Patent Document 1 describes a report creation system that aggregates maintenance inspection results for each work target item of a property based on information based on voice input by a worker to a mobile terminal.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] In work such as inspections at work sites, work may be performed by a work team consisting of one instructor and one or more workers. The instructor and the workers in the work team conduct voice conversations with each other via a communication device such as a transceiver. Hereinafter, as an example of work, an example of the outline procedure of an inspection work will be shown. (Step 1) The instructor has a "procedure manual" in which the work procedures for each inspection item are described. The procedure manual may be in a format for referring to the digitized data from a screen. (Step 2) The instructor conveys the inspection instruction to the worker by reading out the inspection items in the procedure manual to the worker. (Step 3) The worker responds to the instructor that the inspection instruction has been conveyed by repeating the conveyed inspection item. Then, the worker executes the inspection work in response to the inspection instruction. (Step 4) The supervisor confirms whether the worker performed the inspection correctly by reviewing the worker's repetition. Any inspection items that were not repeated may indicate a work error, so the supervisor should confirm them with the worker again.

[0005] Here, in order to reduce the burden on the supervisor in (Step 4), we will consider introducing a voice recognition system to mechanically check whether the content of what the supervisor and worker are repeating is correct. However, if the contents of the manual are read aloud word for word, it will take a long time to communicate, which will be a heavy burden on both the supervisor and the worker. For this reason, workers tend to communicate mainly the important words (hereinafter referred to as "important words") extracted from the manual to avoid misunderstandings about the contents of each inspection item, and omit other words from the manual that are not important. Therefore, in a voice recognition system introduced for inspection work, the key point that the system mechanically checks is whether or not important words were correctly pronounced by both the instructor and the worker.

[0006] With the evolution of speech recognition systems, the accuracy of converting spoken audio data into text data continues to improve. On the other hand, from the perspective of whether the content of inspection items is correctly communicated from the supervisor to the worker, how the key words that characterize the inspection items are defined greatly affects the accuracy of the communication check. The accuracy of the communication check refers to the accuracy of confirming that the work content has been correctly communicated, that is, the accuracy of the system that checks communication using key words.

[0007] For example, suppose the instruction manual includes an inspection item that reads, "Periodic inspection: Disconnect the cable from cable terminal Z1 in terminal block Y1 within panel number X1." The person giving the instructions said, "Start the routine inspection. Disconnect the cable from cable terminal Z1 on panel number X1." The worker reportedly repeated, "I will pull the cable out of cable terminal Z1." In this case, both the supervisor and the worker understand the task to be "pulling the cable out of cable terminal Z1." However, the worker does not repeat the "panel number X1" spoken by the supervisor, and there is a risk that they may mistakenly perform the task for "panel number X2." Furthermore, the worker does not speak the "terminal block number Y1," and there is a risk that the worker may mistakenly perform the task for "terminal block number Y2." On the other hand, the worker did not repeat the supervisor's statement, "Start the periodic inspection," but this discrepancy did not affect the communication of the work instructions at all.

[0008] Thus, simply comparing the content of the supervisor's speech with the content of the worker's speech and detecting whether they match or not may not be sufficient to determine whether the inspection items described in the manual have been correctly communicated from the supervisor to the worker. In the example above, it is desirable to set four key words: "panel number X1," "terminal block number Y1," "cable terminal Z1," and "pull out the cable." If these four key words are covered in the speech of both the supervisor and the worker, it can be considered that the work instructions have been correctly communicated. Furthermore, the word "periodic inspection" can be excluded from the list of key words, so more key words are not always better.

[0009] However, conventional technologies do not provide a mechanism to support the setting of important keywords in terms of how to set important keywords so that the work content is correctly communicated. For example, Patent Document 1 describes summarizing the inspector's speech based on predetermined keywords indicating the work inspection location, predetermined keywords indicating the equipment to be inspected, and predetermined keywords indicating the inspection overview. However, it does not describe how to set the "predetermined keywords". The above examples illustrate inspections, but the problems described above can occur in general work.

[0010] Therefore, the main objective of this invention is to appropriately set important words that indicate the content of the work. [Means for solving the problem]

[0011] In order to solve the above problems, the work support system of the present invention has the following features. The present invention extracts sentence structures from past specifications construction and , among the words included in the past specification, the words repeated in the inspection work where the work instructor speaks first and the worker responds to the speech and repeats The system learns the statistical information attached to the repeated words, and the words that were repeated in the inspection work from among the words included in the past manuals. are output as important words included in the past specification Review an important word extraction device that stores an evaluation model; By inputting the sentence structure extracted from the current specification into the evaluation model, the important words output from the evaluation model are registered in an important word dictionary as keywords to be used in the inspection work based on the current specification. It is characterized by having a work support device. Other means will be described later.

Effect of the Invention

[0012] According to the present invention, important words indicating the work content can be appropriately set.

Brief Description of the Drawings

[0013] [Figure 1] It is a schematic diagram of the inspection work related to this embodiment. [Figure 2] It is a configuration diagram of the inspection support device related to this embodiment. [Figure 3] It is a hardware configuration diagram of the inspection support device related to this embodiment. [Figure 4] It is a flowchart showing an overview of the processing of the dictionary generation unit related to this embodiment. [Figure 5] It is a table showing the repetition confirmation results extracted from the collected data related to this embodiment. [Figure 6] It is an explanatory diagram showing two types of sentence structures extracted from the specification data related to this embodiment. [Figure 7]A table showing the dependency analysis result of FIG. 6 related to this embodiment. [Figure 8] A configuration diagram showing details of the first example of the dictionary generation unit related to this embodiment. [Figure 9] An explanatory diagram showing details of the evaluation model of FIG. 8 related to this embodiment. [Figure 10] A configuration diagram showing details of the second example of the dictionary generation unit related to this embodiment. [Figure 11] An explanatory diagram showing details of the evaluation model of FIG. 10 related to this embodiment. [Figure 12] A configuration diagram showing details of the third example of the dictionary generation unit related to this embodiment. [Figure 13] A configuration diagram showing details of the fourth example of the dictionary generation unit related to this embodiment.

Mode for Carrying Out the Invention

[0014] This embodiment will be described with reference to the drawings.

[0015] FIG. 1 is an overview diagram of the inspection work. Hereinafter, in this specification, the inspection work of equipment is exemplified as an example of work, but the present invention is also applicable to any work site where work is carried out by a team such as a disaster site. This team is such that one inspection instructor 1 instructs one or more inspection workers 2 to perform inspection work, and the inspection worker 2 repeats the important words spoken first by the inspection instructor 1 in response to that speech, and the inspection work is carried out according to the following procedure. (Step 1) From the tablet terminal (inspection terminal 3 in FIG. 2) held by the inspection instructor 1, the inspection instructor 1 is made to confirm the "procedure manual" in which the work procedures for each inspection item are described. (Step 2) The inspection instructor 1 reads out the inspection items in the procedure manual to the inspection worker 2 to convey the inspection instruction to the inspection worker 2. (Step 3) By the inspection worker 2 repeating the conveyed inspection item, the inspection worker 2 replies to the inspection instructor 1 that the inspection instruction has been conveyed. Then, the inspection worker 2 executes the inspection work according to the inspection instruction. (Step 4) The work support system 30 in Figure 2 performs speech recognition on the spoken content 102 of (Step 2) and (Step 3), and provides feedback to the inspection supervisor 1 with the result of the repetition confirmation obtained from the speech recognition, either by voice or on the screen.

[0016] Procedure Data 101 is data that describes the work procedures for the inspection items. In Figure 1, Procedure Data 101 explains the inspection items in text format, but other data formats such as checklists or check sheets are also acceptable. Utterance 102 shows the statements made between Inspector 1 and Inspector 2 during (Procedure 2) and (Procedure 3), in accordance with Procedure Data 101. For example, Inspector 1 says, "Please lift the cable," to which Inspector 2 repeats, "Lift it, right?" This shows that the action of "lifting" during the inspection work is communicated from Inspector 1 to Inspector 2.

[0017] The confirmation screen 103 is a display screen for the repetition confirmation result recognized by the work support system 30 from the spoken content 102 in (step 4), and is displayed on a tablet terminal (inspection terminal 3 in Figure 2) held by the inspection instructor 1. The confirmation screen 103 has a procedure section 103A and a warning section 103B. The procedure section 103A reflects the results of the repetition confirmation. For example, the procedure section 103A underlines important words (such as "lift" and "1-2345A") listed in the procedure data 101, and highlights the important words that were repeated in the spoken content 102 by circling them. The repetition of important words should be performed in the order in which they appear in the procedure data 101, and if there are multiple inspection workers 2, at least one person needs to repeat them. The warning section 103B displays a message to draw attention to important words (in this case, "B67") that were underlined in the instructions section 103A but were not repeated. This confirmation screen 103 allows inspection supervisor 1 to check the inspection items that were not repeated (inspection items with discrepancies) and to confirm them again with inspection worker 2, thereby preventing human error.

[0018] Figure 2 is a diagram of the work system 100. The work system 100 is network-connected to the work support system 30 and the inspection terminal 3. The work support system 30 includes an inspection result notification unit 31, a voice recognition unit 32, a dictionary generation unit 33, and a procedure notification unit 35. The work support system 30 stores an important word dictionary 34 and procedure data 36 in its storage unit. Procedure Data 36 is data that describes the work procedures for each inspection item, such as Procedure Data 101 in Figure 1. The important vocabulary dictionary 34 consists of important words extracted from the procedure data 36. Important words are those that affect the accuracy of the communication check of work instructions, and as shown in the confirmation screen 103 of Figure 1, if an important word is not repeated, it is considered that the work instructions were not communicated. The dictionary generation unit 33 generates an important word dictionary 34 from the manual data 36 (see Figure 8 for details). As described later, the dictionary generation unit 33 learns the content of utterances repeated during past inspection work, thereby reducing the effort required for administrators to manually extract the important word dictionary 34 while extracting important words with high accuracy.

[0019] The procedure notification unit 35 notifies the inspection terminal 3 of the procedure data 101 shown in Figure 1 (Procedure 1). The speech recognition unit 32 converts the spoken audio (such as the speech content 102 in Figure 1) acquired from the inspection instructor 1 and the inspection worker 2 via transceivers, etc., into text using speech recognition, and obtains a repeat confirmation result from the result (step 4). The speech recognition unit 32 then compares the important words registered in the important word dictionary 34 with the words repeated as a repeat confirmation result, determines the inspection result based on the match or mismatch, and transmits the inspection result to the inspection result notification unit 31. The inspection result notification unit 31 notifies the inspection terminal 3 of the inspection results transmitted from the voice recognition unit 32 using the confirmation screen 103 in Figure 1, and also outputs the contents of the warning column 103B in Figure 1 as voice to the transceiver held by the inspection instructor 1.

[0020] Figure 3 is a hardware configuration diagram of the work support system 30. Each device of the work support system 30 (such as the important word extraction device and work support device described later) is configured as a computer 900 having a CPU 901, RAM 902, ROM 903, HDD 904, communication I / F 905, input / output I / F 906, and media I / F 907. The communication interface 905 is connected to an external communication device 915. The input / output interface 906 is connected to the input / output device 916. The media interface 907 reads and writes data to the recording medium 917. Furthermore, the CPU 901 improves and controls each processing unit by executing a program (also called an application or app) loaded into the RAM 902. This program can also be distributed via a communication line or by recording it on a recording medium 917 such as a CD-ROM and distributing it that way. The dictionary generation unit 33 may be configured as a separate device outside the work support system 30.

[0021] The following four examples illustrate the details of the dictionary generation unit 33. (Example 1) A method for creating an evaluation model 17 for words appearing in the manual data 36, ​​based on the sentence structure of word sequences extracted from the manual data 36 (see Figure 6 for details) and the content of utterances during past checks (Figures 8 and 9). (Second example) A method to create an evaluation model 17 of words appearing in the manual data 36 based on the sentence structure of word sequences extracted from the manual data 36 and the content of utterances during past checks and the current exercise (rehearsal) (Figures 10 and 11). (Third example) A method (Figure 12) that creates an evaluation model 17 for words appearing in the manual data 36 based on the sentence structure of dependency relationships between words extracted from the manual data 36 (see Figure 6 for details) and the content of utterances during past checks. (Example 4) A method for creating an evaluation model 17 of words appearing in the manual data 36, ​​based on the sentence structure of dependency relationships between words extracted from the manual data 36 and the content of utterances during past checks and the current exercise (Figure 13).

[0022] Figure 4 is a flowchart showing an overview of the processing in the dictionary generation unit 33. The dictionary generation unit 33 collects data such as utterance content from past inspection work based on past procedure data (S11). The dictionary generation unit 33 creates a past evaluation model from the data collected in S11 (S12). These processes S11 and S12 are performed in (Example 1) to (Example 4). The dictionary generation unit 33 collects data such as utterance content from the current inspection exercise work based on the current procedure data 36 (S13). This processing in S13 is omitted in (Example 1) and (Example 3). The dictionary generation unit 33 generates the current important word dictionary 34 by inputting the current procedure data 36 into the past evaluation model of S12 (S14). In addition, in the second and fourth examples where S13 is executed, the repetition confirmation results extracted from the utterances collected in S13 are also input into the past evaluation model of S12. As explained in Figure 2, the dictionary generation unit 33 supports the current inspection by providing the important word dictionary 34 from S14 to the speech recognition unit 32 (S15). With the support of S15, the work support system 30 checks whether the important words registered in the important word dictionary 34 were spoken during the work based on the procedure data 36, ​​as explained in Figure 2, and warns of any important words that were not spoken (warning column 103B in Figure 1).

[0023] Figure 5 is a table showing the repetition confirmation results extracted from the data collected in S11 and S13. The repetition confirmation results associate each word appearing in the procedure data 36 with the "number of repetitions" repeated by the inspection worker 2 and the "repetition interval," which indicates the number of utterances between the time the inspection supervisor 1 speaks and the time the inspection worker 2 repeats the word. For example, in Figure 1, when the inspection supervisor 1 says "Please lift the cable," the next speech bubble shows the inspection worker 2 repeating "Lift, right?", so the repetition interval = 1 (responding immediately after the call). Also, there are a total of two places where the inspection worker 2 says "lift," so the number of repetitions = 2.

[0024] Figure 6 is an explanatory diagram showing two types of sentence structures extracted from the manual data 36. As shown in Figure 1, the procedure manual data 101 describes the work procedures for the inspection items in written form. Morphological analysis, which divides the text into individual words, is performed as a preprocessing step for both of the two types of sentence structures described below. The word sequence relationships 111 are data obtained by connecting the word sequences after morphological analysis in the order in which they appear in the manual data 101 (arrows in the diagram). The word sequence relationships 111 can be extracted from the manual data 101, for example, by applying a machine learning-based model.

[0025] The dependency parsing result 112 is data that outputs the dependency structure between parts of speech such as subjects, predicates, objects, and modifiers that make up a sentence, for the word sequence after morphological analysis. First, the check term "lift," which is registered in a pre-prepared check word dictionary, is extracted as the verb of the sentence. Next, the object of the verb "lift" (=cable, terminal) and related words that have a dependency structure with the verb "lift," such as the demonstrative pronouns "1-2345A, B67, CD890A12, terminal T1," are also extracted.

[0026] Figure 7 is a table showing the dependency parsing results 112 from Figure 6. This table associates each word after morphological analysis with information indicating whether or not it corresponds to a check term, and information indicating whether or not it corresponds to various dependency structures (subject, object, modifier, clause, etc.) for the verb of the check term. "Yes" in the table indicates that it corresponds.

[0027] Figure 8 is a configuration diagram showing the details of the first example of the dictionary generation unit 33. The dictionary generation unit 33 includes a past processing unit (important word extraction device) 10A that processes data from past inspection work (S11, S12 in Figure 4), and a current processing unit (work support device) 20A that processes data from the current inspection work (S13 to S15 in Figure 4). The past processing unit 10A includes a past speech recognition unit 12, a word sequence analysis unit 13A, and a learning unit 15, and stores the past procedure document 11, the sequenced words 14A, the past repeated words 16, and the evaluation model 17 in its memory unit. Past procedure document 11, as procedure document data 36 in Figure 2, is sample data used in past inspection work, and can be one or multiple (n past inspection work). Note that the work site, the inspection items performed at that work site, and the members of the team that inspects those items may be the same or different between past inspection work and the current inspection work.

[0028] The past speech recognition unit 12, as the speech recognition unit 32 in Figure 2, obtains repetition confirmation results from past spoken audio based on past procedure documents 11 acquired from the inspection supervisor 1 and the inspection worker 2. The past speech recognition unit 12 stores the words spoken (repeated) by both the inspection supervisor 1 and the inspection worker 2 as past repeated words 16. Note that the past repeated words 16 may be limited to certain parts of speech (nouns, verbs, adjectives, etc.). In addition, statistical information for each word (number of repetitions, repetition interval, etc.) may be added to the past repeated words 16, as shown in Figure 5. Furthermore, the past repeated words 16 may be manually entered instead of being extracted by speech recognition. The word sequence analysis unit 13A extracts the word sequence relationships that appear in the past manual 11 as a sentence structure, as explained in the word sequence relationship 111 in Figure 6, and stores the extraction results as sequence words 14A.

[0029] The learning unit 15 uses machine learning to create an evaluation model 17 from the sequence of words 14A and the previously repeated words 16. The learning unit 15 then stores the evaluation model 17 learned from past data as the evaluation model 25 of the current processing unit 20A, and uses the evaluation model 25 for inference processing of the current data. Evaluation model 17 is a model that has learned the relationship between the sentence structure of the manual data 36 and the importance of each word that makes up the sentences in the manual data 36. Importance is the degree to which a word is likely to be selected as an important word, and can take values ​​in the range of 0 to 1. A value of 0 indicates that it is definitely not an important word, and a value of 1 indicates that it is definitely an important word. Furthermore, the order of the words that make up the sentences in the instruction manual data 36 often affects the importance of each word. For example, even the same word "cable" will have a higher importance if it is immediately followed by a verb meaning inspection, such as "lift." On the other hand, if a verb unrelated to inspection, such as "sell," is immediately following the word "cable," the importance of "cable" will decrease.

[0030] Here, the learning unit 15 also uses the previously repeated words 16 as input data for the evaluation model 17, so that the repetition confirmation results are reflected in the importance of each word, as illustrated below. • Increase the importance of words included in the 16 previously repeated words (words with a repeat count of ≥ 1 in the table in Figure 5) compared to the importance of words not included in the 16 previously repeated words. The more frequently a word is repeated in the table in Figure 5, the greater its importance. • In the table in Figure 5, the shorter the repetition interval for a word, the greater the importance of that word.

[0031] The current processing unit 20A includes a word sequence analysis unit 22A and an identification unit 24, and stores the current procedure data 36, ​​sequence words 23A, evaluation model 25, previously repeated words 16, and the current important word dictionary 34 in its memory unit. The word sequence analysis unit 22A, similar to the word sequence analysis unit 13A, extracts the sequence relationships of words appearing in the current procedure data 36 as a sentence structure and stores the extraction result as sequence words 23A. The identification unit 24 functions as the machine learning inference unit, corresponding to the machine learning learning unit 15. In other words, the identification unit 24 inputs the sequence of words 23A to the evaluation model 25, causing the evaluation model 25 to output the importance of each word in the sequence of words 23A. The identification unit 24 then uses a judgment formula, such as "if importance > 0.80, it is an important word," to extract important words from the sequence of words 23A, and uses the extracted results as the important word dictionary 34.

[0032] Figure 9 is an explanatory diagram showing the details of evaluation model 17 in Figure 8. The sequenced word 201 is the input data to the evaluation model 17, corresponding to the sequenced word 14A in Figure 8. The vector transformation 202 is a processing unit that transforms the sequenced word 201 into a word vector 203, and corresponds to the input layer of the evaluation model 17. Word vector 203 is data in which each word in the sequence of words 201 is represented by multiple vector component values. The first word of the sequence of words 201, "below", is converted to the first word vector 203 "x1", and the second word of the sequence of words 201, "cable", is converted to the second word vector 203 "x2". Note that the seventh word vector 203 "x7", corresponding to the seventh word "(2-921A)", has all vector component values ​​of "1", so it is an unknown word, and unknown words are likely to be technical terms and are likely to become important words.

[0033] The hidden layer 204 is a processing unit that outputs an estimated result 205 for each word by inputting the word vector 203 of each word in the order of the sequenced words 201, and corresponds to the hidden layer of the evaluation model 17. In Figure 9, the evaluation model 17 uses a time-series DNN (Deep Neural Network) model such as RNN (Recurrent Neural Network) or LSTM (Long short-term memory). Therefore, the evaluation functions h1, h2, ..., h4, ..., h7 are connected one by one to the evaluation functions of the next stage. The first evaluation function h1 of the hidden layer 204 takes the first word of the sequenced words 201, "below," as input, and the second evaluation function h1 of the hidden layer 204 takes the second word of the sequenced words 201, "cable," as input.

[0034] The estimation result 205, as the evaluation result of the intermediate layer 204, shows the combination of the estimated result vector (y1, y2, ..., y7) and the likelihood (value in the range of 0 to 1) for each word in the sequence of words 201, and corresponds to the input layer of the evaluation model 17. The estimated result vector is a vector that stores two values, upper and lower. The upper value indicates that the word was classified as an "important word" and is represented by the value "1". The lower value indicates that the word was classified as an "important word" and is represented by the value "1". The likelihood is a value that indicates how likely a word is to become an important word; the higher the number, the more likely it is to become an important word.

[0035] The important words 206 is a list of words that correspond to the output data of the evaluation model 17. The identification unit 24 adds words that are classified as "important words" in the estimation result vector and have a likelihood of a predetermined value (0.80) or higher to important words 206 with importance = likelihood. On the other hand, for words that are classified as "important words" in the estimation result vector regardless of the likelihood value, the identification unit 24 excludes them from important words 206 with importance = 0. The learning unit 15 inputs the sequence of words 14A in Figure 8 as the sequence of words 201 in Figure 9 into the evaluation model 17, and trains the evaluation model 17 so that the previously repeated words 16 are output as important words 206. In other words, the previously repeated words 16 are used in the machine learning of the learning unit 15 as teaching material data that shows the correct answers for the important words 206 output by the evaluation model 17.

[0036] The first example of the dictionary generation unit 33 described above with reference to Figures 8 and 9 has the following features. Evaluation model 17 is a model that accepts time-series input data. The past processing unit 10A and the present processing unit 20A extract the word order relationships extracted from the manual data 36 as sentence structures, and input these extracted word order relationships as time-series input data into the evaluation model 17.

[0037] Figure 10 is a configuration diagram showing details of the second example of the dictionary generation unit 33. The dictionary generation unit 33 includes a past processing unit 10B that processes data from past inspection work and a current processing unit 20B that processes data from the current inspection work. The past processing unit 10B is the same as the past processing unit 10A in Figure 8. In the second example in Figure 10, the current processing unit 20B is the same as the current processing unit 20A in Figure 8, with the addition of the exercise speech recognition unit 26B and the exercise repeat word 27B.

[0038] The exercise speech recognition unit 26B, targeting the current inspection work exercise, obtains the exercise repetition words 27B from the spoken audio based on the current procedure data 36 obtained from the inspection supervisor 1 and the inspection worker 2, similar to the past speech recognition unit 12. The exercise repetition words 27B indicate the words that were repeated, similar to the past repetition words 16, but the time of repetition is replaced from the past to the present (current). Note that the exercise repetition words 27B may be manually entered instead of being extracted by speech recognition. The identification unit 24 inputs both the sequence of words 23A and the exercise repetition words 27B into the evaluation model 25, extracting important words from the sequence of words 23A, and uses the extraction results as the important word dictionary 34. This improves the accuracy of important word detection by considering the exercise repetition words 27B in the determination of important words.

[0039] The second example of the dictionary generation unit 33 described above with reference to Figure 10 has the following features. In this instance, the processing unit 20B adds words included in the current procedure manual data 36 that were commonly uttered by multiple people during an exercise based on the current procedure manual data 36, ​​which is performed prior to the inspection work based on the current procedure manual data 36, ​​to the input data of the evaluation model 17.

[0040] Figure 11 is an explanatory diagram showing the details of the evaluation model 17 in Figure 10. The evaluation model 17 in Figure 11 is basically the same as the evaluation model 17 in Figure 9, but the word vector 203B in Figure 11 has an exercise repetition word flag 203B1 added to the end of each word. The exercise repetition word flag 203B1 indicates whether each word is included in the exercise repetition words 27B in Figure 10 (value=1) or not (value=0). This extension of the exercise repetition word flag 203B1 allows the intermediate layer 204 to output the estimation result 205 for each word, taking into account the exercise repetition words 27B as well.

[0041] Figure 12 is a configuration diagram showing the details of the third example of the dictionary generation unit 33. The dictionary generation unit 33 includes a past processing unit 10C that processes data from past inspection work and a current processing unit 20C that processes data from the current inspection work. The third example in Figure 12 is a modified version of the first example in Figure 8, in which the method for extracting sentence structure from the manual data 36 is changed from the method of the word sequence analysis unit 13A to the method of the dependency parsing unit 13C. Therefore, the past processing unit 10C in Figure 12 is modified from the past processing unit 10A in Figure 8 by deleting the word sequence analysis unit 13A and sequenced words 14A, and instead adding the dependency parsing unit 13C, dependent words 14C, and check word dictionary 18C. Inspection vocabulary dictionary 18C and inspection vocabulary dictionary 21C are dictionaries in which verbs that are likely to be uttered during inspections, such as "lift," are pre-registered as inspection terms.

[0042] As shown in the dependency parsing result 112 in Figure 6, the dependency parsing unit 13C focuses on the check term in the check term dictionary 18C from the sentences of the past manual 11 and extracts other words that have a dependency relationship with that check term as dependency words 14C. As shown in Figure 7, in addition to the check term, each of the other words that have a dependency relationship with that check term has information (hereinafter referred to as "word features") indicating what kind of dependency relationship (subject, object, ...) it has. The evaluation model 17 in Figure 12, like the evaluation model 17 in Figure 8, is a model that has learned the relationship between the sentence structure of the manual data 36 and the importance of each word that makes up the sentences in the manual data 36. In the evaluation model 17 in Figure 8, the sequenced words 14A extracted by the word sequence parsing unit 13A were input as the sentence structure of the manual data 36. On the other hand, in the evaluation model 17 in Figure 12, the dependency words 14C extracted by the dependency parsing unit 13C and their features are input as the sentence structure of the manual data 36.

[0043] Furthermore, the evaluation models 17 for the third and fourth examples are implemented using one of the following methods. • Time-series DNN models such as LSTM, as shown in Figure 9, used in the first and second examples. • Modeling is performed using neural networks, support vector machines, regression analysis, etc. • An evaluation function that takes the information from the table in Figure 7 (whether the word is an inspection term, whether the word is the subject of an inspection term, etc.) as input as features for each word, and calculates the importance of each word from those features. This evaluation function is, for example, a fully connected set of features extracted from each word. The learning unit 15 then inputs the dependency word 14C into the evaluation function of the evaluation model 17 and learns (updates) the contents of the evaluation function so that the previously repeated word 16 is output as an important word 206 from the evaluation function.

[0044] The initial state of the evaluation model 17, which has not yet been trained by the learning unit 15, is created, for example, by one of the following methods. (Method 1) Create an evaluation model 17 in which only the check terms in the check word dictionary 18C are output as important words. (Method 2) Create an evaluation model 17 that outputs important words not only from the inspection vocabulary dictionary 18C, but also from the words that have dependency relationships (subject, object, ...) with the inspection vocabulary listed in the sample manual. (Method 3) Create an evaluation model 17 that outputs important words not only from the check term dictionary 18C, but also from words that have dependency relationships with the check term in the utterances extracted during the exercises.

[0045] In the current processing unit 20C shown in Figure 12, the word sequence analysis unit 22A and sequenced words 23A are removed from the current processing unit 20A shown in Figure 8, and instead, the check word dictionary 21C, dependency parsing unit 22C, and dependency words 23C are added. The dependency parsing unit 22C, similar to the dependency parsing unit 13C, focuses on the check terms in the check word dictionary 21C from the sentences of the current manual data 36, ​​and extracts other words that have a dependency relationship with those check terms as dependency words 23C. The identification unit 24 then inputs the dependency words 23C into the evaluation model 25 (time-series DNN model or evaluation function), and registers the important words output from the evaluation model 25 into the current important word dictionary 34.

[0046] The third example of the dictionary generation unit 33 described above with reference to Figures 11 and 12 has the following features. Evaluation model 17 is a model that accepts input data of features that indicate dependency relationships between words. The past processing unit 10C and the current processing unit 20C extract feature quantities that show the dependency relationships between words for inspection terms that indicate the operation of the inspection, extracted from the procedure data 36, ​​as sentence structures, and input these extracted feature quantities as input data to the evaluation model 17. The processing unit 20C may now register verbs from the important words registered in the important word dictionary 34 as terms to be checked next time.

[0047] Figure 13 is a configuration diagram showing the details of the fourth example of the dictionary generation unit 33. The dictionary generation unit 33 includes a past processing unit 10D that processes data from past inspection work and a current processing unit 20D that processes data from the current inspection work. The past processing unit 10D is the same as the past processing unit 10C in Figure 12. In the fourth example in Figure 13, the current processing unit 20D is the same as the current processing unit 20C in Figure 12, with the addition of the exercise speech recognition unit 26D and the exercise repeat word 27D. The exercise speech recognition unit 26D and the exercise repeat word 27D are the same as the exercise speech recognition unit 26B and the exercise repeat word 27B in Figure 10.

[0048] The exercise speech recognition unit 26D, targeting the current inspection work exercise, obtains the exercise repetition words 27D from the spoken voice based on the procedure data 36 obtained from the inspection supervisor 1 and the inspection worker 2, similar to the previous speech recognition unit 12. The evaluation model 25 is a model that uses not only the word-specific features shown in the evaluation model 17 in Figure 12 (such as whether the word is a check term), but also statistical information of the exercise repeat words 27D (such as the number of repetitions and repetition intervals in Figure 5) as input data for the evaluation function. The identification unit 24 inputs both the sequence of words 23A and the exercise repetition words 27D into the evaluation model 25, extracting important words from the sequence of words 23A, and uses the extraction results as the important word dictionary 34. This improves the accuracy of important word detection by considering the exercise repetition words 27D in the determination of important words. Furthermore, the identification unit 24 may add verbs from the extracted important words to the check word dictionary 21C as check terms to be used next time. This makes it possible to increase the number of important words to be extracted next time.

[0049] The fourth example of the dictionary generation unit 33 described above with reference to Figure 13 has the following features. Evaluation model 17 is a model that accepts input data that includes features showing dependency relationships between words, as well as features showing statistical information about words commonly uttered by multiple people. In this instance, the processing unit 20D adds statistical information about words commonly spoken by multiple individuals during an exercise based on the current procedure data 36, ​​which is performed prior to the inspection work based on the current procedure data 36, ​​to the input data of the evaluation model 17.

[0050] The dictionary generation unit 33 of this embodiment, as described above, has past processing units 10A to 10D and current processing units 20A to 20D. The past processing units 10A to 10D train the evaluation model 17, which evaluates important words contained in the manual data 36 from the sentence structure extracted from the manual data 36, ​​to output words contained in past manuals 11 that were commonly uttered by multiple people as important words contained in past manuals 11. In this instance, processing units 20A to 20D input the sentence structure extracted from the current procedure data 36 into the evaluation model 17. The important words output by the evaluation model 17 are then registered in the important word dictionary 34 as keywords to confirm whether or not they were spoken during the inspection work based on the current procedure data 36 (a feature common to the first to fourth examples). As a result, important words that have a proven track record in past inspection work are extracted into the important word dictionary 34 as past repeat words 16, allowing for the appropriate setting of important words that indicate the work content. Furthermore, by extracting important words based on the sentence structure words extracted from past procedure documents 11, words that are not used in the inspection work can be appropriately excluded from the important word dictionary 34 as noise.

[0051] Furthermore, in the dictionary generation unit 33 (first example) and (second example), the word order relationships are extracted as sentence structure. This allows for the extraction of sentence structure with less computation. On the other hand, in the dictionary generation unit 33 (third example) and (fourth example), the dependency relationships between words are extracted as sentence structure. This allows for the extraction of highly accurate sentence structure that goes beyond the meaning of the text. Furthermore, in the dictionary generation unit 33 (second example) and (fourth example), the utterances from the current exercise (rehearsal) as well as those from past inspections are reflected in the evaluation model 17. As a result, the same manual 36 is used for both the exercise and the actual inspection, allowing for the preparation of a highly accurate important word dictionary 34 before the actual inspection.

[0052] Furthermore, the present invention is not limited to the embodiments described above, and it goes without saying that various other applications and modifications can be taken as long as they do not depart from the gist of the invention as described in the claims. For example, the embodiments described above describe the configuration of the work support system 30 in detail and specifically in order to explain the present invention in an easy-to-understand manner, and are not necessarily limited to those that include all the components described. Also, it is possible to replace a part of the configuration of one embodiment with a component of another embodiment. It is also possible to add a component of another embodiment to the configuration of one embodiment. Furthermore, it is possible to add, replace, or delete other components for a part of the configuration of each embodiment.

[0053] Furthermore, some or all of the above configurations, functions, and processing units may be implemented in hardware, for example, by designing them as integrated circuits. Broadly defined processor devices such as FPGAs (Field Programmable Gate Arrays) and ASICs (Application Specific Integrated Circuits) may be used as hardware. Furthermore, each component of the work support system 30 according to the above-described embodiment may be implemented on any hardware, as long as the respective hardware can send and receive information from each other via a network. Also, the processing performed by a certain processing unit may be implemented by a single piece of hardware, or by distributed processing using multiple pieces of hardware. [Explanation of Symbols]

[0054] 1. Inspection Instructor 2. Inspection worker 3. Inspection terminal 10A, 10B, 10C, 10D Past Processing Unit (Important Word Extraction Device) 11. Past Procedures (Past Procedures) 12 Past speech recognition unit 13A Word Sequence Analysis Unit 14A Words in order 13C Dependency Analysis Unit 14C Dependent words 15. Learning Department 16 Past words to repeat 17 Evaluation Models 18C Checking Vocabulary Dictionary 20A, 20B, 20C, 20D Processing unit (work support device) 21C Checking Vocabulary Dictionary 22A Word Sequence Analysis Unit 23A Words in order 22C Dependency Analysis Unit 23C Dependent words 24 Identification Unit 25 Evaluation Models 26B, 26D Exercise Speech Recognition Unit 27B, 27D Exercise: Repeat Vocabulary 30. Work support system 31 Inspection Result Notification Department 32. Voice Recognition Unit 33 Dictionary Generation Unit 34 Important Vocabulary Dictionary 35 Instructions Notification Department 36. Manual Data (This manual) 100 work systems

Claims

1. An important word extraction device that stores an evaluation model which learns using sentence structures extracted from past manuals, words from the past manuals that are repeated in inspection work where the work supervisor speaks first and the worker responds by repeating the spoken word, and statistical information attached to the repeated words, and outputs words from the past manuals that are repeated in the inspection work as important words from the past manuals, The device is characterized by having a work support device that inputs the sentence structure extracted from the current manual into the evaluation model, and registers the important words output from the evaluation model as keywords to be used in the inspection work based on the current manual in an important word dictionary. Work support system.

2. A memory unit that stores the data of the instruction manual, An evaluation model is trained using sentence structures extracted from past manuals, words from those past manuals that are repeated during inspection work where the work supervisor speaks first and the worker responds by repeating the utterance, and statistical information attached to those repeated words. The evaluation model outputs words from those past manuals that are repeated during the inspection work as important words from those past manuals. By inputting the sentence structures extracted from the current manual into the evaluation model, A dictionary generation unit that registers important words output from the evaluation model as keywords to be used in the inspection work based on the current procedure manual in the important word dictionary, A speech recognition unit compares keywords registered by the dictionary generation unit with words spoken first by the work supervisor and repeated by the worker in response to the inspection work based on the current procedure manual, and outputs the result of the comparison. It is characterized by having an inspection result notification unit that displays the output result of the voice recognition unit. Work support system.

3. The aforementioned evaluation model is a model that accepts time-series input data, The aforementioned important word extraction device and the work support device are characterized by extracting the sequence of words extracted from the manual as a sentence structure, and inputting the extracted sequence of words as time-series input data into the evaluation model. The work support system according to claim 1.

4. The work support device is characterized by adding words included in the current procedure manual that were commonly spoken by multiple people during an exercise based on the current procedure manual, which was performed prior to the inspection work based on the current procedure manual, to the input data of the evaluation model. The work support system according to claim 1.

5. The aforementioned evaluation model is a model that accepts input data of feature quantities that indicate dependency relationships between words, The aforementioned important word extraction device and the work support device are characterized by extracting feature quantities that show dependency relationships between words for work terms indicating the actions of the work extracted from the manual, as a sentence structure, and inputting the extracted feature quantities as input data into the evaluation model. The work support system according to claim 1.

6. The work support device is characterized by registering verbs from the important words registered in the important word dictionary as the next work term. The work support system according to claim 5.

7. The work support device is characterized in that, during the inspection work based on the current procedure manual, it checks whether or not important words registered in the important word dictionary have been spoken, and warns of any important words that have not been spoken. The work support system according to claim 1.

8. The system is characterized by having a processing unit that learns using sentence structures extracted from past manuals, words included in the past manuals that are repeated in inspection work where the work supervisor speaks first and the worker responds by repeating the spoken word, and statistical information attached to the repeated words, and outputs the words included in the past manuals that are repeated in the inspection work as important words included in the past manuals, and then inputs sentence structures extracted from the current manual into the evaluation model, thereby registering the important words output by the evaluation model as keywords to be used in the inspection work based on the current manual in the important word dictionary. Work support device.

9. The work support system includes a key word extraction device and a work support device. The aforementioned important word extraction device learns using sentence structures extracted from past manuals, words from the past manuals that are repeated during inspection work in which the work supervisor speaks first and the worker responds by repeating the utterance, and statistical information attached to the repeated words, and stores an evaluation model that outputs words from the past manuals that are repeated during the inspection work as important words included in the past manuals. The aforementioned work support device is characterized by inputting the sentence structure extracted from the current manual into the evaluation model, thereby registering the important words output from the evaluation model as keywords to be used in the inspection work based on the current manual in the important word dictionary. Work support method.

Images