Machine motion command program search device

Abstract

A machine operation command program search device according to the present invention comprises: a machine operation command program reading unit that reads a machine operation command program; a label assigning unit that assigns to a code or a code string of the read machine operation command program a label associated with the work executed by the industrial machinery controlled by the code or the code string; a voice conversion unit that converts an inputted voice signal to a search character string; a label searching unit that searches for the converted search character string from among the labels assigned to the code or the code string of the machine operation command program, and outputs the code or the code string corresponding to the searched label as the search result; and an output that outputs the search result.

Description

Technical Field

[0001] The present invention relates to a mechanical operation command program search device.

Background Art

[0002] When controlling industrial machines such as machine tools and robots, the mechanical operation command program executed has a very long code line count of several hundred lines or more (for example, see Patent Document 1). Each code line of the mechanical operation command program is in a form that primitively represents mechanical operations with instruction codes corresponding to operation commands and variables to be referenced (coordinate values, rotational speeds, etc.). Therefore, although individual operations can be grasped, it is difficult to grasp the outline of operation control performed by combining a plurality of commands. That is, from the perspective of an operator, the mechanical operation command program has low readability / searchability.

[0003] The mechanical operation command program commands a predetermined operation by combining a plurality of code lines. Examples of operations include, for example, an operation of "changing a tool", an operation of "moving to a cutting start point", an operation of "starting drilling", an operation of "starting rough machining (finish machining)", and the like. Such operations can be expressed more abstractly in natural language.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] When creating a machine motion command program, it is possible to embed these work details within the program by associating them with the code in the form of comments. However, this embedding process is burdensome for the creator of the machine motion command program. Furthermore, it is difficult to pre-embed such comments in machine motion command programs that are automatically generated by CAD / CAM equipment, etc. Therefore, when searching for the code line that performs the relevant work within a machine motion command program, the operator generally needs to browse the program from beginning to end, read the code line by line, decipher the work that each code line represents, and proceed line by line until they reach the relevant processing section. This process is extremely time-consuming and prone to errors, such as confusing similar processing sections. Therefore, there is a need for technology that can assist in navigating to the relevant line of code within a program. [Means for solving the problem]

[0006] The machine operation command program search device disclosed herein enables the cursor to be moved to the code line corresponding to the operation in the machine operation command program by inputting a work expression in natural language via voice. For example, if "tool change operation" is voice-input, the cursor can be moved to the code sequence for tool change, or if "drilling operation" is voice-input, the cursor can be moved to the code sequence for drilling.

[0007] Furthermore, one aspect of the present disclosure is a machine operation command program retrieval device that includes commands for controlling the operation of an industrial machine, comprising: a machine operation command program reading unit that reads the machine operation command program; and a unit that, with respect to the code or code sequence of the read machine operation command program, performs the work to be performed by the industrial machine controlled by the code or code sequence. The relationship is a word, phrase, or sentence that indicates the aforementioned operation.The machine operation command program search device comprises: a label assignment unit that assigns labels; an audio conversion unit that converts an input audio signal into a search string; a label search unit that searches for the search string converted by the audio conversion unit among the labels assigned to the code or code sequence of the machine operation command program and outputs the code or code sequence corresponding to the searched label as a search result; and an output unit that outputs the search results. [Effects of the Invention]

[0008] In one aspect of this disclosure, an operator can input a voice expression that signifies a predetermined task, thereby displaying the code line in a machine operation command program for performing that task. [Brief explanation of the drawing]

[0009] [Figure 1] This is a hardware configuration diagram of a machine operation command program search device according to the first embodiment. [Figure 2] This is a block diagram showing the functions of a machine operation command program search device according to the first embodiment. [Figure 3] This diagram shows an example of labeling rules. [Figure 4] This figure shows an example of labeling a machine operation command program. [Figure 5] This is an example of how search results are displayed. [Figure 6] This is a hardware configuration diagram of a machine operation command program search device according to the second embodiment. [Figure 7] This is a block diagram showing the functions of a machine operation command program search device according to the second embodiment. [Figure 8] This is a block diagram showing a modified example of a machine operation command program search device according to the second embodiment. [Figure 9] This block diagram shows the functions of a machine operation command program search device according to other embodiments. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a schematic hardware configuration diagram showing the main parts of a machine operation command program search device according to the first embodiment of the present invention. The machine operation command program search device 1 of the present invention can be implemented, for example, as a control device that controls industrial machinery such as machine tools based on a machine operation command program. Furthermore, the machine operation command program search device 1 of the present invention can be implemented on a personal computer attached to a control device that controls industrial machinery based on a machine operation command program, or on a personal computer, cell computer, fog computer 6, cloud server 7, or other computer connected to the control device via a wired / wireless network. In this embodiment, an example is shown in which the machine operation command program search device 1 is implemented on a personal computer connected to a control device that controls industrial machinery via a network.

[0011] The CPU 11 in the machine operation command program search device 1 according to this embodiment is a processor that controls the machine operation command program search device 1 as a whole. The CPU 11 reads the system program stored in the ROM 12 via the bus 22 and controls the entire machine operation command program search device 1 according to the system program. The RAM 13 temporarily stores temporary calculation data, display data, and various data input from external sources.

[0012] The non-volatile memory 14 is composed of, for example, a memory backed up by a battery not shown in the figure, an SSD (Solid State Drive), etc., and retains its stored state even when the power of the machine operation instruction program search device 1 is turned off. In the non-volatile memory 14, a machine operation instruction program and data read from an external device 72 via an interface 15, a machine operation instruction program and data input via an input device 71, a machine operation instruction program and data acquired from an industrial machine 3, etc. are stored. The data stored in the non-volatile memory 14 may be expanded to the RAM 13 during execution / use. Also, various system programs such as a known analysis program are pre-written in the ROM 12.

[0013] The interface 15 is an interface for connecting the CPU 11 of the machine operation instruction program search device 1 and an external device 72 such as a USB device. From the external device 72 side, for example, a control program stored in advance and data related to the operation of each industrial machine 3 can be read. Also, a control program and setting data edited within the machine operation instruction program search device 1 can be stored in external storage means via the external device 72.

[0014] The interface 20 is an interface for connecting the CPU of the machine operation instruction program search device 1 and a wired or wireless network 5. The network 5 is connected to industrial machines 3, fog computers 6, cloud servers 7, etc., and data is exchanged with the machine operation instruction program search device 1.

[0015] To the display device 70, data obtained as a result of execution of each data, program, etc. read onto the memory, etc. is output and displayed via an interface 17. Also, an input device 71 composed of a keyboard, a pointing device, etc. passes commands, data, etc. based on operations by an operator to the CPU 11 via an interface 18.

[0016] A voice sensor 75 such as a microphone is connected to the interface 19 by wire or wirelessly. The voice sensor 75 collects the voice uttered by the operator, converts it into a voice signal, and passes it to the CPU 11. The voice sensor 75 may be attached to the mechanical operation command program search device 1 or may be wearable by the operator.

[0017] FIG. 2 schematically shows the functions of the mechanical operation command program search device 1 according to the first embodiment of the present invention as a block diagram. Each function of the mechanical operation command program search device 1 according to the present embodiment is realized by the CPU 11 of the mechanical operation command program search device 1 shown in FIG. 1 executing a system program and controlling the operations of each part of the mechanical operation command program search device 1.

[0018] The mechanical operation command program search device 1 of the present embodiment includes a mechanical operation command program reading unit 120, a labeling unit 130, a voice conversion unit 140, a label search unit 150, an output unit 160, and an estimation unit 180. Also, on the RAM 13 to the non-volatile memory 14 of the mechanical operation command program search device 1, there are provided a mechanical operation command program storage unit 210 which is an area for storing the mechanical operation command program, and a label assignment rule storage unit 230 in which a label assignment rule for defining rules for assigning labels to code sequences is stored in advance.

[0019] The mechanical operation command program reading unit 120 reads out the mechanical operation command program 220 specified based on the operator's operation or the like from the mechanical operation command program storage unit 210. The mechanical operation command program 220 is stored in the mechanical operation command program storage unit 210 in advance by the operator's operation. This mechanical operation command program 220 may be, for example, read from an external device 72. It may also be obtained from the industrial machine 3, the fog computer 6, or the cloud server 7 via the network. The mechanical operation command program 220 read by the mechanical operation command program reading unit 120 may be displayed on the display device 70 so that the operator can view it by the output unit 160.

[0020] The labeling unit 130 assigns labels to codes or code sequences included in the machine operation command program 220 based on a plurality of labeling rules stored in the labeling rule storage unit 230. A labeling rule is a rule that defines the correlation between a code or code sequence and a label. This rule may, for example, specify a label to be assigned to a code sequence that matches a predetermined condition. In this case, the predetermined condition may be a pattern matching condition including, for example, a regular expression. On the other hand, the label may be a word, phrase, or sentence that indicates the operation of the industrial machine 3 controlled based on the code sequence.

[0021] Figure 3 shows an example of a label assignment rule stored in the label assignment rule storage unit 230. In the example in Figure 3, pattern matching conditions are used as the conditions for the label assignment rule. For example, a label assignment rule is defined that indicates that if a code sequence satisfies any of conditions 1, 2, or 3, the code sequence is correlated with the label "hole drilling". This means that the control device controls the industrial machine 3 based on any of the code sequences that satisfy condition 1, condition 2, or condition 3, causing the industrial machine 3 to perform the operation of drilling holes in the workpiece. Such label assignment rules can be created in advance by the operator or the developer of the machine operation command program.

[0022] In addition to the above, known rule-based translation techniques may also be used as a method of labeling. In this case, the label for a code or code sequence is identified by translating each code line of the machine operation command program 220 into a sequence of labels. The labeling rule storage unit 230 defines the grammar of the code sequence and the rules (dictionary) for conversion to labels as labeling rules. The labeling unit 130 interprets the role of each code in the code sequence of the machine operation command program 220 based on the grammar defined as the labeling rules, and converts each code into a label by applying the conversion rules considering the interpreted role of each code. Then, it assigns the converted label to the machine operation command program 220 as the label for that code. The grammar of the code sequence and the rules for conversion to labels can be created in advance by the developer or others based on the specifications of the machine operation command program. Note that, just as the same word is translated into different words depending on the context in language translation, the rules are created so that even the same code will be assigned different labels depending on the sequence of preceding and succeeding code sequences. Since rule-based translation methods are well-known from other literature, a detailed explanation will be omitted in this specification.

[0023] Figure 4 shows an example of the labeling unit 130 assigning labels to codes in the machine operation command program 220. As illustrated in Figure 4, one label is assigned to one or more codes. In Figure 4, for example, the label "tool replacement" is assigned to the code "T06" in block N01, the code "M06" in block N02, and the code "T06" in block N03 of the machine operation command program 220. In addition, the label "hole drilling" is assigned to block N08. Furthermore, the label "hole drilling" is also assigned to blocks N09 to N12, which are different code sequences from block N8. This is because, although block N08 and blocks N09 to N12 are different code sequences, they are all code sequences that cause the industrial machine 3 to perform a hole drilling operation. Labels may be assigned on a code basis or to a code sequence. As long as it is possible to understand which labels are assigned to which codes, the correspondence between codes and assigned labels can be managed in any data format. The label assignment unit 130 may assign multiple labels to the same code or code sequence if the conditions of multiple label assignment rules are met for the same code or code sequence. In addition, when assigning a label to a code or code sequence, other labels with similar meanings or other labels indicating higher-level concepts may also be assigned.

[0024] The speech conversion unit 140 performs speech recognition on the audio signal input from the audio sensor 75 and converts it into a search string. The speech conversion unit 140 can recognize the search string from the audio signal using known speech recognition algorithms such as dynamic time stretching, hidden Markov models, or neural network methods. The search string converted by the speech conversion unit 140 is output to the label search unit 150. The speech conversion unit 140 may also output to the label search unit 150 a string converted from a series of words following a predetermined keyword if that keyword is detected in the string converted from the audio signal. For example, if the keyword is "code", and the string converted from the audio signal is "code hole-cleaning", the speech conversion unit 140 will output "hole-cleaning" following the keyword "code" to the label search unit 150 as a search string.

[0025] The label search unit 150 searches for labels assigned to the machine operation command program 220 based on the search string input from the voice conversion unit 140. The search results may include the first row of the code sequence to which the label is assigned in the machine operation command program 220, the range of the code sequence, etc. The search results from the label search unit 150 are output to the output unit 160. The output unit 160 then displays and outputs the machine operation command program 220 based on the search results. Figure 5 shows an example of the machine operation command program being displayed by the output unit 160. The output unit 160 may, for example, display on the display device 70 the first row of the code sequence to which the label corresponding to the search string searched by the label search unit 150 has been assigned. It may also highlight the corresponding code sequence by underlining, using a caret, shading, changing the text color, or changing the background color so that the corresponding code sequence can be easily identified. Furthermore, the string of the label that was the target of the search may also be displayed. In Figure 5, the machine operation command program was searched using the search string "kougukokanshori" and the codes in blocks N01 to N03 of the machine operation command program are found to be the corresponding codes. Therefore, block N01 is displayed as the first row, the cursor is moved to block N01, and blocks N01 to N03 are highlighted.

[0026] By using the machine operation command program search device 1 according to this embodiment, which has the above configuration, the operator can input an expression meaning a predetermined task by voice, and the code line for performing that task in the machine operation command program will be displayed. The labeling unit 130 flexibly assigns labels to the code sequences in the machine operation command program 220 based on labeling rules. The assigned labels are abstract concepts that mean the content of the task. Some tasks only make sense when multiple codes are executed. Therefore, compared to searching using the codes themselves, it becomes easier for the operator to find the intended location. Also, even if the code sequences are different, if the content of the tasks is of the same type, the same label will be assigned. Therefore, compared to searching using the codes themselves, it becomes easier for the operator to flexibly search for the code that performs the task they intend. This tendency is particularly noticeable when the same type of task is performed using different code sequences within the machine operation command program 220.

[0027] Figure 6 is a schematic hardware configuration diagram showing the main parts of a machine operation command program search device according to a second embodiment of the present invention. The machine operation command program search device according to this embodiment differs from the machine operation command program search device according to the first embodiment in that it uses a machine learning machine for label assignment. The hardware configuration of the machine operation command program search device 1 according to this embodiment is the same as that of the machine operation command program search device 1 according to the first embodiment, except that it includes a machine learning machine 100.

[0028] Interface 21 is an interface for connecting the CPU 11 and the machine learning machine 100. The machine learning machine 100 includes a processor 101 that controls the entire machine learning machine 100, a ROM 102 that stores system programs and the like, a RAM 103 for temporary storage in each process related to machine learning, and a non-volatile memory 104 used for storing models and the like. The machine learning machine 100 can observe various pieces of information that can be obtained by the machine operation command program search device 1 via interface 21. The machine operation command program search device 1 also obtains processing results output from the machine learning machine 100 via interface 21, and stores, displays, or transmits the obtained results to other devices via the network 5 or the like.

[0029] Figure 7 is a schematic block diagram showing the functions of the machine operation command program search device 1 according to the second embodiment of the present invention. Each function of the machine operation command program search device 1 according to this embodiment is realized by the CPU 11 of the machine operation command program search device 1 shown in Figure 6 and the processor 101 of the machine learning machine 100 executing a system program and controlling the operation of each part of the machine operation command program search device 1 and the machine learning machine 100.

[0030] The machine operation command program retrieval device 1 of this embodiment includes a machine operation command program reading unit 120, a label assignment unit 130, a voice conversion unit 140, a label retrieval unit 150, an output unit 160, and an estimation unit 180. Furthermore, a machine operation command program storage unit 210, which is an area for storing machine operation command programs, is provided on the RAM 13 to non-volatile memory 14 of the machine operation command program retrieval device 1. In addition, a model storage unit 240, which is an area for pre-storing machine learning models used by the estimation unit 180 when estimating labels, is provided on the RAM 103 to non-volatile memory 104 of the machine learning machine maker 100.

[0031] The functions of the machine operation command program reading unit 120, the voice conversion unit 140, the label search unit 150, and the output unit 160 in this embodiment are the same as the functions of the first embodiment.

[0032] In this embodiment, the labeling unit 130 assigns labels to the code sequences included in the machine operation command program 220 based on the label estimation results by the machine learning model 100. When the machine operation command program 220 is read, the labeling unit 130 inputs the code sequences included in the machine operation command program 220 to the estimation unit 180 of the machine learning model 100.

[0033] The model memory unit 240 stores a pre-trained model that has learned the correlation between code sequences and labels. This model is constructed to machine translate code sequences into label sequences and also output word alignments, and may be a known encoder-decoder model composed of a recurrent neural network, for example. Here, word alignment is information that represents the relationship between each label in the label sequence obtained by machine translation and its range in the corresponding code sequence. This model is constructed by performing machine learning with a code sequence as input and a sequence of assigned labels as output. For the data used during training, operators or developers of machine operation command programs can be asked to assign labels to various operation command programs in advance, and these can be used as training data. The model constructed in this way converts the input code sequence into feature representations in the encoder unit, translates it into a label sequence in the decoder unit, and also outputs word alignments. Since this pre-trained model is well known from other literature, a detailed explanation is omitted in this specification.

[0034] When the estimation unit 180 receives the code sequence of the machine operation command program 220 input from the labeling unit 130, it inputs the code sequence to the trained model stored in the model storage unit 240 and obtains a label sequence and word alignment as the model's output. The estimation unit then outputs the obtained label sequence and word alignment to the labeling unit 130. The labeling unit 130 uses the label sequence and word alignment obtained in this way to assign a label to each code in the code sequence.

[0035] By using the machine operation command program search device 1 according to this embodiment, which has the above configuration, the operator can input an expression meaning a predetermined operation by voice, and the code line for performing that operation in the machine operation command program will be displayed. By using machine learning technology, although there is some learning effort involved, the effort of creating labeling rules based on the specifications of the machine operation command program in advance is eliminated. Furthermore, even if there is ambiguity in the order in which the codes appear in the machine operation command program, it is possible to learn accordingly, so that labels can be flexibly assigned to the code sequences in the machine operation command program 220.

[0036] As a modification of the machine operation command program search device 1 according to the second embodiment, the labeling unit 130 may use feature vectors that represent the characteristics of words, phrases, or sentences instead of natural language words, phrases, or sentences as labels to be assigned. Feature vectors of words, phrases, or sentences represent the relationships between each word, phrase, or sentence as multidimensional vectors. Feature vectors of words, phrases, or sentences can be calculated using known methods such as Paragraph Vector, Doc2vec, Word2vec, Glove, and fastText. In this modification, a feature vector calculation unit 190 is provided on top of the machine learning model 100. The feature vector calculation unit 190 calculates the feature vector for each label estimated by the estimation unit 180 using the above-mentioned known methods with a pre-constructed model for calculating feature vectors, and outputs this to the labeling unit 130. The labeling unit 130 then assigns feature vectors to the codes or code sequences.

[0037] During a search, the label search unit 150 inputs the search string received from the speech conversion unit 140 to the feature vector calculation unit 190 to obtain a search feature vector. At this time, the feature vector calculation unit 190 may convert the feature vector of the search string in the feature vector space in natural language to a feature vector in the feature vector space obtained from the machine operation command program. The unit then calculates the distance between the obtained search feature vector and the feature vectors assigned to the codes or code sequences in the machine operation command program 220. Feature vectors whose distance is calculated to be below a predetermined threshold are considered similar to the search feature vector. The code or code sequence to which the feature vector is assigned is then output as the search result. Alternatively, the system may output the code or code sequence to which the feature vector closest in distance to the search feature vector is assigned among all the feature vectors assigned to the codes or code sequences in the machine operation command program 220 as the search result. Furthermore, a measure other than distance, such as cosine similarity, may be used instead of distance to measure similarity.

[0038] The modified machine operation command program search device 1 uses feature vectors as labels, enabling it to search for conceptually similar codes or code sequences to a search string obtained from input speech. For example, if natural language is used as a label, even if the operator inputs "code drilling" via the voice sensor 75, the code or code sequence with the label "hole cleaning" will not be found. However, when feature vectors are used as labels, the distance between the feature vectors of conceptually similar phrases such as "drilling," "tapping," and "bowling" is small. Therefore, the operator can perform flexible searches using words that indicate similar concepts, without having to use fixed words.

[0039] Although embodiments of the present invention have been described above, the present invention is not limited to the examples of embodiments described above, and can be implemented in various forms by making appropriate modifications. For example, the embodiment described above shows an example in which each function of the machine operation command program search device 1 is implemented on a single personal computer. However, some functions may be configured to be provided on other devices. For example, consider the case in which the machine operation command program search device 1 is implemented as a personal computer attached to the industrial machine 3, as illustrated in Figure 9. In this case, the machine operation command program storage unit 320 may be implemented on the industrial machine 3. In this configuration, the machine operation command program reading unit 120 reads the machine operation command program 220 from the machine operation command program storage unit 320 provided on the industrial machine 3 via the network 5. The operator inputs a search string by voice from the voice sensor 375 connected to the industrial machine 3 and transmits it to the voice conversion unit 140. The search results from the label search unit 150 are then transmitted to the industrial machine 3 via the network 5 by the output unit 160 and displayed on the display device 370.

[0040] Furthermore, in the above-described embodiment, the output unit 160 displays the search results to the display device 70. However, it is not limited to this, and for example, the block number or code sequence of the machine operation command program 220 to which the searched label is assigned may be output as sound from a speaker device (not shown).

[0041] Furthermore, the above-described embodiment showed an example of handling a numerical control program. However, the functions of the present invention can also be applied to, for example, teaching programs for controlling robots, or sequence programs expressed in text format. [Explanation of Symbols]

[0042] 1. Machine operation command program search device 3. Industrial Machinery 5 Network 6. Fog Computer 7 Cloud Server 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15 Interfaces 17,18,19,20,21 Interface 22 buses 70,370 Display device 71 Input device 72 External equipment 75,375 Voice Sensors 100 machine learning machines 101 Processors 102 ROM 103 RAM 104 Non-volatile memory 120 Machine operation command program reading unit 130 Labeling Unit 140 Voice Conversion Unit 150 Label Search Section 160 Output section 180 Estimation Department 210,320 Machine operation command program storage unit 220 Machine motion command program 230 Label assignment rule storage unit 240 Model Memory Unit

Claims

1. A search device for machine operation command programs that include commands for controlling the operation of industrial machinery, A machine operation command program reading unit reads the machine operation command program, A labeling unit assigns labels to the code or code sequence of the machine operation command program that have been read, which are words, phrases, or sentences that represent the work to be performed by the industrial machine controlled by the code or code sequence, A voice conversion unit that converts an input voice signal into a search string, A label search unit searches the search string converted by the voice conversion unit among the labels assigned to the code or code sequence of the machine operation command program, and outputs the code or code sequence corresponding to the searched label as the search result. An output unit that outputs the search results, A machine operation command program search device equipped with the following features.

2. The output unit displays and outputs the code or code sequence as the search result so that the operator can view it. The machine operation command program retrieval device according to claim 1.

3. The system further includes a labeling rule storage unit that stores labeling rules that define rules for assigning natural language labels to the code or code sequence of the machine operation command program, The labeling unit assigns the label to the code or code sequence of the machine operation command program based on the labeling rules. The machine operation command program retrieval device according to claim 1.

4. A model storage unit that stores a model in which the correlation between the code or code sequence of a machine operation command program and a label has been learned in advance, The system further comprises an estimation unit that uses the model stored in the model storage unit to estimate a label to be assigned to a code or code sequence of a machine operation command program, The labeling unit assigns the label to the code or code sequence of the machine operation command program based on the estimation result by the estimation unit. The machine operation command program retrieval device according to claim 1.

5. The aforementioned labels are in natural language. The machine operation command program search device according to claim 4.

6. The aforementioned labels are feature vectors, The system further includes a feature vector calculation unit that calculates a search feature vector based on the aforementioned search string, The label search unit searches for the label that is similar to the feature vector used for the search. The machine operation command program search device according to claim 4.

7. The feature vector calculation unit converts the feature vector of a search string in the feature vector space obtained from natural language into a feature vector in the feature vector space obtained from a machine motion command program. The machine operation command program search device according to claim 6.

Images