Verification assistance device and computer-readable recording medium
The verification support device addresses hallucination issues in AI-generated setting information by iteratively updating settings based on expected operations, ensuring reliable and efficient industrial machine control.
Patent Information
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- FANUC LTD
- Filing Date
- 2024-01-25
- Publication Date
- 2026-06-04
AI Technical Summary
The challenge of using generation AI devices for generating setting information for industrial machines is the risk of hallucination, leading to unreliable output that requires manual correction, increasing man-hours.
A verification support device that operates industrial machinery and control devices, compares test results with expected operations, and uses an interactive response device to iteratively update setting information until matching results are achieved, leveraging a Large Language Model (LLM) for prompt-based corrections.
Ensures reliable setting information by reducing the need for manual correction, thereby decreasing man-hours and ensuring consistent, problem-free operation across multiple industrial setups.
Smart Images

Figure JP2024002130_04062026_PF_FP_ABST
Abstract
Description
Verification support device and computer-readable recording medium
[0001] The present disclosure relates to a verification support device and a computer-readable recording medium.
[0002] In order to start a control device that controls industrial machines such as machine tools and robots, it is necessary to create setting information for the control device. The setting information of the control device includes setting parameters of the control device, each offset value, ladder program, macro program, and the like. In order to generate such setting information, it is conceivable to utilize a generation AI device (for example, Patent Document 1). In order to utilize the generation AI device, it is necessary to create a predetermined prompt and input it into the generation AI device.
[0003] Japanese Patent No. 7353695
[0004] When the generation AI device responds, there is a problem that hallucination may occur. Hallucination is that the generation AI device generates information not based on facts. For this reason, it is difficult to trust and use the setting information generated by the generation AI device as it is. There is a risk in using the setting information generated by the generation AI device in one trial. When setting information with problems is output, it is necessary to manually correct the setting information, which requires man-hours. In the field, a support technique for reducing the burden of verification work of setting information generated by the generation AI device is desired.
[0005] The verification support device described in this disclosure operates models of industrial machinery and control devices to which configuration information has been applied, based on test operations, and compares the results of the operation (test results) with predetermined expected operation. If there is a difference between the test results and the expected operation, a prompt is created to update the configuration information, and an interactive response device equipped with a generation AI such as a Large Language Model (LLM) is requested to generate configuration information that resolves the difference between the test results and the expected operation based on the created prompt. Then, the models of industrial machinery and control devices to which the configuration information received from the interactive response device has been applied are operated again with test operations. These operations are repeated until the test results and the expected operation match, thereby solving the above problem.
[0006] [Correction based on Rule 91, March 17, 2026] Furthermore, one aspect of the present disclosure is a verification support device comprising: an acquisition unit that acquires setting information of a control device for controlling an industrial machine, a test operation in the control device, and the expected operation of the industrial machine assumed from the test operation; a test execution unit that performs a test operation relating to the control device and the industrial machine based on the test operation and acquires a test result relating to the test operation; a determination unit that determines whether the test result matches the expected operation; a prompt creation unit that, if the determination unit determines that the test result does not match the expected operation, creates a prompt for updating the setting information based on the expected operation and the test result; a transmission unit that transmits the prompt to an interactive response device; a reception unit that receives updated setting information based on the prompt from the interactive response device; an update unit that updates the setting information based on the updated information; and an output unit that outputs the setting information when the determination unit determines that the test result matches the expected operation, wherein the device repeats the test operation using the updated setting information until the determination unit determines that the test result matches the expected operation.
[0007] This is a schematic hardware configuration diagram of a verification support device according to one embodiment of the present disclosure. This is a block diagram showing the schematic functions of a verification support device according to the first embodiment. This is a schematic diagram showing examples of multiple templates stored in the template storage unit. This is a schematic diagram showing an example of a template. This is a schematic diagram showing an example of a response from an interactive response device.
[0008] Embodiments of this disclosure will be described below with reference to the drawings. In the following description, components having the same or similar functions will be denoted by the same reference numerals. Duplication of these components may be omitted.
[0009] In this application, "based on XX" means "based on at least XX," and includes cases where it is based on another element in addition to XX. Furthermore, "based on XX" is not limited to cases where XX is used directly, but also includes cases where it is based on something that has been calculated or processed. "XX" is any element (for example, any information).
[0010] [First Embodiment] Figure 1 is a schematic hardware configuration diagram showing the main parts of a verification support device according to the first embodiment of the present disclosure. The verification support device 1 according to this embodiment can be implemented, for example, on a control device that controls industrial machinery. The verification support device 1 can also be implemented on a personal computer attached to the control device, or on a personal computer, cell computer, fog computer 6, cloud server 7, or other computer connected to the control device via a wired / wireless network. In this embodiment, an example is shown in which the verification support device 1 is implemented on a computer connected to a control device that controls industrial machinery via a network.
[0011] The CPU 11 in the verification support device 1 according to this embodiment is a processor that controls the verification support device 1 as a whole. The CPU 11 reads the system program stored in the ROM 12 via the bus 22 and controls the entire verification support device 1 according to the system program. The RAM 13 temporarily stores temporary calculation data, display data, and various data acquired from external sources.
[0012] The non-volatile memory 14 is composed of, for example, a memory backed up by a battery (not shown) or an SSD (Solid State Drive), and its stored state is maintained even when the power to the verification support device 1 is turned off. The non-volatile memory 14 stores various programs and data read from external devices 72 via the interface 15, programs and data input via the input device 71, and various programs and data acquired from the control device 3 that controls the industrial machine 4 and other devices via the network 5. The various programs and data stored in the non-volatile memory 14 may be expanded into the RAM 13 when executed / used. In addition, various system programs, such as known analysis programs, are pre-written in the ROM 12.
[0013] Interface 15 is an interface for connecting the CPU 11 of the verification support device 1 to an external device 72 such as a USB device. From the external device 72, for example, system programs, configuration data, etc., can be read. In addition, various programs and configuration data created and edited within the verification support device 1 can be stored in an external storage means via the external device 72.
[0014] Interface 20 is an interface for connecting the CPU 11 of the verification support device 1 to a wired or wireless network 5. The network 5 may communicate using technologies such as serial communication (RS-485, for example), Ethernet® communication, optical communication, wireless LAN, Wi-Fi®, Bluetooth®, etc. The network 5 is connected to a control device 3 that controls at least one industrial machine 4 to be verified, an interactive response device 2 that responds to user inquiries, a fog computer 6, a cloud server 7, etc., and exchanges data with the verification support device 1.
[0015] The display device 70 displays data loaded into memory, data obtained as a result of executing various programs, etc., which are output via the interface 17. In addition, the input device 71, which consists of a keyboard, pointing device, voice input device, etc., transmits commands and data based on operator operations to the CPU 11 via the interface 18.
[0016] The interactive response device 2 is configured as a device that returns a response sentence in response to a predetermined sentence. The interactive response device 2 is equipped with a Large Language Model (LLM) that has learned response sentences for predetermined sentences. This model can be a known model such as the Transformer model. The large language model learns the probability that each text is a response to a predetermined sentence when that sentence is input. It also learns the probability that each other text follows that text. By repeating this learning process, the large language model learns to output a sequence of texts with a high probability as a response to a predetermined sentence when that sentence is input. Depending on how the interactive response device 2 is trained to learn response documents for predetermined sentences, it can be used for purposes such as dialogue, question and answer, text summarization, text editing, text translation, text transformation, text modification, text optimization, text interpretation, text detection, recognition, prediction, judgment, code generation, image generation, and overall judgment.
[0017] The verification support device 1 described herein is assumed to be connected to the interactive response device 2 via a network 5. The interactive response device 2 is equipped with a model that has been trained based on various information, for example, that is publicly available on the internet. If necessary, additional training (fine-tuning) may be performed in advance using various documents, including the manuals for the control device 3 and the industrial machine 4, and reports that record problems that have occurred in the past with the control device 3 and the industrial machine 4, the causes of those problems, and solutions to those problems. By narrowing down the information used for training, it is possible to prepare an interactive response device 2 that can return responses to inquiries with a certain degree of accuracy.
[0018] Figure 2 is a schematic block diagram showing the functions of the verification support device 1 according to the first embodiment of this disclosure. Each function of the verification support device 1 according to this embodiment is realized by the CPU 11 of the verification support device 1 shown in Figure 1 executing a system program and controlling the operation of each part of the verification support device 1.
[0019] The verification support device 1 of this embodiment includes an acquisition unit 100, a test execution unit 110, a determination unit 120, a prompt creation unit 130, a transmission unit 140, a reception unit 150, an update unit 160, and an output unit 170. In addition, the RAM 13 to the non-volatile memory 14 of the verification support device 1 is provided with a template storage unit 200, which is an area that stores templates related to prompts in advance.
[0020] The acquisition unit 100 acquires setting information of the control device 3 that controls the industrial machine 4, test operations performed by the control device 3, and the expected operation of the industrial machine 4 that can be assumed from the test operations.
[0021] The configuration information is information related to the control of the industrial machine 4 by the control device 3, and examples include numerical control parameters, numerical control programs, offset information related to the operation of the industrial machine 4, and PLC programs. The configuration information may be, for example, what is set in the control device 3 and the industrial machine 4 in the initial state. Alternatively, it may be configuration information created by an operator for the control device 3 and the industrial machine 4, or configuration information created for other control devices and industrial machines. Furthermore, configuration information generated in advance by the interactive response device 2 may be acquired.
[0022] A test operation is a set of commands to cause the industrial machine 4 to perform a test operation. A test operation may be, for example, a numerical control program executed by the control device 3 when controlling the industrial machine 4. The operator may input a test operation by specifying a numerical control program related to a test operation that has been created in advance.
[0023] Expected operation is information that indicates the expected operation of the industrial machine 4, for example, by performing control based on a test operation. The expected operation may specify the state of the industrial machine 4 during or at the end of the test operation, such as, for example, that no alarm is generated and automatic operation ends. The expected operation may also specify target values and target ranges for statistical values (for example, maximum, minimum, and average values of speed, load, temperature, error, etc.) calculated based on sensor values and feedback values detected in each part of the industrial machine 4. These statistical values may be specified for each command included in the test operation or for each command group. For example, information such as "the average value of the position deviation of the X coordinate during axis movement from sequence number N120 to N140 is a [mm] or less", "during the execution of sequence number N200, the torque load increases to c [N・m] or more at a position where the X coordinate is b [mm]", and "signal F, which was 1 before the test operation, changes to 0 after the test operation" can be expected operations. The expected operation may specify target values and target ranges for statistical values and the state of the industrial machine 4 for each command included in the test operation.
[0024] The acquisition unit 100 may display a screen for inputting setting information, test operations, and expected operations to the display device 70, and acquire this information based on the operator's actions. Alternatively, the information may be acquired from the control device 3 that controls the industrial machine 4 or from an external device 72. The acquisition unit 100 outputs the acquired setting information, test operations, and expected operations to the test execution unit 110.
[0025] The test execution unit 110 sets the setting information acquired by the acquisition unit to the control device 3 and the industrial machine 4, and operates the control device 3 based on the test operations acquired by the acquisition unit 100. It then acquires the operating status of the industrial machine 4 controlled by the control device 3 as a test result. The test execution unit 110 may, for example, instruct the control device 3 to perform settings based on the setting information and to execute a numerical control program as a test operation. Alternatively, it may perform a simulation process using a model that virtually simulates the control device 3 and the industrial machine 4 based on the setting information and the test operations. While the industrial machine 4 is performing the test operation, the test execution unit 110 acquires data from each part of the industrial machine 4 from the control device 3. It then acquires test results based on the acquired data. The test execution unit 110 outputs the setting information input from the acquisition unit 100, the test operations input from the acquisition unit 100, the acquired test results, and the expected operation input from the acquisition unit 100 to the determination unit 120.
[0026] The determination unit 120 compares the test results obtained by the test execution unit 110 with the expected operation. It then determines whether the test results and the expected operation match. The determination unit 120 may also determine whether the test results and the expected operation match perfectly. If the determination unit 120 determines that the test results and the expected operation match perfectly, it outputs the setting information at that time to the output unit 170. If the determination unit 120 determines that the test results and the expected operation do not match, it outputs the test operation, expected operation, and test results at that time to the prompt creation unit 130.
[0027] The prompt creation unit 130 creates a prompt to request the interactive response device 2 to update the setting information related to the control device 3 and the industrial machine 4 when the determination unit 120 determines that the test result and the expected operation do not match. The prompt creation unit 130 uses the expected operation and the test result when creating the prompt. The prompt creation unit 130 may assign prompt identification information to the generated prompt so that the prompt can be uniquely identified. The prompt creation unit 130 may, for example, create a prompt by applying the items included in the expected operation and the test result to a prompt template corresponding to the difference between the expected operation and the test result.
[0028] Figure 3 is a schematic diagram showing an example of multiple templates stored in the template storage unit 200. In the example in Figure 3, the template storage unit 200 stores multiple templates 1, 2, ... Each template is associated with multiple keywords, such as the model number of the control device 3 or industrial machine 4 and the control content. The prompt creation unit 130 searches the templates stored in the template storage unit 200 using keywords such as the model number of the control device 3 or industrial machine 4 and the control content. Then, it reads the template with the highest keyword match as the template to be used to create the prompt. It then generates a prompt using the read template. Finally, it sends the generated prompt to the transmission unit 140.
[0029] Figure 4 is a schematic diagram showing an example of a template. In the example in Figure 4, the template includes information keys for embedding predetermined information at predetermined positions in the prompt text. The prompt creation unit 130 generates a prompt by embedding the corresponding information at the positions of each information key. In the example in Figure 4, the model number of the industrial machine 4 is embedded at the position of "MODEL_NUM", and the model number of the tool used in the industrial machine 4 is embedded at the position of "TOOL_NUM". In addition, the numerical control program related to the test operation is embedded at the position of the information key "TEST_COMMAND". Furthermore, the input as the expected operation is embedded at the positions of the information keys "EXPECT_OPER1_1" and "EXPECT_OPER2_1", and the corresponding test results are embedded at the positions of the information keys "RESULT1_1" and "RESULT2_1", respectively. The template may also include information keys corresponding to information calculated by performing predetermined calculations based on predetermined machine information.
[0030] Such templates should be created in advance to match the format of inquiries to the interactive response device 2. For example, if the interactive response device 2 responds to inquiries that enumerate items, it is desirable that the template generate prompts that enumerate information related to specific operations, machine information, and verification objectives, as illustrated in Figure 4. If the system responds to inquiries in natural language, it is desirable that the template generate prompts in the form of natural sentences. It is desirable that each template be created to include information that will be helpful in creating answers to inquiries about the expected verification objectives of the configuration information.
[0031] The transmission unit 140 transmits the prompt generated by the prompt creation unit 130 to the interactive response device 2. Upon receiving the prompt from the transmission unit 140, the interactive response device 2 generates a response to the prompt. It then transmits the generated response to the verification support device 1.
[0032] The receiving unit 150 receives the response transmitted from the interactive response device 2. It then outputs the received response to the updating unit 160.
[0033] The update unit 160 extracts update information for the setting information from the response received by the receiving unit 150 from the interactive response device 2. Figure 5 is a schematic diagram showing an example of a response from the interactive response device 2. The response from the interactive response device 2 includes information relating to the update of at least one setting information. The update unit 160 extracts the update information for the setting information using, for example, known natural language processing or pattern matching techniques. Then, based on the extracted update information for the setting information, it updates the setting information of the control device 3 and the industrial machine 4 acquired by the acquisition unit 100. The setting information to be updated may include numerical control parameters, numerical control programs, offset information related to the operation of the industrial machine 4, PLC programs, etc.
[0034] Subsequently, the acquisition unit 100 instructs the test execution unit 110 to perform a test operation using the updated setting information. These processes are repeated until the determination unit 120 determines that the test result matches the expected operation.
[0035] The output unit 170 outputs the setting information when the determination unit 120 determines that the test result matches the expected operation. The output unit 170 may also output to the display device 70. Alternatively, it may output to a display device (not shown) of the control device 3 that controls the industrial machine 4 via the network 5. Furthermore, it may transmit output via the network 5 to other computers such as the fog computer 6 or the cloud server 7.
[0036] The verification support device 1 according to this embodiment, which has the above configuration, can obtain reliable setting information by repeatedly verifying the setting information related to the industrial machine 4 and the control device 3. Since only setting information that is judged to have no problems with the final operation is output, it can be reused as is while maintaining a reasonable level of reliability. As a result, the operator does not need to modify the setting information, and a reduction in man-hours can be expected.
[0037] The support provided by the verification support device 1 can be effectively utilized in manufacturing sites equipped with multiple sets of control devices 3 and industrial machines 4. For example, there are cases where it is necessary to apply setting information used in one control device 3 and industrial machine 4 to another control device 3 and industrial machine 4 and operate it. In this case, even if the expected operation is not obtained when the setting information is applied to the other control device 3 and industrial machine 4 and a test operation is performed, the setting information can be updated to suit the other control device 3 and industrial machine 4 based on the response from the interactive response device 2. Therefore, the burden of setting work on the control devices 3 and industrial machines 4 by the operator can be reduced.
[0038] As one modification of the verification support device 1 according to this embodiment, the transmission unit 140 may also transmit supplementary information when sending a prompt generated by the prompt creation unit 130 to the interactive response device 2. Examples of supplementary information include the manual for the control device 3, the manual for the industrial machine 4, various information related to the control device 3, various information related to the industrial machine 4, information related to the drive unit that drives the industrial machine 4 (such as the motor model number), information related to the environment in which the industrial machine 4 operates, and test results based on similar setting information. By transmitting supplementary information to the interactive response device 2 along with the prompt, the accuracy of the response by the interactive response device 2 can be improved, and the setting information can be updated more reliably.
[0039] [Other Embodiments] In the embodiments described above, the setting information, test operations, and expected operations acquired by the acquisition unit 100 may be acquired from the operator, the industrial machine 4, the external equipment 72 of the verification support device 1, etc. Alternatively, a storage device may be provided on an external computer such as a fog computer 6 or a cloud server 7 connected via the network 5, and the above information may be acquired from there.
[0040] While embodiments of this disclosure have been described in detail above, this disclosure is not limited to the individual embodiments described above. These embodiments can be added, replaced, modified, partially deleted, etc., in any way that does not depart from the spirit of the invention or from the idea and intent of this disclosure derived from the claims and their equivalents. For example, the order of operations and processes in the embodiments described above are shown as examples only and are not limited thereto. The same applies when numerical values or mathematical formulas are used in the description of the embodiments described above.
[0041] [Correction based on Rule 91 17.03.2026] The following are additions to the embodiments of the present disclosure. (Additional Note 1) The verification support device (1) according to the present disclosure includes an acquisition unit (100) that acquires setting information of a control device (3) that controls an industrial machine (4), a test operation in the control device (3), and the expected operation of the industrial machine (4) assumed from the test operation; a test execution unit (110) that performs a test operation related to the control device (3) and the industrial machine (4) based on the test operation and acquires the test result related to the test operation; a determination unit (120) that determines whether the test result matches the expected operation; and if the determination unit (120) determines that the test result does not match the expected operation, the device determines the setting based on the expected operation and the test result. The system includes a prompt creation unit (130) that creates a prompt for updating the setting information, a transmission unit (140) that transmits the prompt to an interactive response device (2), a reception unit (150) that receives updated setting information based on the prompt from the interactive response device (2), an update unit (160) that updates the setting information based on the updated information, and an output unit (170) that outputs the setting information when the determination unit (120) determines that the test result matches the expected operation, and the system repeats the test operation using the updated setting information until the determination unit (120) determines that the test result matches the expected operation.
[0042] (Note 2) The setting information used by the verification support device (1) according to this disclosure includes at least one of the setting parameters of the control device (3), the numerical control program, the offset information, and the PLC program. (Note 3) The transmission unit (140) of the verification support device (1) according to this disclosure transmits supplementary information to the interactive response device (2) in addition to the prompt. (Note 4) The supplementary information used by the verification support device (1) according to this disclosure includes at least one of the manual information, industrial machine information, control device information, drive unit information for driving the industrial machine, environmental information, and test results based on similar information.
[0043] (Note 5) The acquisition unit (100) of the verification support device (1) according to this disclosure acquires setting information entered by the operator or setting information received as a response from the interactive response device (2). (Note 6) The prompt creation unit (130) of the verification support device (1) according to this disclosure creates a prompt based on the test operation.
[0044] [Correction based on Rule 91, 17.03.2026] (Appendix 7) The computer-readable recording medium according to the present disclosure includes setting information of a control device (3) that controls an industrial machine (4), a test operation in the control device (3), and an acquisition unit (100) that acquires an expected operation of the industrial machine (4) assumed from the test operation, a test execution unit (110) that performs a test operation related to the control device (3) and the industrial machine (4) based on the test operation and acquires a test result of the test operation, a determination unit (120) that determines whether the test result matches the expected operation, and a prompt creation unit (130) that creates a prompt for updating the setting information based on the expected operation and the test result when the determination unit (120) determines that the test result does not match the expected operation, a transmission unit (140) that transmits the prompt to an interactive response device (2), a reception unit (150) that receives update information of the setting information based on the prompt from the interactive response device (2), an update unit (160) that updates the setting information based on the update information, and an output unit (170) that outputs the setting information when the determination unit (120) determines that the test result matches the expected operation. A program for operating a computer is recorded, and the test operation is repeated using the updated setting information until the determination unit (120) determines that the test result and the expected operation match.
[0045] 1 Verification support device 2 Interactive response device 3 Control device 4 Industrial machine 5 Network 6 Fog computer 7 Cloud server 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15, 17, 18, 20 Interface 22 Bus 70 Display device 71 Input device 72 External device 100 Acquisition unit 110 Test execution unit 120 Determination unit 130 Prompt creation unit 140 Transmission unit 150 Reception unit 160 Update unit 170 Output unit 200 Template storage unit
Claims
1. [Correction based on Rule 91 17.03.2026] Verification support device comprising: an acquisition unit that acquires setting information of a control device for controlling an industrial machine, a test operation in the control device, and the expected operation of the industrial machine assumed from the test operation; a test execution unit that performs a test operation relating to the control device and the industrial machine based on the test operation and acquires the test result relating to the test operation; a determination unit that determines whether the test result matches the expected operation; a prompt creation unit that, when the determination unit determines that the test result does not match the expected operation, creates a prompt for updating the setting information based on the expected operation and the test result; a transmission unit that transmits the prompt to an interactive response device; a receiving unit that receives updated setting information based on the prompt from the interactive response device; an update unit that updates the setting information based on the updated information; and an output unit that outputs the setting information when the determination unit determines that the test result matches the expected operation, wherein the determination unit repeats the test operation using the updated setting information until the determination unit determines that the test result matches the expected operation.
2. The verification support device according to claim 1, wherein the setting information includes at least one of the setting parameters of the control device, a numerical control program, offset information, and a PLC program.
3. The verification support device according to claim 1, wherein the transmitting unit transmits supplementary information in addition to the prompt to the interactive response device.
4. The verification support device according to claim 3, wherein the supplementary information includes at least one of manual information, industrial machine information, control device information, drive unit information for driving the industrial machine, environmental information, and test results based on similar information.
5. The verification support device according to claim 1, wherein the acquisition unit acquires setting information input by the operator or setting information received as a response from the interactive response device.
6. The verification support device according to claim 1, wherein the prompt generation unit generates a prompt based on the test operation.
7. [Correction based on Rule 91 17.03.2026] A computer-readable recording medium on which a program is recorded that causes a computer to operate as follows: an acquisition unit that acquires setting information of a control device that controls an industrial machine, a test operation in the control device, and the expected operation of the industrial machine assumed from the test operation; a test execution unit that performs a test operation relating to the control device and the industrial machine based on the test operation and acquires the test result of the test operation; a determination unit that determines whether the test result matches the expected operation; a prompt creation unit that, if the determination unit determines that the test result does not match the expected operation, creates a prompt for updating the setting information based on the expected operation and the test result; a transmission unit that transmits the prompt to an interactive response device; a reception unit that receives updated setting information based on the prompt from the interactive response device; an update unit that updates the setting information based on the updated information; and an output unit that outputs the setting information if the determination unit determines that the test result matches the expected operation; and the program repeats the test operation using the updated setting information until the determination unit determines that the test result matches the expected operation.