History management device

Abstract

This history management device comprises a display unit, and a history display control unit that displays, on the display unit, two or more pieces of history information relating to a result of executing a predetermined process by an industrial machine together with statistical information relating to the two or more pieces of history information.

Description

History Management Device

[0001] The present disclosure relates to a history management device.

[0002] There are known robot systems that use a visual sensor mounted on the robot to detect workpieces and perform tasks such as picking up the workpieces. In such robot systems, execution history data is output as the program is executed, and the execution results are used to analyze the execution results. Regarding the storage of execution history in a mechanical system, for example, Patent Document 1 describes a molding machine management system that stores and displays history data including inspection result data of molded products by an injection molding machine. Patent Document 2 describes a picking robot system configured to collect and store data representing the operation results of a picking robot.

[0003] JP 2022-170960 A JP 2020-040159 A

[0004] When analyzing execution history collected in a robot system that detects workpieces using a visual sensor and performs specified tasks, it is generally necessary to output the execution history data in a general-purpose format such as a text file and analyze it using application software on a personal computer. This type of work requires time and effort from the worker. There is a need for a history management device that allows workers to accurately and efficiently analyze the execution history output from the machine system.

[0005] One aspect of the present disclosure is a history management device that includes a display unit and a history display control unit that displays two or more pieces of history information regarding the results of execution of a specified process by an industrial machine, together with statistical information regarding the two or more pieces of history information, on the display unit.

[0006] These and other objects, features and advantages of the present invention will become more apparent from the detailed description of exemplary embodiments of the invention illustrated in the accompanying drawings.

[0007] 1 is a diagram showing the equipment configuration of a robot system including a history management device according to one embodiment. FIG. 2 is a functional block diagram related to a robot control device and an image processing device. FIG. 3 is a diagram showing an example of a history display screen generated by a history display control unit and displayed on a display unit. FIG. 4 is a diagram showing a situation in which one statistical value is selected on the history display screen and the history information corresponding to that statistical value is highlighted. FIG. 5 is a diagram showing a situation in which one statistical value is selected on the history display screen and the statistical value corresponding to that historical information is highlighted. FIG. 6 is a diagram showing a situation in which an operation screen for specifying a threshold or numerical range for that statistical value is displayed in response to the selection of one statistical value in the statistics display area. FIG. 7 is a diagram showing an example of history information including a history image. FIG. 8 is a diagram showing an example of a setting adjustment screen presented by a playback function. FIG. 9 is a flowchart showing a process in which a detection program is executed to detect a workpiece in a robot system and store history information. FIG. 10 is a flowchart showing a basic flow related to the display of history information by a history display control unit.

[0008] Next, embodiments of the present disclosure will be described with reference to the drawings. In the drawings, like components or functional parts are designated by like reference numerals. The scales of these drawings have been changed appropriately to facilitate understanding. Furthermore, the embodiment shown in the drawings is one example for implementing the present invention, and the present invention is not limited to the illustrated embodiment.

[0009] FIG. 1 is a diagram showing the equipment configuration of a robot system 100 including a history management device 60 according to one embodiment. As shown in FIG. 1, the robot system 100 includes a robot 30 equipped with a hand 33 at the end of its arm, a robot controller 20 for controlling the robot 30, a teaching pendant 10 connected to the robot controller 20, a visual sensor 70 attached to the end of the arm of the robot 30, an image processing device 50, and an external storage device 90. The robot system 100 can perform processes such as detection and determination of an object 1 (hereinafter referred to as workpiece 1) placed on a workbench 2 using the visual sensor 70, and handle the workpiece 1. The robot system 100 also has a function for outputting and saving history information as a result of work execution.

[0010] In this embodiment, the function of the history management device 60 is incorporated into the image processing device 50, which is responsible for the control of the visual sensor 70 and the image processing function. The history management device 60 is responsible for storing history information and for displaying the history information and related information.

[0011] The robot 30 as an industrial machine is assumed to be a vertical articulated robot here, but other types of robots, such as a Cartesian coordinate robot, a SCARA robot, or a parallel link robot, may also be used. The robot controller 20 controls the operation of the robot 30 in accordance with an operation program loaded into the robot controller 20 or commands input from the teaching pendant 10.

[0012] The image processing device 50 is connected to the robot control device 20, and has the function of controlling the visual sensor 70 and performing various image processing according to commands from the robot control device 20. For example, the image processing device 50 can detect the position of the workpiece 1 from an image captured by the visual sensor 70, and provide the detected position of the workpiece 1 to the robot control device 20. This allows the robot control device 20 to correct the taught position and perform operations such as picking up the workpiece 1.

[0013] The visual sensor 70 may be a camera that captures grayscale or color images, or a stereo camera or 3D sensor that can acquire distance images or 3D point clouds. The image processing device 50 stores a model pattern of the workpiece 1 and can perform image processing to detect the workpiece by pattern matching between the image of the workpiece in the captured image and the model pattern. The visual sensor 70 is assumed to be calibrated, and the image processing device 50 stores calibration data that defines the relative positional relationship between the visual sensor 70 and the robot 30. This allows positions on the image captured by the visual sensor 70 to be converted to positions on a coordinate system (such as a robot coordinate system) fixed to the workspace.

[0014] The image processing device 50 may be a dedicated image processing device or may be configured as a personal computer. The image processing device 50 may have a hardware configuration as a general computer, in which a processor, memory (ROM, RAM (volatile memory), non-volatile memory, etc.), a display unit 52, an operation unit 53 configured with input devices such as a keyboard and a mouse, an input / output interface, etc. are connected via a bus (see FIG. 2). The display unit 52 includes, for example, a liquid crystal display.

[0015] The robot control device 20 may have a hardware configuration as a general computer, in which a processor is connected to memory (ROM, RAM, non-volatile memory, etc.), a storage unit 22, an input / output interface, an operation unit including various operation switches, etc. via a bus.

[0016] The teaching pendant 10 may have a hardware configuration as a general computer, in which a processor is connected to a memory (ROM, RAM, non-volatile memory, etc.), a display unit, an operation unit configured with input devices such as a keyboard and a touch panel (software keys), an input / output interface, etc. via a bus. Note that instead of the teaching pendant 10, various information processing devices such as a tablet terminal, a smartphone, a personal computer, etc. may be used as the teaching device.

[0017] During operation of the robot system 100, one or more workpieces 1 are repeatedly input into the robot system 100, and detection and handling of the workpieces are repeatedly performed, and history information for each execution of the detection process is saved.

[0018] 2 is a functional block diagram of the robot control device 20 and the image processing device 50. As shown in FIG. 2, the robot control device 20 has a storage unit 22 and an operation control unit 121. The storage unit 22 is configured, for example, with a non-volatile memory or a hard disk drive, and stores operation programs and other various information. The operation control unit 121 controls the operation of the robot 30 in accordance with the operation program or commands from the teaching pendant 10. The operation control unit 121 has the function of interpreting the operation program, etc., to plan the trajectory of the robot 30, generating commands to control each axis of the robot 30, and executing servo control of the servo motors of each axis.

[0019] The operation control unit 121 can provide machine control information of the robot 30 to the image processing device 50 (history management device 60). The machine control information of the robot 30 includes the position (rotational position) of each joint axis and the position and orientation of a predetermined control part (for example, TCP (tool center point)). The image processing device 50 (history management device 60) can include the machine control information obtained from the robot control device 20 as part of history information related to the execution results of processing using the visual sensor 70.

[0020] The image processing device 50 includes a visual sensor control unit 151 and an image processing unit 152. The visual sensor control unit 151 has a function of controlling the visual sensor 70 in accordance with a program related to image processing. The image processing unit 152 has a function of performing various image processing, such as detecting the position and orientation of a workpiece from a captured image by pattern matching. The visual sensor control unit 151 and the image processing unit 152 may be functional blocks realized by a processor of the image processing device 50 executing a program. The image processing device 50 includes a storage unit (not shown) that stores model data, calibration data, and other various information used for pattern matching. Note that FIG. 2 illustrates a display unit 52 and an operation unit 53 as hardware components of the image processing device 50.

[0021] The history management device 60 includes a first storage unit 161, a storage control unit 162, a history display control unit 163, an output unit 164, and a setting adjustment unit 165. The storage control unit 162, the history display control unit 163, the output unit 164, and the setting adjustment unit 165 of the history management device 60 may be functional blocks realized by the processor of the image processing device 50 executing software (software related to history management and display).

[0022] The first storage unit 161 is a relatively high-speed memory that stores the results of processing performed by the visual sensor 70 (including historical images, processing results, parameter values, etc.) and historical information, including machine control information at the time of image capture. Note that terms such as high-speed and low-speed refer to read / write speeds. The first storage unit 161 may be configured using, but is not limited to, volatile memory such as DRAM or SRAM. The first storage unit 161 has the capacity to store multiple pieces of historical information. This allows current image data captured by the visual sensor 70 to be processed while past historical information is transferred to the second storage unit 191 and erased from the first storage unit 161. This prevents delays in the operation of the robot system 100 due to processing performed by the first storage unit 161. Furthermore, the ability to store multiple pieces of historical information allows the stored historical information to be used to investigate problems by tracing back to past executions of processing performed by the visual sensor 70.

[0023] The second storage unit 191 of the external storage device 90 is a memory that is slower and has a larger capacity than the first storage unit 161. The second storage unit 191 is not particularly limited, and may be configured with a non-volatile memory such as a flash memory, a solid-state drive (SSD), a hard disk drive, or an optical disk drive. The second storage unit 191 has a capacity that can store history information acquired during relatively long-term operation of the robot system 100. The second storage unit 191 may store history information transferred from the first storage unit 161, or history information may be directly written to the second storage unit 191 as a result of processing by the visual sensor 70.

[0024] The external storage device 90 (second storage unit 191 ) may be defined as one of the functional elements that make up the history management device 60 .

[0025] The storage control unit 162 executes control related to storage of history information, such as storage of history information by the first storage unit 161, storage of history information by the second storage unit 191, and transfer of history information from the first storage unit 161 to the second storage unit 191. The storage control unit 162 can link image data obtained from the visual sensor 70, execution results of processing (detection, determination, etc.) using the visual sensor 70, and machine control information, etc., and store them in the first storage unit 161 or the second storage unit 191 as a single piece of history information.

[0026] In this embodiment, the storage control unit 162 may store history information using the following operational procedure: (a1) Each time an image processing program (including photography) using a visual sensor is executed, the history information is stored in the first storage unit 161. (a2) When the amount of data stored in the first storage unit 161 becomes full or exceeds a predetermined amount of data, the history information is transferred to the second storage unit 191 in chronological order starting with the oldest, and the history information transferred to the second storage unit 191 is deleted from the first storage unit 161. Through the above operations, all of the history information temporarily stored in the first storage unit 161 can be stored in the second storage unit 191 without any omissions.

[0027] The history display control unit 163 has a function of displaying two or more pieces of history information and statistical information related to the two or more pieces of history information on the display unit 52. The history display control unit 163 can read out history information from the first storage unit 161 and the second storage unit 191, calculate statistical information related to the history information, and display the statistical information together with the history information on the display unit 52. Furthermore, the history display control unit 163 can display two or more pieces of history information included in specified selection conditions from the multiple pieces of history information stored in the first storage unit 161 and the second storage unit 191, together with the statistical information related to the two or more pieces of history information on the display unit 52.

[0028] The output unit 164 provides a function of outputting the history information and statistical information displayed on the display unit 52 by the history display control unit 163 as a file in a predetermined format.

[0029] The setting adjustment unit 165 has the function of restoring the parameters contained in the displayed history information to an image processing program (detection program, etc.) in response to a specified operation being performed on the displayed history information, and presenting a setting adjustment screen for adjusting the program and parameters.

[0030] 3 shows an example of a history display screen 300 generated by the history display control unit 163 and displayed on the display unit 52. As shown in Fig. 3, the history display screen 300 includes a history display area 310 that displays history information, and a statistics display area 320 that displays statistical information related to the history information. The history display screen 300 also includes an operation area 330 that includes a field for specifying selection conditions for the history information to be displayed and various operation buttons.

[0031] The history display area 310 displays history information included in the selection conditions specified in the operation area 330, for example, in chronological order. The size of the history display area 310 increases or decreases depending on the number of history information included in the selection conditions. FIG. 3 shows a situation in which three pieces of history information are displayed in order from most recent to least recent. Note that if the number of pieces of history information included in the selection conditions exceeds the number that can be simultaneously displayed on the history display screen 300, all of the history information included in the selection conditions may be displayed by scrolling the screen. In the example of the history display screen 300, each piece of history information includes, as display items related to the detection results, time (date and time), event, duration (processing time), and X (mm), Y (mm), Z (mm), W (deg), P (deg), and R (deg) as position and orientation information of the detected object.

[0032] The statistics display area 320 displays statistical information related to the history information included in the history display area 310 (i.e., included in the selection conditions). In the example of the history display screen 300, the statistics display area 320 displays a plurality of types of statistical values ​​related to each of the detection results (display items) included in the history information, namely, the average value, minimum value, maximum value, range (numerical range), and variance, such as time (processing time), X (mm), Y (mm), Z (mm), W (deg), P (deg), and R (deg).

[0033] Note that the display items of the history information shown in FIG. 3 are examples, and the history information may include various information. For example, the history information may include the number of detections, detection score, contrast, distortion, angle, etc. as detection results. Note that here, the number of detections represents the number of workpieces detected in one detection process. The detection score represents an evaluation value regarding the degree of match between the object on the image and the workpiece model in the detection process. The contrast represents the contrast of the outline of the object on the image against the background. The distortion represents the degree of distortion of the object on the image relative to the workpiece model. The angle represents the deviation in the rotational direction of the object on the image relative to the workpiece model. The statistical information may include statistical values ​​(average value, minimum value, maximum value, range (numerical range), variance) for each of the number of detections, detection score, contrast, distortion, and angle.

[0034] The operation area 330 includes a date and time specification field 331 for specifying the date and time range of the history information to be displayed, and an event specification field 332 for specifying the event of the history information to be displayed. By specifying a date and time range in the date and time specification field 331, history information included in the specified date and time range can be displayed and can also be used to obtain statistical information. The event specification field 332 provides a function for narrowing down the history information to be displayed (to be used as the subject of statistical calculations) by specifying the type of processing (detection, determination, etc.). It is also possible to specify that all history information contained in the storage unit (first storage unit 161 and second storage unit 191) be displayed.

[0035] The operation area 330 further includes a download button 333, an auto update button 334, an update button 335, and a playback button 336. Pressing the download button 333 outputs the displayed history information and statistical information as a tabular data file. Pressing the download button 333 may display a screen for specifying an output device or folder to which the data file is to be output. The function of outputting the history information and statistical information as a file by pressing the download button 333 is provided by the output unit 164.

[0036] The automatic update button 334 is a button for specifying whether to update the displayed history information and statistical information each time new history information is added to the first storage unit 161. In other words, the automatic update button 334 is a button for specifying whether to automatically update the history information and statistical information in real time. For example, if the date and time specification field 331 specifies that the display target is from a certain point in the past to the "present time," and the automatic update button 334 is pressed in this state, the history display control unit 163 adds new history information to the history display area each time the new history information is added to the first storage unit 161 and updates the statistical information using the new history information. If the automatic update button 334 is not pressed, the history information and statistical information are not automatically updated. This function of updating the history information and statistical information in real time allows the user to constantly check the latest execution results and quickly check execution results after adjusting parameters of a detection program, etc.

[0037] The update button 335 provides a function for the user to manually update the history information. For example, if the update button 335 is pressed when the display target is specified to be from a certain point in the past to the "present time" in the date and time specification field 331 and the automatic update button 334 has not been pressed, the history display control unit 163 updates the history information and statistical information to the latest status.

[0038] The playback button 336 provides a function for restoring various setting parameters included in the history information to a program and adjusting the program and parameters. For example, when the user selects desired history information and presses the playback button 336, the setting adjustment unit 165 restores various setting parameters included in the selected history information to the program and presents a setting adjustment screen for adjusting the program.

[0039] 3, the history display control unit 163 can display history information and statistical information in a tabular format within the same table. Furthermore, on the history display screen 300, values ​​for the same display item are displayed side by side in the same column. This display format allows the user to quickly and accurately grasp the relationship between each display item of the history information and its associated statistical value.

[0040] 3, multiple types of detection results (display items) included in each piece of history information are displayed aligned in the row direction, and multiple types of statistical values ​​related to each detection result are displayed aligned in the column direction at display positions in the alignment direction (row direction) of each detection result. Instead of this display format, multiple types of detection results included in each piece of history information may be displayed aligned in the column direction on the history display screen 300, and multiple types of statistical values ​​related to each detection result may be displayed aligned in the row direction at display positions in the alignment direction (column direction) of each detection result.

[0041] By displaying the history information and the related statistical information in the same table on the history display screen 300, the user can quickly and accurately grasp the detection accuracy status of the detection program and take necessary measures. For example, if the user recognizes a trend in the displayed history information that shows large contrast variations and unstable detection scores, the user can determine that lighting is affecting the detection accuracy and take measures such as adding more lighting. Alternatively, the user may change the parameters of the detection program based on the statistical values ​​included in the statistical information. For example, the user may check the minimum detection score and determine a detection score threshold based on that minimum value and reflect it in the detection program. Alternatively, the user may set the detection score threshold of the detection program to a value within 3 sigma of the average detection score.

[0042] Note that, when each piece of history information displayed in the history display area 310 includes further information such as a history image, the further information such as the history image may be displayed by selecting the history information and performing a predetermined operation. FIG. 7 shows an example of an information display screen 210 that is displayed by selecting one piece of history information included in the history display area 310 and performing a predetermined operation. The information display screen 210 includes a history image G and display fields 211 for the number of detections, score (detection score), contrast, distortion, and angle. The information display screen 210 may be displayed simultaneously with the history display screen 300 in a window format, or may be displayed independently as a display screen separate from the history display screen 300. The history display control unit 163 controls these functions related to the presentation of the information display screen 210.

[0043] The history display control unit 163 has a function of improving the convenience of user confirmation by displaying the statistical information displayed in the statistics display area 320 in association with the history information displayed in the history display area 310. For example, the history display control unit 163 can provide the following functions: (F1) A function of highlighting history information corresponding to (including) a selected statistical value (e.g., a maximum value) displayed in the statistics display area 320. (F2) A function of highlighting a statistical value corresponding to a piece of history information included in the history display area 310 when that piece of history information is selected. For example, if the selected history information has a detection result (item) that is the maximum statistical value, the corresponding maximum value in the statistics display area 320 is highlighted.

[0044] FIG. 4 shows a display state in which the maximum Y value, "13.5," in the statistical display area 320 is selected by the above-mentioned function (F1), and the history information 312 having this maximum value is highlighted. Note that highlighting can take various forms, such as changing the color, thickness, or font of the text, or adding a cell color or pattern. For example, as shown in FIG. 4, selecting one of the statistical values ​​may be performed by placing a cursor C1 on one of the statistical values ​​and clicking the mouse. By highlighting the corresponding history information in response to a selected statistical value, the user can instantly confirm the history information corresponding to the selected statistical value (e.g., the maximum value). Furthermore, the user can instantly adjust the parameters of the history information 312 by pressing the playback button 336 while the history information 312 is highlighted.

[0045] In addition, in function (F1), if the history information corresponding to the selected statistical value includes further information such as a history image, a transition to an information display screen that displays all history information including the history image may be made depending on the selection of the statistical value.

[0046] FIG. 5 illustrates a situation in which, when history information 311 is selected in the history display area 310 using the above function (F2), statistical values ​​corresponding to the detection results (display items) included in the selected history information 311 are highlighted. For example, selecting a piece of history information may be performed by placing a cursor C1 on the piece of history information and clicking the mouse, as shown in FIG. 5 . In this example, the Y value ("11.1") included in the selected history information 311 is the statistical minimum, and the X value ("-50.3"), Z value ("80.5"), W value ("-31.2"), P value ("-90"), and R value ("0.1") are the statistical maximums, so these statistical values ​​are highlighted. In this case, the user can immediately confirm that the selected history information contains a statistically characteristic value (in this case, the maximum or minimum value).

[0047] The history display control unit 163 may further have the following function (F3): (F3) A function of presenting an operation screen for receiving an operation to specify a threshold value (or a numerical range) for the selected statistical value in response to the selection of a statistical value included in the statistical information, and highlighting, from among the history information displayed in the history display area 310, the history information selected by the threshold value (or numerical range) specified via the operation screen.

[0048] FIG. 6 illustrates a situation in which an operation screen 361 for specifying a threshold value for the statistical value is displayed in response to the selection of the "average value of X" included in the statistical information by function (F3). The operation screen 361 may be displayed, for example, by placing a cursor C1 on the statistical value and performing a specific operation (right-clicking and selecting the operation screen from the menu that appears). The operation screen 361 is configured to accept the specification of a threshold value for the X value. For example, if the user specifies the threshold value for the X value as "±10 from the average value" via the operation screen 361, the history display control unit 163 may highlight, among the history information included in the history display area 310, history information whose X value exceeds the specified threshold value (+10) (or, in the case of a negative value, whose absolute value is greater than the threshold value (−10)).

[0049] For example, function (F3) allows a user to quickly check a group of historical information whose statistical values ​​are greater than a specific value, a group of historical information whose statistical values ​​are less than a specific value, or a group of historical information whose statistical values ​​fall within a specific range. Furthermore, if necessary, the user can instantly adjust parameters for the highlighted historical information using the playback function. Thus, function (F3) provides a function for searching historical information based on statistical value selection criteria.

[0050] The history display control unit 163 may further include a function (F4) for displaying statistical information based on a formula independently set by the user. The history display screen 300 in FIG. 3 illustrates, as an example, a state in which a formula (columns F + G + H) is set by the user in column L (denoted by reference numeral 321) using function (F4), and the calculation result based on this formula is displayed in column L. The history display screen 300 in FIG. 3 also illustrates, as an example, a state in which a formula is specified in row 322 to calculate the sum of the numerical data for each display item of the first to third pieces of history information. Based on this formula, columns E to L of row 322 display the sum of the numerical values ​​for the first to third pieces of history information corresponding to each column. The output data output by pressing the download button 333 also includes statistical information based on the formula set by the user.

[0051] The history display control unit 163 may have a function (F5) that allows the user to exclude history information included in the history display area 310 from the calculations used to obtain statistical information in the statistics display area 320. For example, as shown in FIG. 3 , a check box 315 may be added to each piece of history information to specify that the information be excluded from statistical calculations. In this case, the history display control unit 163 excludes the history information for which the check box 315 is checked from the statistical calculations and recalculates and displays the statistical information. This function (F5) allows the user to update the statistical information by excluding specific history information selected by viewing the statistical information (e.g., history information whose detection position is significantly different from other history information).

[0052] In this way, the history display function via the history display screen 300 as a user interface allows the user to evaluate the execution history based on the information the user values. In addition, the function that allows the user to set mathematical expressions also allows the user to evaluate the history information based on their own evaluation criteria. In other words, the history display screen 300 is also customizable.

[0053] As described above, the history display control unit 163 has a function of restoring parameters included in the history information to a program and displaying a setting adjustment screen for adjusting the program and parameters when the playback button 336 is pressed while a piece of history information displayed in the history display area 310 is selected or highlighted. Fig. 8 shows an example of a setting adjustment screen 400 that is presented when the playback button 336 is pressed while a piece of history information displayed in the history display area 310 is selected or highlighted. As shown in Fig. 8, the setting adjustment screen 400 includes a detection program display area 411 that displays the detection program, an image display area 412 that displays the history image, a parameter display area 413, a detection result display area 414, and an operation button display area 415 that displays various operation buttons.

[0054] As an example, the detection program display area 411 displays a detection program titled "Two-dimensional correction for one camera." This detection program includes a snapshot icon 431, a pattern match icon 432, and a correction data calculation icon 433. The snapshot icon 431 represents a command to perform imaging using the visual sensor 70. The pattern match icon 432 represents a command to perform workpiece detection by pattern matching on the captured image. The correction data calculation icon 433 represents a command to calculate correction data for correcting the teaching position based on the workpiece position obtained by pattern matching.

[0055] In the initial state after transitioning from the history display screen 300 to the setting adjustment screen 400, the image display area 412 displays a history image 452 of the history information that was selected or highlighted on the history display screen 300.

[0056] The parameter display area 413 displays, as various parameters of the detection program, the restored values ​​of the parameters in the history information that were selected or highlighted on the history display screen 300. FIG. 8 shows a state in which a pattern match icon 432 is selected in the detection program display area 411, and various parameters related to the pattern match icon 432 are displayed in the parameter display area 413. The parameters of the pattern match icon 432 include a score threshold, a contrast threshold, and various other parameters. By selecting another icon in the detection program display area 411, the parameters (restored parameters) for that icon can be displayed in the parameter display area 413 and adjusted.

[0057] The processing results of the detection program are displayed in the detection result display area 414. Fig. 8 shows a state in which the detection results (number of detections, position, score, contrast, etc.) included in the history information that was selected or highlighted on the history display screen 300 are displayed in the detection result display area 414.

[0058] Operation buttons for performing various operations are displayed in the operation button display area 415. After adjusting various parameters in the parameter display area 413, the user can press the detection button 441 to execute the detection program with the new parameters and check the results.

[0059] In this way, the user can restore parameters to the program from the history display screen 300 and transition to the setting adjustment screen 400 where the parameters can be adjusted. Since the history information and statistical information are associated with each other on the history display screen 300, the user can select history information that needs adjustment by looking at the statistical information, and smoothly transition to the parameter adjustment screen for adjusting the parameter value of that history information.

[0060] FIG. 9 is a flowchart illustrating the process by which the robot system 100 detects the workpiece 1 and stores the history information through the execution of the detection program. As shown in FIG. 9 , the detection program executes, causing the visual sensor 70 to capture an image of the workpiece 1 (step S11). Then, the image processing unit 152 of the image processing device 50 executes the detection process (step S12). The detection result (detected position of the workpiece 1) is used to control the robot 30 in the operation control unit 121. Next, the history information is stored in the storage units (first storage unit 161 and second storage unit 191) by the above-described storage function (operation procedures (a1) and (a2)) of the storage control unit 162 of the history management device 60 (step S13). Workpieces are repeatedly input into the robot system 100, and this process is repeatedly executed accordingly. As a result, history information is accumulated in the storage units (first storage unit 161 and second storage unit 191), and the latest history information is added.

[0061] 10 is a flowchart showing the basic flow of displaying history information by the history display control unit 163. This process (history display process) is executed under the control of the processor of the history management device 60 (the processor of the image processing device 50 in this embodiment). The history display process is started, for example, when a predetermined operation is performed to instruct the history management device 60 (image processing device 50) to display history. The history display control unit 163 reads history information from the storage units (the first storage unit 161 and the second storage unit 191) according to selection conditions specified in the date and time specification field 331, etc., and displays the information in the history display area 310 on the history display screen 300 (step S21).

[0062] Next, the history display control unit 163 calculates statistical information regarding the displayed history information, and displays the calculated statistical information in the same table as the history information (step S22).

[0063] When the operation update mode is activated by pressing the automatic update button 334, which updates history information in real time (S23: YES), the history display control unit 163 reads history information according to the selection conditions from the storage unit and repeats the process of updating the history information and statistical information (steps S21 and S22). Note that the "automatic update mode" in step S23 corresponds to the case where displaying history up to the present is specified in the date and time specification field 331 and the automatic update button 334 is pressed. When the automatic update mode is not specified (S23: NO), this process is terminated. Thereafter, this history display process may be started in response to the occurrence of various events that should start this history display process (such as an operation to specify selection conditions in the operation area 330, an operation to press the update button 335, or an operation to press the automatic update button 334).

[0064] The above-described embodiment is an example of a configuration in which the robot system includes a visual sensor and the history information relates to the results of a detection process based on an image captured by the visual sensor. However, the above-described embodiment can also be applied to configurations in which the robot system includes various sensors and the history information includes at least one of the sensor output and a processing result based on the sensor output. For example, consider a robot system equipped with a force sensor between a flange at the end of a robot arm and a tool (e.g., a hand) and configured to execute a force control program based on the detected value of the force sensor. The force sensor outputs the detected value of an external force acting on the robot (tool). In this case, the force control program may include programs for precision fitting, polishing, and other tasks. A control device functioning as a history management device in this robot system executes the force control program and records history information as the execution results. In this case, the history information may include the execution start date and time, the force control program name, the position where force control was executed (the position of the control point), and the force sensor detection values ​​at that position (force and moment in each axial direction). Note that the parameters of the force control program (force control parameters) include, for example, a target pressing force and a force control gain. The control device can calculate statistical information for multiple pieces of history information (detected values) recorded by multiple executions of the force control program, as in the above-described embodiment, and display the statistical information together with the history information. Note that instead of force sensors, torque sensors disposed on each axis of the robot may be used as sensors for detecting external forces acting on the robot (tool).

[0065] The configuration of the above-described embodiment can be applied to the display of history information output by the execution of work in an industrial machinery system involving measurements by various sensors.

[0066] In the above-described embodiment, a configuration in which the function of the history management device 60 is realized within the image processing device 50 has been described. However, this configuration is merely an example, and a configuration example in which the function of the history management device 60 is realized within the robot control device 20 is also possible. A configuration example in which the entire function of the image processing device 50, including the function of the history management device 60, is realized within the robot control device 20 is also possible. When the function of the history management device 60 is realized within the robot control device 20, the functional blocks constituting the history management device 60 may be arranged within the robot control device, and the external storage device 90 may be connected to the robot control device 20. In this case, the history display control unit 163 may display various screens, such as a history display screen, on the display unit of the teaching operation panel 10, and operation inputs to the history display screen, etc., may be received via the operation unit of the teaching operation panel 10 (or a touch panel screen integrated with the display unit, etc.).

[0067] Alternatively, the function of the history management device 60 may be realized in the teaching pendant 10. In this case, the function block of the history management device 60 is arranged in the teaching pendant 10. The external storage device 90 may be connected to the teaching pendant 10.

[0068] In the above-described embodiment, a configuration is exemplified in which an external storage device 90 is connected to the history management device 60 as the second storage unit 191, but a non-volatile memory that functions as the second storage unit 191 may also be built into the history management device 60.

[0069] The functional blocks of the history management device shown in Figure 2 may be realized by one or more processors of the history management device executing various software stored in a storage device, or may be realized by a hardware-based configuration such as an ASIC (Application Specific Integrated Circuit).

[0070] The programs for executing various processes related to history management, such as the history display process in the above-described embodiments, can be recorded on various computer-readable recording media (e.g., semiconductor memory such as ROM, EEPROM, flash memory, magnetic recording media, optical disks such as CD-ROM, DVD-ROM, etc.).

[0071] As described above, according to each embodiment, it becomes possible for an operator to accurately and efficiently analyze the execution history information output from the machine system.

[0072] Although the present disclosure has been described in detail, the present disclosure is not limited to the individual embodiments described above. Various additions, substitutions, modifications, partial deletions, etc. are possible in these embodiments without departing from the gist of the present disclosure or the spirit of the present disclosure derived from the content of the claims and their equivalents. These embodiments can also be implemented in combination. For example, in the above-described embodiments, the order of each operation and the order of each process are shown as examples and are not limited to these. The same applies when numerical values ​​or mathematical expressions are used in the description of the above-described embodiments.

[0073] The following supplementary notes are provided regarding the above-described embodiment and modified examples. (Supplementary Note 1) A history management device (60) comprising a display unit (52) and a history display control unit (163) that displays, on the display unit (52), two or more pieces of history information related to execution results of a predetermined process by an industrial machine and statistical information related to the two or more pieces of history information. (Supplementary Note 2) The history management device (60) according to Supplementary Note 1 further comprises a memory unit (161, 191) that stores multiple pieces of history information related to execution results of the predetermined process by the industrial machine, and the history display control unit (163) displays, from the multiple pieces of history information stored in the memory unit (161, 191), the two or more pieces of history information included in specified selection conditions and the statistical information related to the two or more pieces of history information. (Supplementary Note 3) The history management device (60) according to Supplementary Note 1 or 2, wherein the statistical information includes multiple types of statistical values ​​related to at least one item included in each piece of history information. (Supplementary Note 4) The history management device (60) according to any one of Supplements 1 to 3, wherein the history display control unit (163) displays the two or more pieces of history information and statistical information relating to the two or more pieces of history information in the same table. (Supplementary Note 5) The history management device (60) according to Supplementary Note 4, wherein the table displays multiple types of items included in each piece of history information aligned in one direction of rows or columns, and multiple types of statistical values ​​relating to each of the items aligned in the other direction of the rows or columns at display positions in the alignment direction of each of the items. (Supplementary Note 6) The history management device (60) according to any one of Supplements 1 to 5, wherein the history display control unit (163), in response to an operation of selecting one statistical value included in the statistical information, highlights the history information corresponding to the selected one statistical value from among the two or more pieces of displayed history information. (Appendix 7) The history management device (60) described in any one of Appendices 1 to 6, wherein the history display control unit (163) highlights a statistical value in the statistical information corresponding to one piece of displayed history information in response to an operation of selecting one piece of displayed history information.(Supplementary Note 8) The history management device (60) according to any one of Supplements 1 to 7, wherein the history display control unit (163) displays, on the display unit, an operation screen for specifying a threshold or a numerical range for a selected statistical value in response to an operation of selecting a statistical value included in the statistical information, and highlights, from the two or more pieces of displayed history information, history information selected by the specified threshold or numerical range for the selected statistical value. (Supplementary Note 9) The history management device (60) according to any one of Supplements 1 to 8, wherein the history display control unit (163) accepts specification of a calculation formula using the statistical information or numerical data included in the history information, and displays a calculation result using the specified calculation formula. (Supplementary Note 10) The history management device (60) according to any one of Supplements 1 to 9, wherein the history display control unit (163) accepts an operation of specifying, from the two or more pieces of displayed history information, history information to be excluded from calculation of the statistical information. (Supplementary Note 11) The history management device (60) according to Supplementary Note 2, wherein new history information is stored in the storage unit (161, 191) each time the industrial machine executes the predetermined process, and wherein the history display control unit (163) adds new history information and displays it on the display unit in response to new history information that is history information included in the specified selection condition being added to the storage unit, and updates the statistical information based on the new history information. (Supplementary Note 12) The history management device (60) according to any one of Supplements 1 to 11, further comprising: an output unit (164) that outputs the two or more pieces of history information and the statistical information displayed on the display unit as a file in a predetermined format. (Supplementary Note 13) The history management device (60) according to any one of Supplements 1 to 12, further comprising: a setting adjustment unit (165) that restores parameters included in one piece of history information to a program and displays a setting adjustment screen on the display unit for adjusting the program and parameters in response to a predetermined operation being performed while one piece of history information is selected or highlighted on the display unit.(Supplementary Note 14) The history management device (60) according to any one of Supplementary Notes 1 to 13, wherein the predetermined processing by the industrial machine includes processing using a sensor, and the history information includes at least one of an output from the sensor or a result of processing the output from the sensor. (Supplementary Note 15) The history management device (60) according to Supplementary Note 14, wherein the sensor is a visual sensor (70), and the history information includes a result of executing detection using the visual sensor (70).

[0074] REFERENCE SIGNS LIST 1 Workpiece 2 Workbench 10 Teaching operation panel 20 Robot control device 22 Memory unit 30 Robot 33 Hand 50 Image processing device 52 Display unit 53 Operation unit 60 History management device 70 Visual sensor 90 External storage device 100 Robot system 151 Visual sensor control unit 152 Image processing unit 161 First storage unit 162 Storage control unit 163 History display control unit 164 Output unit 165 Setting adjustment unit 191 Second storage unit 300 History display screen 310 History display area 320 Statistics display area 330 Operation area 400 Setting adjustment screen 411 Detection program display area 412 Image display area 413 Parameter display area 414 Detection result display area 415 Operation button display area

Claims

1. A display unit; a history display control unit that displays, on the display unit, two or more pieces of history information related to execution results of a predetermined process by the industrial machine and statistical information related to the two or more pieces of history information; A history management device comprising:

2. a storage unit that stores a plurality of pieces of history information related to execution results of predetermined processes by the industrial machine; The history management device according to claim 1, wherein the history display control unit displays the two or more pieces of history information included in the specified selection conditions from among the plurality of pieces of history information stored in the memory unit, and statistical information relating to the two or more pieces of history information.

3. The history management device according to claim 1 , wherein the statistical information includes a plurality of types of statistical values ​​relating to at least one item included in each piece of history information.

4. The history management device according to claim 1 , wherein the history display control unit displays the two or more pieces of history information and statistical information relating to the two or more pieces of history information in the same table.

5. The history management device of claim 4, wherein the table displays multiple types of items contained in each of the history information aligned in one direction of rows or columns, and multiple types of statistical values ​​related to each of the items aligned in the other direction of the rows or columns at display positions in the alignment direction of each of the items.

6. The history management device according to any one of claims 1 to 5, wherein the history display control unit highlights, in response to an operation of selecting one statistical value included in the statistical information, the history information corresponding to the selected statistical value among the two or more displayed history information.

7. The history management device according to any one of claims 1 to 5, wherein the history display control unit highlights a statistical value in the statistical information corresponding to one piece of displayed history information in response to an operation of selecting one piece of displayed history information.

8. The history display control unit displays an operation screen on the display unit for specifying a threshold or numerical range for one statistical value included in the statistical information in response to an operation of selecting the one statistical value included in the statistical information, and highlights, from the two or more pieces of displayed history information, the history information selected by the threshold or numerical range specified for the one selected statistical value.

9. 6. The history management device according to claim 1, wherein the history display control unit accepts specification of a calculation formula using numerical data included in the statistical information or the history information, and displays the calculation results using the specified calculation formula.

10. The history management device according to claim 1 , wherein the history display control unit accepts an operation to specify, from among the two or more pieces of displayed history information, history information to be excluded from calculation of the statistical information.

11. new history information is stored in the storage unit every time the predetermined process is executed by the industrial machine; The history management device of claim 2, wherein the history display control unit adds new history information that is included in the specified selection conditions to the storage unit, displays the new history information on the display unit, and updates the statistical information based on the new history information.

12. The history management device according to claim 1 , further comprising an output unit that outputs the two or more pieces of history information and the statistical information displayed on the display unit as a file in a predetermined format.

13. A history management device as described in any one of claims 1 to 5, further comprising a setting adjustment unit that, in response to a predetermined operation being performed while one piece of history information is selected or highlighted on the display unit, restores parameters contained in the one piece of history information to a program and displays a setting adjustment screen on the display unit for adjusting the program and parameters.

14. the predetermined process by the industrial machine includes a process using a sensor, The history management device according to claim 1 , wherein the history information includes at least one of an output from the sensor and a result of processing the output from the sensor.

15. the sensor is a visual sensor; The history management device according to claim 14 , wherein the history information includes a result of detection performed using the visual sensor.