History management device

Abstract

This process management device has a display unit and a process display control unit that displays on the display unit two or more individual process information relating to a result of an execution of a specified process by an industrial machine, together with statistical information relating to the two or more individual process information.

Description

field of technology

[0001] The present disclosure relates to a history management device. State of the art

[0002] A robotic system is known for detecting a workpiece by means of a visual sensor attached to the robot and for performing a task such as removing the workpiece. In such a robotic system, execution history data is output when an operating program is executed and used to perform an analysis with respect to an execution result. With regard to storing an execution history in a machine system, PTL 1, for example, describes a management system for injection molding machines for storing and displaying history data, including inspection result data and the like, with respect to a molded part formed by an injection molding machine. PTL 2 describes a removal robot system designed to collect and accumulate data representing the results of removal robot operations. List of quotations Patent literature [PTL 1] Unexamined Japanese patent publication (Kokai) No. 2022-170960 A [PTL 2] Unexamined Japanese patent publication (Kokai) No. 2020-040159 A Overview: Technical Task

[0003] To analyze an execution history collected by a robot system trained to detect a workpiece using a visual sensor and perform a predefined operation based on the detection result, the execution history data generally needs to be output by the robot system in a universal format, such as a text file, so that it can be analyzed using application software running on a personal computer. However, this work requires a significant investment of effort and time from the operator. A history management device is desired that allows an operator to analyze an execution history output by a machine system in a suitable and efficient manner. Technical solution

[0004] One aspect of the present disclosure is a history management device comprising: a display unit; and a history display control unit configured to display on the display unit two or more individual history pieces of information relating to an execution result of a predetermined processing by an industrial machine, together with statistical information relating to the two or more individual history pieces of information.

[0005] The tasks, features and advantages and other tasks, features and advantages will become more apparent from the detailed description of typical embodiments of the present invention, which are illustrated in the accompanying drawings. Brief description of the drawings Fig. Figure 1 is a graphical representation depicting a configuration of a robot system that includes a process management device according to one embodiment. Fig. Figure 2 is a graphical function block representation relating to a robot control device and an image processing device. Fig. Figure 3 is a graphical representation that shows an example of a history display screen generated by a history display control unit and displayed on a display unit. Fig. Figure 4 is a graphical representation depicting a situation in which one of the statistics is selected on the history display screen and history information corresponding to the statistic is highlighted. Fig. Figure 5 is a graphical representation depicting a situation in which a single piece of history information is selected on the history display screen and a statistic corresponding to the history information is highlighted. Fig. Figure 6 is a graphical representation depicting a situation in which, in response to the selection of one of the statistics in a statistics display area, an operator screen is displayed for determining a threshold or numerical range with respect to the statistic. Fig. Figure 7 is a graphical representation that provides an example of the progress information, including a progress image. Fig. Figure 8 is a graphical representation that shows an example of a settings adjustment screen displayed by a playback function. Fig. Figure 9 is a flowchart that specifies a process for recognizing a workpiece and storing the progress information in the robot system by executing a recognition program. Fig. 10 is a flowchart that represents a basic process regarding the display of history information by the history display control unit. Description of the embodiments

[0006] Next, embodiments of the present disclosure are described with reference to the drawings. A similar configuration or functional area is indicated by the same reference numeral in the aforementioned drawings. For clarity, the scale in the drawings has been appropriately modified. An aspect shown in the drawing is an example of an implementation of the present invention, and the present invention is not limited to the aspect shown.

[0007] Fig. Figure 1 is a graphical representation depicting a configuration of a robot system 100 that includes a process management device 60 according to one embodiment. As shown in Fig. As shown in Figure 1, the robot system 100 comprises a robot 30, which includes a hand 33 mounted on an arm tip, a robot control unit 20 that controls the robot 30, a programming handheld device 10 connected to the robot control unit 20, a visual sensor 70 attached to the arm tip of the robot 30, an image processing device 50, and an external storage device 90. The robot system 100 can perform processing, such as detection and identification, of a target object 1 (hereinafter referred to as workpiece 1) arranged on a worktable 2 using the visual sensor 70 and can handle the workpiece 1. Furthermore, the robot system 100 has functions for outputting and storing progress information as a result of the work performed.

[0008] In the present embodiment, the function as a history management device 60 is integrated into the image processing device 50, which has functions for controlling the visual sensor 70 and for image processing. The history management device 60 manages storage of the history information and also has a function for displaying the history information and information relating to the history information.

[0009] The robot 30 is assumed to be a vertical articulated robot, but a different type of robot, such as an orthogonal coordinate robot, a SCARA robot, or a parallel linkage robot, could be used. The robot control unit 20 controls the movement of the robot 30 according to a motion program loaded into the robot control unit 20 or a command entered from the programming handheld device 10.

[0010] The image processing device 50 is connected to the robot control unit 20 and has a function for controlling the visual sensor 70 and performing various types of image processing according to a command from the robot control unit 20. For example, the image processing device 50 can detect the position of the workpiece 1 from an image captured by the visual sensor 70 and provide the detected position of the workpiece 1 to the robot control unit 20. In this way, the robot control unit 20 can correct a teach-in position and perform removal and similar operations of the workpiece 1.

[0011] The visual sensor 70 can be a camera that captures a grayscale or color image, or it can be a stereo camera or a three-dimensional sensor capable of capturing a distance image and a three-dimensional point group. The image processing device 50 holds a model pattern of a workpiece 1 and can perform image processing to recognize a workpiece by pattern matching between an image of the workpiece in a captured image and the model pattern. It is assumed that calibration has been performed on the visual sensor 70, and it is assumed that the image processing device 50 has calibration data that defines a relative positional relationship between the visual sensor 70 and the robot 30.In this way, a position on an image captured by the visual sensor 70 can be converted into a position in a coordinate system defined in a work area (such as a robot coordinate system).

[0012] The image processing device 50 can be a device specifically designed for image processing, or it can be a personal computer. The image processing device 50 can have a hardware configuration as a general-purpose computer, in which a memory (such as ROM and RAM (volatile memory) and non-volatile memory), a display unit 52, an operator unit 53 consisting of an input device such as a keyboard and a mouse, an input / output interface, and the like are connected to a processor via a bus (see Fig. 2) The display unit 52 includes, for example, a liquid crystal display.

[0013] The robot control device 20 can have a hardware configuration as a general computer in which a memory (such as a ROM, a RAM and a non-volatile memory), a storage unit 22, an input / output interface, an operator unit which includes various operating switches and the like are connected to a processor via a bus.

[0014] The programming handheld device 10 can have a hardware configuration as a general-purpose computer, in which a memory (such as ROM, RAM, and non-volatile memory), a display unit, an operator interface consisting of an input device such as a keyboard and a touch panel (software button), an input / output interface, and the like are connected to a processor via a bus. It should be noted that various information processing devices, such as a tablet computer, a smartphone, and a personal computer, can be used as a training device instead of the programming handheld device 10.

[0015] At the time of operation of the robot system 100, one or more workpieces 1 are repeatedly inserted into the robot system 100 and recognition and handling of the workpiece are repeatedly carried out, so that the progress information is stored as a result of the execution of each part of the recognition processing.

[0016] Fig. Figure 2 is a graphical function block representation relating to the robot control unit 20 and the image processing device 50. As in Fig. As shown in Figure 2, the robot control unit 20 includes a memory unit 22 and a motion control unit 121. The memory unit 22 is, for example, a non-volatile memory or a hard disk device and stores a motion program and various other types of information. The motion control unit 121 controls the movement of the robot 30 according to a motion program or a command from a programming handheld device 10. The motion control unit 121 has functions for interpreting a motion program and the like, for planning a trajectory of the robot 30, for generating a command to control each axis of the robot 30, and for performing servo control of a servo motor of each axis.

[0017] The motion control unit 121 can provide machine control information relating to the robot 30 to the image processing device 50 (process management device 60). This machine control information for the robot 30 includes the position (rotational position) of each articulation axis and the position and orientation of a predefined control area (for example, a tool center point (TCP)). The image processing device 50 (process management device 60) can incorporate the machine control information acquired by the robot control unit 20 into the process information relating to the execution result of the processing using the visual sensor 70.

[0018] The image processing device 50 includes a control unit 151 for the visual sensor and an image processing unit 152. The control unit 151 for the visual sensor has a function for controlling the visual sensor 70 according to a program with respect to image processing. The image processing unit 152 has a function for performing various types of image processing, such as processing to detect the position and orientation of a workpiece from a captured image by pattern matching. The control unit 151 for the visual sensor and the image processing unit 152 can be functional blocks that are achieved by executing a program by the processor of the image processing device 50. The image processing device 50 includes a memory unit (not shown) that stores model data used for pattern matching, calibration data, and various other types of information.It should be noted that... Fig. 2 the display unit 52 and the operating unit 53 are represented as hardware components of the image processing device 50.

[0019] The history management device 60 includes a first storage unit 161, a storage control unit 162, a history display control unit 163, an output unit 164, and a setting adjustment unit 165. The storage control unit 162, the history display control unit 163, the output unit 164, and the setting adjustment unit 165 of the history management device 60 can be functional blocks that are achieved by executing software (software relating to the management and display of a history) by the processor of the image processing device 50.

[0020] The first memory unit 161 is a relatively high-speed memory that stores historical information, including the execution result of processing by the visual sensor 70 (including a historical image, an execution result of processing, a parameter value, and the like) and machine control information at the time of image acquisition. It should be noted that when terms like "high speed," "low speed," and the like are used to describe the performance of a memory, these terms refer to read / write speeds. The first memory unit 161 can be, but is not limited to, volatile memory such as DRAM or SRAM. The first memory unit 161 has the capacity to store a plurality of individual pieces of historical information.Since historical information from the first storage unit 161 can be transferred to the second storage unit 191 and historical information from the first storage unit 161 can be deleted simultaneously with the processing of current image data acquired by the visual sensor 70, a delay in the movement of the robot system 100 due to processing with respect to the first storage unit 161 can be prevented. Furthermore, since the majority of individual historical information can be stored, an investigation of a problem caused by the processing performed by the visual sensor 70 in the past can be carried out using the stored historical information.

[0021] The second storage unit 191 of the external storage device 90 is a memory that has a lower speed and a larger capacity than the first storage unit 161. The second storage unit 191 can be configured as non-volatile memory, such as flash memory, a solid-state drive (SSD), a hard disk drive, or an optical disk drive, but is not limited to these. The second storage unit 191 has the capacity to store multiple individual history data points collected during long-term operation of the robot system 100. The second storage unit 191 can store the history data transferred from the first storage unit 161, or the history data can be written directly to the second storage unit 191 as a result of processing by the visual sensor 70.

[0022] It should be noted that the external storage device 90 (the second storage unit 191) can be defined as one of the functional components that form the history management device 60.

[0023] The storage control unit 162 performs control with regard to the storage of historical information, such as storing historical information in the first storage unit 161, storing historical information in the second storage unit 191, and transferring historical information from the first storage unit 161 to the second storage unit 191. The storage control unit 162 can store image data acquired by the visual sensor 70, the result of processing (such as detection and identification) using the visual sensor 70, machine control information, and the like, in conjunction with each other, as a single piece of historical information in either the first storage unit 161 or the second storage unit 191.

[0024] In the present embodiment, the storage control unit 162 can perform a storage of the history information by procedures as follows. (a1) Each time an image processing program (including image acquisition) is executed using the visual sensor, the history information is stored in the first memory unit 161. (a2) When the amount of data stored in the first storage unit 161 is full or exceeds a predetermined amount of data, the history information is transferred to the second storage unit 191 in order from the oldest to the newest, and the history information transferred to the second storage unit 191 is deleted from the first storage unit 161.

[0025] The procedures described above allow the history information temporarily stored in the first storage unit 161 to be stored in the second storage unit 191 without omission.

[0026] The history display control unit 163 has a function for displaying two or more individual history pieces of information together with statistical information relating to the two or more individual history pieces of information on the display unit 52. The history display control unit 163 can read the history information from the first memory unit 161 and the second memory unit 191, calculate the statistical information relating to the history information, and display the statistical information together with the history information on the display unit 52.Furthermore, the history display control unit 163 can display on the display unit 52 two or more individual history information items that satisfy a specific selection condition from the plurality of individual history information items stored in the first memory unit 161 and the second memory unit 191, together with the statistical information relating to the two or more individual history information items.

[0027] The output unit 164 has a function for outputting the history information and statistical information displayed by the history display control unit 163 on the display unit 52 as a file in a predefined format.

[0028] The settings adjustment unit 165 has functions for restoring parameters contained in the history information in an image processing program (such as a recognition program) and for displaying a settings adjustment screen to adjust the program and parameters in response to a predefined operation performed on the displayed history information.

[0029] Fig. Figure 3 is a graphical representation showing an example of a history display screen 300, generated by the history display control unit 163 and displayed on the display unit 52. As in Fig. As shown in Figure 3, the history display screen 300 includes a history display area 310 for displaying the history information and a statistics display area 320 for displaying statistical information related to the history information. Furthermore, the history display screen 300 includes an operation area 330, which contains a field for defining a selection condition for the history information to be displayed and various control buttons.

[0030] In the history display area 310, the history information that meets the selection condition specified in the control area 330 is displayed, for example, in chronological order. The size of the history display area 310 increases and decreases according to the number of individual history information items that meet the selection condition. Fig. Figure 3 represents a situation in which three individual history pieces of information are arranged and displayed in order from newest to oldest, from top to bottom. It should be noted that if the number of individual history pieces of information that meet the selection condition exceeds the number of individual history pieces of information that can be displayed simultaneously on the history display screen 300, all history pieces of information that meet the selection condition can be displayed in a scrolling manner on the screen. In the history display screen 300 example, each individual history piece of information includes, as display elements relating to recognition results, a time (date and time), an event, a time (processing duration), and X (mm), Y (mm), Z (mm), W (degrees), P (degrees), and R (degrees) as position / orientation information relative to a recognized target object.

[0031] In statistics display area 320, statistical information relating to the historical information contained in historical display area 310 is displayed (i.e., fulfilling the selection condition). In the example for historical display screen 300, statistics display area 320 shows an average, a minimum, a maximum, a range (numeric range), and a variance as multiple types of statistics, each relating to time (processing time), X (mm), Y (mm), Z (mm), W (degrees), P (degrees), and R (degrees), which are the detection results (display elements) contained in the historical information.

[0032] It should be noted that the in Fig. The three displayed display elements of the history information are an example, and various types of information can be included in the history information. For example, a recognition count, a recognition score, contrast, distortion, angle, and the like can be included as recognition results in the history information. It should be noted that here, the recognition count represents the number of workpieces recognized when the recognition processing is executed once. The recognition score represents a rating regarding the degree of correspondence between a target object in an image and a workpiece model during the recognition processing. The contrast represents the contrast of an outline of the target object relative to the background of the image. The distortion represents the degree of distortion of the target object relative to the workpiece model in the image.The angle represents a deviation in a rotational direction of the target object from the workpiece model in the image. The statistical information can include statistics (an average value, a minimum value, a maximum value, a range (numerical range), and a variance) for the recognition count, the recognition score, the contrast, the distortion, and the angle.

[0033] Control area 330 includes a field 331 for specifying the date and time to define a range of date and time for the historical information to be displayed, and a field 332 for specifying an event to define within the historical information to be displayed. By specifying a date and time range in field 331, the historical information that meets the specified date and time range can be set as the display target and can also be set as the target for obtaining statistical information. Field 332 for specifying an event provides a function for limiting the individual pieces of historical information to be displayed (which must be set as the target of a statistical calculation) by specifying the type of processing (such as detection or determination).It should be noted that it is also possible to determine all individual history information contained in the storage unit (the first storage unit 161 and the second storage unit 191) as history information to be displayed.

[0034] Control area 330 also includes a download button 333, an automatic update button 334, an update button 335, and a playback button 336. Pressing the download button 333 outputs the displayed history and statistical information as a tabular data file. When the download button 333 is pressed, a screen may appear for specifying a device or folder as the output destination for the data file. The function to output history and statistical information as a file by pressing the download button 333 is provided by output unit 164.

[0035] The automatic update button 334 is a button for determining whether the displayed history and statistical information should be updated each time new history information is added to the first storage unit 161. In other words, the automatic update button 334 is a button for determining whether the history and statistical information should be updated automatically in real time.For example, if the date and time in field 331 are set to determine the date and time such that a time span from a specific point in the past to a "present time" is set as the display target, and the automatic update button 334 is pressed in this state, the history display control unit 163 adds new history information to the history display area each time the new history information is added to the first memory unit 161, and also updates the statistical information using the new history information. In a state where the automatic update button 334 is not pressed, the automatic update of the history and statistical information does not occur.In this way, a user can always confirm the latest situation regarding an execution result by using the function to update history information and statistical information in real time, and can also quickly confirm an execution result after a parameter adjustment of the detection program and the like.

[0036] The refresh button 335 provides a function for the user to manually update the history information. For example, if the date and time in field 331 are set to determine the date and time such that a time span from a specific point in the past to a "present time" is set as the display target, and if the refresh button 335 is pressed while the automatic refresh button 334 is not pressed, the history display control unit 163 will update the history and statistical information to the latest state.

[0037] The Playback Button 336 has functions for restoring various setting parameters contained in the history information for a program and for adjusting the program and its parameters. For example, if the user selects desired history information and presses the Playback Button 336, the Settings Adjustment Unit 165 restores various setting parameters contained in the selected history information for a program and displays a settings adjustment screen for customizing the program.

[0038] As in Fig. As shown in Figure 3, the history display control unit 163 can display history information and statistical information in tabular form within the same table. Furthermore, in the history display screen 300, values ​​related to the same display element are aligned and displayed in the same column. This display format allows the user to quickly and easily identify the relationship between each history information display element and the statistics related to that element.

[0039] It should be noted that the history display screen shows 300 in Fig. 3. A plurality of types of recognition results (display elements), included in each of the history information, are arranged and displayed in a row direction, and a plurality of types of statistics relating to each of the recognition results are arranged and displayed in a column direction at a display position for each of the recognition results in the arrangement direction (row direction) of the recognition results. Alternatively, on the history display screen 300, a plurality of types of recognition results, included in each of the history information, can be arranged and displayed in a column direction, and a plurality of types of statistics relating to each of the recognition results in the row direction can be arranged and displayed at a display position for each of the recognition results in the arrangement direction (column direction).

[0040] The History Display screen 300 is designed to display history information and related statistics in the same table. This allows the user to quickly and appropriately assess the recognition program's accuracy and take necessary action. For example, if the displayed history information shows a tendency towards large contrast fluctuations and an unstable recognition score, the user can determine that lighting, for instance, is affecting the result and can take corrective action by increasing lighting or similar factors. Alternatively, the user can modify the recognition program's parameters based on the statistics provided.For example, the user can confirm a minimum value for a recognition score, set a threshold for the recognition score based on this minimum value, and apply the threshold to the recognition program. Alternatively, the user can set a value that defines a range of three sigmas from an average recognition score as the recognition program's recognition score threshold.

[0041] It should be noted that if each of the history information displayed in the history display area 310 contains additional information such as a history image, the additional information such as the history image can be displayed by selecting the history information and performing a predefined operation. Fig. Figure 7 shows an example of an information display screen 210, which is displayed by selecting a single piece of history information contained in the history display area 310 and performing a predefined operation. The information display screen 210 includes a history image G and a display field 211 for a recognition number, a score (recognition score), a contrast, a distortion, and an angle. The information display screen 210 can be displayed simultaneously with the history display screen 300 in the form of a window or can be displayed alone as a separate display screen. The history display control unit 163 performs the functions relating to the display of the information display screen 210.

[0042] The history display control unit 163 has a function to improve user convenience when reviewing displayed history information by linking the statistical information displayed in the statistics display area 320 with the history information displayed in the history display area 310. For example, the history display control unit 163 may have the following functions. (F1) A function to highlight, in response to a selection of a statistic (for example, a maximum value) displayed in a field in the statistics display area 320, the historical information corresponding to the selected statistic (including the selected statistic). (F2) A function to highlight a statistic that corresponds to the selected history information, in response to the selection of a single piece of history information contained in the history display area 310. For example, if the selected history information includes a detection result (element) that has a maximum value as a statistic, the corresponding maximum value will be highlighted in the statistics display area 320.

[0043] Fig. Figure 4 represents a display state in which a maximum value of "13.5" of the Y-value is selected in the statistics display area 320, and historical information 312, which exhibits the maximum value, is highlighted by the function (F1) described above. It should be noted that various methods, such as changing the color, thickness, or font of a character, and adding a color or pattern to a cell, can be used as a highlighting method. As in Fig. As shown in Figure 4 as an example, the process of selecting one of the statistics can be performed by placing the cursor (C1) on one of the statistics and clicking with the mouse. By highlighting the historical information corresponding to the selected statistic, the user can immediately confirm the historical information associated with the selected statistic (such as a maximum value). Furthermore, the user can also immediately adjust the parameters of the historical information (312) by pressing the playback button (336) while the historical information (312) is highlighted.

[0044] It should be noted that if the history information corresponding to a selected statistic includes additional information such as a history image when using the (F1) function, a transition to an information display screen, showing all the contents of the history information including the history image, may occur in response to the selection of the statistic.

[0045] Fig. Figure 5 represents a situation in which, in response to the selection of the history information 311 in the history display area 310, a statistic corresponding to a recognition result (display element) contained in the history information 311 is highlighted by the function (F2) described above. As in Fig. As shown in Figure 5 as an example, selecting a single piece of historical data can be done by placing the cursor (C1) on that piece of data and clicking with the mouse. In the selected piece of historical data 311, the Y-value ("11.1") is statistically a minimum value, while the X-value ("-50.3"), Z-value ("80.5"), W-value ("-31.2"), P-value ("-90"), and R-value ("0.1") are statistically maximum values, and therefore these statistics are highlighted. In this case, the user can immediately confirm that the selected piece of historical data contains a statistically significant value (in this case, the maximum or minimum value).

[0046] The progress display control unit 163 may also have the following function (F3). (F3) A function to display, in response to the selection of a statistic included in the statistical information, an operating screen to accept an operation to determine a threshold (or numerical range) with respect to the selected statistic, and to highlight the history information selected from the history information displayed in the history display area 310 by the threshold (or numerical range) determined via the operating screen.

[0047] Fig. Figure 6 describes a situation in which, in response to a selection of "Average Value of X" contained in the statistical information, a control screen 361 is displayed for determining a threshold value with respect to the statistic using the function (F3). Control screen 361 can be displayed, for example, by placing the cursor C1 on a statistic and performing a specific operation (an operation to select a control screen from a menu that appears after a right-click). Control screen 361 is configured to accept a determination of a threshold value with respect to the X value.For example, if the user sets a threshold for the X-value as “+10 from the average value” via the operating screen 361, the history display control unit 163 can highlight from the history information contained in the history display area 310 the history information whose X-value exceeds the specified threshold (+10) (in the case of a negative value, an absolute value of this is greater than an absolute value of the threshold (-10)).

[0048] For example, using function (F3), the user can quickly select a group of individual history entries whose statistics are greater than a specific value, a group of individual history entries whose statistics are less than a specific value, or a group of individual history entries whose statistics fall within a specific numerical range. Furthermore, the user can also immediately adjust parameters based on the highlighted history entries using the playback function. In this way, function (F3) provides a function for searching for history entries based on a selection condition related to the statistics.

[0049] The history display control unit 163 may also have a function (F4) for displaying statistical information calculated by a numerical expression uniquely set by the user. This information is displayed on the history display screen 300. Fig. Figure 3 illustrates a state in which a numerical expression (column F + column G + column H) is entered by the user in an L-th column (indicated by the reference 321) using the function (F4), and a calculation result is displayed in the L-th column based on the numerical expression. Furthermore, the history display screen shows 300 in Fig. Figure 3 illustrates a situation where a numerical expression is used to calculate the total value of numerical data for each display element of the first through third historical information in a row designated by reference 322. The numerical expression displays the total value of the numerical values ​​in each column of the first through third historical information in each of the columns from the E-th to the L-th in the row designated by reference 322. The statistical information calculated by the user-defined numerical expression is also included in the output data, which is downloaded by pressing button 333.

[0050] The history display control unit 163 can have a function (F5) that is capable of excluding the history information contained in the history display area 310 from a goal of a calculation for obtaining the statistical information in the statistics display area 320. As in Fig. As shown in Figure 3, for example, a checkbox 315 can be added to each piece of history information to specify its exclusion from statistical calculation. In this case, the history display control unit 163 excludes the history information whose checkbox 315 is selected from a statistical calculation target and recalculates and displays the statistical information. According to such a function (F5), the user can exclude specific history information (such as history information where a detection position differs significantly from the other history information) by reviewing the statistical information and updating the statistical information.

[0051] In this way, the user can evaluate a performance history based on information of their choosing, using the history display function provided via the History Display Screen 300 as a user interface. Furthermore, the history information can be evaluated using a user-defined rating criterion by setting a numerical expression. In other words, the History Display Screen 300 is customizable.

[0052] As described above, the history display control unit 163 has the functions to restore a parameter included in the history information displayed in the history display area 310 for a program in response to pressing the button 336 to play it back in a state where a single piece of history information is selected or highlighted, and to display the settings adjustment screen to adjust the program and the parameter. Fig. Figure 8 shows an example of a settings adjustment screen 400 that appears in response to pressing the Play button 336 in a state where a single piece of history information displayed in the history display area 310 is selected or highlighted. As in Fig. As shown in Figure 8, the settings adjustment screen 400 includes a recognition program display area 411 for displaying a recognition program, an image display area 412 for displaying a history image, a parameter display area 413, a recognition result display area 414, and an operating button display area 415 for displaying various operating buttons.

[0053] As an example, a recognition program named "Two-Dimensional Camera Correction" is displayed in the recognition program display area 411. The recognition program includes a snapshot icon 431, a pattern matching icon 432, and a correction data calculation icon 433. The snapshot icon 431 represents a command to execute an image capture using the visual sensor 70. The pattern matching icon 432 represents a command to perform workpiece detection in a captured image by pattern matching. The correction data calculation icon 433 represents a command to calculate correction data for correcting a teach-in position based on the workpiece position detected by pattern matching.

[0054] In the first phase of a transition from the history display screen 300 to the settings adjustment screen 400, the history image 452 of the history information that is selected or highlighted on the history display screen 300 is displayed in the image display area 412.

[0055] Parameter display area 413 shows parameters of the detection program that are recovered from the history information selected or highlighted on the history display screen 300. Fig. Figure 8 represents a state in which the pattern matching icon 432 is selected in the recognition program display area 411, and various parameters related to the pattern matching icon 432 are displayed in the parameter display area 413. The parameters of the pattern matching icon 432 include a score threshold, a contrast threshold, and various other parameters. By selecting the "Other Icon" option in the recognition program display area 411, parameters (recovered parameters) for the selected icon can be displayed and adjusted in the parameter display area 413.

[0056] The recognition result display area 414 shows a processing result of the recognition program. Fig. 8 represents a state in which a recognition result (such as a recognition number, a position, a score, and a contrast) that is included in the history information selected or highlighted on the history display screen 300 is displayed in the recognition result display area 414.

[0057] The control button display area 415 shows control buttons for performing various operations. By adjusting various parameters in the parameter display area 413 and then pressing a button 441 for recognition, the recognition program can be executed with new parameters and the result of the execution can be confirmed.

[0058] This allows the user to restore program parameters from the History Display screen 300 and also to transition to the Settings Adjustment screen 400, where the parameters can be adjusted. On the History Display screen 300, the history information and the statistical information are linked, allowing the user to select the history information that requires adjustment by reviewing the statistical information. This ensures a smooth transition to the Parameter Adjustment screen for modifying the parameter values ​​of the selected history information.

[0059] Fig. Figure 9 is a flowchart that describes a process for recognizing workpiece 1 by executing a recognition program and storing progress information in the robot system 100. As in Fig. As shown in Figure 9, an image of the workpiece 1 is acquired by the visual sensor 70 through the execution of the recognition program (step S11). The recognition processing is then carried out by the image processing unit 152 of the image processing device 50 (step S12). A recognition result (a recognized position of the workpiece 1) is used to control the robot 30 by the motion control unit 121. Next, the history information is stored in the memory unit (the first memory unit 161 and the second memory unit 191) by the memory functions (procedures (a1) and (a2)) of the memory control unit 162 of the history management device 60, as described above (step S13). A workpiece is repeatedly inserted into the robot system 100, and the current machining process is repeated accordingly.In this way, the history information is accumulated in the memory unit (the first memory unit 161 and the second memory unit 191) and the latest history information is added.

[0060] Fig. Figure 10 is a flowchart that describes a basic sequence of events for displaying history information by the history display control unit 163. The actual processing (history display processing) is executed under the control of the processor of the history management device 60 (in the present embodiment, the processor of the image processing device 50). The history display processing is started, for example, by a predefined operation to determine the history display on the history management device 60 (image processing device 50).The history display control unit 163 reads the history information from the memory unit (the first memory unit 161 and the second memory unit 191) according to a selection condition specified in field 331 for determining date and time and the like, and displays the history information in the history display area 310 of the history display screen 300 (step S21).

[0061] Next, the history display control unit 163 calculates the statistical information related to the displayed history information and displays the calculated statistical information in the same table as the history information (step S22).

[0062] If the automatic update mode for updating history information in real time is selected by pressing the automatic update button 334 (S23: YES), the history display control unit 163 repeats the processing to read the history information from the memory unit according to a selection condition for updating history and statistical information (steps S21 and S22). Note that the "automatic update mode" in step S23 corresponds to a case where an operation to display history up to the present is performed on field 331 to determine the date and time, and the automatic update button 334 is pressed. If the automatic update mode is not selected (S23: NO), the current processing is terminated.The current history display processing can then be started in response to the occurrence of various events (such as an operation to determine a selection condition in control area 330, an operation to press button 335 to update, and an operation to press button 334 to automatically update), on the basis of which the current history display processing should be called.

[0063] The embodiment described above is a configuration example in which the robot system includes a visual sensor, and the history information relates to a result of recognition processing based on an image captured by the visual sensor. The embodiment described above can also be applied to a case in which the robot system includes various sensors, and the history information includes an output from the sensor and / or a result of processing based on the sensor output. For example, a case is assumed in which the robot system is configured to include a force sensor mounted between a flange of a robot arm tip and a tool (such as a hand), and to execute a force control program based on a recognition value from the force sensor.The force sensor outputs an external force acting on a robot (tool) as a detection value. The force control program, in this case, can include a program for performing precision fitting, polishing, and similar operations. A control unit, acting as a history management device within the robot system, executes the force control program and records the history information as the execution result. This history information can include the date and time of execution start, the name of the force control program, the position at which force control is performed (position of a control point), the force sensor's detection value at that position (force and torque in each axis direction), and similar data. It should be noted that parameters (force control parameters) of the force control program include, for example, a target pressing force, force control amplification, and the like.For example, the control unit can calculate statistical information from a plurality of individual trend data (detection values) recorded by repeatedly executing the force control program and display this statistical information along with the trend data, as in the embodiment described above. It should be noted that instead of a force sensor, a torque sensor located on each axis of a robot (tool) can be used as a sensor to detect an external force acting on the robot.

[0064] The configuration of the embodiments described above can be used to display the progress information output by performing operations in an industrial machine system in which a measurement is carried out by various sensors.

[0065] The above description describes the configuration when the function as a history management device 60 is integrated into the image processing device 50. However, this configuration is just one example, and a configuration example is also possible in which the function as a history management device 60 is integrated into the robot control unit 20. A configuration example is also possible in which the entire function of the image processing device 50, including the function as a history management device 60, is integrated into the robot control unit 20. When the function as a history management device 60 is integrated into the robot control unit 20, the functional blocks that constitute the history management device 60 can be located within the robot control unit, and the external storage device 90 can be connected to the robot control unit 20.In this case, the history display control unit can display 163 different screens, such as the history display screen on the display unit of the programming handheld device 10, and accept input into the history display screen and the like via the control unit of the programming handheld device 10 (or a touch panel screen integrated into the display unit and the like).

[0066] Alternatively, the function can be integrated into the programming handheld device 10 as a history management device 60. In this case, the function blocks of the history management device 60 are arranged in the programming handheld device 10. Then the external storage device 90 can be connected to the programming handheld device 10.

[0067] The embodiments described above represent an exemplary configuration in which the external storage device 90 is connected to the history management device 60 as a second storage unit 191; however, a non-volatile memory functioning as a second storage unit 191 may be included in the history management device 60.

[0068] The in Fig. The two functional blocks of the history management device shown can be achieved by executing different types of software stored in a memory device by one or more processors of the history management device, or can be achieved by a configuration in which a hardware component such as an application-specific integrated circuit (ASIC) is a major component.

[0069] The program for performing various types of processing with regard to history management, such as history display processing in the embodiments described above, can be recorded in various computer-readable recording media (for example, a ROM, an EEPROM, a semiconductor memory such as a flash memory, a magnetic recording medium, or an optical disc such as a CD-ROM and a DVD-ROM).

[0070] As described above, according to each of the embodiments, a worker can analyze the execution history information output by a machine system in a suitable and efficient manner.

[0071] Although the present disclosure has been described in detail above, it is not limited to the individual embodiments described above. Various types of additions, substitutions, modifications, partial deletions, and the like may be made to the embodiments without deviating from the purpose of the present disclosure or from the content and scope of the present disclosure as described in the claims, which are derived from their correspondences. Furthermore, the embodiments may also be carried out in combination. For example, the embodiments described above include a sequence of operations and a sequence of parts of a processing operation as an example, which is not limited to these.Furthermore, the same applies to a case in which a numerical value or a numerical expression is used in the description of the embodiments described above.

[0072] With regard to the embodiments and modification examples described above, further supplementary remarks are described below. Supplementary Note 1

[0073] A process management device (60) comprising: a display unit (52); and a process display control unit (163) configured to display on the display unit (52) two or more individual process information relating to an execution result of a predetermined processing by an industrial machine together with statistical information relating to the two or more individual process information. Supplementary Note 2

[0074] The history management device (60) according to Supplementary Note 1, which further includes a storage unit (161, 191) configured to store a plurality of individual history information relating to an execution result of a predetermined processing by the industrial machine, wherein the history display control unit (163) is configured to display the two or more individual history information pieces that satisfy a specific selection condition from the plurality of individual history information pieces stored in the storage unit (161, 191), and the statistical information relating to the two or more individual history information pieces. Supplementary Note 3

[0075] The history management device (60) according to Supplementary Note 1 or 2, wherein the statistical information includes a plurality of types of statistics relating to at least one element that is included in each of the history information. Supplementary note 4

[0076] The history management device (60) according to any one of Supplementary Notes 1 to 3, wherein the history display control unit (163) is configured to display the two or more individual pieces of history information and the statistical information relating to the two or more individual pieces of history information in the same table. Supplementary note 5

[0077] The history management device (60) according to Supplementary Note 4, wherein a plurality of types of elements, which are included in each of the history information, are arranged and displayed in the table in one direction of a row direction and a column direction, and a plurality of types of statistics relating to each of the elements are arranged and displayed in the other direction of the row direction and the column direction at a display position of each of the elements in an arrangement direction of the elements. Supplementary Note 6

[0078] The history management device (60) according to any one of Supplementary Notes 1 to 5, wherein the history display control unit (163) is configured to highlight, in response to an operation to select a statistic included in the statistical information, the history information that corresponds to the selected statistic from the two or more individual history information displayed. Supplementary note 7

[0079] The history management device (60) according to any one of Supplementary Notes 1 to 6, wherein the history display control unit (163) is configured to highlight, in response to an operation to select a single history information from the two or more individual history information displayed, a statistic in the statistical information that corresponds to the selected individual history information. Supplementary Note 8

[0080] The history management device (60) according to any one of Supplementary Notes 1 to 7, wherein the history display control unit (163) is configured to display on the display unit, in response to an operation to select a statistic included in the statistical information, an operating screen for determining a threshold or numerical range with respect to the selected statistic, and to highlight from the two or more individual pieces of history information displayed the history information selected by the determined threshold or numerical range with respect to the selected statistic. Supplementary note 9

[0081] The history management device (60) according to any one of Supplementary Notes 1 to 8, wherein the history display control unit (163) is configured to accept a determination of a numerical expression using numerical data contained in the statistical information or the history information and to display a calculation result by means of the determined calculation expression. Supplementary Note 10

[0082] The history management device (60) according to any one of Supplementary Notes 1 to 9, wherein the history display control unit (163) is configured to accept a process for determining the history information to be excluded from a goal of calculating the statistical information from the two or more individual history information displayed. Supplementary Note 11

[0083] The history management device (60) according to Supplementary Note 2, wherein new history information is stored in the storage unit (161, 191) each time the specified processing is performed by the industrial machine, and the history display control unit (163) is configured to additionally display the new history information on the display unit in response to the addition of new history information to the storage unit that meets the specified selection condition, and to update the statistical information based on the new history information. Supplementary Note 12

[0084] The history management device (60) according to any one of Supplementary Notes 1 to 11, which further includes an output unit (164) configured to output the two or more individual pieces of history information and the statistical information displayed on the display unit as a file in a specified format. Supplementary Note 13

[0085] The history management device (60) according to any of Supplementary Notes 1 to 12, further comprising a settings adjustment unit (165) configured to restore a parameter contained in a single piece of history information for a program in response to a predefined operation performed in a state in which a single piece of history information is selected or highlighted in the display unit (52), and to display a settings adjustment screen on the display unit for adjusting the program and the parameter. Supplementary Note 14

[0086] The history management device (60) according to any one of Supplementary Notes 1 to 13, wherein the predefined processing performed by the industrial machine includes processing using a sensor and the history information includes an output from the sensor or at least a result of processing the output from the sensor. Supplementary Note 15

[0087] The history management device (60) according to Supplementary Note 14, wherein the sensor is a visual sensor (70) and the history information includes an execution result of the detection using the visual sensor (70). List of reference symbols 1 workpiece 2 work tables 10 Programming handheld device 20 Robot control unit 22 storage units 30 robots 33 Hand 50 Image processing device 52 Display unit 53 Control unit 60 History Management Device 70 visual sensor 90 external storage devices 100 robot systems 151 Control unit of the visual sensor 152 Image processing unit 161 first storage unit 162 Storage control unit 163 History Display Control Unit 164 output units 165 Setting adjustment unit 191 second storage unit 300 History Display Screen 310 History display area 320 Statistics display area 330 operating area 400 Settings Adjustment Screen 411 Recognition program display area 412 Image display area 413 Parameter display area 414 Recognition result display area 415 Control buttons - Display area QUOTES INCLUDED IN THE DESCRIPTION

[0000] This list of documents cited by the applicant was automatically generated and is included solely for the reader's convenience. The list is not part of the German patent or utility model application. The DPMA accepts no liability for any errors or omissions. Cited patent literature

[0000] JP 2022-170960 A

[0002] JP 2020-040159 A

[0002]

Claims

[1] History management device which includes: a display unit; and a progress display control unit designed to display on the display unit two or more individual progress information relating to an execution result of a given processing by an industrial machine, together with statistical information relating to the two or more individual progress information. [2] A process management device according to claim 1, further comprising a storage unit designed to store a plurality of individual pieces of progress information relating to an execution result of a given processing by the industrial machine, wherein The history display control unit is designed to display the two or more individual pieces of history information that meet a specific selection condition from the plurality of individual pieces of history information stored in the memory unit, and the statistical information relating to the two or more individual pieces of history information. [3] History management device according to claim 1 or 2, wherein the statistical information includes a plurality of types of statistics relating to at least one element that is included in each of the history information. [4] History management device according to any one of claims 1 to 3, wherein the history display control unit is configured to display the two or more individual history information and the statistical information relating to the two or more individual history information in the same table. [5] History management device according to claim 4, wherein, in the table, a plurality of types of elements, which is included in each of the history information, are arranged and displayed in a row direction and a column direction, and a plurality of types of statistics relating to each of the elements are arranged and displayed in the other row direction and column direction at a display position of each of the elements in an arrangement direction of the elements. [6] History management device according to any one of claims 1 to 5, wherein the history display control unit is configured to highlight, in response to a process for selecting a statistic included in the statistical information, the history information that corresponds to the selected statistic from the two or more individual history information displayed. [7] History management device according to any one of claims 1 to 6, wherein the history display control unit is configured to highlight a statistic in the statistical information corresponding to the selected individual of the history information in response to an operation to select one of the history information from the two or more individual history information displayed. [8] History management device according to any one of claims 1 to 7, wherein the history display control unit is configured to display an operating screen for determining a threshold or numerical range with respect to the selected statistic on the display unit in response to an operation for selecting a statistic included in the statistical information, and to highlight the history information selected by the determined threshold or numerical range with respect to the selected statistic from the two or more individual pieces of history information displayed. [9] History management device according to any one of claims 1 to 8, wherein the history display control unit is configured to accept a determination of a numerical expression using numerical data contained in the statistical information or the history information and to display a calculation result by means of the determined calculation expression. [10] History management device according to any one of claims 1 to 9, wherein the history display control unit is configured to accept a process for determining the history information to be excluded from a goal of calculating the statistical information from the two or more individual history information displayed. [11] History management device according to claim 2, wherein New history information is stored in the storage unit each time the specified processing is carried out by the industrial machine, and The history display control unit is designed to additionally display the new history information on the display unit and update the statistical information based on the new history information in response to the addition of new history information to the storage unit that meets the specific selection condition. [12] History management device according to any one of claims 1 to 11, which further comprises an output unit configured to output the two or more individual history information items and the statistical information displayed on the display unit as a file in a predetermined format. [13] History management device according to any one of claims 1 to 12, further comprising a settings adjustment unit configured to restore a parameter contained in a single piece of history information for a program in response to a predetermined operation performed in a state in which a single piece of history information is selected or highlighted in the display unit, and to display a settings adjustment screen on the display unit for adjusting the program and the parameter. [14] History management device according to any one of claims 1 to 13, wherein the specified processing carried out by the industrial machine includes processing using a sensor, and The history information includes an output from the sensor or at least a result of processing the output from the sensor. [15] History management device according to claim 14, wherein the sensor is a visual sensor, and The progress information includes an execution result of the detection using the visual sensor.

Images