Image generation device and computer-readable storage medium

Abstract

A video generation device for generating a recorded video of an industrial machine acquires data indicating a state of the industrial machine, searches for the data indicating the state of the industrial machine, detects a trigger event indicating a work executed by the industrial machine, determines a viewpoint corresponding to the trigger event, and generates a video based on the viewpoint determination.

Description

Technical Field

[0001] The invention relates to an image generating device and a computer readable storage medium. Background Art

[0002] Currently, there is a technology that monitors the status of industrial machinery for efficient control and maintenance of machinery. In this technology, by collecting data related to industrial machinery and analyzing the collected data, the status of industrial machinery is understood, failures of industrial machinery are prevented, product quality is confirmed, and it is applied to improve production management.

[0003] Industrial machinery and the machinery detected by sensors installed around them are recorded in a data collection device called a data logger. Many data loggers are installed in the factory to record the status of the machinery in the factory.

[0004] In addition, a PLC (Programmable Logic Controller) that controls the operation sequence of a machine and a numerical control device (Numerical Control) that controls a machine tool also have a recording unit inside to store control information and status information.

[0005] The collected data is converted into a form that is easy for the operator to visually recognize. Patent document 1 states the following: "If the time load unit obtains control data obtained from the control device, image data obtained from the camera, and sound data obtained from the controlled object device, time-added control data, time-added image data, and time-added sound data with the time point at which each data was obtained are generated, and these data are stored in a temporary file. Then, the log data forming unit reads the time-added control data, time-added image data, and time-added sound data of the group with the closest time among the time-added control data, time-added image data, and time-added sound data stored in the temporary file to form log data with time, control data, image data, and sound data, so that each data can be reproduced while being synchronized." According to Patent document 1, the image, sound, and log of the controlled object device are reproduced while being synchronized in terms of time.

[0006] Patent Document 2 states the following: "A camera for photographing the robot is connected to the controller, and image data representing the robot's behavior obtained by photographing is provided to the controller in the form of an electrical signal," and "When an error (event) occurs, an error record associated with the error is generated separately from the action log." According to Patent Document 2, when an event occurs while an image is being photographed, the event is associated with the image and recorded.

[0007] Prior art literature

[0008] Patent Literature

[0009] Patent Document 1: Japanese Patent Application Publication No. 2001-125612

[0010] Patent Document 2: Japanese Patent Application Publication No. 2019-150923 Summary of the invention

[0011] Problems to be solved by the invention

[0012] Patent document 1 displays images and logs at the same time, and Patent document 2 records logs (data) when events occur. If there are logs and images when errors occur, it is possible to understand the situation when the errors occur, which helps to investigate the cause of the errors. Similarly, if images with good visibility are produced, they can also be flexibly used for maintenance of industrial machinery and improvement of productivity.

[0013] In the field of data management for industrial machinery, there is a challenge to improve visibility.

[0014] Means for solving problems

[0015] An image generating device as one embodiment of the present invention comprises: a data acquiring unit, which acquires data indicating a mechanical state of an industrial machine; a viewpoint determining unit, which searches for data indicating a state of the industrial machine, detects a trigger event indicating a job performed by the industrial machine, and determines a viewpoint corresponding to the trigger event; and an image generating unit, which generates an image of the industrial machine based on the viewpoint determined by the viewpoint determining unit.

[0016] As one method of the present invention, a storage medium stores processor-readable commands, and one or more processors execute the commands to perform the following processing: obtain data representing the status of an industrial machine, search for data representing the status of the industrial machine, detect a trigger event representing a job performed by the industrial machine, determine a viewpoint corresponding to the trigger event, and generate an image based on the determination of the viewpoint.

[0017] Effects of the Invention

[0018] According to one aspect of the present invention, the visibility of an image can be improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 This is a diagram showing the application range of the image generating device.

[0020] Figure 2 This is a block diagram of a first disclosed image generating device.

[0021] Figure 3 This is a conceptual model of the first disclosed viewpoint determination unit.

[0022] Figure 4 This is a block diagram of a second disclosed image generating device.

[0023] Figure 5 This is a conceptual model of the viewpoint determination unit disclosed in the second disclosure.

[0024] Figure 6 This is a diagram showing an example of an event set.

[0025] Figure 7 This is a diagram showing an example of work before starting machining on a lathe.

[0026] Figure 8 This is a diagram showing an example of an image generated by the image generating unit.

[0027] Fig. 9 This is a diagram showing an example of work before starting machining in a machining center.

[0028] Fig.10 This is a diagram showing an example of an image generated by the image generating unit.

[0029] Fig.11 This is a diagram showing trigger events and event sets for workpiece measurement operations (end faces).

[0030] Fig.12 This is a diagram showing trigger events and event sets for workpiece measurement (hole-to-hole measurement).

[0031] Fig.13 It is a diagram showing the event set search method.

[0032] Fig.14 This is a diagram showing the relationship between work units and viewpoints.

[0033] Fig.15 This is a diagram showing an example of a progress bar in which work units are arranged on a time axis.

[0034] Fig.16 This figure shows an example of combining two images into one.

[0035] Fig.17 It is a hardware structure diagram of the image generating device. DETAILED DESCRIPTION

[0036] The following is a description of an image generation device 100 to which the present disclosure is applied. The image generation device 100 acquires data representing the state of industrial machinery arranged in a factory. The image generation device 100 uses the stored data to determine the viewpoint of an image representing the movement of the industrial machinery. Examples of industrial machinery that are targets include machine tools, numerical control devices, robots, etc., but are not limited thereto.

[0037] The image generation device 100 is applied to information processing devices such as PCs (personal computers), servers, and portable terminals. Figure 1 As shown, the image generation device 100 is applied to a PC used by a factory manager, a server on the cloud, a PC used by a machine manufacturer or an operator controlling the machine manufacturer, a portable terminal used by a factory manager, etc., but is not limited thereto. The image generated by the image generation device 100 is flexibly applied to the management (maintenance, production management, etc.) of industrial machinery.

[0038] Furthermore, the image generating device 100 can also be applied to information processing devices such as numerical control devices and PLCs (Programmable Logic Controllers).

[0039] [First Publication]

[0040] Figure 2 1 is a block diagram of a video generation device 100 according to the first disclosure. The video generation device 100 includes a data acquisition unit 1 , a video generation unit 2 , and a viewpoint determination unit 3 .

[0041] The data acquisition unit 1 acquires data related to the industrial machine. The data acquired by the data acquisition unit 1 includes state information of the industrial machine, operation input information, control information of the industrial machine, and the like.

[0042] In the present disclosure, numerical control devices and machine tools are taken as examples of industrial machinery. The status information of industrial machinery includes the mode of the numerical control device, tool offset, alarm, time, number of processed parts, modal information, position information (coordinate system), power consumption of the machine tool, load of the servo motor, data measured by the sensor (temperature, pressure), etc. The operation input information is the operation input of the operator to the touch panel, button, dial, steering wheel, etc.

[0043] The image generation unit 2 generates a simulated image representing the action of the industrial machine using existing simulation technology. The viewpoint of the simulated image can be switched. In the present disclosure, an example of generating a simulated image is given, but the image generated by the image generation unit 2 may not be a simulated image but a two-dimensional image based on the result of the viewpoint determination unit 3.

[0044] The viewpoint determination unit 3 determines the viewpoint of the simulated video generated by the video generation unit 2 . Figure 3 This is a conceptual model of the viewpoint determination unit 3. The viewpoint determination unit 3 monitors the data acquired by the data acquisition unit 1 and detects a trigger event. The viewpoint determination unit 3 determines a viewpoint corresponding to the detected trigger event. A specific example of the trigger event will be described later.

[0045] The video generation unit 2 generates a simulated video at the viewpoint determined by the viewpoint determination unit 3. Alternatively, instead of generating a simulated video, a two-dimensional image may be generated based on the result of the viewpoint determination unit 3. In this way, a recorded video at a viewpoint suitable for confirming the operation of the industrial machine is generated.

[0046] As described above, the first disclosed image generation device 100 acquires data related to industrial machinery, detects trigger events, and generates a recorded image from a viewpoint suitable for confirming the operation of the industrial machinery. According to the present disclosure, the viewpoint of the image can be switched according to the operation of the industrial machinery to generate an image with high visibility.

[0047] [Second Publication]

[0048] Figure 4 This is a block diagram of a video generation device 100 according to the second disclosure. The video generation device 100 includes a data acquisition unit 1 , a video generation unit 2 , a data history storage unit 7 , a viewpoint determination unit 3 , and an event set storage unit 8 .

[0049] The video generation device 100 of the second disclosure includes a data history storage unit 7 and an event set storage unit 8. The data history storage unit 7 stores the history of data acquired by the data acquisition unit 1. The event set storage unit 8 stores a combination of data (event set) found when executing a job.

[0050] Figure 5 It is a conceptual model of the viewpoint determination unit 3. The viewpoint determination unit 3 searches the data acquired by the data acquisition unit 1 to detect a trigger event. The viewpoint determination unit 3 reads out an event set corresponding to the detected trigger event, searches past data stored in the data history storage unit 7, and searches whether an event set corresponding to the detected trigger event has been found in the past. If an event set corresponding to the detected trigger event has been found in the past, it is determined that a certain job has been executed.

[0051] An event set corresponds to a unit of work from the start to the end of a job. In the present disclosure, an event found when a job starts is referred to as a start event. From the start event to the trigger event, a unit of work is formed. A specific example of an event set will be described later.

[0052] The video generation unit 2 generates a video based on the result of the viewpoint determination unit 3 .

[0053] Furthermore, the time at which an event is discovered may be added to the image generated by the image generating unit 2. This time can be used for generating a progress bar described later or for merging with other images.

[0054] As described above, the second disclosed video generation device 100 determines a viewpoint based on whether an event (trigger event and event set) associated with a task is found. The video generation device 100 generates a simulated video based on the determined viewpoint. By switching the viewpoint, a highly visible video can be generated.

[0055] [Specific example of the second disclosure]

[0056] Hereinafter, a specific example of the video generation device 100 according to the second disclosure will be described. In this example, the events are limited to the operation signal of the operator and the screen display.

[0057] Reference Figure 6 , Fig.11 , Fig.12 , explaining the switching of viewpoints in a numerical control device for controlling a machine tool.

[0058] As a premise, the viewpoint determination unit 3 searches for a trigger event from the data acquired by the data acquisition unit 1 .

[0059] The viewpoint determination unit 3 detects the trigger event “pressing of the loop start button”. Figure 6 is an example of an event set. Figure 6 In the example, the trigger event "Press the cycle start button" is associated with an event set consisting of three events: "Press the button to change the NC mode to MEM", "Display the automatic operation screen", and "Press the cycle start button".

[0060] Among these three events, "pressing the button to change the NC mode to MEM" is the start event. If events such as "displaying the automatic operation screen" and "pressing the cycle start button" are executed in sequence from the start event, it can be seen that the work unit "processing start" is executed.

[0061] The viewpoint determination unit 3 detects "pressing the cycle start button" and "pressing the button to change the NC mode to MEM" based on the operation signal. "Displaying the automatic operation screen" can be detected based on the "screen ID" contained in the history of the operation signal. "Screen ID" refers to the ID assigned to each screen. It can be detected based on the "screen ID" that the "automatic operation screen" is displayed. If the events are executed in the order defined in the event set, the viewpoint determination unit 3 can determine that the operation such as "processing start" has been executed.

[0062] Furthermore, in the above description, the order in which events are discovered is fixed, but the order may be different.

[0063] As a preparation before starting machining, the tool is moved to the machining start position by rapid feed. When machining using a new machining program that has no previous machining results, the tool is generally stopped temporarily at the machining start position, or the feed rate multiplier on the industrial machinery operation panel is used to decelerate. This is because, at the machining start position, if the distance between the workpiece and the tool is the expected distance, it can be determined that the "machining program" created by the programmer, the "tool offset" set at the machining site, and the "workpiece coordinate setting" setting values ​​are correct.

[0064] In the work before starting machining, an image that allows easy visual recognition of the distance between the workpiece and the tool is useful.

[0065] The appropriate viewpoint varies depending on the type of machine tool and the direction in which the tool moves.

[0066] Figure 7 This shows an example of a lathe.

[0067] Generally, the lathe sets the direction parallel to the rotation axis of the spindle as the Z axis, and the Z axis as the middle finger, and the direction of the thumb as the X axis in the right-handed orthogonal coordinate system. In order to confirm the distance between the tool and the workpiece before cutting, a viewpoint perpendicular to the XZ plane is better.

[0068] Figure 8 The image is generated from a viewpoint perpendicular to the XZ plane. By switching the viewpoint, it is easy to visually identify the distance between the workpiece and the tool. Figure 8 The image is a two-dimensional image based on the result of the viewpoint determination unit 3.

[0069] Fig. 9 This shows an example of a machining center.

[0070] Similarly, in a machining center, the tool moves to the cutting start position by rapid feed and "temporarily stops" or "decelerates". In this example, the tool that starts cutting moves parallel to the X axis of the workpiece coordinate system. In this example, if the viewpoint is set perpendicular to the XZ plane, it is easy to confirm the distance between the tool and the workpiece.

[0071] Fig.10 The image is generated from a viewpoint perpendicular to the XZ plane. By switching the viewpoint, it is easy to visually identify the distance between the workpiece and the tool. Fig.10 The image is a two-dimensional image based on the result of the viewpoint determination unit 3.

[0072] The viewpoint determination unit 3 may analyze the machining program and determine the viewpoint based on the moving direction of the tool.

[0073] Fig.11This is a diagram showing the trigger event and event set of the workpiece measurement operation (end face). The trigger event of the workpiece measurement operation (end face) is "workpiece coordinate data setting". The trigger event "workpiece coordinate data setting" is associated with the event set consisting of the three events "display workpiece measurement screen", "select workpiece measurement (end face) in the workpiece measurement screen", and "set workpiece coordinate data".

[0074] Among the three events, "displaying the workpiece measurement screen" is the start event. If the events "selecting workpiece measurement (end face) in the workpiece measurement screen" and "setting workpiece coordinate data" are executed in sequence from the start event, it can be seen that the work unit "starting workpiece measurement (end face)" has been executed.

[0075] The viewpoint determination unit 3 searches the data acquired by the data acquisition unit 1 and detects data indicating a discovery event set.

[0076] Next, a method of identifying a work unit by searching for an event while tracing back the time starting from a trigger event will be described.

[0077] Fig.12 This is a diagram showing the trigger events and event sets for workpiece measurement (hole-to-hole measurement). Fig.12 The trigger event in is "Set workpiece coordinate data". The trigger event "Set workpiece coordinate data" is associated with the event set consisting of the six events "Change to touch probe", "Display workpiece coordinate measurement screen", "Select hole-to-hole measurement", "Axis operation based on jog", "Touch probe contacts workpiece", and "Set workpiece coordinate data".

[0078] Fig.12 The trigger event "Set workpiece coordinate data" is related to Fig.11 In other words, sometimes multiple event sets correspond to one trigger event. In such a case, when searching for events while tracing back the time from the trigger event as the starting point, it will eventually converge to one event set (operation unit).

[0079] Fig.13 A specific example is shown. When a trigger event such as "data for setting workpiece coordinates" is detected, the previous event "touch probe contacts workpiece" is searched. Then, if there is an event "touch probe contacts workpiece", the previous event "axis operation based on jog" is searched. If there is no event "touch probe contacts workpiece", other event sets (such as Fig.11 ) is included in the event. Events are searched while going back in time, and when the start event is reached, the work unit corresponding to the event set is determined.

[0080] If the operating unit is determined, the viewpoint is determined. Fig.14Indicates the relationship between the work unit and the viewpoint. In the "Start machining" operation (machining center), the Z axis is set as the longitudinal direction, and the viewpoint for observing the workpiece and tool from the horizontal direction. In the "Workpiece measurement (end face)" operation, the Z axis is set as the longitudinal direction, and the viewpoint for observing the workpiece and tool from the horizontal direction. In the "Workpiece measurement (hole measurement)", the Z axis is set as the longitudinal direction, and the viewpoint for observing from above is determined.

[0081] The viewpoint can be defined by the direction and distance relative to the object. For example, the viewpoint can be selected from predefined viewing directions and distances of the tool and workpiece. Alternatively, the viewpoint can be automatically determined by simulating the direction and distance between the object and the obstacle without entering.

[0082] If the work unit is configured on the time axis, Fig.15 As shown, a progress bar of the image can be created. By creating a progress bar, the overall operation flow can be confirmed, and the visibility of the data is improved.

[0083] In addition, even if the discovery time of the event is arranged on the timeline, a progress bar of the image can be generated. By making a progress bar, not only can the flow of the work be visually grasped, but the work can also be confirmed not only by images but also by text, which improves the visibility of the data.

[0084] [Third Publication]

[0085] In the third disclosure, an image on the display screen of the numerical controller and an image generated by the image generating unit 2 are displayed side by side. Fig.16 This is an example of arranging and displaying an image on the display screen of the numerical controller and an image generated by the image generating unit 2. The image generating device 100 combines the image on the display screen of the numerical controller and the generated image in terms of time, and generates a single image by combining the two images synchronized in time.

[0086] exist Fig.16 In the example, the image of the display screen of the numerical control device and the image generated by the image generating unit 2 are arranged and displayed. The image of the display screen can also be captured without shooting with a camera. By arranging the image of the display screen and the simulated image, the operator's operation and the state of the industrial machine can be confirmed at the same time.

[0087] Hereinafter, the hardware configuration of the video generation device 100 to which the present disclosure is applied will be described. Fig.17 is a hardware structure diagram of the image generating device 100. Fig.17As shown, the image generating device 100 includes a CPU 111 for overall control of the image generating device 100, a ROM 112 for recording programs and data, and a RAM 113 for temporarily expanding data. The CPU 111 reads the system program recorded in the ROM 112 via a bus and controls the overall image generating device 100 according to the system program.

[0088] Nonvolatile memory 114 is backed up by a battery (not shown), for example, and maintains storage even when image generating apparatus 100 is powered off. Nonvolatile memory 114 stores various data such as programs read from external devices via interfaces 115 , 118 , and 119 and user operations input via input unit 71 .

[0089] The interface 115 mediates the connection between the video generating device 100 and an external device such as an adapter, and reads programs, various parameters, and the like from the external device.

[0090] Description of Reference Numerals

[0091] 100 Image Generating Device

[0092] 1Data acquisition unit

[0093] 2Image generation unit

[0094] 3. Viewpoint determination unit

[0095] 7. Data history storage unit

[0096] 8Event set storage unit

[0097] 111CPU

[0098] 113RAM

[0099] 114 non-volatile memory.

Claims

1. An image generating device, characterized in that: The image generating device comprises: a data acquisition unit that acquires data indicating a machine state of the industrial machine; a viewpoint determination unit that searches for data indicating a state of the industrial machine, detects a trigger event indicating a job performed by the industrial machine, and determines a viewpoint corresponding to the trigger event; as well as An image generating unit generates an image of the industrial machine based on the viewpoint determined by the viewpoint determining unit.

2. The image generation device according to claim 1, characterized in that: The image generation device includes: a data history storage unit that stores the history of data acquired by the data acquisition unit; The viewpoint determination unit stores an event set which is a combination of multiple events. When the trigger event is detected, the viewpoint determination unit searches whether there is data indicating that the event included in the event set is found in the data history storage unit. When there is data indicating that the event included in the event set is found in the data history storage unit, the operation unit corresponding to the event set is determined, and the viewpoint corresponding to the operation unit is determined.

3. The image generation device according to claim 1, characterized in that: The viewpoint determination unit searches for data indicating events included in a discovery event set while tracing back in time, starting from the trigger event.

4. The image generation device according to claim 2, characterized in that: The video generation device adds the time at which the event is discovered to the video generated by the video generation unit.

5. The image generation device according to claim 2, characterized in that: The video generation device creates a progress bar indicating a discovery time of an event included in the event set.

6. The image generation device according to claim 2, characterized in that: The image generation device creates a progress bar indicating a time at which the work unit is discovered.

7. The image generation device according to claim 1, characterized in that: The image acquisition unit is an image generation unit that generates a simulated image representing the operation of the industrial machine or a two-dimensional image based on the result of the viewpoint determination unit.

8. A storage medium storing processor-readable commands, characterized in that: The following processing is performed by executing the command by one or more processors: obtaining data representing the status of industrial machinery; searching for data representing the state of the industrial machine, detecting a trigger event representing a job performed by the industrial machine, and determining a viewpoint corresponding to the trigger event; as well as An image based on the viewpoint determination is generated.

Images