Image generation system and computer-readable recording medium

Abstract

This image generation device generates an image for ascertaining the state of an industrial machine, and is provided with: a coupling unit that couples first image data relating to the state of the industrial machine and data indicating the internal state of the industrial machine into image data; a first data storage unit that associates and stores the image data coupled by the coupling unit with a time at which the first image data is acquired or a time at which data indicating the internal state of the industrial machine is acquired; a second data storage unit that stores second image data relating to the state of the industrial machine, which is different from the first image data, in association with the time at which the second image data is acquired; and an output section that outputs the image data stored in the first data storage section or the second image data stored in the second data storage section in accordance with a request from the outside.

Description

Technical Field

[0001] The present invention relates to an image generation system and a computer-readable recording medium. Background Art

[0002] At a manufacturing site where industrial machinery is installed, operators can understand the operating status of industrial machinery by checking the screen of the industrial machinery and the operation of the industrial machinery. During the operation of the industrial machinery, data collected from various parts of the industrial machinery, data detected by sensors installed on the industrial machinery and its surroundings, etc. are recorded. The recorded data is used by users located at a distance from the industrial machinery and engineers who perform maintenance on the industrial machinery to understand or analyze the operating status of the industrial machinery.

[0003] In order to understand the operating status of industrial machinery, there is a technology that associates an image obtained by photographing the industrial machinery with an image of a screen showing the results of measuring physical quantities involved in the operation of the industrial machinery, thereby making it easy to understand information related to past manufacturing (for example, Patent Document 1, etc.).

[0004] Prior art literature

[0005] Patent Literature

[0006] Patent Document 1: Japanese Patent Application Publication No. 2018-073185 Summary of the invention

[0007] Problems to be solved by the invention

[0008] Even if the captured images are combined for presentation, the operating state of the industrial machine may not be accurately understood. In addition, if the client side performs processing to combine a plurality of images or data, it is necessary to develop the necessary processing separately.

[0009] Means for solving problems

[0010] The image generation device of the present invention solves the above-mentioned problems by appropriately transmitting an image that provides information necessary for understanding the operating state of an industrial machine from a server side.

[0011] Furthermore, one embodiment of the present disclosure is an image generation system that generates an image for grasping the state of an industrial machine, the image generation system comprising: a first data storage unit that stores first image data related to the state of the industrial machine and a time when the first image data is acquired in association; a second data storage unit that stores second image data related to the state of the industrial machine that is different from the first image data and is associated with the time when the second image data is acquired; a subtitle data generation unit that generates data related to subtitles representing the internal state of the industrial machine based on data representing the internal state of the industrial machine; a third data storage unit that stores data related to the subtitles and a time when the internal state of the industrial machine is acquired in association; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit according to an acquisition request for the first image data or the second image data; an output unit that outputs the data searched by the search processing unit; a display data acquisition unit that acquires the data output by the output unit as data related to screen display; a combining unit that combines the data related to screen display acquired by the display data acquisition unit into image data; and a screen display unit that displays the image data combined by the combining unit.

[0012] Another aspect of the present disclosure is a computer-readable recording medium having a program recorded thereon for causing a computer to operate as an image generating device for generating an image for grasping the state of an industrial machine, the program causing the computer to operate as follows: a first data storage unit for storing first image data related to the state of the industrial machine in association with a time when the first image data is acquired; a second data storage unit for storing second image data related to the state of the industrial machine that is different from the first image data in association with a time when the second image data is acquired; a subtitle data generating unit for generating data related to subtitles representing the internal state of the industrial machine based on data representing the internal state of the industrial machine; a third data storage unit for storing data related to the subtitles in association with the time when the internal state of the industrial machine is acquired; a search processing unit for searching the first data storage unit or the second data storage unit and the third data storage unit in accordance with a request for acquiring the first image data or the second image data; and an output unit for outputting the data searched by the search processing unit.

[0013] Effects of the Invention

[0014] According to one embodiment of the present disclosure, the data representing the internal state of the industrial machine is combined with the image data on the server side, so there is no need to prepare a dedicated function for each action environment on the client side, and it is expected that the development work hours can be reduced. In addition, by using a search processing unit that searches for the first image data, the second image data, and the third data stored at different times, it is possible to obtain each data at approximately the same time by only requesting the data involved at one time on the data acquisition request side, and it is expected that the development work hours and processing load related to the request processing on the client side can be reduced. BRIEF DESCRIPTION OF THE DRAWINGS

[0015] Figure 1 This is a schematic hardware configuration diagram of an image generation system according to one embodiment of the present invention.

[0016] Figure 2 This is a block diagram showing the general functions of the image generation system according to the first embodiment of the present invention.

[0017] Figure 3 It is a diagram showing an example of image data of a display image of the industrial machine according to the first embodiment.

[0018] Figure 4 This is a diagram showing an example of image data obtained by capturing an image of an operating state of an industrial machine according to the first embodiment.

[0019] Figure 5 It is a diagram showing an example of image data after being combined by the combining unit according to the first embodiment.

[0020] Figure 6 It is a diagram showing another example of image data after being combined by the combining unit according to the first embodiment.

[0021] Figure 7 This is a diagram showing an example in which data output by the output unit of the first embodiment is displayed on a terminal.

[0022] Figure 8 This is a diagram showing another example of displaying the data output by the output unit of the first embodiment on a terminal.

[0023] Fig. 9 This is a block diagram showing the general functions of the image generation system according to the second embodiment of the present invention.

[0024] Fig.10 This is a block diagram showing the general functions of an image generation system according to another embodiment of the present invention.

[0025] Fig.11 It is a diagram showing an example of image data combined by a combining unit according to another embodiment. DETAILED DESCRIPTION

[0026] Hereinafter, embodiments of the present invention will be described with reference to the accompanying drawings.

[0027] Figure 1 This is a schematic hardware structure diagram of the main parts of the image generation system of one embodiment of the present invention. The image generation system 300 of this embodiment is composed of an image generation device 1 and a terminal 8 connected via a network 5. The image generation device 1 can be installed on a control device that controls an industrial machine 4 set up at a manufacturing site such as a factory. In addition, the image generation device 1 can be installed on a personal computer installed in conjunction with the control device, a fog computer 6 connected to the control device via a wired or wireless network, a cloud server 7, and other computers. The following is an example in which the image generation device 1 of this embodiment is installed on a personal computer that is connected to the control device that controls the industrial machine 4 via a network and acts as a server.

[0028] The CPU 11 included in the image generating device 1 of the present embodiment is a processor that controls the entire image generating device 1. The CPU 11 reads a system program stored in the ROM 12 via the bus 22, and controls the entire image generating device 1 according to the system program. The RAM 13 temporarily stores temporary calculation data, display data, and various data input from the outside.

[0029] The non-volatile memory 14 is composed of, for example, a memory backed up by a battery (not shown), an SSD (Solid State Drive), etc., and maintains the storage state even when the power of the image generating device 1 is turned off. The non-volatile memory 14 stores data obtained from the industrial machine 4, image data obtained from the imaging sensors 9a and 9b, programs and data read from the external device 72 via the interface 15, programs and data input via the input device 71, and programs and data obtained from other devices via the network 5. The programs and data stored in the non-volatile memory 14 can also be expanded in the RAM 13 when executed / used. In addition, various system programs such as well-known analysis programs are pre-written in the ROM 12.

[0030] The interface 15 is an interface for connecting the CPU 11 of the image generating device 1 and an external device 72 such as a USB device. For example, programs and setting data are read from the external device 72. In addition, programs and setting data edited in the image generating device 1 can be stored in an external storage unit via the external device 72.

[0031] The interface 20 is an interface for connecting the CPU 11 of the image generating device 1 and the wired or wireless network 5. The network 5 is connected to industrial machines 4, fog computers 6, cloud servers 7, etc., and the image generating device 1 and the image generating device 1 can exchange data with each other.

[0032] The display device 70 outputs and displays the data read into the memory and the data obtained as a result of executing the program, etc., via the interface 17. The display device 70 may also include an LED indicator and a warning lamp to indicate the state of the machine. In addition, the input device 71 composed of a keyboard, a pointing device, etc. transmits instructions and data based on the operator's operation to the CPU 11 via the interface 18.

[0033] The image generating device 1 , together with at least one industrial machine 4 and at least one terminal 8 connected via a network 5 , constitute an image generating system 300 .

[0034] Industrial machinery 4 is a machine tool, an electrical discharge machine, a robot, etc. installed at a manufacturing site such as a factory. Photo sensors 9a and 9b are installed near the industrial machinery 4. Photo sensor 9a is configured to be able to photograph the processing of the workpiece performed in the industrial machinery 4, the movement of the industrial machinery 4 itself, the surrounding conditions, etc. Photo sensor 9a may be, for example, a camera, etc. Photo sensor 9b is configured to be able to photograph the display screen of the control device that controls the industrial machinery 4. Photo sensor 9b may be, for example, a camera, a capture device, etc. Photo sensors 9a and 9b generate image data such as still images and dynamic images. The image data generated by photo sensors 9a and 9b is transmitted to CPU 11 via network 5 and interface 20.

[0035] The terminal 8 operates as a client of the image generating device 1. The terminal 8 may be, for example, a personal computer installed at a distance from the industrial machine 4 and the image generating device 1. Alternatively, the terminal 8 may be a portable device carried by an operator or the like. The terminal 8 has at least a function of acquiring image data generated by the image generating device 1 via the network 5 and displaying the image data.

[0036] The CPU 811 included in the terminal 8 is a processor that controls the entire terminal 8. The CPU 811 reads a system program stored in the ROM 812 via the bus 822, and controls the entire terminal 8 according to the system program. The RAM 813 temporarily stores temporary calculation data, display data, and various data input from the outside.

[0037] The non-volatile memory 814 is composed of, for example, a memory backed up by a battery (not shown), an SSD (Solid State Drive), etc., and maintains the storage state even when the power of the terminal 8 is turned off. In the non-volatile memory 814, programs or data obtained from the image generation device 1 or other devices via the network 5, programs or data input via the input device 871, etc. are stored. The programs and data stored in the non-volatile memory 814 can also be expanded in the RAM 813 when executed / used. In addition, various system programs such as well-known image processing programs are pre-written in the ROM 812.

[0038] The interface 815 is an interface for connecting the CPU 811 of the terminal 8 to an external device 872 such as a USB device. For example, programs and data are read from the external device 872. In addition, programs and data generated in the terminal 8 can be stored in an external storage unit via the external device 872.

[0039] The interface 820 is an interface for connecting the CPU 811 of the terminal 8 and the wired or wireless network 5. The image generation device 1, industrial machine 4, fog computer 6, cloud server 7, etc. are connected to the network 5, and data is exchanged between the terminal 8 and the network 5.

[0040] The display device 870 outputs and displays various data read into the memory, data obtained as a result of executing the program, etc., via the interface 817. In addition, the input device 871 composed of a keyboard, a pointing device, a touch panel, etc. transmits instructions and data based on the operator's operation to the CPU 811 via the interface 818.

[0041] Figure 2 The functions of the image generating device 1 and the terminal 8 according to the first embodiment of the present invention are shown as a schematic block diagram. The functions of the image generating device 1 and the terminal 8 according to the present embodiment are shown as follows. Figure 1 This is achieved by the CPU 11 included in the image generating device 1 and the CPU 811 included in the terminal 8 respectively executing system programs to control the operations of the respective components of the image generating device 1 and the terminal 8 .

[0042] The image generating device 1 of this embodiment includes a state data acquisition unit 100, a display image data acquisition unit 110, a machine image data acquisition unit 120, a subtitle data generation unit 130, a search processing unit 140, and an output unit 150. In addition, the image generating device 1 has a RAM 13 or a nonvolatile memory 14, which is prepared in advance: a first data storage unit 210, which is a region for storing image data obtained by photographing a display screen of a control device for controlling the industrial machine 4; a second data storage unit 220, which is a region for storing image data obtained by photographing an operating state of the industrial machine 4; and a third data storage unit 230, which is a region for storing data related to subtitles indicating the internal state of the industrial machine 4.

[0043] The state data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the industrial machine 4. The state data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the control device of the industrial machine 4. The data indicating the internal state of the industrial machine 4 may be, for example, the value of the operation signal of the operator of the industrial machine 4, the value of the action signal, the value of the macro variable, the name of the tool used, the size value of the tool used, the workpiece coordinate value, the set value of the predetermined parameter, the instruction of the processing program being executed, etc. The data indicating the internal state of the industrial machine 4 may be a sequence of values ​​obtained at a predetermined period, that is, time series data. The data indicating the internal state of the industrial machine 4 is associated with at least the time when the data is obtained or the time from when the industrial machine 4 starts to operate. The state data acquisition unit 100 outputs the acquired data indicating the internal state of the industrial machine 4 to the subtitle data generation unit 130.

[0044] The display image data acquisition unit 110 acquires image data obtained by capturing an image of a display screen of a control device that controls the industrial machine 4. The display image data acquisition unit 110 acquires image data of a display image from the imaging sensor 9b. Figure 3 This is an example of the image data 410 of the display image acquired by the display image data acquisition unit 110. The image data 410 of the display image may be time series data of a still image acquired at a predetermined period. Alternatively, it may be data of a dynamic image. The image data 410 of the display image is associated with at least the time when the data is acquired or the time from when the industrial machine 4 starts to operate. The display image data acquisition unit 110 stores the acquired image data 410 of the display image in the first data storage unit 210.

[0045] The machine image data acquisition unit 120 acquires image data of the operating state of the industrial machine 4, such as the processing of the workpiece, the operation of the industrial machine 4 itself, the surrounding conditions, etc. The machine image data acquisition unit 120 acquires image data of the operating state of the industrial machine 4 from the imaging sensor 9a. Figure 4 An example of image data 420 obtained by photographing the operating state of the industrial machine 4 and obtained by the machine image data acquisition unit 120 is shown. The image data 420 obtained by photographing the operating state of the industrial machine 4 may be time series data of still images obtained at a predetermined period. Alternatively, it may be data of a dynamic image. The image data 420 obtained by photographing the operating state of the industrial machine is associated with at least the time when the data is obtained or the time from when the industrial machine 4 starts to operate. The machine image data acquisition unit 120 stores the image data 420 obtained by photographing the operating state of the industrial machine 4 in the second data storage unit 220.

[0046] The subtitle data generating unit 130 generates data related to subtitles indicating the internal state of the industrial machine 4 based on the data indicating the internal state of the industrial machine 4 input from the state data acquiring unit 100. The data related to the subtitles may be, for example, a character string indicating the internal state of the industrial machine 4. In addition, the character string indicating the internal state of the industrial machine 4 may be visualized. The data related to the subtitles generated by the subtitle data generating unit 130 may represent a character string indicating a predetermined state indicated by the data indicating the internal state of the industrial machine 4. For example, when the automatic operation signal is turned on (ON) in the industrial machine 4, data related to subtitles such as "Automatic Operation" may be generated. In addition, when the value of the override is set to 100% in the industrial machine 4, data related to subtitles such as "Rapid Traverse Override 100%" may be generated. The data related to the subtitles may represent a character string in a predetermined language or a plurality of languages. The subtitle data generating unit 130 stores the generated data related to the subtitles in the third data storage unit 230 in association with the time data indicating the time when the industrial machine 4 becomes the predetermined state and the predetermined period determined in advance, in the case.

[0047] When the output unit 150 receives a request for obtaining image data from the terminal 8, the search processing unit 140 searches the first data storage unit 210, the second data storage unit 220, and the third data storage unit 230 for the image data requested by the acquisition request. Then, the data obtained as a result of the search is output to the output unit 150. The acquisition request from the terminal 8 includes at least a request for the type of image data. The request for the type of image data is either a request for image data of a display image or a request for image data obtained by photographing the operating state of the industrial machine 4. When the acquisition request includes a request for image data of a display image, the search processing unit 140 searches the data stored in the first data storage unit 210 for the image data of the requested display image. In addition, when the acquisition request includes a request for image data obtained by photographing the operating state of the industrial machine 4, the search processing unit 140 searches the data stored in the second data storage unit 220 for the image data of the requested display image. Then, the acquired image data is output to the output unit 150. In either case, the search processing unit 140 can search for data related to subtitles stored in the third data storage unit 230 based on the searched image data, and output the data to the output unit 150 .

[0048] The acquisition request from the terminal 8 may include information uniquely identifying the industrial machine 4. When the identification information of the industrial machine 4 is included, the search processing unit 140 searches the image data stored in each storage unit for image data related to the industrial machine 4 corresponding to the requested identification information.

[0049] The acquisition request from the terminal 8 may include a request related to time. In the case of including a request related to time, the search processing unit 140 searches the image data stored in each storage unit for image data captured at a time closest to the requested time. The search processing unit 140 may also search for image data captured at the most recent time before the requested time. Then, the obtained image data is output to the output unit 150. In either case, the search processing unit 140 may search for data related to subtitles at a time close to the image data from the third data storage unit 230 based on the searched image data, and output it to the output unit 150.

[0050] The output unit 150 instructs the search processing unit 140 to search for image data based on a request from the terminal 8. Then, the image data and data related to the subtitles searched by the search processing unit 140 are output via the network 5. The output unit 150 only needs to be able to output any one of the data stored in the first data storage unit 210 or the second data storage unit 220 at the same timing in response to a request from one terminal 8. The output unit 150 can also process the output data according to the type of terminal 8. For example, a template that specifies the configuration and size of image data for each type of terminal 8 is prepared in advance. Then, the type of terminal 8 included in the request from the terminal 8 can be determined, and the data can be processed according to the template corresponding to the determined type of terminal 8, and then the processed data can be output to the terminal 8. The data output by the output unit 150 is displayed by the terminal 8. This display can be displayed as a still image. In addition, it can also be displayed as a dynamic image.

[0051] The terminal 8 of the present embodiment includes a display data acquisition unit 880 , a combining unit 885 , a screen display unit 890 , and a display switching unit 895 .

[0052] The display data acquisition unit 880 makes a request to the image generating device 1 to acquire data related to the display of the screen. Then, the display data related to the display of the screen sent from the image generating device 1 via the network 5 as a response to the acquisition request is acquired. The acquisition request sent by the display data acquisition unit 880 to the image generating device 1 at least includes a request for the type of image data. The request for the type of image data is a request for image data of a display image or a request for image data obtained by photographing the operating state of the industrial machine 4. For example, when the acquisition request includes a request for image data of a display image, the display data acquisition unit 880 can acquire the requested image data of the display image and data related to the subtitles. In addition, when the acquisition request includes a request for image data obtained by photographing the operating state of the industrial machine 4, the display data acquisition unit 880 can acquire the requested image data obtained by photographing the operating state of the industrial machine 4 and data related to the subtitles. The display data acquisition unit 880 can include information that uniquely identifies the industrial machine 4 in the request to the image generating device 1. In addition, the display data acquisition unit 880 can include a request for time in the request to the image generating device 1. The display data acquisition unit 880 outputs the acquired data related to the screen display to the combining unit 885 .

[0053] The combining unit 885 combines the data related to the screen display acquired by the display data acquiring unit 880 into image data. Figure 5 885 represents an example of image data after being combined. Figure 5 As illustrated, the combining unit 885 generates, for example, image data 430 related to subtitles based on data representing the internal state of the industrial machine 4. Then, the image data 410 of the display image and the data representing the internal state of the industrial machine 4 can be combined by overlapping the generated image data 430 related to the subtitles with the image data 410 of the display image. The background of the image data 430 related to the subtitles can also be a semi-transparent image that allows the image data 410 of the display image to be seen. In addition, the combining unit 885 can also generate, for example, image data related to a graph showing changes in the data representing the internal state of the industrial machine 4, image data representing a predetermined internal state (for example, a warning mark, etc.), and combine them with the image data 410 of the display image. The combining unit 885 can combine data with the same or close time points associated with each data. Figure 6 Another example of image data after being combined by the combining unit 885 is shown. Figure 6As illustrated, the combining unit 885 can combine the image data 420 obtained by photographing the operating state of the industrial machine 4 and the data indicating the internal state of the industrial machine 4 by superimposing the image data 430 related to the captions on the image data 420 obtained by photographing the operating state of the industrial machine 4. The combining unit 885 outputs data related to the display of the combined screen to the screen display unit 890.

[0054] The screen display unit 890 displays data related to the display of the screen connected by the connection unit 885 on the display device 870 .

[0055] Figure 7 An example of a screen displayed on the terminal 8 is shown. Figure 7 In FIG. 1 , an example of display in which image data of a display image and data related to subtitles are combined as data related to screen display is shown. Figure 7 In the example, the image data 410 related to the display screen is a dynamic image, and the associated time is superimposed and displayed in the upper left of the image data. The image data of the display image and the image data 430 related to the subtitles are updated as the display time passes. Figure 7 As shown in the example, the display screen of the control device can be viewed on the terminal 8, and the internal state of the industrial machine 4 can be observed through the subtitles. Figure 7 As illustrated, an operation panel 450 is included. The operation panel 450 includes at least one operable object and accepts user operations.

[0056] The display switching unit 895 switches the data related to the display of the screen acquired by the display data acquisition unit 880 based on the user's operation. The display switching unit 895 can switch the data related to the display of the screen acquired by the display data acquisition unit 880 based on the user's operation on the input device 871. The operation can be, for example, the operation of a keyboard, a pointing device, a touch panel, etc. For example, Figure 7 The operable objects included in the illustrated operation panel 450 may include an operation button for switching the image data 410 related to the display screen to the image data 420 obtained by photographing the operating state of the industrial machine 4. In this case, when the user operates the operation button, the display switching unit 895 issues an instruction to the display data acquisition unit 880 based on the user's operation to switch the data related to the display of the screen to be acquired to the image data obtained by photographing the operating state of the industrial machine 4. Then, based on the image data obtained by photographing the operating state of the industrial machine 4 and the data related to the subtitles obtained by the display data acquisition unit 880, Figure 8As illustrated, the screen display unit 890 displays a screen in which the image data 430 related to the captions is superimposed on the image data 420 obtained by capturing the operating state of the industrial machine 4 . Figure 8 The illustrated operation panel 450 may include an operation button for switching the display of the image data 410 to a display image.

[0057] In this way, when switching the display of the image data 410 of the display image and the image data 420 obtained by photographing the operating state of the industrial machine 4, the display switching unit 895 can instruct the display data acquisition unit 880 to include the request for the currently displayed time in the acquisition request to the image generation device 1. With this structure, when the user is viewing a predetermined image at a certain time, by performing a display switching operation, he can view other images captured at the same time. In addition, the display switching unit 895 can also instruct the display data acquisition unit 880 to include the request for a time predetermined time (for example, 5 seconds ago) that is predetermined before the currently displayed time in the acquisition request to the image generation device 1. With this structure, when the user is viewing a predetermined image at a certain time, by performing a display switching operation, he can view other images captured at a slightly earlier time.

[0058] The display switching unit 895 may receive an operation for switching the industrial machine 4 to be displayed. The display switching unit 895 instructs the display data acquisition unit 880 to include information uniquely identifying the industrial machine 4 specified by the user's operation in the request to the image generating device 1 .

[0059] The image generating device 1 having the above configuration can appropriately switch the image data to be displayed to other image data acquired at substantially the same time for browsing by operation on the client side, so that the temporal relationship with other image data can be grasped as needed while referring to the details of the image data.

[0060] Fig. 9 This is a diagram showing the functions of the image generating device 1 and the terminal 8 according to the second embodiment of the present invention as a schematic block diagram. The functions of the image generating device 1 and the terminal 8 according to this embodiment are shown in FIG. Figure 1 This is achieved by the CPU 11 included in the image generating device 1 and the CPU 811 included in the terminal 8 respectively executing system programs to control the operations of the respective components of the image generating device 1 and the terminal 8 .

[0061] The image generating device 1 and the terminal 8 of this embodiment are different from the image generating device 1 and the terminal 8 of the first embodiment in that the image generating device 1 has the function of the combining unit 885 that the terminal 8 of the first embodiment has.

[0062] The image generating device 1 of this embodiment includes a combining unit 160. The combining unit 160 combines the image data searched by the search processing unit 140 and the data related to the subtitles into one image data. The combining unit 160 has the same function as the combining unit 885 included in the terminal 8 of the first embodiment. The combining unit 160 outputs the combined image data to the output unit 150.

[0063] Then, the output unit 150 of the present embodiment transmits the input image data to the terminal 8 for output.

[0064] The image generation device 1 having the above structure combines the data representing the internal state of the industrial machine 4 with the image data on the image generation device 1 side as a server, so it is not necessary to prepare a dedicated function for each operation environment of the client, and it can be expected to reduce the man-hours of development. In addition, when browsing the image data, the load of the operation of the terminal 8 as the client is also reduced.

[0065] As mentioned above, although embodiment of this invention was described, this invention is not limited only to the example of the said embodiment, It can implement in various forms by adding appropriate changes.

[0066] For example, in the above-mentioned embodiment, the image data obtained by photographing the action state of the industrial machine 4 uses the image data obtained by photographing the industrial machine 4 by the photographing sensor 9a. However, the image data may also use image data generated by a known processing simulation technology. Depending on the type of industrial machine 4, sometimes it is not possible to smoothly photograph the state of processing of the workpiece due to cutting fluid or the like. In such a case, by using an image generated by a processing simulation technology instead of the image data photographed by the photographing sensor 9a, the action state of the industrial machine 4 can be easily grasped.

[0067] In addition, in the above-mentioned embodiment, an example is shown in which the image generating device 1 and the terminal 8 are installed on different computers. Fig.10 As shown, each function can be realized on one computer. Fig.10 In the embodiment, the display data acquisition unit 170, the combining unit 160, the screen display unit 180, and the display switching unit 190 respectively have the same functions as the display data acquisition unit 880, the combining unit 885, the screen display unit 890, and the display switching unit 895 of the first embodiment. With such a configuration, a system for a user to confirm the state of the industrial machine 4 can be constructed with only the image generating device 1 at a low cost.

[0068] Furthermore, in the above-described embodiment, the combining unit generates image data obtained by overlapping the image data 410 showing the image or the image data 420 obtained by photographing the operating state of the industrial machine 4 with the captions indicating the internal state of the industrial machine 4. However, for example, Fig.11 As shown, after setting a predetermined size area 440 outside the image data, image data with subtitles indicating the internal state of the industrial machine 4 superimposed on the area 440 may be generated. The area 440 may be set at any position above, below, left, or right of the image data. By configuring in this way, the subtitles will not hide the image data, so it is expected that the readability of the information will be improved.

[0069] Explanation of symbols

[0070] 1. an image generating device;

[0071] 4. Industrial machinery;

[0072] 5. Network;

[0073] 6Fog computer;

[0074] 7. Cloud Server;

[0075] 8 terminals;

[0076] 9a, 9b photographing sensors;

[0077] 11CPU;

[0078] 12 ROM;

[0079] 13 RAM;

[0080] 14 non-volatile memory;

[0081] 15, 17, 18, 20 interfaces;

[0082] 22 buses;

[0083] 70 display device;

[0084] 71 input device;

[0085] 72 external devices;

[0086] 100 status data acquisition unit;

[0087] 110 display image data acquisition unit;

[0088] 120: a mechanical image data acquisition unit;

[0089] 130 subtitle data generating unit;

[0090] 140 search processing unit;

[0091] 150 output unit;

[0092] 160 joint;

[0093] 811CPU;

[0094] 812 ROM;

[0095] 813 RAM;

[0096] 814 non-volatile memory;

[0097] 815, 817, 818, 820 interfaces;

[0098] 822 bus;

[0099] 870 display device;

[0100] 871 input device;

[0101] 872 external devices;

[0102] 880 display data acquisition unit;

[0103] 885 joint;

[0104] 890 screen display unit;

[0105] 895 displays a switching unit.

Claims

1. An image generation system for generating an image for understanding the state of an industrial machine, characterized in that: The image generation system comprises: a first data storage unit for storing first image data related to the state of the industrial machine in association with a time when the first image data was acquired; a second data storage unit for storing second image data related to the state of the industrial machine, which is different from the first image data, in association with a time when the second image data is acquired; a subtitle data generating unit that generates data related to subtitles representing the internal state of the industrial machine based on the data representing the internal state of the industrial machine; a third data storage unit for storing data related to the subtitles in association with a time when the internal state of the industrial machine is acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit according to a request for obtaining the first image data or the second image data; an output unit, which outputs the data searched by the search processing unit; a display data acquisition unit that acquires the data output by the output unit as data related to the display of the screen; a combining unit that combines the data related to the display of the screen acquired by the display data acquiring unit into image data; and A screen display unit displays the image data combined by the combining unit.

2. The image generation system according to claim 1, characterized in that: The first image data is image data representing a display image of the industrial machine. The second image data is image data obtained by capturing an image of an operating state of the industrial machine.

3. The image generation system according to claim 1, characterized in that: The first image data is image data obtained by photographing the operating state of the industrial machine. The second image data is image data representing a display image of the industrial machine.

4. The image generation system according to claim 2 or 3, characterized in that: The data representing the internal state of the industrial machine includes at least any one of the values ​​of the operation signals of the operator of the industrial machine, the values ​​of the action signals, the values ​​of the macro variables, the names of the tools used, the dimension values ​​of the tools used, the coordinate values ​​of the workpieces, the set values ​​of the predetermined parameters, and the instructions of the machining program being executed.

5. The image generation system according to claim 2 or 3, characterized in that: The combining unit combines the data indicating the internal state of the industrial machine in the form of at least one of image information and subtitle information.

6. The image generation system according to claim 1, characterized in that: The acquisition request is any one of a request for the first image data and a request for the second image data. When the acquisition request is a request for the first image data, the search processing unit searches the first data storage unit for the first image data, and searches the third data storage unit for data related to subtitles associated with the searched first image data. When the acquisition request is a request for the second image data, the search processing unit searches the second data storage unit for the second image data, and searches the third data storage unit for data related to subtitles associated with the searched second image data.

7. The image generation system according to claim 6, characterized in that: The acquisition request also includes a request for time. The search processing unit searches for data acquired at a time close to the requested time.

8. A computer-readable recording medium having a program recorded thereon for causing a computer to operate as an image generating device for generating an image for understanding the state of an industrial machine, wherein: The program causes the computer to operate as the following components: a first data storage unit for storing first image data related to the state of the industrial machine in association with a time when the first image data was acquired; a second data storage unit for storing second image data related to the state of the industrial machine, which is different from the first image data, in association with a time when the second image data is acquired; a subtitle data generating unit that generates data related to subtitles representing the internal state of the industrial machine based on the data representing the internal state of the industrial machine; a third data storage unit for storing data related to the subtitles in association with a time when the internal state of the industrial machine is acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit according to a request for obtaining the first image data or the second image data; as well as An output unit outputs the data searched by the search processing unit.

Images