Image generation system and computer-readable recording medium

Abstract

An image generation device according to the present disclosure generates images for ascertaining the state of an industrial machine, the image generation device comprising: a merging unit that merges first image data pertaining to the state of the industrial machine and data indicating an internal state of the industrial machine as image data; a first data storage unit that stores the image data merged by the merging unit, in association with a date and time at which the first image data was acquired or a date and time at which the data indicating the internal state of the industrial machine was acquired; a second data storage unit that stores second image data pertaining to a different state of the industrial machine than that of the first image data, in association with a date and time at which the second image data was acquired; and an output unit that, in response to an external request, outputs the image data that is stored in the first data storage unit or the second image data that is stored in the second data storage unit.

Description

Technical Field

[0001] The present invention relates to an image generation system and a computer-readable recording medium. Background Art

[0002] At a manufacturing site where industrial machinery is installed, an operator grasps the operating state of the industrial machinery by checking the screen of the industrial machinery and the operation of the industrial machinery respectively. While the industrial machinery is operating, data collected from each part of the industrial machinery and data detected by sensors attached to the industrial machinery and its surroundings are recorded. This recorded data is used by users located remotely from the industrial machinery and engineers who perform maintenance on the industrial machinery to grasp and analyze the operating state of the industrial machinery.

[0003] In order to grasp the operating state of industrial machinery, there is a technology that makes information related to past manufacturing easy to understand by associating and displaying an image obtained by photographing the industrial machinery and an image of a screen displaying results of measuring physical quantities related to the operation of the industrial machinery (for example, Patent Document 1). Prior Art Documents Patent Documents

[0004] Patent Document 1 Japanese Unexamined Patent Application Publication No. 2018-073185 Summary of the Invention Problems to be Solved by the Invention

[0005] Even when captured images are combined and presented, there are cases where the operating state of the industrial machinery cannot be accurately grasped. Furthermore, if processing for combining a plurality of images and data is performed on the client side, it is necessary to separately develop the required processing. Means for Solving the Problems

[0006] The image generation device according to the present invention solves the above problem by appropriately transmitting images from the server side that provide information necessary to understand the operating status of industrial machinery.

[0007] Furthermore, one aspect of the present disclosure is an image generation system for generating images for understanding the state of an industrial machine, comprising: a first data storage unit that stores first image data relating to the state of the industrial machine in association with the time the first image data was acquired; a second data storage unit that stores second image data relating to a state of the industrial machine different from the first image data in association with the time the second image data was acquired; a subtitle data creation unit that creates subtitle data indicating the internal state of the industrial machine based on data indicating the internal state of the industrial machine; a third data storage unit that stores the subtitle data in association with the time the internal state of the industrial machine was acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; an output unit that outputs the data searched by the search processing unit; a display data acquisition unit that acquires the data output by the output unit as data relating to the display of a screen; a merging unit that combines the data relating to the display of a screen acquired by the display data acquisition unit as image data; and a screen display unit that displays the combined image data by the merging unit. Another aspect of the present disclosure is a computer-readable recording medium that records a program for operating a computer as an image generation device for generating images for understanding the state of an industrial machine, wherein the computer records a program for operating the computer as a first data storage unit that stores first image data relating to the state of the industrial machine in association with the time the first image data was acquired; a second data storage unit that stores second image data relating to a state of the industrial machine different from the first image data in association with the time the second image data was acquired; a subtitle data creation unit that creates data relating to subtitles indicating the internal state of the industrial machine based on data indicating the internal state of the industrial machine; a third data storage unit that stores the data relating to subtitles in association with the time the internal state of the industrial machine was acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; and an output unit that outputs the data searched by the search processing unit. [Effects of the Invention]

[0008] In one aspect of this disclosure, since the server side combines image data with data indicating the internal state of the industrial machine, it is not necessary to prepare dedicated functions for each client-side operating environment, and development man-hours are expected to be reduced. Furthermore, by using a search processing unit to retrieve first, second, and third image data stored at different time points, the data acquisition requesting party can acquire each data point at approximately the same time by simply requesting data related to a single time point, which is expected to reduce development man-hours and processing load related to request processing on the client side. [Brief explanation of the drawing]

[0009] [Figure 1] This is a schematic hardware diagram of an image generation system according to one embodiment of the present invention. [Figure 2] This block diagram shows the schematic functions of an image generation system according to the first embodiment of the present invention. [Figure 3]This figure shows an example of image data for the display screen of an industrial machine according to the first embodiment. [Figure 4] This figure shows an example of image data captured of the operating state of an industrial machine according to the first embodiment. [Figure 5] This figure shows an example of image data with a combined portion according to the first embodiment. [Figure 6] This figure shows another example of image data joined by the joint according to the first embodiment. [Figure 7] This figure shows an example of data output by the output unit according to the first embodiment being displayed on a terminal. [Figure 8] This figure shows another example of displaying data output by the output unit according to the first embodiment on a terminal. [Figure 9] This block diagram shows the schematic functions of an image generation system according to a second embodiment of the present invention. [Figure 10] This block diagram shows the schematic functionality of an image generation system according to another embodiment of the present invention. [Figure 11] This figure shows an example of image data with combined parts according to another embodiment. [Modes for carrying out the invention]

[0010] Embodiments of the present invention will be described below with reference to the drawings. Figure 1 is a schematic hardware configuration diagram showing the main components of an image generation system according to one embodiment of the present invention. The image generation system 300 according to this embodiment is configured such that an image generation device 1 and a terminal 8 are connected via a network 5. The image generation device 1 can be mounted on a control device that controls industrial machinery 4 installed in a manufacturing site such as a factory. Alternatively, the image generation device 1 can be mounted on a personal computer attached to the control device, or on a computer such as a fog computer 6 or cloud server 7 connected to the control device via a wired or wireless network. In the following description, the image generation device 1 according to this embodiment will be explained based on an example in which it is mounted on a personal computer that is connected via a network to a control device that controls industrial machinery 4 and operates as a server.

[0011] A CPU 11 included in the image generation apparatus 1 according to the present embodiment is a processor that generally controls the image generation apparatus 1. The CPU 11 reads a system program stored in a ROM 12 via a bus 22, and controls the entire image generation apparatus 1 in accordance with the system program. Temporary calculation data, display data, various types of data input from the outside, and the like are temporarily stored in a RAM 13.

[0012] A non-volatile memory 14 is configured of, for example, a memory backed up by a battery (not shown), an SSD (Solid State Drive), or the like, and retains a stored state even when the power of the image generation apparatus 1 is turned off. The non-volatile memory 14 stores data acquired from an industrial machine 4, image data acquired from imaging sensors 9a and 9b, programs and data read from an external device 72 via an interface 15, programs and data input via an input device 71, programs and data acquired from another device via a network 5, and the like. The programs and data stored in the non-volatile memory 14 may be expanded to the RAM 13 at the time of execution or use. In addition, various system programs such as known analysis programs are pre-written in the ROM 12.

[0013] An interface 15 is an interface for connecting the CPU 11 of the image generation apparatus 1 to an external device 72 such as a USB device. For example, programs, setting data, and the like are read from the external device 72 side. In addition, programs, setting data, and the like edited in the image generation apparatus 1 can be stored in an external storage means via the external device 72.

[0014] An interface 20 is an interface for connecting the CPU 11 of the image generation apparatus 1 to a wired or wireless network 5. The industrial machine 4, a fog computer 6, a cloud server 7, and the like are connected to the network 5, and exchange data with the image generation apparatus 1 mutually.

[0015] The display device 70 outputs and displays, via the interface 17, various data read onto a memory, data obtained as a result of execution of programs, and the like. The display device 70 may include an LED indicator indicating a machine state and a warning lamp. Furthermore, the input device 71 configured of a keyboard, a pointing device, or the like passes commands, data, and the like based on an operation by an operator to the CPU 11 via the interface 18.

[0016] The image generation device 1 constitutes an image generation system 300 together with at least one industrial machine 4 and at least one terminal 8 connected via a network 5. The industrial machine 4 is a machine tool, an electric discharge machine, a robot, or the like installed at a manufacturing site such as a factory. Imaging sensors 9a and 9b are installed in the vicinity of the industrial machine 4. The imaging sensor 9a is installed so as to be capable of capturing images of the state of workpiece processing performed by the industrial machine 4, the operation of the industrial machine 4 itself, surrounding conditions, and the like. The imaging sensor 9a may be, for example, a camera. The imaging sensor 9b is installed so as to be capable of capturing an image of a display screen of a control device that controls the industrial machine 4. The imaging sensor 9b may be, for example, a camera or a capture device. The imaging sensors 9a and 9b generate image data such as still images and moving images. The image data generated by the imaging sensors 9a and 9b is passed to the CPU 11 via the network 5 and the interface 20.

[0017] The terminal 8 operates as a client of the image generation device 1. The terminal 8 may be, for example, a personal computer installed remotely from the industrial machine 4 or the image generation device 1. The terminal 8 may also be a portable device carried by an operator or the like. The terminal 8 has at least a function of acquiring and displaying image data generated by the image generation device 1 via the network 5.

[0018] The CPU 811 in terminal 8 is a processor that controls terminal 8 as a whole. The CPU 811 reads the system program stored in ROM 812 via bus 822 and controls the entire terminal 8 according to the system program. RAM 813 temporarily stores temporary calculation data, display data, and various data input from external sources.

[0019] The non-volatile memory 814 is composed of, for example, a memory backed up by a battery (not shown) or an SSD (Solid State Drive), and its memory state is maintained even when the terminal 8 is powered off. The non-volatile memory 814 stores programs and data acquired from the image generation device 1 and other devices via the network 5, as well as programs and data input via the input device 871. The programs and data stored in the non-volatile memory 814 may be loaded into the RAM 813 when executed or used. In addition, various system programs, such as known image processing programs, are pre-written into the ROM 812.

[0020] Interface 815 is an interface for connecting the CPU 811 of terminal 8 to an external device 872 such as a USB device. Programs and data, for example, can be read from the external device 872. Programs and data created within terminal 8 can also be stored in external storage means via the external device 872.

[0021] Interface 820 is an interface for connecting the CPU 811 of terminal 8 to a wired or wireless network 5. The network 5 is connected to an image generation device 1, industrial machinery 4, fog computer 6, cloud server 7, etc., and exchanges data with terminal 8.

[0022] The display device 870 displays data, programs, and other data obtained as a result of their execution, which are read into memory, via the interface 817. The input device 871, consisting of a keyboard, pointing device, touch panel, etc., transmits commands and data based on operator input to the CPU 811 via the interface 818.

[0023] Figure 2 is a schematic block diagram showing the functions of the image generation device 1 and terminal 8 according to the first embodiment of the present invention. Each function of the image generation device 1 and terminal 8 according to this embodiment is realized by the CPU 11 of the image generation device 1 and the CPU 811 of the terminal 8, respectively, executing system programs and controlling the operation of each part of the image generation device 1 and terminal 8, as shown in Figure 1.

[0024] The image generation device 1 according to this embodiment includes a state data acquisition unit 100, a display image data acquisition unit 110, a machine image data acquisition unit 120, a subtitle data creation unit 130, a search processing unit 140, and an output unit 150. Furthermore, the RAM 13 to non-volatile memory 14 of the image generation device 1 are pre-configured with a first data storage unit 210 for storing image data of the display screen of the control device that controls the industrial machine 4, a second data storage unit 220 for storing image data of the operating state of the industrial machine 4, and a third data storage unit 230 for storing data related to subtitles that indicate the internal state of the industrial machine 4.

[0025] The status data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the industrial machine 4. The status data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the control device of the industrial machine 4. The data indicating the internal state of the industrial machine 4 may be, for example, the value of the operator's operation signal, the value of the operating signal, the value of a macro variable, the name of the tool used, the dimensional value of the tool used, the work coordinate value, the setting value of a predetermined parameter, or the command of the machining program being executed. The data indicating the internal state of the industrial machine 4 may be time-series data, which is a series of values ​​acquired at a predetermined period. The data indicating the internal state of the industrial machine 4 is associated with at least the time the data was acquired or the time since the industrial machine 4 started operating. The status data acquisition unit 100 outputs the acquired data indicating the internal state of the industrial machine 4 to the subtitle data creation unit 130.

[0026] The display image data acquisition unit 110 acquires image data of the display screen of the control device that controls the industrial machine 4. The display image data acquisition unit 110 acquires image data of the display screen from the imaging sensor 9b. Figure 3 shows an example of image data 410 of the display image acquired by the display image data acquisition unit 110. The image data 410 of the display image may be time-series data of still images acquired at a predetermined period. It may also be data of moving images. The image data 410 of the display screen is associated with at least the time the data was acquired or the time since the industrial machine 4 started operating. The display image data acquisition unit 110 stores the acquired image data 410 of the display screen in the first data storage unit 220.

[0027] The machine image data acquisition unit 120 acquires image data capturing the operating state of the industrial machine 4, such as the workpiece processing, the operation of the industrial machine 4 itself, and the surrounding conditions. The machine image data acquisition unit 120 acquires image data capturing the operating state of the industrial machine 4 from the imaging sensor 9a. Figure 4 shows an example of image data 420 capturing the operating state of the industrial machine 4 acquired by the machine image data acquisition unit 120. The image data 420 capturing the operating state of the industrial machine 4 may be time-series data of still images acquired at a predetermined period. It may also be data of moving images. The image data 420 capturing the operating state of the industrial machine is associated with at least the time the data was acquired or the time since the industrial machine 4 started operating. The machine image data acquisition unit 120 stores the acquired image data 420 capturing the operating state of the industrial machine 4 in the second data storage unit 220.

[0028] The subtitle data creation unit 130 creates subtitle data indicating the internal state of the industrial machine 4 based on data indicating the internal state of the industrial machine 4 input from the state data acquisition unit 100. The subtitle data may be, for example, a string representing the internal state of the industrial machine 4. Alternatively, it may be an image representation of the string indicating the internal state of the industrial machine 4. The subtitle data created by the combination unit 130 may represent a string meaning a predetermined state indicated by the data indicating the internal state of the industrial machine 4. For example, if the automatic operation signal is ON in the industrial machine 4, the unit may create subtitle data such as "Automatic Operation". Also, if the override value is set to 100% in the industrial machine 4, the unit may create subtitle data such as "Rapid Traverse Override 100%". The subtitle data may represent a string in one predetermined language or a string in multiple languages. The subtitle data creation unit 130 associates the time when the industrial machine 4 entered the predetermined state with period data indicating a predetermined period and stores the created subtitle data in the third data storage unit 230.

[0029] When the output unit 150 receives a request for image data acquisition from the terminal 8, the search processing unit 140 searches for the image data requested by the acquisition request from the first data storage unit 210, the second data storage unit 220, and the third data storage unit 230. The search processing unit 140 then outputs the data obtained as a result of the search to the output unit 150. The acquisition request from the terminal 8 includes at least a request for the type of image data. The request for the type of image data is either a request for image data of a display image or a request for image data capturing the operating state of the industrial machine 4. If the acquisition request includes a request for image data of a display image, the search processing unit 140 searches for the requested image data of the display image from the data stored in the first data storage unit 210. If the acquisition request includes a request for image data capturing the operating state of the industrial machine 4, the search processing unit 140 searches for the requested image data of the display image from the data stored in the second data storage unit 220. The search processing unit 140 then outputs the obtained image data to the output unit 150. In either case, the search processing unit 140 may search for subtitle-related data stored in the third data storage unit 230 in accordance with the searched image data and output it to the output unit 150.

[0030] The acquisition request from terminal 8 may include information that uniquely identifies the industrial machine 4. If it includes identification information for the industrial machine 4, the search processing unit 140 searches for image data related to the industrial machine 4 that corresponds to the requested identification information among the image data stored in each memory unit.

[0031] The acquisition request from terminal 8 may include a request related to time. If the request includes a request related to time, the search processing unit 140 searches for image data among the image data stored in each storage unit that was captured at the time closest to the requested time. The search processing unit 140 may also search for image data captured at the time closest to the requested time. The obtained image data is then output to the output unit 150. In either case, the search processing unit 140 may search the third data storage unit 230 for subtitle data from a time close to the retrieved image data and output it to the output unit 150.

[0032] The output unit 150 instructs the search processing unit 140 to search for image data based on a request from the terminal 8. The output unit 150 then outputs the image data and subtitle data found by the search processing unit 140 via the network 5. The output unit 150 should be configured to exclusively output either the data stored in the first data storage unit 210 or the second data storage unit 220 at the same time in response to a request from a single terminal 8. The output unit 150 may also process the data to be output according to the type of terminal 8. For example, templates that define the arrangement and size of image data for each type of terminal 8 may be prepared in advance. The output unit 150 may then identify the type of terminal 8 included in the request from the terminal 8, process the data based on the template corresponding to the identified type of terminal 8, and then output the processed data to the terminal 8. The data output by the output unit 150 is displayed on the terminal 8. This display may be as a still image or as a moving image.

[0033] The terminal 8 according to this embodiment includes a display data acquisition unit 880, a coupling unit 885, a screen display unit 890, and a display switching unit 895.

[0034] The display data acquisition unit 880 requests the image generation device 1 to acquire data related to the display of the screen. In response to the acquisition request, it acquires the data related to the display of the screen that has been transmitted from the image generation device 1 via the network 5. The acquisition request that the display data acquisition unit 880 transmits to the image generation increment 1 includes at least a request for the type of image data. The request for the type of image data is a request for image data of the display image, or a request for image data capturing the operating state of the industrial machine 4. For example, if the acquisition request includes a request for image data of the display image, the display data acquisition unit 880 can acquire the requested image data of the display image and data related to subtitles. Also, if the acquisition request includes a request for image data capturing the operating state of the industrial machine 4, the display data acquisition unit 880 can acquire the requested image data capturing the operating state of the industrial machine 4 and data related to subtitles. The display data acquisition unit 880 may include information that uniquely identifies the industrial machine 4 in its request to the image generation device 1. The display data acquisition unit 880 may also include a request for the time in its request to the image generation device 1. The display data acquisition unit 880 outputs the acquired screen display data to the merging unit 885.

[0035] The merging unit 885 combines the data related to the screen display acquired by the display data acquisition unit 880 as image data. Figure 5 shows an example of image data combined by the merging unit 885. As illustrated in Figure 5, the merging unit 885 creates image data 430 related to subtitles based on data indicating the internal state of the industrial machine 4, for example. The created image data 430 related to subtitles may be superimposed on the image data 410 of the display screen to combine the image data 410 of the display screen with the data indicating the internal state of the industrial machine 4. The background of the image data 430 related to subtitles may be a semi-transparent image that allows the image data 410 of the display screen to be seen. In addition, the merging unit 885 may create image data related to a graph showing changes in the data indicating the internal state of the industrial machine 4, or image data indicating a predetermined internal state (e.g., a warning mark), and combine them with the image data 410 of the display screen. The merging unit 885 may combine data whose associated time is the same or nearby. Figure 6 shows another example of image data combined by the merging unit 885. As illustrated in Figure 6, the coupling unit 885 may combine the image data 420 capturing the operating state of the industrial machine 4 with the data indicating the internal state of the industrial machine 4 by superimposing the image data 430 related to subtitles onto the image data 420 capturing the operating state of the industrial machine 4. The coupling unit 885 outputs the data related to the display of the combined screen to the screen display unit 890.

[0036] The screen display unit 890 displays data related to the display of the combined screens of the combining unit 885 to the display device 870. Figure 7 shows an example of a screen displayed on terminal 8. Figure 7 shows an example of a display where image data of the display image and data related to subtitles are combined as data related to the display of the screen. In the example of Figure 7, the image data 410 related to the display screen is a moving image, and the associated time is superimposed on the upper left of the image data. The image data of the display image and the image data 430 related to subtitles are updated as the display time progresses. As illustrated in Figure 7, on terminal 8, the display screen of the control device can be viewed while the internal state of the industrial machine 4 can be viewed as subtitles. The display screen that the screen display unit 890 displays on the display device 870 may include an operation panel 450, as illustrated in Figure 7. The operation panel 450 has at least one operable object and accepts user input.

[0037] The display switching unit 895 switches the data related to the screen display acquired by the display data acquisition unit 880 based on user operation. The display switching unit 895 may switch the data related to the screen display acquired by the display data acquisition unit 880 based on user operation on, for example, the input device 871. This operation may be, for example, operation on a keyboard, pointing device, touch panel, etc. For example, the operable objects included in the operation panel 450 illustrated in Figure 7 may include an operation button for switching the image data 410 related to the display screen to image data 420 capturing the operating state of the industrial machine 4. In such a case, when the user operates the operation button, the display switching unit 895 instructs the display data acquisition unit 880 to switch the data related to the screen display to be acquired to image data capturing the operating state of the industrial machine 4 based on the user's operation. Then, based on the image data of the operating state of the industrial machine 4 and the subtitle data acquired by the display data acquisition unit 880, the screen display unit 890 displays a screen in which the image data 430 related to subtitles is superimposed on the image data 420 of the operating state of the industrial machine 4, as illustrated in Figure 8. The operation panel 450 illustrated in Figure 8 may include operation buttons for switching the display to the screen data 410 of the display image.

[0038] Thus, when switching the display between the screen data 410 of the display image and the image data 420 capturing the operating state of the industrial machine 4, the display switching unit 895 may instruct the display data acquisition unit 880 to include a request for the currently displayed time in the acquisition request to the image generation device 1. With this configuration, when a user is viewing a predetermined image at a certain time, they can switch the display to view other images captured at the same time. Alternatively, the display switching unit 895 may instruct the display data acquisition unit 880 to include a request for a time predetermined a certain time before the currently displayed time (for example, 5 seconds before) in the acquisition request to the image generation device 1. With this configuration, when a user is viewing a predetermined image at a certain time, they can switch the display to view other images captured at a slightly earlier time.

[0039] The display switching unit 895 may be configured to accept an operation to switch the industrial machine 4 to be displayed. The display switching unit 895 instructs the display data acquisition unit 880 to include information that uniquely identifies the industrial machine 4 specified by the user in the request to the image generation device 1.

[0040] The image generation device 1, equipped with the above configuration, allows the client to appropriately switch the displayed image data to other image data acquired at approximately the same time, enabling them to refer to the details of the image data while, if necessary, understanding the temporal relationship with the other image data.

[0041] Figure 9 is a schematic block diagram showing the functions of the image generation device 1 and terminal 8 according to the second embodiment of the present invention. The functions of the image generation device 1 and terminal 8 according to this embodiment are realized by the CPU 11 of the image generation device 1 and the CPU 811 of the terminal 8, as shown in Figure 1, executing system programs and controlling the operation of each part of the image generation device 1 and terminal 8.

[0042] The image generation device 1 and terminal 8 according to this embodiment differ from the image generation device 1 and terminal 8 according to the first embodiment in that the image generation device 1 is equipped with the function of the coupling portion 885 that is provided in the terminal 8 according to the first embodiment.

[0043] The image generation device 1 according to this embodiment includes a merging unit 160. The merging unit 160 merges the image data searched by the search processing unit 140 and the data related to subtitles into a single image data. The merging unit 160 has the same function as the merging unit 885 provided in the terminal 8 according to the first embodiment. The merging unit 160 outputs the merged image data to the output unit 150. The output unit 150 in this embodiment then transmits the input image data to the terminal 8.

[0044] The image generation device 1, equipped with the above configuration, performs the integration of data indicating the internal state of the industrial machine 4 with the image data on the server side (image generation device 1). This eliminates the need to prepare dedicated functions for each client's operating environment, and is expected to reduce development time. Furthermore, the load on the client terminal 8 during image data viewing is also reduced.

[0045] Although embodiments of the present invention have been described above, the present invention is not limited to the examples of embodiments described above, and can be implemented in various forms by making appropriate modifications. For example, in the embodiment described above, the image data capturing the operating state of the industrial machine 4 was obtained by capturing the industrial machine 4 with the imaging sensor 9a. However, this image data may be created using known machining simulation technology. Depending on the type of industrial machine 4, it may not be possible to properly capture the state of workpiece machining due to cutting fluid or other factors. In such cases, by using images created with machining simulation technology instead of image data captured by the imaging sensor 9a, it becomes easier to understand the operating state of the industrial machine 4.

[0046] Furthermore, the above-described embodiment shows an example in which the image generation device 1 and the terminal 8 are implemented on different computers. However, as illustrated in Figure 10, each function may be implemented on a single computer. In Figure 10, the display data acquisition unit 170, the coupling unit 160, the screen display unit 180, and the display switching unit 190 each have the same functions as the display data acquisition unit 880, the coupling unit 885, the screen display unit 890, and the display switching unit 895 according to the first embodiment. With this configuration, a system for the user to check the status of the industrial machine 4 can be constructed at a low cost using only the image generation device 1.

[0047] Furthermore, in the above-described embodiment, the coupling unit created image data by superimposing image data 410 of the display screen or image data 420 capturing the operating state of the industrial machine 4 with subtitles indicating the internal state of the industrial machine 4. However, as shown in Figure 11, for example, an area 440 of a predetermined size may be provided outside the image data, and image data may be created by superimposing subtitles indicating the internal state of the industrial machine 4 onto that area 440. The area 440 may be provided at any position above, below, to the left, or to the right of the image data. With this configuration, the subtitles do not obscure the image data, and thus, the readability of the information is expected to improve. [Explanation of Symbols]

[0048] 1. Image generation device 4. Industrial Machinery 5 Network 6. Fog Computer 7 Cloud Server 8 devices 9a, 9b Imaging sensors 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15,17,18,20 Interface 22 buses 70 Display device 71 Input device 72 External equipment 100 Status data acquisition unit 110 Display image data acquisition unit 120 Machine Image Data Acquisition Unit 130 Subtitle Data Creation Department 140 Search Processing Unit 150 Output section 160 Joint 811 CPU 812 ROM 813 RAM 814 Non-volatile memory 815, 817, 818, 820 Interface 822 Bus 870 Display device 871 Input device 872 External equipment 880 Display Data Acquisition Unit 885 Joint 890 Screen display section 895 Display switching section

Claims

1. An image generation system that generates images for understanding the status of industrial machinery, A first data storage unit stores first image data relating to the state of the industrial machine, associated with the time the first image data was acquired. A second data storage unit stores second image data relating to the state of the industrial machine, which is different from the first image data, in association with the time when the second image data was acquired. A subtitle data creation unit creates subtitle data indicating the internal state of the industrial machine based on data indicating the internal state of the industrial machine, A third data storage unit stores the data related to the subtitles in association with the time when the internal state of the industrial machine was acquired, A search processing unit searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data, The aforementioned search processing unit outputs the data it has searched, A display data acquisition unit acquires the data output by the output unit as data related to the display on the screen, A combining unit combines the data related to the screen display acquired by the display data acquisition unit into image data, The aforementioned coupling unit includes a screen display unit that displays the combined image data, Equipped with, The first image data is image data showing a display image of the industrial machine, The second image data is image data capturing the operating state of the industrial machine. Image generation system.

2. An image generation system for generating images for understanding the state of industrial machinery, A first data storage unit stores first image data relating to the state of the industrial machine, associated with the time the first image data was acquired. A second data storage unit stores second image data relating to the state of the industrial machine, which is different from the first image data, in association with the time when the second image data was acquired. A subtitle data creation unit creates subtitle data indicating the internal state of the industrial machine based on data indicating the internal state of the industrial machine, A third data storage unit stores the data related to the subtitles in association with the time when the internal state of the industrial machine was acquired, A search processing unit searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data, The aforementioned search processing unit outputs the data it has searched, A display data acquisition unit acquires the data output by the output unit as data related to the display on the screen, A combining unit combines the data related to the screen display acquired by the display data acquisition unit into image data, The aforementioned coupling unit includes a screen display unit that displays the combined image data, Equipped with, The first image data is image data capturing the operating state of the industrial machine, The second image data is image data showing a display image of the industrial machine. Image generation system.

3. The data indicating the internal state of the industrial machine includes at least one of the following: the value of the operator's control signal, the value of the operating signal, the value of a macro variable, the name of the tool being used, the dimensions of the tool being used, the workpiece coordinate values, the setting values ​​of predetermined parameters, and the commands of the currently executing machining program. The image generation system according to claim 1 or 2.

4. The coupling unit combines data indicating the internal state of the industrial machine in at least one form: image information or subtitle information. The image generation system according to claim 1 or 2.

5. The acquisition request is either a request for the first image data or a request for the second image data. The search processing unit, If the acquisition request is for the first image data, the first image data is searched for in the first data storage unit, and data related to subtitles associated with the retrieved first image data is searched for in the third data storage unit. If the acquisition request is for the second image data, the second image data is searched for in the second data storage unit, and data related to subtitles associated with the retrieved second image data is searched for in the third data storage unit. The image generation system according to claim 1.

6. The aforementioned acquisition request further includes a time request, The search processing unit searches for data acquired at a time close to the requested time. The image generation system according to claim 5.

Images