Image generation system and computer-readable recording medium

Abstract

An image generation device according to the present disclosure generates images for ascertaining the state of an industrial machine, the image generation device comprising: a merging unit that merges first image data pertaining to the state of the industrial machine and data indicating an internal state of the industrial machine as image data; a first data storage unit that stores the image data merged by the merging unit, in association with a date and time at which the first image data was acquired or a date and time at which the data indicating the internal state of the industrial machine was acquired; a second data storage unit that stores second image data pertaining to a different state of the industrial machine than that of the first image data, in association with a date and time at which the second image data was acquired; and an output unit that, in response to an external request, outputs the image data that is stored in the first data storage unit or the second image data that is stored in the second data storage unit.

Description

Image generation system and computer-readable recording medium

[0001] The present invention relates to an image generation system and a computer-readable recording medium.

[0002] At manufacturing sites where industrial machines are installed, operators understand the operating status of the machines by checking their screens and the operation of the machines. While the machines are operating, data collected from each part of the machines and data detected by sensors attached to the machines and their surroundings are recorded. This recorded data is used by users located remotely from the machines and engineers who maintain the machines to understand and analyze the operating status of the machines.

[0003] In order to understand the operating status of industrial machinery, there is a technology that makes it easier to understand information about past manufacturing by correlating and displaying an image of the industrial machinery with an image of a screen that displays the results of measuring physical quantities related to the operation of the industrial machinery (for example, Patent Document 1).

[0004] Japanese Patent Application Laid-Open No. 2018-073185

[0005] [Amendment pursuant to Rule 91 10.10.2024] Even if captured images are combined and presented, it may not be possible to accurately grasp the operating status of industrial machinery. In addition, if the process of combining multiple images and data is performed on the client side, a separate process will be required.

[0006] The image generating device according to the present invention solves the above-mentioned problems by appropriately transmitting images that provide information necessary to understand the operating state of industrial machinery from the server side.

[0007] One aspect of the present disclosure is an image generation system that generates an image for grasping a state of an industrial machine, the image generation system including: a first data storage unit that stores first image data relating to the state of the industrial machine in association with a time when the first image data was acquired; a second data storage unit that stores second image data relating to the state of the industrial machine, different from the first image data, in association with a time when the second image data was acquired; a subtitle data creation unit that creates data relating to subtitles indicating the internal state of the industrial machine based on data indicating the internal state of the industrial machine; a third data storage unit that stores the data relating to the subtitles in association with the time when the internal state of the industrial machine was acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; an output unit that outputs the data searched by the search processing unit; a display data acquisition unit that acquires the data output by the output unit as data relating to screen display; a combination unit that combines the data relating to screen display acquired by the display data acquisition unit as image data; and a screen display unit that displays the image data combined by the combination unit. Another aspect of the present disclosure is a computer-readable recording medium having recorded thereon a program that causes a computer to operate as an image generating device that generates images for grasping a state of an industrial machine, the computer comprising: a first data storage unit that stores first image data relating to the state of the industrial machine in association with a time when the first image data was acquired; a second data storage unit that stores second image data relating to the state of the industrial machine, different from the first image data, in association with a time when the second image data was acquired; a subtitle data creation unit that creates data relating to subtitles that indicate the internal state of the industrial machine based on data that indicates the internal state of the industrial machine; a third data storage unit that stores the data relating to the subtitles in association with the time when the internal state of the industrial machine was acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; and an output unit that outputs the data searched by the search processing unit.

[0008] [Amendment based on Rule 91 10.10.2024] According to one aspect of the present disclosure, the server side combines image data with data indicating the internal state of an industrial machine, eliminating the need for dedicated functions for each client side operating environment, which is expected to reduce development efforts. Furthermore, a search processing unit that searches for first image data, second image data, and third data stored at different times enables the data requester to simply request data related to a single time to obtain each piece of data stored at approximately the same time, which is expected to reduce development efforts and processing load related to request processing on the client side.

[0009] [Correction based on Rule 91 10.10.2024] This is a schematic hardware configuration diagram of an image generation system according to one embodiment of the present invention. This is a block diagram showing the schematic functions of an image generation system according to a first embodiment of the present invention. This is a diagram showing an example of image data of a display image of an industrial machine according to the first embodiment. This is a diagram showing an example of image data captured of the operating state of an industrial machine according to the first embodiment. This is a diagram showing an example of image data combined by a combining unit according to the first embodiment. This is a diagram showing another example of image data combined by a combining unit according to the first embodiment. This is a diagram showing an example of data output by an output unit according to the first embodiment displayed on a terminal. This is a block diagram showing the schematic functions of an image generation system according to a second embodiment of the present invention. This is a block diagram showing the schematic functions of an image generation system according to another embodiment of the present invention. This is a diagram showing an example of image data combined by a combining unit according to another embodiment.

[0010] An embodiment of the present invention will now be described with reference to the drawings. FIG. 1 is a schematic hardware configuration diagram showing the main components of an image generation system according to an embodiment of the present invention. An image generation system 300 according to this embodiment is configured by connecting an image generation device 1 and a terminal 8 via a network 5. The image generation device 1 can be implemented on a control device that controls industrial machinery 4 installed at a manufacturing site such as a factory. The image generation device 1 can also be implemented on a personal computer attached to the control device, or on a computer such as a fog computer 6 or cloud server 7 connected to the control device via a wired or wireless network. The following description will be based on an example in which the image generation device 1 according to this embodiment is implemented on a personal computer that is connected via a network to the control device that controls the industrial machinery 4 and operates as a server.

[0011] The CPU 11 provided in the image generation device 1 according to this embodiment is a processor that controls the entire image generation device 1. The CPU 11 reads a system program stored in the ROM 12 via the bus 22 and controls the entire image generation device 1 in accordance with the system program. The RAM 13 temporarily stores temporary calculation data, display data, various data input from outside, and the like.

[0012] The non-volatile memory 14 is composed of, for example, a battery-backed memory (not shown) or an SSD (Solid State Drive), and retains its stored state even when the image generation device 1 is powered off. The non-volatile memory 14 stores data acquired from the industrial machine 4, image data acquired from the imaging sensors 9 a and 9 b, programs and data read from an external device 72 via the interface 15, programs and data input via the input device 71, programs and data acquired from other devices via the network 5, and the like. The programs and data stored in the non-volatile memory 14 may be loaded into the RAM 13 when executed / used. In addition, various system programs such as known analysis programs are written in the ROM 12 in advance.

[0013] The interface 15 is an interface for connecting the CPU 11 of the image generating device 1 to an external device 72 such as a USB device. For example, programs, setting data, etc. are read from the external device 72. In addition, programs, setting data, etc. edited within the image generating device 1 can be stored in external storage means via the external device 72.

[0014] The interface 20 is an interface for connecting the CPU 11 of the image generation device 1 to a wired or wireless network 5. The network 5 is connected to industrial machines 4, fog computers 6, cloud servers 7, etc., and exchanges data with the image generation device 1.

[0015] The display device 70 displays various data loaded into the memory, data obtained as a result of executing programs, etc., output via the interface 17. The display device 70 may also be equipped with an LED indicator or a warning lamp that indicates the machine status. An input device 71, which is comprised of a keyboard, pointing device, etc., transmits commands, data, etc., based on operations by an operator, to the CPU 11 via the interface 18.

[0016] The image generation device 1, together with at least one industrial machine 4 and at least one terminal 8 connected via a network 5, constitutes an image generation system 300. The industrial machine 4 is a machine tool, an electric discharge machine, a robot, or the like installed at a manufacturing site such as a factory. Image sensors 9a and 9b are installed near the industrial machine 4. The image sensor 9a is installed so as to capture images of the state of workpiece processing being performed by the industrial machine 4, the operation of the industrial machine 4 itself, and the surrounding conditions. The image sensor 9a may be, for example, a camera. The image sensor 9b is installed so as to capture images of the display screen of a control device that controls the industrial machine 4. The image sensor 9b may be, for example, a camera or a capture device. The image sensors 9a and 9b generate image data such as still images and videos. The image data generated by the image sensors 9a and 9b is passed to the CPU 11 via the network 5 and an interface 20.

[0017] Terminal 8 operates as a client of image generation device 1. Terminal 8 may be, for example, a personal computer installed remotely from industrial machine 4 or image generation device 1. Terminal 8 may also be a portable device carried by a worker or the like. Terminal 8 has at least the function of acquiring image data generated by image generation device 1 via network 5 and displaying it.

[0018] The CPU 811 provided in the terminal 8 is a processor that controls the entire terminal 8. The CPU 811 reads a system program stored in the ROM 812 via the bus 822 and controls the entire terminal 8 in accordance with the system program. The RAM 813 temporarily stores temporary calculation data, display data, various data input from outside, and the like.

[0019] The non-volatile memory 814 is configured, for example, by a memory backed up by a battery (not shown) or an SSD (Solid State Drive), and maintains its storage state even when the power to the terminal 8 is turned off. The non-volatile memory 814 stores programs and data acquired from the image generation device 1 or other devices via the network 5, programs and data input via the input device 871, etc. The programs and data stored in the non-volatile memory 814 may be expanded into the RAM 813 when executed / used. In addition, various system programs such as well-known image processing programs are written in the ROM 812 in advance.

[0020] The interface 815 is an interface for connecting the CPU 811 of the terminal 8 to an external device 872 such as a USB device. For example, programs, data, etc. are read from the external device 872. In addition, programs, data, etc. created within the terminal 8 can be stored in external storage means via the external device 872.

[0021] The interface 820 is an interface for connecting the CPU 811 of the terminal 8 to a wired or wireless network 5. The image generation device 1, the industrial machine 4, the fog computer 6, the cloud server 7, etc. are connected to the network 5, and data is exchanged between the network 5 and the terminal 8.

[0022] The display device 870 displays data and the like obtained as a result of executing various data and programs loaded into memory, output via an interface 817. An input device 871, which is comprised of a keyboard, pointing device, touch panel, and the like, transfers instructions, data, and the like based on operations by an operator to the CPU 811 via an interface 818.

[0023] 2 is a schematic block diagram showing functions provided in the image generation device 1 and terminal 8 according to the first embodiment of the present invention. The functions provided in the image generation device 1 and terminal 8 according to this embodiment are realized by the CPU 11 provided in the image generation device 1 and the CPU 811 provided in the terminal 8 shown in FIG. 1 executing system programs and controlling the operation of each part of the image generation device 1 and terminal 8.

[0024] The image generating device 1 according to this embodiment includes a status data acquiring unit 100, a display image data acquiring unit 110, a machine image data acquiring unit 120, a subtitle data creating unit 130, a search processing unit 140, and an output unit 150. In addition, on the RAM 13 to the non-volatile memory 14 of the image generating device 1, a first data storage unit 210 which is an area for storing image data of an image of the display screen of a control device that controls the industrial machine 4, a second data storage unit 220 which is an area for storing image data of an image of the operating state of the industrial machine 4, and a third data storage unit 230 which is an area for storing data related to subtitles that show the internal state of the industrial machine 4 are prepared in advance.

[0025] The status data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the industrial machine 4. The status data acquisition unit 100 acquires data indicating the internal state of the industrial machine 4 from the control device of the industrial machine 4. The data indicating the internal state of the industrial machine 4 may be, for example, the value of an operation signal of an operator of the industrial machine 4, the value of an operation signal, the value of a macro variable, the name of a tool used, the dimensional values ​​of the tool used, the workpiece coordinate values, the setting values ​​of predetermined parameters, commands of a machining program being executed, etc. The data indicating the internal state of the industrial machine 4 may be time-series data, which is a series of values ​​acquired at a predetermined cycle. The data indicating the internal state of the industrial machine 4 is associated with at least the time when the data was acquired or the time since the industrial machine 4 started operating. The status data acquisition unit 100 outputs the acquired data indicating the internal state of the industrial machine 4 to the subtitle data creation unit 130.

[0026] [Correction based on Rule 91 10.10.2024] The display image data acquisition unit 110 acquires image data captured from the display screen of a control device that controls the industrial machine 4. The display image data acquisition unit 110 acquires image data of the display image from the imaging sensor 9b. FIG. 3 shows an example of image data 410 of the display image acquired by the display image data acquisition unit 110. The image data 410 of the display image may be time-series data of still images acquired at a predetermined cycle. It may also be data of moving images. The image data 410 of the display image is associated with at least the time when the data was acquired or the time since the industrial machine 4 started operating. The display image data acquisition unit 110 stores the acquired image data 410 of the display image in the first data storage unit 210.

[0027] The machine image data acquisition unit 120 acquires image data of the operating state of the industrial machine 4, such as the state of workpiece machining, the operation of the industrial machine 4 itself, and surrounding conditions. The machine image data acquisition unit 120 acquires image data of the operating state of the industrial machine 4 from the image sensor 9a. FIG. 4 shows an example of image data 420 of the operating state of the industrial machine 4 acquired by the machine image data acquisition unit 120. The image data 420 of the operating state of the industrial machine 4 may be time-series data of still images acquired at a predetermined cycle. It may also be moving image data. The image data 420 of the operating state of the industrial machine is associated with at least the time when the data was acquired or the time since the industrial machine 4 started operating. The machine image data acquisition unit 120 stores the acquired image data 420 of the operating state of the industrial machine 4 in the second data storage unit 220.

[0028] [Correction based on Rule 91 10.10.2024] The caption data creation unit 130 creates data related to captions indicating the internal state of the industrial machine 4 based on the data indicating the internal state of the industrial machine 4 input from the state data acquisition unit 100. The data related to captions may be, for example, a character string indicating the internal state of the industrial machine 4. It may also be an image of the character string indicating the internal state of the industrial machine 4. The data related to captions created by the caption data creation unit 130 may represent a character string meaning a predetermined state indicated by the data indicating the internal state of the industrial machine 4. For example, if the automatic operation signal is ON in the industrial machine 4, caption data such as "Automatic Operation" may be created. Furthermore, if the override value is set to 100% in the industrial machine 4, caption data such as "Rapid Traverse Override 100%" may be created. The data related to captions may represent a character string in one predetermined language or may represent character strings in multiple languages. The subtitle data creation unit 130 associates the time when the industrial machine 4 entered a specified state and period data indicating a predetermined period of time with the data related to the created subtitles, and stores the data in the third data storage unit 230.

[0029] When the output unit 150 receives an acquisition request for image data from the terminal 8, the search processing unit 140 searches the first data storage unit 210, the second data storage unit 220, and the third data storage unit 230 for the image data requested by the acquisition request. The search processing unit 140 then outputs the data obtained as a result of the search to the output unit 150. The acquisition request from the terminal 8 includes at least a request for the type of image data. The request for the type of image data is either a request for image data of a display image or a request for image data of an image of the operating state of the industrial machine 4. If the acquisition request includes a request for image data of a display image, the search processing unit 140 searches for the image data of the requested display image from the data stored in the first data storage unit 210. If the acquisition request includes a request for image data of an image of the operating state of the industrial machine 4, the search processing unit 140 searches for the image data of the requested display image from the data stored in the second data storage unit 220. The search processing unit 140 then outputs the obtained image data to the output unit 150. In either case, the search processing unit 140 may search for data related to subtitles stored in the third data storage unit 230 in accordance with the searched image data, and output the data to the output unit 150.

[0030] The acquisition request from the terminal 8 may include information that uniquely identifies the industrial machine 4. When the identification information of the industrial machine 4 is included, the search processing unit 140 searches for image data related to the industrial machine 4 that corresponds to the requested identification information from among the image data stored in each memory unit.

[0031] The acquisition request from the terminal 8 may include a request related to time. When the request related to time is included, the search processing unit 140 searches for image data stored in each storage unit that was captured at the time closest to the requested time. The search processing unit 140 may also search for image data that was captured at the time closest to the requested time. The search processing unit 140 then outputs the obtained image data to the output unit 150. In either case, the search processing unit 140 may search the third data storage unit 230 for data related to subtitles captured at a time close to the searched image data, in accordance with the searched image data, and output the data to the output unit 150.

[0032] The output unit 150 instructs the search processing unit 140 to search for image data based on a request from the terminal 8. The search processing unit 140 then outputs the image data and subtitle data searched for by the search processing unit 140 via the network 5. The output unit 150 may be configured to exclusively output either the data stored in the first data storage unit 210 or the data stored in the second data storage unit 220 at the same time in response to a request from one terminal 8. The output unit 150 may process the data to be output in accordance with the type of terminal 8. For example, templates that define the layout and size of image data for each type of terminal 8 may be prepared in advance. The output unit 150 may then identify the type of terminal 8 included in the request from the terminal 8, process the data based on the template corresponding to the identified type of terminal 8, and output the processed data to the terminal 8. The data output by the output unit 150 is displayed on the terminal 8. This display may be a still image. Alternatively, it may be a moving image.

[0033] The terminal 8 according to this embodiment includes a display data acquisition unit 880 , a combining unit 885 , a screen display unit 890 , and a display switching unit 895 .

[0034] [Correction based on Rule 91 10.10.2024] The display data acquisition unit 880 requests the image generation device 1 to acquire data related to the screen display. Then, the display data acquisition unit 880 acquires the data related to the screen display transmitted from the image generation device 1 via the network 5 as a response to the acquisition request. The acquisition request transmitted by the display data acquisition unit 880 to the image generation device 1 includes at least a request for the type of image data. The request for the type of image data is a request for image data of a display image or a request for image data capturing the operating state of the industrial machine 4. For example, if the acquisition request includes a request for image data of a display image, the display data acquisition unit 880 can acquire the requested image data of the display image and data related to subtitles. Furthermore, if the acquisition request includes a request for image data capturing the operating state of the industrial machine 4, the display data acquisition unit 880 can acquire the requested image data capturing the operating state of the industrial machine 4 and data related to subtitles. The display data acquisition unit 880 may include information that uniquely identifies the industrial machine 4 in the request to the image generation device 1. Furthermore, the display data acquisition unit 880 may include a request for the time in the request to the image generation device 1. The display data acquisition unit 880 outputs the acquired data related to the display of the screen to the combining unit 885.

[0035] [Correction Based on Rule 91 10.10.2024] The combining unit 885 combines the data related to the screen display acquired by the display data acquiring unit 880 as image data. FIG. 5 shows an example of image data combined by the combining unit 885. As illustrated in FIG. 5, the combining unit 885 creates image data 430 related to subtitles based on data indicating the internal state of the industrial machine 4, for example. The created image data 430 related to subtitles may then be superimposed on the image data 410 of the display screen, thereby combining the image data 410 of the display screen with the data indicating the internal state of the industrial machine 4. The background of the image data 430 related to subtitles may be a semi-transparent image through which the image data 410 of the display image can be seen. Alternatively, the combining unit 885 may create, for example, image data related to a graph showing changes in data indicating the internal state of the industrial machine 4 or image data indicating a predetermined internal state (e.g., a warning mark), and combine the created image data with the image data 410 of the display image. The combining unit 885 may combine data associated with the same or nearby times. Fig. 6 shows another example of image data combined by the combination unit 885. As illustrated in Fig. 6 , the combination unit 885 may combine the image data 420 capturing the image of the operating state of the industrial machine 4 with data indicating the internal state of the industrial machine 4 by superimposing image data 430 relating to subtitles on the image data 420 capturing the image of the operating state of the industrial machine 4. The combination unit 885 outputs the combined data relating to the screen display to the screen display unit 890.

[0036] The screen display unit 890 displays the data related to the screen display combined by the combining unit 885 on the display device 870. FIG. 7 shows an example of a screen displayed on the terminal 8. FIG. 7 shows an example of a display when image data of a display image and data related to subtitles are combined as data related to the screen display. In the example of FIG. 7, image data 410 related to the display screen is a moving image, and an associated time is superimposed in the upper left corner of the image data. The image data of the display image and image data 430 related to subtitles are updated as the display time passes. As exemplified in FIG. 7, while viewing the display screen of the control device on the terminal 8, the internal state of the industrial machine 4 can be viewed in subtitles. The display screen displayed on the display device 870 by the screen display unit 890 may include an operation panel 450, as exemplified in FIG. 7. The operation panel 450 has at least one operable object and accepts user operations.

[0037] [Correction Based on Rule 91 10.10.2024] The display switching unit 895 switches the data related to the screen display acquired by the display data acquisition unit 880 based on a user operation. The display switching unit 895 may switch the data related to the screen display acquired by the display data acquisition unit 880 based on, for example, a user operation on the input device 871. This operation may be, for example, an operation of a keyboard, a pointing device, a touch panel, or the like. For example, the operable objects included in the operation panel 450 illustrated in FIG. 7 may include an operation button for switching the image data 410 related to the display screen to image data 420 of the captured operating state of the industrial machine 4. In such a case, when the user operates the operation button, the display switching unit 895 instructs the display data acquisition unit 880 to switch the data related to the screen display acquired to the image data of the captured operating state of the industrial machine 4 based on the user operation. Then, based on the image data of the captured operating state of the industrial machine 4 and the data related to subtitles acquired by the display data acquisition unit 880, the screen display unit 890 displays a screen in which image data 430 related to subtitles is superimposed on image data 420 of the captured operating state of the industrial machine 4, as exemplified in Fig. 8. The operation panel 450 exemplified in Fig. 8 may include an operation button for switching the display to image data 410 of the display image.

[0038] [Correction Based on Rule 91 10.10.2024] In this way, when switching the display between the image data 410 of the display image and the image data 420 capturing the operating state of the industrial machine 4, the display switching unit 895 may instruct the display data acquisition unit 880 to include a request for the currently displayed time in the acquisition request to the image generation device 1. With this configuration, while a user is viewing a specific image at a certain time, the user can view other images captured at the same time by performing a display switching operation. Furthermore, the display switching unit 895 may instruct the display data acquisition unit 880 to include a request for a time a predetermined time before the currently displayed time (e.g., 5 seconds before) in the acquisition request to the image generation device 1. With this configuration, while a user is viewing a specific image at a certain time, the user can view other images captured a short time before by performing a display switching operation.

[0039] The display switching unit 895 may be configured to accept an operation to switch the displayed industrial machine 4. The display switching unit 895 instructs the display data acquisition unit 880 to include, in a request to the image generation device 1, information that uniquely identifies the industrial machine 4 specified by the user's operation.

[0040] The image generating device 1 having the above configuration allows the client to switch the displayed image data to other image data acquired at approximately the same time and view it as needed, so that the client can refer to the details of the image data and understand the temporal relationship with other image data as needed.

[0041] 9 is a schematic block diagram showing functions provided in the image generation device 1 and terminal 8 according to the second embodiment of the present invention. The functions provided in the image generation device 1 and terminal 8 according to this embodiment are realized by the CPU 11 provided in the image generation device 1 and the CPU 811 provided in the terminal 8 shown in FIG. 1 executing system programs and controlling the operation of each part of the image generation device 1 and terminal 8.

[0042] The image generation device 1 and terminal 8 according to this embodiment differ from the image generation device 1 and terminal 8 according to the first embodiment in that the image generation device 1 has the function of the connection unit 885 that the terminal 8 according to the first embodiment has.

[0043] The image generating device 1 according to this embodiment includes a combining unit 160. The combining unit 160 combines the image data searched by the search processing unit 140 and the data related to the subtitles into one piece of image data. The combining unit 160 has the same function as the combining unit 885 included in the terminal 8 according to the first embodiment. The combining unit 160 outputs the combined image data to the output unit 150. Then, the output unit 150 according to this embodiment transmits and outputs the input image data to the terminal 8.

[0044] The image generation device 1 having the above configuration combines image data with data indicating the internal state of the industrial machine 4 on the image generation device 1 side, which is the server, so there is no need to prepare dedicated functions for each client operating environment, which is expected to reduce development man-hours. In addition, the load related to the operation of the client terminal 8 when viewing image data is also reduced.

[0045] Although the embodiments of the present invention have been described above, the present invention is not limited to the above-described embodiments and can be embodied in various forms by making appropriate modifications. For example, in the above-described embodiments, image data capturing an image of the operating state of the industrial machine 4 is obtained by capturing an image of the industrial machine 4 with the imaging sensor 9a. However, this image data may be generated using image data generated using a known machining simulation technique. Depending on the type of industrial machine 4, it may be difficult to capture a good image of the state of workpiece machining due to cutting fluid or the like. In such cases, the operating state of the industrial machine 4 can be more easily understood by using an image generated using a machining simulation technique instead of the image data captured with the imaging sensor 9a.

[0046] In the above-described embodiment, an example was shown in which the image generation device 1 and the terminal 8 were implemented on different computers. However, as illustrated in FIG. 10 , each of these functions may be implemented on a single computer. In FIG. 10 , the display data acquisition unit 170, the combining unit 160, the screen display unit 180, and the display switching unit 190 have the same functions as the display data acquisition unit 880, the combining unit 885, the screen display unit 890, and the display switching unit 895 according to the first embodiment, respectively. With this configuration, a system that allows a user to check the status of an industrial machine 4 using only the image generation device 1 can be constructed at low cost.

[0047] [Correction based on Rule 91 10.10.2024] Furthermore, in the above-described embodiment, the combining unit creates image data in which image data 410 of a display image or image data 420 capturing an image of the operating state of the industrial machine 4 is superimposed with subtitles indicating the internal state of the industrial machine 4. However, as shown in FIG. 11 , for example, it is also possible to create image data in which an area 440 of a predetermined size is provided outside the image data and subtitles indicating the internal state of the industrial machine 4 are superimposed on that area 440. The area 440 may be provided above, below, left, or right of the image data. This configuration is expected to improve the readability of information because the subtitles do not obscure the image data.

[0048] REFERENCE SIGNS LIST 1 Image generation device 4 Industrial machine 5 Network 6 Fog computer 7 Cloud server 8 Terminal 9a, 9b Image sensor 11 CPU 12 ROM 13 RAM 14 Non-volatile memory 15, 17, 18, 20 Interface 22 Bus 70 Display device 71 Input device 72 External device 100 Status data acquisition unit 110 Display image data acquisition unit 120 Machine image data acquisition unit 130 Subtitle data creation unit 140 Search processing unit 150 Output unit 160 Combining unit 811 CPU 812 ROM 813 RAM 814 Non-volatile memory 815, 817, 818, 820 Interface 822 Bus 870 Display device 871 Input device 872 External device 880 Display data acquisition unit 885 Combining unit 890 Screen display unit 895 Display switching unit

Claims

1. An image generation system for generating an image for grasping a state of an industrial machine, comprising: a first data storage unit which stores first image data relating to a state of the industrial machine in association with a time when the first image data was acquired; a second data storage unit which stores second image data relating to a state of the industrial machine different from the first image data in association with a time when the second image data was acquired; a subtitle data creation unit which creates data relating to subtitles showing the internal state of the industrial machine based on data showing the internal state of the industrial machine; a third data storage unit which stores the data relating to the subtitles in association with the time when the internal state of the industrial machine was acquired; a search processing unit which searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; an output unit which outputs the data searched by the search processing unit; a display data acquisition unit which acquires the data output by the output unit as data relating to screen display; a combination unit which combines the data relating to screen display acquired by the display data acquisition unit as image data; and a screen display unit which displays the image data combined by the combination unit.

2. [Correction under Rule 91 10.10.2024] The image generating system according to claim 1, wherein the first image data is image data showing a display image of the industrial machine, and the second image data is image data capturing an operating state of the industrial machine.

3. [Correction under Rule 91 10.10.2024] The image generating system according to claim 1, wherein the first image data is image data capturing an operating state of the industrial machine, and the second image data is image data showing a display image of the industrial machine.

4. An image generating system as described in claim 2 or 3, wherein the data indicating the internal state of the industrial machine includes at least any of the following information: the value of an operation signal of an operator of the industrial machine, the value of an operation signal, the value of a macro variable, the name of a tool used, the dimensional value of the tool used, the work coordinate value, the setting value of a specified parameter, and the command of a machining program being executed.

5. The image generating system according to claim 2 or 3, wherein the combining unit combines data indicating the internal state of the industrial machine in the form of at least one of image information and subtitle information.

6. The image generating system of claim 1, wherein the acquisition request is either a request for the first image data or a request for the second image data, and the search processing unit, when the acquisition request is a request for the first image data, searches for the first image data from the first data storage unit and searches for data relating to subtitles related to the searched first image data from the third data storage unit, and when the acquisition request is a request for the second image data, searches for the second image data from the second data storage unit and searches for data relating to subtitles related to the searched second image data from the third data storage unit.

7. The image generating system according to claim 6, wherein the acquisition request further includes a time request, and the search processing unit searches for data acquired at a time close to the requested time.

8. A computer-readable recording medium having recorded thereon a program for operating a computer as an image generating device that generates images for grasping the state of industrial machinery, comprising: a first data storage unit that stores first image data relating to the state of the industrial machinery in association with the time when the first image data was acquired; a second data storage unit that stores second image data relating to the state of the industrial machinery different from the first image data in association with the time when the second image data was acquired; a subtitle data creation unit that creates data relating to subtitles showing the internal state of the industrial machinery based on data showing the internal state of the industrial machinery; a third data storage unit that stores the data relating to the subtitles in association with the time when the internal state of the industrial machinery was acquired; a search processing unit that searches the first data storage unit or the second data storage unit and the third data storage unit based on a request to acquire the first image data or the second image data; and an output unit that outputs the data searched by the search processing unit.

Images