Image generating device

Abstract

An image generating device (10) comprises: an input unit (20) that acquires video from a camera (3) inserted into a pipeline (2); and a control unit (30) that generates, from the video, a developed view depicting the interior of the pipeline (2). The control unit (30) divides the video into a plurality of first frame images, determines the presence or absence of blurring in each of the first frame images, and selects a plurality of second frame images to be used for creating the developed view from the plurality of first frame images. When the selected second frame images include a blurred frame image, the control unit (30) replaces the blurred second frame image with a blur-free first frame image selected from the plurality of first frame images, and creates the developed view on the basis of the second frame images after the replacement.

Description

Image Generation Device

[0001] The present disclosure relates to image generation devices.

[0002] Conventionally, underground pipelines have been inspected by visually inspecting the interior of the pipeline using a pipe camera. For example, as shown in Figure 12, an inspector 102 inserts a pipe camera 104 into a pipeline 103 in a manhole 101 and captures images of the interior of the pipeline 103 while towing the pipe camera 104. An imaging optical system and an imaging element are provided at the tip of the pipe camera 104, and the image is captured in the direction opposite to the direction in which the pipe camera 104 is being towed. The worker 102 can check the image of the interior of the pipeline 103 using a monitor device installed in the manhole 101 or on the ground.

[0003] Furthermore, in order to record the location and degree of deterioration such as rust, facility managers perform inspections by extracting multiple frame images 106 from the image captured by the pipe camera 104 and creating a development 107 along the circumferential direction of the inner wall, as shown in Fig. 13 (see, for example, Non-Patent Document 1). In the example of Fig. 13, developments 107 are generated from frame images 106 acquired at three points, A, B, and C, within the pipeline 103. By combining the developments generated from the frame images from these multiple points, it becomes possible to grasp the overall condition of the interior of the pipeline 103.

[0004] Tsunoda Masayoshi et al., "Development of Software for Efficient Sewer Pipe Inspection," Proceedings of the Institute of Infrastructure Maintenance Practice Research, Vol. 1, No. 1, pp. 356-362 (March 2022).

[0005] However, as shown in FIG. 14 , the inside of the pipeline 103 may have irregularities due to rust bumps 105 or the like. When the pipe camera 104 photographs the inner surface of the pipeline 103, the pipe camera 104 may get caught on the irregularities and move irregularly, causing blurring in the image captured by the pipe camera 104. Blurred frame images are not suitable for creating a development diagram. Therefore, when creating a development diagram, it is possible to replace blurred frame images with unblurred frame images selected from the vicinity of the image in question. However, depending on the conditions inside the pipeline, the number of blurred images may be large. Therefore, manually determining whether or not there is blur and replacing the images may result in a huge amount of work.

[0006] Therefore, the purpose of the present disclosure, which has been made with these points in mind, is to provide an image generation device that can automatically remove blurred frame images and create a development drawing from only clear, unblurred images.

[0007] An image generating device according to one embodiment includes an input unit that acquires images from a camera inserted into a pipeline, and a control unit that generates an unfolded view of the inside of the pipeline from the images, wherein the control unit divides the images into a plurality of first frame images, determines whether each of the first frame images is blurred, selects a plurality of second frame images to be used to create the unfolded view from the first frame images, and, when the second frame images include a blurred frame image, replaces the blurred second frame image with an unblurred first frame image selected from the plurality of first frame images, and creates the unfolded view based on the replaced second frame image.

[0008] According to the image generating device of the present disclosure, even if the selected second frame image contains a blurred image, it is possible to create a development view from only clear, unblurred images by replacing it with an unblurred first frame image. This makes it possible to improve the quality of inspection records of underground pipelines and reduce the workload.

[0009] 1 is a block diagram showing a schematic configuration of a system for capturing images of the inside of a pipeline according to one embodiment. FIG. 2 is a block diagram showing a schematic configuration of the image generating device of FIG. 1. FIG. 3 is a block diagram showing a schematic configuration of the moving image dividing unit of FIG. 2. FIG. 4 is a block diagram showing a schematic configuration of the blur learning unit of FIG. 2. FIG. 5 is a block diagram showing a schematic configuration of the blur determination unit of FIG. 2. FIG. 6 is a block diagram showing a schematic configuration of the development drawing creation unit of FIG. 2. FIG. 7 is a diagram explaining an example of a method for replacing a blurred image when selecting second frame images at equal time intervals. FIG. 8 is a diagram explaining an example of a method for replacing a blurred image when selecting second frame images at equal distance intervals. FIG. 9 is a diagram explaining an example of processing when a blurred image cannot be replaced. FIG. 10 is a diagram explaining a method for creating a development drawing from a second frame image. FIG. 11 is a diagram explaining a method for creating a development drawing from a second frame image. FIG. 12 is a flowchart showing the procedure for generating a development drawing according to one embodiment. FIG. 13 is a diagram explaining a method for capturing images of the inside of a pipeline using a pipe camera. FIG. 14 is a diagram explaining a method for creating a development drawing from images from a pipe camera. FIG. 15 is a diagram explaining an example of a reason why blur occurs when capturing images using a pipe camera.

[0010] Hereinafter, embodiments of the present disclosure will be described with reference to the drawings. The drawings used in the following description are schematic. The dimensional ratios and the like in the drawings do not necessarily correspond to the actual ones.

[0011] (Overall Configuration) In one embodiment, a system including an image generating device 10 of the present disclosure is used to inspect the inside of a pipeline 2 connecting two manholes 1A and 1B, as shown in Fig. 1. The manholes 1A and 1B are spaces that humans can enter, provided for the management and maintenance of underground facilities such as the pipeline 2 installed underground. To inspect the pipeline 2, a pipe camera 3 is inserted into the pipeline 2 from the manhole 1A. The pipeline 2 includes various types of pipelines, such as a pipeline that houses communication cables, a pipeline that houses power cables, and a sewer pipeline.

[0012] The pipe camera 3 may be a wide-angle camera with a wide angle of view. The pipe camera 3 can photograph the inner wall of the pipeline 2 to observe corrosion, damage, and the like. The pipe camera 3 may have a lighting device for illuminating the direction of photography. The pipe camera 3 is retracted, for example, by pulling a cable 4 connected to the pipe camera 3 with a driving mechanism 5 such as a winch disposed on the ground. The driving mechanism 5 can retract the pipe camera 3 by pulling the cable 4 at a constant speed.

[0013] The pipe camera 3 captures images of the interior of the pipeline 2 ahead while moving backward within the pipeline 2. The images captured by the pipe camera 3 are transmitted to the image generating device 10 by wired or wireless communication means. The image generating device 10 can display the images captured by the pipe camera 3 in real time to an operator inspecting the pipeline 2. Note that a monitor device may be provided separately from the image generating device 10 to display the images captured by the pipe camera 3 in real time.

[0014] A sensor for measuring the distance traveled by the pipe camera 3 may be provided inside the manhole 1A. Alternatively, information on the distance traveled by the pipe camera 3 may be measured by the driving unit 5. The image generating device 10 is capable of acquiring position information of the pipe camera 3 inside the pipeline 2.

[0015] (Image Generation Device) Image generation device 10 is a device that generates an unfolded view of the inside of pipeline 2 from images captured by pipe camera 3. Image generation device 10 may be placed in any location, such as inside manhole 1A, on the ground near manhole 1A, or in a remote location. Image generation device 10 may generate an unfolded view by sequentially acquiring images captured by pipe camera 3 as the images are captured, or may generate an unfolded view by acquiring images some time after the image capturing is completed.

[0016] Image generating device 10 may be a dedicated device, or may be a device in which a general-purpose computer such as a PC (Personal Computer) is installed with a program for image generating device 10. As shown in FIG. 2 , image generating device 10 includes an input unit 20, a control unit 30, an output unit 40, and a storage unit 50.

[0017] The input unit 20 includes a communication interface for acquiring the video captured by the pipe camera 3. The communication interface may be either a wired or wireless communication interface. In the case of wired communication, the communication interface may include, for example, a Universal Serial Bus (USB) or a Local Area Network (LAN) interface. In the case of wireless communication, the communication interface may include, for example, an interface compatible with a mobile communication standard such as Bluetooth (registered trademark), IEEE 802.11, LTE (Long Term Evolution), 4G (4th generation), or 5G (5th generation). The image generating device 10 may acquire the video via a recording medium such as a USB memory or an SD card rather than via communication. In this case, a recording medium reader functions as the input unit 20.

[0018] The input unit 20 can acquire RGB video (moving image) with a width w (px) and a height h (px). For example, the width is 1080 (px) and the height is 720 (px). The video is a moving image captured of the inside of a pipeline or the like buried underground. The number of frames per second (fps) of the video is arbitrary, but may be, for example, 30 fps or 60 fps. In addition to the video captured by the pipe camera 3, the input unit 20 may also acquire information on the distance or position traveled by the pipe camera 3. Information such as the video acquired by the input unit is stored in the memory unit 50.

[0019] The control unit 30 includes at least one processor, at least one dedicated circuit, or a combination thereof. The processor is a general-purpose processor such as a central processing unit (CPU) or a graphics processing unit (GPU), or a dedicated processor specialized for a specific process. The dedicated circuit is, for example, a field-programmable gate array (FPGA) or an application-specific integrated circuit (ASIC). The control unit 30 controls each unit of the image generation device 10 and executes processes related to the operation of the image generation device 10. The processes executed by the control unit 30 can be considered to be processes executed by the image generation device 10.

[0020] The output unit 40 outputs information to a user of the image generation device 10. The output unit 40 may include a display, a speaker, a printer, etc. The display may be, for example, a liquid crystal display (LCD), an organic electroluminescence (EL) display, or an inorganic EL display. The output unit 40 may output information to a portable storage medium. The output unit 40 may transmit information to an external information processing device using wired or wireless communication means.

[0021] The storage unit 50 includes at least one semiconductor memory, at least one magnetic memory, at least one optical memory, or a combination of at least two of these. The semiconductor memory is, for example, a random access memory (RAM) or a read-only memory (ROM). The RAM is, for example, a static random access memory (SRAM) or a dynamic random access memory (DRAM). The ROM is, for example, an electrically erasable programmable read-only memory (EEPROM). The storage unit 50 functions as, for example, a main storage device, an auxiliary storage device, or a cache memory. The storage unit 50 stores programs and data used in the operation of the image generation device 10, as well as data obtained by the operation of the image generation device 10. For example, an image acquired by the image generation device 10 via the input unit 20 is stored in the storage unit 50. The control unit 30 performs various processes on the image stored in the storage unit 50.

[0022] The image generating device 10 may further include a user interface that accepts input operations from an operator inspecting the pipeline 2. The user interface may include a mouse, a keyboard, and a touch panel integrated with the display of the output unit 40.

[0023] The functions of the image generation device 10 are realized by executing a program relating to the image generation method of this embodiment on a processor corresponding to the control unit 30. That is, the functions of the image generation device 10 are realized by software. The program causes a computer to execute the operations of the image generation device 10, thereby causing the computer to function as the image generation device 10. That is, the computer functions as the image generation device 10 by executing the operations of the image generation device 10 in accordance with the program.

[0024] In this embodiment, the program can be recorded on a computer-readable recording medium. The computer-readable recording medium includes non-transitory computer-readable media, such as a magnetic recording device, an optical disc, a magneto-optical recording medium, or a semiconductor memory. The program can be distributed, for example, by selling, transferring, or lending a portable recording medium, such as a DVD (digital versatile disc) or a CD-ROM (compact disc read only memory), on which the program is recorded. The program can also be distributed by storing the program in the storage of an external server and transmitting the program from the external server to another computer. The program can also be provided as a program product.

[0025] (Processing Executed by the Control Unit) The control unit 30 includes a moving image division unit 31, a blur learning unit 32, a blur determination unit 33, and a development drawing creation unit 34. Each of the components of the moving image division unit 31, the blur learning unit 32, the blur determination unit 33, and the development drawing creation unit 34 may be a hardware module or a software module. The processing executed by each component can be rephrased as processing executed by the control unit 30. As described below, the processing executed by the blur learning unit 32 does not have to be executed in the same device as the image generation device 10. The processing executed by the blur learning unit 32 may be executed in advance by a computer different from the image generation device 10.

[0026] The video segmentation unit 31 segments the video (moving image) of the interior of the pipeline 2 acquired by the input unit 20 into frame images. Hereinafter, the frame images segmented by the video segmentation unit 31 are referred to as first frame images. As shown in FIG. 3 , the video segmentation unit 31 includes a frame number reading unit 31a and a frame segmentation unit 31b. The frame number reading unit 31a reads the number of frames per second (fps) from the input video. The frame segmentation unit 31b segments the video into first frame images based on the number of frames per second read by the frame number reading unit 31a and stores the first frame images. Instead of using all frames as first frame images, the number of first frame images to be stored may be reduced by thinning out the frames as needed. The stored first frame images are then passed to the blur determination unit 33.

[0027] The blur learning unit 32 performs machine learning using a plurality of prepared frame images with blur and without blur as training data to generate a trained model. The generated trained model becomes a determiner 32d that determines whether or not a frame image is blurred. The determiner generated by the blur learning unit 32 is loaded into the blur determination unit 33. The blur learning unit 32 executes the learning phase of machine learning. The blur learning unit 32 does not need to be included in the image generation device 10. The processing of the blur learning unit 32 may be executed in advance by a computer separate from the image generation device 10. In the image generation device 10, a determiner generated by a separate computer may be incorporated into the blur determination unit 33.

[0028] As shown in FIG. 4, the blur learning unit 32 includes an image input unit 32a, a machine learning unit 32b, and an evaluation unit 32c.

[0029] The image input unit 32a receives training data including a plurality of pre-prepared blurred and unblurred images. The training data may be an image set in which frame images extracted from a video of the interior of the pipeline 2 are divided into two categories: blurred and unblurred frame images. The number of first frame images included in the image set is 10 or more, preferably 100 or more, and more preferably 1,000 or more. The classification of blurred and unblurred frame images is performed according to a predetermined definition. For example, a blurred frame image can be defined as an image in which the entire image is blurred and deterioration inside the pipeline cannot be visually recognized. An unblurred frame image can be defined as an image in which the contours and colors of the subject are clearly visible. However, the definition of blurred or unblurred is not limited to the above and can be defined arbitrarily. The image input unit 32a divides the image set into learning and evaluation images and passes them to the machine learning unit 32b and the evaluation unit 32c, respectively.

[0030] The machine learning unit 32b performs machine learning using the image set as training data to generate a trained model. As the machine learning method, for example, a convolutional neural network can be used. Alternatively, a recurrent neural network, a support vector machine, or the like can be used. The machine learning unit 32b passes the trained model to the evaluation unit 32c.

[0031] The evaluation unit 32c evaluates the trained model created by the machine learning unit 32b using the evaluation image set acquired from the image input unit 32a. The evaluation unit 32c uses the trained model passed from the machine learning unit 32b to determine whether or not the frame images included in the evaluation image set are blurred, and calculates the accuracy rate. If the accuracy rate does not meet the user's required standard, the blur learning unit 32 returns to the machine learning unit 32b and repeats learning until the user's required standard is met. The user's required standard is, for example, a correct answer rate of 90% or more. Once a trained model that meets the user's required standard is created, the evaluation unit 32c passes the created trained model to the blur determination unit 33 as a determiner 32d.

[0032] The blur determination unit 33 determines the presence or absence of blur in the first frame image using the determiner 32d created by the blur learning unit 32. As shown in FIG. 5 , the blur determination unit 33 includes a determination unit 33a and a blurred image tagging unit 33b. The determination unit 33a may be equipped with inference software incorporating the determiner 32d. The determination unit 33a determines the presence or absence of blur for the first frame image received from the video segmentation unit 31 using the determiner 32d. In other words, the determination unit 33a executes the inference phase of machine learning.

[0033] The blurred image tagging unit 33b assigns information indicating the presence of blur to first frame images that have been determined to be blurred based on the determination result of the determination unit 33a, making it possible to easily classify whether or not a first frame image is blurred. The process of assigning information indicating the presence of blur to a first frame image is called tagging. After tagging all first frame images that are blurred, the blurred image tagging unit 33b passes the first frame images to the development drawing creation unit 34.

[0034] The development drawing creation unit 34 creates a development drawing using second frame images selected at regular intervals from the plurality of first frame images received from the blur determination unit 33. The second frame images are images for creating a development drawing. The second frame images are included in the first frame images. Furthermore, the development drawing creation unit 34 creates a development drawing by replacing second frame images determined to be blurred with first frame images that are not blurred. As shown in FIG. 6, the development drawing creation unit 34 includes an image selection unit 34a and an image development unit 34b.

[0035] The image selection unit 34a selects a plurality of second frame images to be used for creating a development view from a plurality of first frame images so that overlap of the imaged ranges within the pipeline 2 is small. For example, the image selection unit 34a can select second frame images at equal time intervals from the start of image capture. The image selection unit 34a can set the time intervals based on the output of a sensor that measures the towing speed of the pipe camera 3 or the travel distance of the pipe camera 3 so that overlap of the imaged ranges within the pipeline 2 is small and no range is not captured. The time interval for selecting second frame images may also be input by an operator operating the image generation device 10. In this case, the image selection unit 34a may be configured to automatically select second frame images from a plurality of first frame images with reference to the frame rate (fps) of the video from the pipe camera 3.

[0036] If the selected second frame image includes a blurred frame image tagged by the blurred image tagging unit 33b, the image selecting unit 34a replaces the unblurred first frame image taken at an earlier point in time than the blurred second frame image with the blurred second frame image. The image selecting unit 34a selects subsequent second frame images from the first frame images at equal time intervals, starting from the newly selected unblurred second frame image.

[0037] For example, Fig. 7 shows an example of selecting a second frame image at four-second intervals. In Figs. 7 to 9, the shaded images indicate images with blur. The unshaded images indicate images without blur. The image selection unit 34a selects the second frame image at time t ０ The first frame image at time t (seconds) is selected as the second frame image img01. ０ If the second frame image img02 at +4 (seconds) is a blurred frame image, the image selection unit 34a ０ (seconds) and t ０ +4 (seconds) ０ The image selection unit 34a searches for a first frame image without blur, starting from the image closest to +4 (seconds). ０The blurred second frame image img02 is replaced with the blurred first frame image img03 found at +4-x (seconds) (x<4). The image selection unit 34a selects subsequent second frame images at equal time intervals, starting from the replaced second frame image img03. For example, the next second frame image img03 is the one at time t ０ The second frame image img04, which is +8-x seconds, is selected. By doing this, it is possible to create a development using only clear, unblurred second frame images. It is also possible to prevent gaps from appearing in the development due to the time interval between the second frame images being widened.

[0038] Fig. 7 illustrates a case where the image selection unit 34a extracts second frame images at equal time intervals from the start of imaging. The image selection unit 34a may select second frame images at equal distance intervals from the start of imaging. Here, the distance may be, for example, the distance from the end of the pipe camera 3 on the manhole 1A side or 1B side of the pipeline 2. Fig. 8 illustrates an example where second frame images are selected at 4 (m) intervals. The image selection unit 34a extracts second frame images at equal distance intervals from the end of the pipe camera 3 on the manhole 1A side or 1B side of the pipeline 2. ０ The first frame image at (m) is selected as the second frame image img11. ０ If the second frame image img12 at +4 (m) is a blurred frame image, the image selection unit 34a ０ (m) and d ０ +4(m) and d ０ The image selection unit 34a searches for a first frame image without blur from the one closest to the distance d ０ The unblurred first frame image img13 found at +4-x(m) (x<4) is replaced with the blurred second frame image img12. The image selection unit 34a selects subsequent second frame images at equal distance intervals, starting from the replaced second frame image img13. For example, the next second frame image img13 is the one at a distance d ０The second frame image img04 at +8-x(m) is selected. By doing this, it is possible to create a development using only clear, unblurred second frame images. It is also possible to prevent gaps from appearing in the development due to the increased distance between the second frame images.

[0039] In the example of FIG. 7, the time t ０ (seconds) and t ０ For example, when the inner surface of the pipe 2 has an uneven state over a certain length of the pipe 2, as shown in FIG. ０ (seconds) to t ０ The image captured by the pipe camera 3 may remain blurred for a certain period of +4 (seconds). In this way, if all first frame images from the capture time of a blurred second frame image to the capture time of the immediately preceding second frame image are blurred images, the image selection unit 34a does not need to replace the blurred second frame image. Furthermore, the times of subsequent second frame images do not need to be changed. However, when creating a development drawing, the development drawing creation unit 34 indicates that the image is blurred.

[0040] The image development unit 34b creates a development view from each of the selected second frame images. Any method can be used to create the development view. As an example, assume that the image development unit 34b selects the second frame image of FIG. 10A from the video captured by the pipe camera 3. In FIG. 10A, the image is centered in the distant direction where the pipeline 2 extends. FIG. 10A is a drawing created for explanatory purposes and includes parts that differ from the actual image of the inside of the pipeline 2. For example, if the inside of the pipeline 2 is captured with the pipe camera 3 equipped with a lighting device, the central part of the image in front of the pipe camera 3 is far from the pipe camera 3 and is not illuminated, resulting in a dark and unclear image.

[0041] As shown in Fig. 10B, the image development unit 34b divides the second frame image into multiple angular ranges in a radial direction from the center, excluding the central portion where the image is dark and unclear. For example, the image development unit 34b divides the second frame image into eight regions at 45-degree intervals from the center. The number of regions into which the second frame image is divided is not limited to eight. The image development unit 34b may divide the second frame image into seven or fewer regions or nine or more regions.

[0042] 10C , the image development unit 34b generates a development in which the horizontal axis is the direction going around the pipeline 2 and the vertical axis is the extension direction of the pipeline 2. Furthermore, the image development unit 34b can generate a development of the entire pipeline 2 by combining developments created from multiple second frame images in the vertical axis direction. The development generated by the image development unit 34b is output from the output unit 40.

[0043] (Method for Generating an Image Inside a Pipeline) Hereinafter, a procedure for generating a development view of the inside of a pipeline 2, which is executed by the control unit 30 of the image generating device 10 in one embodiment, will be described with reference to FIG.

[0044] The control unit 30 acquires the video captured by the pipe camera 3 via the input unit 20 (step S101). The pipe camera 3 captures the video while retreating inside the pipeline 2. The video stored by the control unit 30 is stored in the storage unit 50.

[0045] The control unit 30 divides the acquired video into a plurality of first frame images (step S102).

[0046] The control unit 30 determines whether or not each of the first frame images is blurred using the blurred image determiner 32d generated by machine learning (step S103). The control unit 30 tags the blurred images.

[0047] The control unit 30 selects a plurality of second frame images to be used for generating a development view from the plurality of first frame images (step S104). The second frame images are selected at equal time intervals or equal distance intervals so as to minimize overlap between the images.

[0048] The control unit 30 determines whether any of the second frame images is blurred (step S105). The control unit 30 can efficiently determine whether or not the second frame image is blurred based on the information added to the frame image by tagging.

[0049] If the second frame image contains a blurred image (step S105: Yes), the control unit 30 replaces the blurred second frame image with the unblurred first frame image taken before the second frame image (step S106).

[0050] If there is no blurred image in the second frame image (step S105: No), and after replacing the blurred second frame image with a non-blurred frame image in step S106, the control unit 30 creates a development based on the second frame image (step S107). The control unit 30 causes the output unit 40 to output the created development.

[0051] As described above, according to the image generating device 10 of this embodiment, even if the selected second frame image contains a blurred image, it is possible to replace the selected second frame image with an unblurred first frame image, thereby creating a development view from only clear, unblurred images. This improves the quality of inspection records of underground pipelines and reduces the workload. Furthermore, the image generating device 10 uses a determiner 32d generated by machine learning to determine whether or not the frame image is blurred, allowing for stable judgment without imposing a burden on the worker inspecting the pipeline.

[0052] Although the above-described embodiments have been described as typical examples, it will be apparent to those skilled in the art that many modifications and substitutions can be made within the spirit and scope of the present invention. Therefore, the present invention should not be construed as being limited by the above-described embodiments, and various modifications and / or alterations are possible without departing from the scope of the claims. For example, multiple building blocks described in the embodiments and examples can be combined into one, or one building block can be divided.

[0053] In the above embodiment, the pipe camera 3 is towed from the manhole 1A by a cable 4 or the like. However, the pipe camera may be a self-propelled camera equipped with a battery, a motor, tires, etc., and capable of traveling within the pipeline under remote control. The pipe camera may be configured to transmit images to the image generation device via wireless communication. In this case, a wireless router for relaying the wireless communication may be installed inside the manhole. Furthermore, the pipe camera is not limited to having a single camera at its tip, but may have multiple cameras, including one facing the sidewall of the pipeline. The image generation device may generate a development view by combining frame images acquired from multiple pipe cameras.

[0054] The following additional notes are provided regarding the above-described embodiments.

[0055] (Supplementary Item 1) An image generation device comprising: an input unit that acquires video from a camera inserted into a pipeline; and a control unit that generates a development view of an image of the inside of the pipeline from the video, wherein the control unit divides the video into a plurality of first frame images, determines whether or not each of the first frame images is blurred, selects a plurality of second frame images to be used for creating the development view from the first frame images, and when the second frame images include a frame image with blur, replaces the blurred second frame image with an unblurred first frame image selected from the plurality of first frame images, and creates the development view based on the replaced second frame image. (Supplementary Item 2) The image generation device according to Supplementary Item 1, wherein the control unit comprises a blur determination unit having a classifier that has learned using frame images classified into blurred images and unblurred images as training data, and the first frame images are input to the blur determination unit to determine whether or not the first frame images are blurred. (Supplementary Item 3) The image generating device according to Supplementary Item 1 or 2, wherein the control unit selects the second frame images captured at equal time intervals or equal distance intervals to create the development view, and when one of the second frame images is a blurred frame image, selects one unblurred first frame image between the second frame image and another second frame image immediately preceding the second frame image as the first frame image to be replaced. (Supplementary Item 4) The image generating device according to Supplementary Item 3, wherein the control unit selects, starting from the replaced second frame image, the second frame images subsequent to the replaced second frame image at equal time intervals or equal distance intervals.

[0056] 1A, 1B Manhole 2 Pipe 3 Pipe camera (camera) 4 Cable 5 Driving unit 10 Image generating device 20 Input unit 30 Control unit 31 Video dividing unit 31a Frame number reading unit 31b Frame dividing unit 32 Blur learning unit 32a Image input unit 32b Machine learning unit 33c Evaluation unit 32d Determinator 33 Blur determination unit 33a Determination unit 33b Blur image tagging unit 34 Development drawing creation unit 34a Image selection unit 34b Image development unit 40 Output unit 50 Storage unit

Claims

1. An image generation device comprising: an input unit that acquires images from a camera inserted inside a pipeline; and a control unit that generates an unfolded view of the inside of the pipeline from the images, wherein the control unit divides the images into a plurality of first frame images, determines whether each of the first frame images is blurred, selects a plurality of second frame images to be used for creating the unfolded view from the first frame images, and, when the second frame images include a blurred frame image, replaces the blurred second frame image with an unblurred first frame image selected from the plurality of first frame images, and creates the unfolded view based on the replaced second frame image.

2. The image generating device according to claim 1, wherein the control unit includes a blur determination unit having a classifier that has learned using, as training data, frame images classified into blurred and unblurred images, and the first frame image is input to the blur determination unit to determine whether or not the first frame image is blurred.

3. The image generating device according to claim 1 or 2, wherein the control unit selects the second frame images captured at equal time intervals or equal distance intervals to create the unfolded view, and when one of the second frame images is a blurred frame image, selects one of the first frame images that is not blurred and is located between the second frame image and another second frame image immediately preceding the second frame image, as the first frame image to be replaced.

4. The image generating device according to claim 3, wherein the control unit selects the second frame images after the replaced second frame image at equal time intervals or equal distance intervals, starting from the replaced second frame image.

Images