Development diagram creation device, development diagram creation method, and program

Abstract

To provide a development diagram creation device or the like that can create development diagrams with little distortion in real time.SOLUTION: A development diagram creation device 3 includes: an image acquisition unit 311 configured to acquire, for each frame, images captured by an imaging vehicle 2 while it moves inside a pipe 10 in a pipe line direction; a development image generation unit 312 configured to generate, for each frame, a development image in which the inside of a tubular structure is developed in a planar form from a part of the captured images; a calculation unit 313 configured to calculate, for each frame, a shift amount between development images of adjacent frames; and a development image combination unit 314 configured to combine the development images of respective frames based on the calculated shift amount to create a development diagram.SELECTED DRAWING: Figure 2

Description

[Technical field]

[0001] The present invention relates to a development view creating device, a development view creating method, and a program, and more particularly to image generation for assisting in the investigation of tubular structures. [Background technology]

[0002] Inspections are being conducted for the presence or absence of damage and the condition of tubular structures such as water supply and sewerage pipes, cable pipes, tunnels, etc. In these inspections, the inner wall surface of the pipe is photographed, and a computer is used to create an exploded view of the image, which is cut into a flat surface with the pipe running longitudinally. Workers visually inspect the exploded view to determine the presence of damage and to prepare an inspection report.

[0003] Patent Document 1 describes a pipe inner wall image development system that captures all-sky image data while moving inside a tubular structure and creates a development diagram based on this. In the pipe inner wall image development system of Patent Document 1, a cable connected to an image capture means running inside the pipe is held so as not to slacken, the position of the image capture means is detected from the amount of cable feed to obtain the travel distance (position information), and the travel distance when the image data is input from the image capture means is inserted into the image data as a digital signal to generate a development diagram with little distortion. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] Publication No. 2008-090782 [Patent Document 2] Publication No. 2021-197677 Summary of the Invention [Problem to be solved by the invention]

[0005] However, in the system of Patent Document 1, hardware such as a cable is required to measure the travel distance of the image capturing means traveling inside the pipe. Also, the travel distance cannot be obtained until the cable is stretched. Therefore, there is a method of estimating the travel distance by assuming that the image capturing means moves at a constant speed until the cable is stretched. However, since there are obstacles such as garbage and puddles on the road surface inside the pipe, even if the image capturing means is made to travel at a constant speed, the speed is not necessarily constant, which causes image distortion.

[0006] On the other hand, there is a method for creating a development diagram with less distortion based on the movement distance calculated from an image captured by a camera. Patent Document 2 describes an image processing device that calculates the rotation amount and movement distance (movement amount) of the optical axis of a camera from an image captured by the camera of an imaging device moving inside a tubular structure, and generates a development image based on the rotation amount and movement distance, thereby making it possible to create a development diagram with less distortion even when a moving body equipped with a camera takes an image while shaking inside the tubular structure. However, the process of calculating the movement distance from an image captured by such a camera has a problem that it is not suitable for real-time processing because of its high calculation cost.

[0007] The present invention has been made in consideration of the above problems, and has an object to provide a development drawing device and the like capable of creating a development drawing with less distortion in real time. [Means for solving the problem]

[0008] The first invention for solving the above-mentioned problems is an unfolded view creation device comprising: an acquisition unit that acquires images captured frame by frame while moving inside a tubular structure in a pipeline direction; a generation unit that generates an unfolded image of the inside of the tubular structure unfolded in a planar form from a portion of the captured images frame by frame; a calculation unit that calculates a shift amount between the unfolded images of adjacent frames frame by frame; and a combination unit that creates an unfolded view by combining the unfolded images of each frame based on the calculated shift amount.

[0009] According to the first aspect of the present invention, the shift amount of the developed images between adjacent frames is calculated, and the developed images of each frame are combined based on the shift amount to create a developed view. This corrects the shift in the display position of the developed images due to changes in the moving speed during shooting, etc., and allows the creation of a developed view with less distortion. In addition, since there is no need to calculate the moving distance from the captured images when creating the developed view, calculation costs can be reduced and it is also suitable for real-time processing.

[0010] The calculation unit may calculate a horizontal shift amount between the exfoliated images of adjacent frames for each frame, and the combining unit may combine the exfoliated images of each frame based on the horizontal shift amount. For example, the combining unit combines the exfoliated images of each frame while cutting them out based on the horizontal shift amount. Also, the combining unit combines the exfoliated images of each frame while overlapping them based on the horizontal shift amount. This allows the exfoliated images of each frame to be smoothly connected in the horizontal direction, resulting in a development with little distortion.

[0011] The calculation unit may calculate, for each frame, a vertical shift amount between the exfoliated images of adjacent frames, and the combining unit may combine the exfoliated images of each frame by rotating them vertically based on the vertical shift amount. This allows the exfoliated images of each frame to be smoothly connected vertically, resulting in a development with little distortion.

[0012] When it is determined that the pipeline is being retracted, the image of the retracted amount may be removed from the developed view created up to that point. In this way, when the pipeline is retracted, the image of the retracted amount is deleted from the developed view, so that a developed view that follows the retracting movement can be obtained.

[0013] A second invention is a method for creating an unfolded diagram, comprising the steps of: acquiring, by a computer, images captured by the computer while moving inside a tubular structure in a pipeline direction for each frame; generating, for each frame, an unfolded image in which the inside of the tubular structure is unfolded in a planar form from a portion of the captured images; calculating, for each frame, an amount of shift between the unfolded images of adjacent frames; and creating an unfolded diagram by combining the unfolded images of each frame based on the calculated amount of shift.

[0014] The third invention is a program that causes a computer to function as an acquisition unit that acquires images captured frame by frame while moving inside a tubular structure in a pipeline direction, a generation unit that generates an expanded image of the inside of the tubular structure expanded in a planar form from a portion of the captured images frame by frame, a calculation unit that calculates the amount of shift between the expanded images of adjacent frames frame by frame, and a combination unit that creates an expanded diagram by combining the expanded images of each frame based on the calculated shift amount. Effect of the Invention

[0015] According to the present invention, it is possible to provide a development drawing device and the like capable of creating a development drawing with less distortion in real time. [Brief description of the drawings]

[0016] [Figure 1] FIG. 1 is a diagram illustrating an example of the overall configuration of an exfoliated image generating system. [Diagram 2] 2 is a block diagram showing an example of a functional configuration of a photographing vehicle and a development view creating device; [Diagram 3] 13 is a flowchart showing the overall processing flow. [Figure 4] 13 is a flowchart showing the flow of a development view creation process. [Diagram 5] FIG. 2 is a diagram showing the relationship between a captured image and a developed image. [Figure 6] 13A and 13B are diagrams illustrating a process of combining expanded images during progression. [Figure 7]13A and 13B are diagrams illustrating a circulation process of an expanded image during rotation. [Figure 8] FIG. 13 is a diagram showing a process of cutting out the development view when reversing; [Figure 9] FIG. 13 is a diagram showing an example of a development circle setting screen. [Figure 10] FIG. 13 is a diagram showing a state in which the development circle is aligned with a joint portion on the development circle setting screen. [Figure 11] FIG. 13 is a diagram showing an example of a deployment screen. [Figure 12] FIG. 13 is a diagram showing an example of a confirmation screen. [Figure 13] 11 is a diagram comparing a development created by the method of this embodiment and a development created by a conventional method. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0017] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the drawings.

[0018] Fig. 1 is a diagram showing an example of the overall configuration of a development view creation system 1 according to the present invention. In the following description, as an example, as shown in Fig. 1, a development view creation system 1 in which a photographing vehicle 2 and a development view creation device 3 are connected via a wired or wireless communication interface will be described. Note that the system configuration example in Fig. 1 is an example, and the present invention is not limited thereto. For example, a configuration in which the photographing vehicle 2 and the development view creation device 3 are connected to a server (not shown) via a network, and photographed data can be shared via the server, may be used.

[0019] The photographing vehicle 2 (photographing device) photographs video with a camera 21 while moving along the pipe direction inside a tubular structure 10 to be inspected (such as water supply and sewerage pipes, intake and exhaust pipes, cable pipes, and tunnels; hereafter referred to as pipes 10). In this embodiment, the photographing vehicle 2 moving leftward of the pipe 10 in Fig. 1 is defined as "advancing," and moving rightward is defined as "retreating."

[0020] As shown in FIG. 2, the photographing vehicle 2 has a camera 21 and a vehicle driving unit 22. The camera 21 has a lens, and a CCD (Charge Coupled Devices), a CMOS (Complementary Metal Oxide Semiconductor) The camera is equipped with an imaging element such as a semiconductor laser (Semiconductor Laser) and is capable of photographing the inner wall surface around the camera 21 by photographing the longitudinal direction of the tube 10 as the depth direction. Note that it is preferable that the photographing vehicle 2 is provided with lighting (not shown) for illuminating the inside of the tube 10.

[0021] The traveling vehicle driving unit 22 includes wheels, a motor, a power source, and the like for driving the imaging vehicle 2. In addition, a floating body for dealing with a puddle inside the pipe 10 may be provided.

[0022] The development drawing creation device 3 is a computer terminal that creates a development drawing 5 based on the captured image P(t) (t: frame number) captured as a video using the camera 21 of the photographing vehicle 2. The development drawing 5 is an image obtained by cutting an image of the inside of the pipe 10 in the pipe direction and developing it into a flat surface. The development drawing 5 is created by combining development images E(t) generated from the captured image P(t) of each frame t. The method of creating the development drawing 5 will be described later.

[0023] 2, the development drawing creation device 3 is a PC (Personal Computer) in which a control unit 31, a storage unit 32, a communication unit 33, an input unit 34, a display unit 35, and a peripheral device I / F (interface) unit 36, etc. are connected via a bus. Note that the development drawing creation device 3 may be a tablet, a smartphone, etc., in addition to a PC. A development drawing creation program is installed in the development drawing creation device 3, and the control unit 31 executes processing in accordance with the development drawing creation program to realize each function described below.

[0024] The control unit 31 includes a CPU (Central Processing Unit). The CPU loads programs stored in the memory unit 32, ROM, etc. into a work memory area on the RAM and executes them, and drives and controls each unit (memory unit 32, communication unit 33, input unit 34, display unit 35, peripheral device I / F unit 36) connected via a bus. The ROM permanently stores programs such as a boot program and BIOS, data, etc. The RAM temporarily stores loaded programs and data, and is provided with a work area that the control unit 31 uses to perform various processes.

[0025] The storage unit 32 is a storage device such as a flash memory or a hard disk, and stores the acquired image data, the input information on the pipe 10, etc. The storage unit 32 also stores processing programs (development drawing creation programs) related to each function described later.

[0026] The communication unit 33 has a communication port for a wireless communication unit such as a WiFi antenna or Bluetooth (registered trademark) or a wired communication unit such as a LAN, and a communication control device, and is an interface that mediates communication with external devices.

[0027] The input unit 34 includes, for example, a touch panel, a keyboard, a pointing device such as a mouse, an input device such as a numeric keypad, and the like, and inputs input data to the control unit 31.

[0028] The display unit 35 is composed of a display such as a liquid crystal panel and a logic circuit (such as a video adapter) for executing display processing in cooperation with the display, and causes the display to display the display data input under the control of the control unit 31. The display unit 35 may be a touch panel display in which an input device (input unit 34) such as a touch panel is integrally provided on the display screen.

[0029] The peripheral device I / F unit 36 ​​is a port for connecting a peripheral device, and the control unit 31 transmits and receives data to and from the peripheral device via the peripheral device I / F unit 36. The peripheral device I / F unit 36 ​​is configured with a USB (Universal Serial Bus) or the like. The connection with the peripheral device may be wired or wireless.

[0030] Next, the functional configuration of the developed view creation device 3 will be described with reference to Fig. 2. The developed view creation device 3 includes, as functional units, an image acquisition unit 311, a developed image generation unit 312, a shift amount calculation unit 313, a developed image combination unit 314, etc. These functional units are realized by the CPU of the control unit 31 reading a processing program (developed image generation program) stored in the storage unit 32, calling it up into a work memory area on the RAM, and executing it.

[0031] The image acquisition unit 311 acquires the captured image P(t) captured by the camera 21 of the photographing vehicle 2 for each frame t via the communication unit 33 or the peripheral device I / F unit 36. Alternatively, the image acquisition unit 311 may read and acquire the captured image P(t) stored in the memory unit 32 or another storage medium, or the communication unit 33 may acquire the captured image P(t) stored in another computer or a storage server (not shown) via a network such as a LAN or the Internet, and import the image into the control unit 31.

[0032] The exfoliated image generating unit 312 generates an exfoliated image E(t) for each frame t by exposing an image of the inside of the pipe 10 onto a plane from a part of the captured image P(t) (in this embodiment, the exfoliated area 44 shown in FIG. 5).

[0033] The shift amount calculation unit 313 calculates the shift amount between the exfoliated images of adjacent frames for each frame t. In this embodiment, the shift amount calculation unit 313 calculates the horizontal shift amount Sx(t) and the vertical shift amount Sy(t) between the exfoliated images of adjacent frames (in this embodiment, the exfoliated image E(t) and the exfoliated image E(t+1)) for each frame t.

[0034] The exfoliated image combination unit 314 creates an exfoliated drawing 5 by combining the exfoliated images E(t) of each frame t based on the shift amounts (horizontal shift amount Sx(t) and vertical shift amount Sy(t)) calculated by the shift amount calculation unit 313.

[0035] Next, a development drawing creating method using the development drawing creating system 1 will be described with reference to Fig. 3 to Fig. 12. The process of each step in the flowchart of Fig. 3 to Fig. 4 is executed by the control unit 31 of the development drawing creating device 3.

[0036] An operator carries the photographing vehicle 2 and the development drawing creation device 3 to the investigation site, and prepares to take a video of the inside of the pipe 10. The operator starts the development drawing creation program of the development drawing creation device 3 to start processing. In this embodiment, as an example, a development drawing is created in real time while taking a video.

[0037] 3 shows the overall flow of the developed image creation method. When the developed image creation program is started (step S1), the control unit 31 of the developed image creation device 3 displays a pipe information input screen (not shown) on the display unit 35 and accepts input of pipe information, etc. (step S2). For example, various information such as the pipe type, pipe diameter, upstream manhole, downstream manhole, etc. of the pipe 10 is input.

[0038] Next, the control unit 31 displays the development circle setting screen 62 shown in FIG. 9 and accepts the setting of the development circle (step S3). The development circle setting screen 62 is provided with a camera image display area 62a and a camera information input field 62b. The camera image display area 62a is an area for displaying images input from the cameras. The camera information input field 62b is an input field for designating which camera images are to be captured. In the camera information input field 62b, cameras that are connected for communication via the communication unit 33 or the peripheral device I / F unit 36 ​​of the development drawing creation device 3 and are in a communicable state are displayed in a list and can be selected. When the camera images to be captured are input, the development drawing creation device 3 captures the images of the designated camera and displays them in the camera image display area 62a. In the camera image display area 62a, the development circle 4 for aligning the camera images (photographed images) is displayed superimposed on the camera images (photographed images).

[0039] The developed circle 4 is for specifying the position of the inner wall surface of the pipe 10 on the captured image, and is an elliptical or circular object having four pointers 41a, 41b, 41c, and 41d for specifying four points of the pipe 10, i.e., the pipe top, the pipe bottom, the left end, and the right end, and a center mark 43 for specifying the center position. Each pointer 41 and center mark 43 can be moved by operating a mouse or the like. The control unit 31 draws an ellipse or circle (developed circle) whose vertical diameter is the length between the pointers 41a and 41b and whose horizontal diameter is the length between the pointers 41c and 41d. That is, the shape of the developed circle 4 can be changed by moving each of the pointers 41a to 41d. When the center mark 43 of the developed circle 4 is moved by operating a mouse or the like, the position of the developed circle 4 is moved.

[0040] The worker aligns the position of the developed circle 4 while viewing the image (photographed image) displayed in the camera image display area 62a. In aligning the position, for example, as shown in Fig. 10, the worker aligns the positions of the pointers 41a, 41b, 41c, 41d and the center mark 43 so that the joints of the pipe 10 shown in the photographed image displayed in the camera image display area 62a overlap with the developed circle 4.

[0041] Thereafter, when the "To development" button 62c is operated, the development drawing creation device 3 displays the development screen 63 shown in FIG. 11 and starts the development drawing creation process (step S4). The development screen 63 is provided with a camera image display area 63a, a development drawing display area 63b, a status display area 63c, a "Start development / Stop" button 63d, an "End" button 63e, a confirmation button 63f, and the like. The camera image display area 63a displays the image (captured image) input from the camera 21. The development drawing display area 63b displays the created development drawing 5. The "Start development / Stop" button 63d displays "Start development" in the initial state, and when the button is operated in this state, the development process starts. During the development process, the display of the "Start development / Stop" button 63d changes to "Stop development", and when the button is operated in this state, the development process is paused. The status display area 63c displays the status of the camera image, such as "Playing", "Paused", and the like. The "End" button 63e is a button operated when ending the development process. The confirmation button 63f is a button that is operated when moving to a confirmation screen 64, which is the next step.

[0042] The development drawing creation process in step S4 will be described with reference to the flowchart in Fig. 4. When the "development start / stop" button 63d is operated on the development screen 63 and video shooting by the photography vehicle 2 is started (step S11; Yes), the control unit 31 acquires the photographed image P(t) photographed by the camera 21 in real time for each frame t, and stores it in the RAM or the storage unit 32.

[0043] Then, the control unit 31 creates the unfolded view 5 by repeatedly executing the frame processing of the following steps S12 to S19. First, the control unit 31 initializes the frame number t to 1 (step S12). Next, the control unit 31 reads the captured image P(t) of the t-th frame from the RAM or the storage unit 32, and generates the unfolded view E(t) of the t-th frame (step S13). The generated unfolded view E(t) is temporarily stored in the RAM or the storage unit 32. Note that if t≧2, the unfolded view E(t) has been generated during the processing of the previous frame, and therefore the control unit 31 obtains the generated unfolded view E(t) from the RAM or the storage unit 32.

[0044] Fig. 5 is a diagram showing the relationship between a captured image P(t) and a generated exfoliated image E(t). As shown in Fig. 5, in the captured image P(t), a region between the exfoliated circle 4 and a circle 4a whose diameter is reduced by a predetermined amount from the exfoliated circle 4 toward the center position indicated by the center mark 43 is set as an exfoliated region 44, and the exfoliated region 44 is expanded into a plane to generate an exfoliated image E(t). The horizontal direction (X-axis direction) of the exfoliated image E(t) corresponds to the depth direction Z of the pipe 10, and the vertical direction (Y-axis direction) of the exfoliated image E(t) corresponds to the circumferential direction θ of the pipe 10.

[0045] The control unit 31 also reads the captured image P(t+1) of the t+1th frame from the RAM or storage unit 32, and generates an exfoliated image E(t+1) of the t+1th frame (step S14). The generated exfoliated image E(t+1) is temporarily stored in the RAM or storage unit 32. By the processing of steps S13 and S14, exfoliated images of adjacent frames (exfoliated image E(t) and exfoliated image E(t+1)) are generated.

[0046] Next, the control unit 31 calculates a shift amount based on the exfoliated picture E(t) and the exfoliated picture E(t+1) (step S15). The shift amount refers to the difference between the position where the same object is displayed in the exfoliated picture E(t) and the position where it is displayed in the exfoliated picture E(t+1) (the positional deviation between the images). In this embodiment, the control unit 31 calculates a horizontal shift amount Sx(t), which is the difference (shift) in the horizontal position between the images of the exfoliated image E(t) and the exfoliated image E(t+1), and a shift amount Sy(t), which is the difference (shift) in the vertical position. As a method for calculating the shift amount Sx(t) and the shift amount Sy(t), a known method capable of calculating the position difference (deviation) between images (e.g., template matching or a feature-based method for matching feature points between images) can be used. Furthermore, the control unit 31 judges whether the photographing vehicle 2 (camera 21) is moving forward or backward based on the shift direction of the shift amount Sx(t) (whether the unfolded image E(t+1) is shifted to the right or left relative to the unfolded image E(t)). When the shift amount Sx(t) and the shift amount Sy(t) are both 0 (when there is no difference (shift) between the unfolded image E(t) and the unfolded image E(t+1)), the control unit 31 judges that the photographing vehicle 2 (camera 21) is stopped.

[0047] Next, the control unit 31 performs a process of combining the unfolded images based on the shift amount (shift amount Sx(t) and shift amount Sy(t)) calculated in step S15 (step S16). Specifically, when the photographing vehicle 2 (camera 21) is moving, the control unit 31 cuts out the unfolded image E(t) by the shift amount Sx(t) and sets the cut-out image as the unfolded image E'(t). When the photographing vehicle 2 (camera 21) is rotating (when the shift amount Sy(t) is not 0), the control unit 31 rotates the unfolded image E'(t) based on the shift amount Sy(t). Then, the control unit 31 combines the unfolded image E'(t) with the unfolded image 5 created up to the processing of the previous frame (t-1) and updates the unfolded image 5. When t=1 (when the first frame is processed), the control unit 31 sets the unfolded image E'(t) as the unfolded image 5.

[0048] 6 is a diagram showing the process of combining exfoliated images during progression. As shown in the figure, the exfoliated image E'(t) obtained by cutting out the exfoliated image E(t) by the shift amount Sx(t) is combined with the exfoliated image 5 to update the exfoliated image 5. In this way, the image portion of the exfoliated image E(t) that does not overlap with the exfoliated image E(t+1) of the next frame is combined with the exfoliated image 5. By repeating this combining process for each frame, the exfoliated images of each frame are smoothly connected in the horizontal direction, and a development 5 with little distortion is generated.

[0049] FIG. 7 is a diagram showing a rotation process of an exfoliated image during rotation. FIG. 7 (left diagram) shows an example of an exfoliated image E'(t) that is vertically shifted downward by Sy(t). In this case, as shown in FIG. 7 (center diagram), an image of Sy(t) is cut from the top end of the exfoliated image E'(t) and pasted at the bottom end of the exfoliated image E'(t), thereby rotating the exfoliated image E'(t). Then, as shown in FIG. 7 (right diagram), the rotated exfoliated image E'(t) is combined with the exfoliated image 5. This makes it possible to correct the shift and combine the exfoliated images even if they are shifted vertically due to the rotation of the camera 21.

[0050] On the other hand, when the photographing vehicle 2 (camera 21) is moving backward, the control unit 31 cuts off the end of the development 5 by the shift amount Sx(t).

[0051] FIG. 8 is a diagram showing the process of cutting out the development when reversing. FIG. 8 (left diagram) shows the development 5 created up to the processing of the previous frame. As shown in the figure, an image is cut out from the end of the development 5 by the shift amount Sx(t), and the development 5 is updated. In this way, when the photographing vehicle 2 reverses, the image of the retreating portion is deleted, so that a development that follows the operation of the photographing device can be obtained. Note that the cut-out area is drawn again when the photographing vehicle 2 advances.

[0052] When both the shift amount Sx(t) and the shift amount Sy(t) are 0, that is, when the photographing vehicle 2 (camera 21) is stopped, the control unit 31 does not update the development view 5 and ends the processing of step S16.

[0053] When the developed view 5 is updated, the control unit 31 displays the updated developed view 5 in the developed view display area 63b of Fig. 11, and ends the combining process of step S16 in Fig. 4. When moving to the next frame process (in the case of development) (step S17; No), the control unit 31 updates the frame number to t←t+1, returns to the process of step S13, and performs the processes (frame processes) of steps S13 to S16 for the next frame. As described above, by repeatedly performing the processes (frame processes) of steps S13 to S16, the developed view 5 is updated and displayed in real time in the developed view display area 63b.

[0054] When all frame processing is completed, or when the "Start / Stop Development" button 63d or the "End" button 63e is operated, the control unit 31 determines that development is completed (step S17; Yes), and ends the development drawing creation process. The series of images input from the camera 21 are temporarily stored in the RAM or the storage unit 32 as captured images (video), and the created development drawing 5 is also temporarily stored in the RAM or the storage unit 32.

[0055] When the development drawing process is completed, the control unit 31 proceeds to step S5 of the flowchart in FIG. 3. When the confirmation button 63f of the development screen 63 is operated, the control unit 31 displays the confirmation screen 64 shown in FIG. 12. In the confirmation screen 64, the image captured by the camera 21 is displayed in the display area 64a, and the created development 5 (part or all) is displayed in the display area 64b. The development 5 is displayed so as to be scrollable left and right (in the pipe direction), and the display range can be changed by operating the scroll bar 64d with a mouse or the like. In addition, the captured image at the position corresponding to the development 5 displayed in the display area 64b is displayed in the display area 64a, and the two images can be compared and checked. In addition, when the play / stop button 64c below the display area 64a is operated, the control unit 31 plays back the captured image as a video and displays the development 5 in the display area 64b so as to include the development image at the position corresponding to the displayed captured image.

[0056] When the data save button 64e is operated (step S6; Yes), the control unit 31 stores the photographed image and the development drawing 5 in the storage unit 32 of the development drawing creation device 3 (step S7). When the development drawing generation is to be redone (step S6; No), the process returns to step S3.

[0057] FIG. 13 shows an example of an actually created development. FIG. 13(a) shows a part of the development created by the method of this embodiment, and FIG. 13(b) shows a part of the development created by the conventional method. The development in FIG. 13 was created from a video taken while a filming vehicle was traveling inside a tubular structure in which a scale with numbers displayed at regular intervals was arranged in the pipeline direction. Here, the conventional method is a method of creating a development assuming that the filming vehicle is moving in the pipeline direction at a constant speed. If it can be assumed that the vehicle is moving at a constant speed, the horizontal shift amount between frames is constant. In the conventional method, the development is created by cutting out the development image E(t) of each frame (t) with the same shift amount and combining the cut-out development images E(t). As is clear from FIG. 13(b), it can be seen that the display intervals of the numbers are not constant in the development created by the conventional method. In other words, the image is significantly distorted due to the influence of the speed change of the filming vehicle, etc. On the other hand, as shown in FIG. 13(a), in the development created by the method of this embodiment, the display intervals of the numbers are approximately constant, and it is clear that the image distortion is small.

[0058] The present embodiment has been described above. According to this embodiment, the photographed image P(t) photographed by the photographing vehicle 2 while moving inside the pipe 10 in the pipe line direction is acquired for each frame t, and an expanded image E(t) in which the inside of the pipe 10 is expanded in a planar shape is generated from a part of the photographed image P(t) for each frame t. Then, for each frame (t), the horizontal shift amount Sx(t) and the vertical shift amount Sy(t) of the expanded images of adjacent frames (expanded image E(t) and expanded image E(t+1)) are calculated, and the expanded image E(t) of each frame t is combined based on the calculated shift amount Sx(t) and shift amount Sy(t) to create the expanded view 5. This corrects the shift in the display position of the expanded image due to a change in the moving speed during shooting, etc., and it is possible to create the expanded view 5 with little distortion. In addition, in the method of this embodiment, it is not necessary to estimate the moving distance of the photographing vehicle 2 (camera 21) from the photographed image P(t), and the calculation load is small. Therefore, it is possible to create the expanded view 5 in real time while shooting the pipe 10.

[0059] In this embodiment, an example of a process for creating an unfolded view 5 in real time while driving the photographing vehicle 2 and taking video is described. However, it is also possible to first only take video and store the video in the memory unit 32, and then read the taken video from the memory unit 32 after the video is taken to create the unfolded view 5.

[0060] In the present embodiment, as shown in FIG. 1, an example has been described in which the camera 21 is installed in front of the photographing vehicle 2 (in the direction of travel). However, the camera 21 can also be installed behind the photographing vehicle 2.

[0061] In the present embodiment, the developed image combining process of FIG. 7 has been described as an example in which the developed image 5 is created by cutting out the developed image E(t) by the shift amount Sx(t) in the horizontal direction and combining the cut out developed image E'(t). However, the developed images can be combined by other methods. For example, the developed image 5 can be created by combining the developed image E(t) and the developed image E(t+1) while overlapping them based on the shift amount Sx(t). In this case, the control unit 31 combines the developed image E(t+1) with the developed image E(t) by overlapping them at the position of Sx(t) of the developed image E(t). For the pixels in the overlapping area of ​​the developed image E(t) and the developed image E(t+1), for example, the average value or weighted average value of each pixel can be used. Similarly, for exfoliated image E(t+1), exfoliated image E(t+2) is superimposed and combined at the position Sx(t+1) of exfoliated image E(t+1), and for exfoliated image E(t+2), exfoliated image E(t+3) is superimposed and combined at the position Sx(t+2) of exfoliated image E(t+2). By superimposing and combining exfoliated images of adjacent frames in this manner, it is also possible to create a development 5. In addition, when the exfoliated images to be superimposed are shifted vertically, for example, when exfoliated image E(t) and exfoliated image E(t+1) are misaligned in the vertical direction by the shift amount Sy(t), either exfoliated image E(t) or exfoliated image E(t+1) is rotated by the shift amount Sy(t) to eliminate the misalignment in the vertical direction between the exfoliated images, and then the exfoliated images are superimposed.

[0062] Furthermore, in this embodiment, frame processing (steps S13 to S18 in FIG. 4) is executed for each frame t of the captured image P(t), but the frame rate may be downsampled or upsampled as necessary.

[0063] Furthermore, the photographing vehicle 2 may be any moving body capable of photographing the inside of the tubular structure to be photographed. For example, in the case of a water supply or sewerage pipe, a self-propelled or non-self-propelled robot camera equipped with a drive unit may be used. In the case of a road tunnel, a car equipped with a camera may be used, and in the case of a railway tunnel, a train equipped with a camera may be used. Also, instead of a moving vehicle, an air vehicle such as a drone may be used. Furthermore, instead of using a moving vehicle or an air vehicle, a person may hold a camera and photograph the inside of the tubular structure while moving about.

[0064] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0065] 1. Development diagram creation system 2....................Photographic vehicle 3. Development drawing device 4………………Expanded circle 5………………Expanded view 10………Tubular structure (pipe) 21……………Camera 22.....Travel vehicle drive unit 31...Control unit 311: Image acquisition unit 312: Exploded image generating unit 313: Shift amount calculation unit 314....Expanded image joining section t…………Frame number E(t)…Expanded image E'(t) ... Extracted image P(t)…Captured image Sx(t) …… Horizontal shift amount Sy(t) …… Vertical shift amount

Claims

1. an acquisition unit that acquires images frame by frame while moving inside the tubular structure in a pipeline direction; a generation unit that generates an expanded image in which the inside of the tubular structure is expanded in a planar form from a part of the photographed image for each frame; a calculation unit that calculates, for each frame, a shift amount between exfoliated images of adjacent frames; a combining unit that combines the developed images of the frames based on the calculated shift amount to create a development diagram; A development drawing creation device comprising:

2. the calculation unit calculates, for each frame, a horizontal shift amount between the exfoliated images of adjacent frames; 2. The development drawing device according to claim 1, wherein the combining unit combines the development images of the respective frames based on a horizontal shift amount.

3. 3. The development drawing device according to claim 2, wherein the combining unit combines the development images of the frames while cutting them out based on the horizontal shift amount.

4. 3. The development drawing device according to claim 2, wherein the combining unit combines the development images of the frames while overlapping them on the basis of the amount of shift in the horizontal direction.

5. the calculation unit calculates, for each frame, a vertical shift amount between exfoliated images of adjacent frames; 2. The development drawing device according to claim 1, wherein the combining unit combines the developed images of the frames by rotating them vertically based on the vertical shift amount.

6. 2. The development drawing device according to claim 1, wherein, when it is determined that the pipeline is being retracted, images of the amount of retraction are removed from the development drawings created up to that point.

7. The computer acquiring images frame by frame while moving inside the tubular structure in a pipeline direction; generating a developed image of the inside of the tubular structure developed in a planar manner from a part of the photographed image for each frame; calculating, for each frame, a shift amount between exfoliated images of adjacent frames; creating a development by combining the development images of the frames based on the calculated shift amount; A development drawing creating method comprising:

8. Computer, an acquisition unit that acquires images frame by frame while moving inside the tubular structure in a pipeline direction; a generation unit that generates, for each frame, an expanded image in which the inside of the tubular structure is expanded in a planar form from a part of the photographed image; a calculation unit that calculates, for each frame, a shift amount between exfoliated images of adjacent frames; A combining section that combines the developed images of each frame based on the calculated shift amount to create a development diagram; A program characterized by causing the program to function as a

Images