DUCT INSPECTION SYSTEM

Description

[0001] The invention relates to a canal inspection system with at least one airborne inspection device.

[0002] Sewer inspection systems are used to inspect sewer structures such as sewer pipes or manholes, to detect damage, and, if necessary, to carry out work within the sewer structure. Mobile systems are known for this purpose, which travel through the sewer pipe as a powered carriage. Systems are also known in which a probe, i.e., an inspection device, is advanced in the sewer pipe using a push rod. Meanwhile, developments with aerial inspection devices, which fly through the sewer pipe like a drone, are also underway. CN 113 212 732 B discloses an aerial inspection device in the form of a drone, which has several cameras: one camera at the front and another on a rotating wheel, which is radially oriented to the direction of flight and whose viewing direction rotates with the wheel around a longitudinal axis corresponding to the direction of flight.A 360° scan of the surroundings is performed while the drone moves in its direction of flight. Simultaneously moving forward, this captures a spiral image. The disadvantages are that a large number of images must be taken, and the rotating camera arrangement also results in a more complex drone design.

[0003] The object of the invention is to further develop a canal inspection system with at least one airborne inspection device in such a way that a complete image model of a canal structure can be created in a simple manner.

[0004] This problem is solved by a channel inspection system having the features specified in claim 1. Preferred embodiments are described in the dependent claims, the following description, and the accompanying figures.

[0005] The sewer inspection system according to the invention comprises at least one flyable inspection device. This device is designed to fly through a sewer structure, such as a sewer pipe and / or a manhole, for inspection purposes. The at least one inspection device has at least two cameras and a control unit. The two cameras serve to capture images within the sewer structure, e.g., in the sewer pipe or the manhole, as the device flies through it. The control unit is specifically designed to control the cameras so that images are captured at desired times. Preferably, the cameras are controlled by the control unit to capture individual images at desired times. Alternatively or additionally, it would also be conceivable to control the cameras to record video footage.Furthermore, the control unit can be designed to also control the flight of the inspection device, enabling it to fly preferably autonomously through a channel structure such as a sewer pipe or a shaft. In doing so, the control unit can interact with sensors to guide the inspection device along a desired flight path.

[0006] The sewer inspection system also includes an image processing unit, which may be fully or partially integrated into the inspection device or may be fully or partially external to the inspection device. The image processing unit can, for example, be a software application running on a computer system. The image processing unit is designed to further process the individual images or videos captured by the camera.

[0007] According to the invention, the cameras and the control unit of the inspection device are designed to generate 360° images. This is achieved by the cameras capturing images in a first viewing direction and images in at least a second viewing direction. These are opposing viewing directions, so that together they achieve a 360° field of view. The image processing unit is designed to combine the images captured in the at least two viewing directions into a 3D image model, thereby generating a three-dimensional model of the environment captured by the cameras. This can be accomplished by first performing independent 3D reconstructions for both viewing directions and then transforming the resulting 3D models into a unified coordinate system, and thus a unified 3D model, using corresponding image points.This 3D image model then makes it possible to virtually view the entire canal structure from all desired perspectives. According to the invention, images are not only captured from one perspective, but from at least two, positioned to create a 360° view, which is then assembled from the individual images in the 3D reconstruction. This allows for a very fast and comprehensive inspection of the entire canal structure, as the airborne inspection device can fly very quickly through the structure, such as a sewer pipe or manhole, and capture the necessary number of images to create a 3D image model. Subsequently, any point within the canal structure can be examined or investigated in more detail within the 3D image model.

[0008] According to the invention, in 3D reconstruction, a known 3D reconstruction method is used to create an actual three-dimensional reconstruction of the space or spatial contour captured in the images from the at least one camera. In a canal structure, the actual three-dimensional shape of the canal wall can thus be reconstructed in the model without having to base it on a known pipe or canal geometry. That is, according to the invention, images are not merely projected onto an idealized pipe geometry in the form of a cylinder.

[0009] The described image processing in the image processing unit can take place immediately after the images are captured, virtually in real time. Alternatively, all captured images can first be stored, and the necessary image processing can then be carried out after all required images have been captured, for example, after the memory in the inspection device, where the captured images are stored, has been read.

[0010] The at least one flight-capable inspection device comprises at least one lift device and at least one propulsion device. The lift device serves to keep the inspection device suspended, while the propulsion device serves to move the inspection device forward, and optionally backward and / or sideways. The propulsion device is preferably designed as at least one rotor. Such a rotor can generate thrust, as is known from drones, helicopters, and / or airplanes. Rotors can also be used as lift devices, as is known from helicopters or drones. Alternatively or additionally, other types of lift devices can be used, for example, aerostatic lift devices such as lift-generating gas fillings like those used in airships.

[0011] According to a preferred embodiment of the invention, the at least one flight-capable inspection device has at least one combined lift and propulsion device designed as a rotor. This can be a rotor of the type used in helicopters or an arrangement of several rotors, as is common in drones, for example, in the type of a quadcopter. Such multi-rotor aircraft are easy and precise to control. In particular, they enable movement in all directions.

[0012] Both cameras are fisheye cameras, meaning they have a fisheye lens offering a field of view of at least 180°. With such a camera, half of a 360° image can be captured. If the two viewing directions are 180° apart, meaning the central optical axes in each direction are oriented 180° away from each other, then a 360° image can be captured from the images taken in these two directions.

[0013] According to the invention, the two cameras have opposite viewing directions. If these two cameras are each designed as a fisheye camera, they can thus jointly capture a 360° image. The two are arranged in a common inspection device, meaning they are moved together with an inspection device through the canal structure, with preferably one camera facing forward in the direction of movement and the second camera facing backward in the direction of movement.

[0014] Preferably, the at least two cameras are spaced apart and further preferably arranged at opposite ends of an inspection device. The inspection device can, for example, have a drive with propulsion elements such as rotors and / or electronic components for control and data transmission. The two cameras particularly preferably have a defined, previously known distance between them, which can be taken into account during 3D reconstruction to generate a true-to-scale 3D image model. The defined distance enables correct scaling of the 3D model based on this distance.Furthermore, if the two cameras are arranged in a defined relative position on a common inspection device, it can be ensured that both cameras always remain aligned in opposite directions when the inspection device is moved, so that ideally, as described above, two fisheye cameras can always capture a 360° image, i.e., two 180° images.

[0015] The control unit is preferably designed to control at least two or more cameras such that, during movement of the inspection device, an image is captured at time intervals, preferably one image in the first viewing direction and one image in the second viewing direction. For example, it is possible to capture an image at predetermined intervals during movement, such as every five centimeters. Alternatively, it would be conceivable to capture an image every second, which, at a constant movement speed, would result in regular spatial intervals. It would also be possible to record a video, which in turn comprises a multitude of individual frames. For subsequent image processing, it is advantageous if the control unit records timestamps and / or position data along with the image data when capturing the images.Location information of the inspection device is stored.

[0016] Preferably, the control device is configured to cause at least two cameras to simultaneously capture an image. This is particularly advantageous when the two cameras are arranged at a defined distance on the same inspection device. This ensures that two images are always captured at the same time, so that the two cameras maintain a defined, predetermined distance from each other for these two images. As will be explained in more detail below, this allows the image model to be scaled based on this predetermined distance.

[0017] According to a further possible embodiment of the invention, the inspection device and / or the image processing device includes a position detection system, wherein the position detection system is preferably configured to determine the position of the inspection device, and in particular the position of the cameras, and the paths traveled by them, based on the images captured by the at least one camera. The position detection system can thus detect changes in position and / or orientation. For this purpose, individual pixels or image information, for example characteristic pixels, can be considered and, if necessary, retrieved as corresponding image information in different images.

[0018] Preferably, the position detection system is designed to determine positions and paths within a 3D image model created by the image processing unit. This has the advantage that additional orientation and position sensors can ideally be dispensed with. This is particularly advantageous because conventional navigation systems, such as those used in drones (e.g., GPS), function only to a limited extent or not at all in enclosed shafts or sewer pipes.

[0019] In a specific embodiment, at least one image processing unit can be at least partially integrated into at least one inspection device. This is advantageous for enabling continuous image processing, i.e., image processing in quasi-real time.

[0020] According to another possible embodiment, at least one inspection device can have at least one motion and / or position sensor, and the control unit can be designed such that measured values ​​from such a motion and / or position sensor, captured by the camera at the time of image acquisition, are stored together with the captured images and taken into account by the image processing unit for generating the 3D image model. This means that the location or spatial orientation of the camera at which the image was captured can be considered when generating the 3D image model.

[0021] According to a further possible embodiment of the invention, the image processing device has a structure recognition function configured to recognize corresponding pixels or image content in the captured images. Furthermore, in this embodiment, the image processing device is configured to assemble the images into a 3D image model based on the recognized pixels or image content. That is, the image processing device recognizes corresponding pixels or image information in several captured images and assembles the images according to this information. Various methods, including AI-supported ones, can be used for model generation, i.e., the creation of a 3D model. For example, a "structure from motion" method or a SLAM (Simultaneous Localization and Mapping) method could be employed.

[0022] According to another possible embodiment of the invention, the two cameras are arranged at a defined distance from each other and have opposite viewing directions, e.g., forwards and backwards or upwards and downwards. The image evaluation device is designed such that it scales the 3D image model based on this defined distance between the two cameras.

[0023] To accurately determine distances and dimensions in this virtual 3D image model, it is essential that the image model is scaled correctly. This requires a reference distance to scale the overall model accordingly. Preferably, the predetermined distance between the two cameras is used as such a reference distance. This predetermined camera distance results from the design of the inspection device and can be pre-dimensioned and, if necessary, measured very precisely. This known camera distance can then be used as the basis for scaling during the 3D reconstruction process, in which the virtual three-dimensional image model is generated. This means that all geometric relationships in the model can be scaled and thus precisely determined based on this known distance.In this way, additional measuring systems that measure the canal structure in addition to taking pictures can be dispensed with, or the accuracy of such measuring systems can be increased by additionally scaling the image information based on the previously known camera distance.

[0024] For the reconstruction of the 3D image model, corresponding image points or image content are preferably recognized or captured in the recorded images, and the individual images, which were previously captured by the two cameras at time intervals, are combined into a 3D image model based on the recognized image points or image content that correspond in several images. The image content or image points can be characteristic image points or image content, which can be, for example, edges, image content resulting from the structure of a pipe wall, or other objects in the image. Image evaluation methods are known with which such characteristic or distinctive image content or image content can be identified.Image points can be identified and recognized again, allowing them to be located in different images. Based on these corresponding points found in multiple images, the images can then be assembled into a three-dimensional model. The geometric relationships between individual image points or objects define the three-dimensional structure or model of the canal structure. These geometric relationships are scaled based on the known distance between the two cameras, enabling the derivation of absolute measurements from the image.

[0025] The scaling can preferably be performed by determining a cloud of pixels within the images, including the relative geometric relationships between the pixels. An arbitrary or initial scale is first applied to this cloud, and the distance between the two cameras is calculated based on this scale. This calculated distance is then compared with the known distance, and subsequently, the cloud of pixels is scaled so that the calculated value for the camera distance corresponds to the actual value, i.e., the known distance between the cameras. In this way, absolute measurements for the geometric relationships between individual image objects or pixels can be determined, and the entire 3D image model can be scaled true to scale.

[0026] In a preferred embodiment of the invention, the described scaling of the 3D image model enables the determination of the path traveled during the movement of the inspection device based on the scaled 3D image model. For this purpose, image evaluation or reconstruction can particularly preferably be performed directly during the movement of the inspection device, ideally in essentially real time, so that the 3D image model can be used for orientation and positioning of the inspection device during movement. The image evaluation can be performed directly in the inspection device in a suitable computer system and / or externally in a computer system connected to the inspection device, either via a wired connection or a wireless connection, for example, a radio connection.

[0027] The invention is described below by way of example with reference to the accompanying figures. These show: Figure 1 schematically shows a channel inspection system according to the invention when used in a channel pipe, and Figures 2 and 3 schematically show the scaling of a 3D image model.

[0028] Figure 1 Figure 2 schematically shows a sewer pipe 2 as an example of a sewer structure with an inspection device 4 arranged within it. The inspection device 4 is designed like a drone with rotors 6 for hovering and locomotion. The rotors 6 serve both as a lift and a propulsion system. However, the inspection device 4 could also be designed to fly or hover in a different way, e.g., with separate propulsion and lift systems.

[0029] The inspection device 4 has a camera 8, 10 at each of its two opposite ends. The optical axes B, i.e., the mid-axes of the viewing directions of the two cameras 8 and 10, are aligned in this arrangement, so that the cameras 8 and 10 have two viewing directions 180 degrees apart. The lenses of the cameras 8 and 10 are each designed as fisheye lenses with a field of view of at least 180 degrees, so that together the two cameras 8 and 10 capture a 360-degree image. The inspection device 4 is moved through the culvert 2 in the feed direction S by means of the rotors 6. The optical axes B are directed in the direction of feed S, so that camera 8 faces forward and camera 10 faces backward. Due to the overall length of the inspection device 4, the two cameras 8 and 10 are spaced apart by a distance d.

[0030] The inspection device 4 has a control unit 12 which can control the movement along the feed direction S. In particular, the control unit 12 controls the cameras 8 and 10 such that they each simultaneously capture images at time intervals. The time intervals are preferably regular, so that with a constant feed movement in the feed direction S, images are captured by both cameras at regular intervals, for example, every 2 or 5 cm. Since the cameras 8 and 10 each capture an image simultaneously, there are always two images in which the cameras were at the defined distance d. This can be used for scaling.

[0031] The images captured by cameras 8 and 10 are transmitted to a computer system 14, which includes an image evaluation and processing unit. Data transmission can occur in real time immediately after capture. Alternatively, the image data can be stored in a suitable memory within the inspection device 4 and read out after passing through the sewer pipe 2 and transmitted to the computer system 14 with the image processing unit. The image processing unit generates a 3D image model from the numerous images captured by cameras 8 and 10 using a standard 3D reconstruction method. For this purpose, a structure recognition function can be provided in the image processing unit, which identifies corresponding pixels or image information in the captured images in order to merge individual images or 3D models into a unified 3D model based on these corresponding pixels.The 3D reconstruction is based solely on the captured image information, thus generating an actual three-dimensional model of the canal structure, i.e., the inner wall bounding the canal structure, without relying on an idealized form of the structure. In this way, the actual shape or form of a canal pipe can be determined from the image information, making it possible to detect changes or deformations.

[0032] The image processing unit generates a point cloud of pixels, which is then processed. Figures 2 and 3The image points P1, P2, P3, and P4 are shown schematically. These image points have a defined geometric position, and in particular, a defined geometric relationship to each other. That is, the relative distances to each other result from the 3D reconstruction. However, to determine the absolute dimensions, it is necessary to use a scale for correct scaling. In this example, the known length d of the inspection device 4, i.e., the known distance d between cameras 8 and 10, is used for this purpose. The scaling is carried out such that, for the point cloud, as in Figure 2As shown, an initial scale is used, which can be a predetermined or arbitrary scale. For example, an arbitrary dimension, such as D₁ = 50 cm, is initially assumed for the pipe diameter D. Based on the defined relative relationships, this dimension yields a calculated distance d₁ for the two cameras 8 and 10, which here, for example, is d₁ = 10 cm. If it is now known that the actual distance d = 5 cm, the entire 3D image model can be scaled accordingly, i.e., reduced by half in this example, so that the calculated distance d₂ corresponds to the actual distance d, in this example, d₂ = 5 cm. Based on the given geometric relationships or those determined in the 3D reconstruction, the actual pipe diameter D₂ is then calculated to be 25 cm.According to the invention, the distance d between the two cameras 8 and 10 is ideally used as the only previously known correct dimension, and the entire determined 3D image model is scaled on the basis of these known dimensions, so that all other dimensions, for example positions of detected damage, dimensions of house connections or sleeves, can then be determined or derived from it.

[0033] If the images for the 3D image model are not captured by two cameras connected at a predetermined distance d, a different scale can be used to scale the geometric relationships found in the 3D image model. For example, a distance in the canal structure, such as in canal pipe 2, could be measured using a different method. Furthermore, surveying systems for determining diameters or similar measurements could be integrated into the inspection device to obtain precise dimensions upon which the image model can be scaled.

[0034] Additional sensors could be used to determine the position and orientation of the inspection device 4, for example a position sensor 16, which detects the angular position, in particular the angular position of the optical axes B in space. The measured values ​​acquired by the position sensor 16 can be stored together with the images, so that the angular position of the optical axes B during image acquisition can later be taken into account during 3D reconstruction. Reference symbol list

[0035] 2 Sewer pipe 4 Inspection device 6 Rotors 8, 10 Cameras 12 Control unit 14 Computer system, image processing unit 16 Position sensor S Feed directions B Viewing directions d, d1, d2 Camera spacing 8, 10 D1, D2 Pipe diameter P1, P2, P3, P4 Pixels

Claims

1. Channel inspection system comprising at least one inspection device (4) capable of flight, which is provided to fly through a channel structure to inspect said structure and two cameras (8, 10) and a control device (12), as well as comprising an image processing device (14), characterized in that the two cameras (8, 10) are fisheye cameras and have oppositely directed viewing directions, the two cameras (8, 10) and the control device (12) are configured for generating 360° images in such a manner that images in a first viewing direction and images in at least a second viewing direction are captured by the cameras (8, 10), and the image processing device (14) is configured in such a manner that it combines the images captured in the two viewing directions in a 3D reconstruction to form a 3D image model.

2. Channel inspection system, characterized in that the at least one inspection device (4) capable of flight has at least one lifting device (6) and at least one propulsion device (6), of which at least one propulsion device (6) is preferably configured as a rotor.

3. Channel inspection system according to Claim 1 or 2, characterized in that the at least one inspection device (4) capable of flight has at least one combined lifting and propulsion device configured as a rotor (6).

4. Channel inspection system according to one of the preceding claims, characterized in that the two cameras (8, 10) are spaced apart from one another and are preferably arranged at opposite ends of an inspection device (4).

5. Channel inspection system according to one of the preceding claims, characterized the control device (12) is configured in such a manner that it controls the cameras (8, 10) in such a manner that during a movement of the inspection device (4) at temporal intervals an image is captured in the first viewing direction and an image is captured in the second viewing direction.

6. Channel inspection system according to one of the preceding claims, characterized in that the control device (12) is configured in such a manner that it causes the at least two cameras (8, 10) to each capture one image simultaneously.

7. Channel inspection system according to one of the preceding claims, characterized in that the inspection device (4) and / or the image processing device (14) includes a position detection system, wherein the position detection system is preferably configured in such a manner that it detects the position of the inspection device (4) and the paths travelled by it on the basis of the images captured by the at least one camera (8, 10).

8. Channel inspection system according to Claim 7, characterized in that the position detection system is configured in such a manner that it determines positions and paths in a 3D image model created by the image processing device (14).

9. Channel inspection system according to one of the preceding claims, characterized in that the image processing device (14) is at least partially integrated into the at least one inspection device (4).

10. Channel inspection system according to one of the preceding claims, characterized in that the at least one inspection device (4) has at least one motion and / or position sensor (16) and the control device (12) is configured in such a manner that measured values of such a motion and / or position sensor (16) captured at the time of image acquisition by the cameras (8, 10) are stored together with the captured images and are taken into account by the image processing device (14) when generating the 3D image model.

11. Channel inspection device according to one of the preceding claims, characterized in that the image processing device (14) has a structure recognition function which is configured in such a manner that it recognizes corresponding image points (P1, P2, P3, P4) or image content in the captured images, and is configured in such a manner that it combines the images into a 3D image model based on the recognized image points (P1, P2, P3, P4) or image content.

12. Channel inspection system according to one of the preceding claims, characterized in that the two cameras (8, 10) are arranged at a defined distance (d) from each other and the image processing device (14) is configured in such a manner that it scales the 3D image model on the basis of this defined distance (d) between the two cameras (8, 10).