Method and system for detecting structures in a channel

Abstract

The invention relates to a method for detecting and classifying structures (S) in a channel (K), in particular on the inner wall (W) of the channel, the method comprising at least the following steps: - detecting movement of an inspection unit (10) in the channel by means of a sensor unit (20) associated with the inspection unit (10), wherein the sensor unit generates sensor data (D) corresponding to the detected movement, and - transmitting the sensor data (D) to a structure detection module (M) of an evaluation unit (30), wherein, on the basis of the transmitted sensor data, the structure detection module carries out a structure detection by means of which structures in the channel, in particular on the inner wall of the channel, are detected and classified. The invention also relates to a correspondingly designed channel inspection and / or maintenance system (1).

Description

[0001] February 24, 2026 iPEK International GmbH I56178PC AB

[0002] Method and system for structure detection in a channel

[0003] Field of invention

[0004] The invention relates to a method and a system for structure detection in a channel. In particular, the invention relates to a method and a system for detecting and classifying structures in a channel, especially on the inner wall of the channel, for example, a sewer.

[0005] Background of the invention

[0006] Sewer inspection and / or maintenance systems are well-established technologies. Such systems are used, for example, to inspect and, if necessary, repair sewer pipes, such as wastewater sewers. Sewer inspection systems typically include an inspection unit, which may be designed as a crawler. The inspection unit is usually inserted into the sewer from outside and can then be moved within it. The purpose of a sewer inspection is to assess and, if necessary, document the condition of a sewer pipe. This includes identifying defects, such as deposits, cracks in the pipe, or intrusions (e.g., roots). It is also desirable to identify and document certain features of the sewer pipe itself, such as joints or branches.

[0007] It is known to use optical systems, such as cameras, to detect, for example, defects or other features of sewer pipes. For instance, German patent DE 20 308 761 Ul discloses an inspection system that uses cameras to detect diameter deformations in sewer pipes. Light points projected onto the inner wall of the pipe are captured and analyzed by the camera. The structure of the sewer pipe can then be deduced from the results of this analysis.

[0008] Inspection systems using optical systems, such as cameras, to detect defects or other features of the sewer pipe, such as the system known from EP 3 599 510 Al, have two major disadvantages.

[0009] On the one hand, suboptimal lighting conditions can lead to insufficient illumination of damaged areas, which in turn can result in damage being missed or incorrectly detected in the image or video data. Creating better lighting conditions to improve camera recordings, however, requires larger or more powerful lighting units. For small-diameter sewer pipes, however, only small lighting units can usually be used. More powerful lighting units, on the other hand, mean higher energy consumption, which can significantly shorten the inspection process.

[0010] On the other hand, video processing requires significant computing power, meaning that automatic detection of defects in the video data is usually not possible directly within the inspection unit. In practice, the video data is typically transmitted to a processing unit outside the sewer, where the necessary video processing takes place. However, transmitting video data, especially high-resolution video data, requires high bandwidth, which is not always available. For this reason, it is common practice to compress the video data before transmission, but this cannot be done without some loss of quality. The information loss associated with compression can, in turn, lead to defects in the video data not being detected or being detected incorrectly.

[0011] - In water-bearing canals, defects or other structures located in the canal bed and underwater are usually not detectable at all with optical systems. Objective of the invention

[0012] The object of the invention is therefore to provide an alternative solution with which the aforementioned disadvantages can be at least partially avoided, while still enabling the reliable and efficient detection of defects or other features of the sewer pipe.

[0013] Inventive solution

[0014] This problem is solved by a method for detecting and classifying structures in a channel, as well as by a corresponding channel inspection and / or maintenance system according to the independent claims. Advantageous embodiments of the invention are specified in the respective dependent claims.

[0015] Accordingly, a method for detecting and classifying structures (also referred to as structure detection and classification methods) in a channel, in particular on the inner wall of the channel, is provided, wherein the method comprises at least the following steps:

[0016] Detecting movement of an inspection unit in the channel by means of a sensor unit assigned to the inspection unit, wherein the sensor unit generates sensor data corresponding to the detected movement, and

[0017] - Transmitting sensor data to a structure recognition module of an evaluation unit,

[0018] wherein the structure recognition module performs structure recognition based on the transmitted sensor data, with which structures in the channel, in particular on the inner wall of the channel, are detected and classified.

[0019] The sensor unit is not an optical camera. Optical cameras cannot generate sensor data corresponding to the detected movement. This eliminates the need for computationally intensive image processing while still effectively detecting and classifying structures within the pipe. The lighting conditions in the pipe are therefore irrelevant – it is even possible to perform structure detection and classification in a completely dark pipe. A further advantage is that the required sensor unit is significantly more compact than a camera and can be installed directly in or integrated into the crawler.

[0020] The structure recognition module can use pattern recognition, in particular a neural network, for detection and classification.

[0021] It can be advantageous if

[0022] The movement of the inspection unit is continuously detected by the sensor unit, and the sensor unit continuously generates corresponding sensor data.

[0023] The continuously generated sensor data are continuously transmitted to the structure recognition module of the evaluation unit.

[0024] wherein the structure recognition module performs structure recognition based on a block of transmitted sensor data, wherein the block of transmitted sensor data comprises the last N transmitted sensor data, where N is a predetermined number of transmitted sensor data.

[0025] In one embodiment of the invention, it is advantageous if

[0026] A currently transmitted sensor data is added to the block, whereby when adding the currently transmitted sensor data to the block, the oldest sensor data is removed from the block, provided the block contains N sensor data, thereby forming an updated block of transmitted sensor data, and

[0027] The structure recognition module performs structure recognition continuously for each updated block.

[0028] It can be advantageous if, for several consecutive blocks for which the structure recognition module has detected a predetermined structure, the sensor data of these blocks are combined to determine an average sensor datum from the sensor data of the several consecutive blocks.

[0029] It is advantageous to assign a marker to a block in which a structure has been detected, with the marker containing information about the detected structure. The marker can include information indicating which structures were detected and with what confidence level.

[0030] When checking whether a predetermined structure has been detected for several consecutive blocks, those structures that were detected with the highest confidence can be used as detected structures.

[0031] It is advantageous if a structure within a block is only positively identified if the confidence for it exceeds a predetermined minimum value.

[0032] It can be advantageous if the sensor data are transmitted to the structure recognition module of the evaluation unit in the form of quaternions, with the structure recognition being carried out on the basis of the quaternions.

[0033] It is advantageous if a displacement sensor is used to continuously detect the distance traveled by the inspection unit in the channel and a detected distance is assigned to each sensor signal or sensor data.

[0034] Furthermore, it can be advantageous to determine the mean sensor datum based on the detected distances traveled, which are assigned to the respective sensor data of the several consecutive blocks.

[0035] The mean sensor datum of the sensor data from the several successive blocks can be determined as that sensor datum whose associated detected distance traveled corresponds to the mean distance traveled by the respective sensor data of the several successive blocks. In one embodiment of the invention, it can be advantageous if fused sensor data is generated by fusing the detected distances traveled with the detected movements, which represent a three-dimensional path of the moving inspection unit in the channel.

[0036] It has proven advantageous to detect the distance traveled by the inspection unit in the channel and the movement of the inspection unit in the channel synchronously.

[0037] The structures can include sockets, cracks in the channel wall, deposits, intrusions, bends and / or obstructions. Other structures are nevertheless covered by the invention.

[0038] It is advantageous to use an inertial measuring unit as the sensor unit.

[0039] Furthermore, a channel inspection and / or maintenance system is provided, comprising an inspection unit (also structure recognition and classification unit) with drive means and an evaluation unit, wherein the inspection unit can be moved in a channel by means of the drive means, wherein

[0040] - wherein the inspection unit has a sensor unit, wherein the sensor unit is adapted to detect the movement of the inspection unit in the channel and to generate sensor data corresponding to the detected movement,

[0041] the sensor unit is operationally coupled with the evaluation unit, wherein the sensor unit is adapted to transmit the generated sensor data to the evaluation unit, and the evaluation unit is adapted to receive the sensor data transmitted by the sensor unit,

[0042] The evaluation unit includes a structure recognition module that is set up to perform structure recognition based on the received sensor data, with which structures in the channel, in particular on the inner wall of the channel, are detected and classified.

[0043] The structure recognition module can use or include pattern recognition, in particular a neural network, for detection and classification. The channel inspection and / or maintenance system can have a displacement sensor with which the distance traveled by the inspection unit in the channel can be continuously detected. The displacement sensor can be located on the inspection unit. Alternatively, the displacement sensor can also be located outside the channel, for example on a cable reel whose cable is connected to the inspection unit.

[0044] The inspection unit may include a carriage, the propulsion means comprising a number of wheels and / or a number of tracks.

[0045] It is advantageous if the sensor unit includes an inertial measuring unit.

[0046] The invention makes it possible for the first time to perform structure recognition in the canal completely without optical systems.

[0047] Brief description of the characters

[0048] Further details and features of the invention, as well as specific, particularly advantageous embodiments of the invention, will become apparent from the following description in conjunction with the drawing. It shows:

[0049] Fig. 1 shows two embodiments of a channel inspection and / or maintenance system according to the invention;

[0050] Fig. 2 shows an inspection unit of a channel inspection and / or maintenance system according to the invention in a channel to illustrate the inventive method for detecting and classifying structures in a channel;

[0051] Fig. 3 shows a flowchart with the essential steps of the inventive method;

[0052] Fig. 4 shows a data set for training a structure recognition system, in particular for training a neural network; and Fig. 5 shows a block diagram to illustrate structure recognition using a neural network;

[0053] Detailed description of the invention

[0054] The inventive method and the inventive channel inspection and / or maintenance system make it possible to detect and classify defects in the channel or features of the channel or the channel pipe.

[0055] Damage can include, for example, cracks in the sewer wall, deposits, ingrown roots, or similar issues. Features of the sewer or sewer pipe can include, for example, sockets, bends, branches, liners, or similar components.

[0056] Such defects and features are hereinafter collectively referred to as "structures".

[0057] Fig. 1 shows two embodiments of a channel inspection and / or maintenance system according to the invention, wherein a first embodiment is shown in figure (a) and a second embodiment is shown in figure (b).

[0058] The two embodiments of a channel inspection and / or maintenance system shown in Figures (a) and (b) are described together below.

[0059] The sewer inspection and / or maintenance system 1 includes an inspection unit 10 that can be moved in a sewer K. The inspection unit 10 is designed as a carriage and has drive elements 11 in the form of wheels. In the examples shown in Fig. 1, the carriage has four wheels. However, six, eight, or more wheels can also be provided. Alternatively, tracks can be used as drive elements 11. In a further alternative, both wheels and tracks can be used as drive elements 11 – for example, tracks can be provided at the rear of the carriage and wheels at the front. The electrical energy for the drive elements 11 can be supplied by a battery (not shown) located in the inspection unit 10.Alternatively, the electrical energy can also be provided by a control unit 40 located outside of channel K and transmitted to the inspection unit 10 via a power cable.

[0060] For data transmission between the inspection unit 10 and the control unit 40, the inspection unit 10 can have a transceiver 50 which is coupled to a transceiver 51 of the control unit 40 (not shown in Fig. 1). The transceivers 50 and 51 are advantageously configured for bidirectional data transmission. For example, control commands can be transmitted via the control unit 40 to the inspection unit 10 in order to control the inspection unit 10 itself or inspection equipment, such as a camera, attached to it. Image data or other sensor data could be transmitted from the inspection unit 10 to the control unit 40 via the transceivers 50 and 51. The communication link between the transmit-receive device 50 of the inspection unit 10 and the transmit-receive device 51 of the control unit 40 can be either wired or wireless.

[0061] A sensor unit 20 (which is not a camera) is arranged on or in the inspection unit 10. The sensor unit 20 is designed to detect the movement of the inspection unit 10 in channel K. The sensor unit 20 may include accelerometers, gyroscopes, magnetic field sensors, shock sensors, vibration sensors, or combinations thereof.

[0062] In one embodiment of the invention, the sensor unit 20 can be an inertial measurement unit (IMU) that spatially combines several inertial sensors, for example, three mutually orthogonal accelerometers for detecting linear motion in the x, y, and z axes, and three mutually orthogonal gyroscopes for detecting rotational motion around the x, y, and z axes. Optionally, the inertial measurement unit can also include magnetic field sensors. In one embodiment of the invention, the inertial measurement unit (IMU) can be implemented using MEMS (micro-electromechanical systems), which is advantageous for use in inspection units 10, which have very limited installation space, due to its very compact design. For larger inspection units 10, optically based inertial measurement units (IMUs) can also be used.Regardless of the specific design of the sensor unit 20, it should be firmly connected to the inspection unit 10 to ensure that when the inspection unit 10 moves, the sensor unit 20 also moves accordingly.

[0063] The sensor unit 20 detects or records the movements of the inspection unit 10 in channel K, for example the movement of the inspection unit 10 when it passes over one of the aforementioned structures.

[0064] When the inspection unit 10 moves over one of the aforementioned structures, it may, for example, tilt slightly to the side or be slightly lifted at the front or rear, which is detected by the sensor unit 20. How the inspection unit 10 moves when moving over a particular structure ultimately depends on the structure's specific shape. Such events (i.e., moving over structures) manifest themselves in the sensor data as spikes. Surprisingly, it was found that each of these spikes can be assigned to a specific event. This means that each event leaves a characteristic trace in the sensor data. According to the invention, the corresponding event or structure can therefore be determined by means of a suitable evaluation of the sensor data.

[0065] The sensor unit 20 can provide the respective sensor data as raw data for each sensor. Alternatively, the sensor data from the sensors can be fused by the sensor unit 20, so that the sensor unit 20 provides fused sensor data. For example, an inertial measurement unit (IMU) can provide the fused sensor data as quaternions. The channel inspection and / or maintenance system 1 further comprises an evaluation unit 30 with a structure recognition module M. According to the embodiment of the invention shown in Figure (a), the evaluation unit 30 is arranged in the inspection unit 10. Alternatively, the evaluation unit 30 can also be arranged outside the channel K, for example in the control unit 40, as shown in Figure (b).

[0066] According to a further embodiment of the invention (not shown in Fig. 1), the evaluation unit 30 can be part of a cloud or a cloud infrastructure.

[0067] According to a further embodiment of the invention (not shown in Fig. 1), the evaluation unit 30 can be arranged outside the channel, as shown in Figure (b), while the structure recognition module M is part of a cloud or cloud infrastructure and is operationally coupled or can be operationally coupled to the evaluation unit 30. In this case, the evaluation unit 30, as defined in the invention, also includes the structure recognition module M.

[0068] The evaluation unit 30 is directly or indirectly coupled to the sensor unit 20.

[0069] The sensor data provided by the sensor unit 20 are transmitted to the evaluation unit 30, where structure recognition is performed based on the sensor data. The structure recognition module M of the evaluation unit 30 is provided for this purpose. In one embodiment of the invention, the evaluation unit 30 itself can constitute the structure recognition module M. According to the invention, the structure recognition module M is adapted to both recognize and classify structures based on the sensor data from the sensor unit 20. This means that the structure recognition module M can not only recognize structures themselves, but also identify the specific structure in each case.

[0070] According to the embodiment of the invention shown in Figure (a), structure recognition takes place in the inspection unit 10. The evaluation unit 30 receives the sensor data from the sensor unit 20 and performs structure recognition using the structure recognition module M and based on the received sensor data. The result of the structure recognition can be transmitted to the control unit 40 via the transceiver 50. If, as shown in Figure (a), the evaluation unit 30 is arranged in the inspection unit 10, it can be advantageous to use the raw data from the sensors of the sensor unit 20 for structure recognition.

[0071] Depending on a detected structure, a control unit (not shown in Fig. 1) of the inspection unit 10 can cause the inspection unit 10 to perform a predetermined action. For example, the inspection unit 10 can be caused to travel a certain distance back and take an image of the channel using an inspection camera, or to remove the structure (such as a deposit) using a milling cutter.

[0072] The sensor data and / or information on detected structures can be stored in a storage device 27. The storage device 27 can be located in the inspection unit 10 and coupled to the evaluation unit 30. Alternatively or additionally, the storage device 27 can also be coupled to the sensor unit 20. Providing a storage device 27 has the advantage, for example, that the sensor data can be recorded as needed, so that it is later available for documenting an inspection process.

[0073] According to the embodiment of the invention shown in Figure (b), structure recognition takes place outside the channel in the evaluation unit 30 of the control unit 40. The sensor data is transmitted from the sensor unit 20 to the control unit 40 via the transceiver 50 of the inspection unit 10. The evaluation unit 30 of the control unit 40 receives the sensor data and performs structure recognition using the structure recognition module M and based on the received sensor data. Depending on a detected structure, the control unit 40 can cause the inspection unit 10 to execute a predetermined action, as explained above. The control unit 40 can transmit corresponding control commands to the inspection unit 10 via the transceiver 50 and 51.If, as shown in Figure (b), the evaluation unit 30 is located outside the channel and not in the inspection unit 10, but in the control unit 40, it can be advantageous to transmit fused sensor data, for example in the form of quaternions, to the control unit 40 instead of the raw data from the sensors of the sensor unit 20. This is advantageous, for example, if the bandwidth of the communication link between the inspection unit 10 and the control unit 40 is insufficient for transmitting the raw data. The fusion of the sensor data can be performed by the sensor unit 20. It can be advantageous to store the raw data in a storage device 27 of the inspection unit 10 so that it is available, in addition to the fused sensor data, for later documentation of an inspection process.Furthermore, the raw data can be used at a later time to validate the structures determined based on the fused sensor data.

[0074] In one embodiment of the invention, the channel inspection and / or maintenance system 1 has a displacement sensor 25 with which the distance traveled by the inspection unit 10 in the channel can be recorded. Preferably, the distance traveled is recorded continuously.

[0075] The displacement sensor 25 can be arranged in or on the inspection unit 10 and can, for example, detect the distance traveled based on the revolutions of the drive means 11.

[0076] In an alternative embodiment of the invention, the displacement sensor 25 can also be arranged outside the channel K. For example, the displacement sensor 25 can be arranged on a cable drum from which a cable leading to the inspection unit 10 is unwound. The distance traveled by the inspection unit 10 can be determined based on the unwound length of the cable.

[0077] In one embodiment of the invention, the distance detected by the displacement sensor 25 can be combined with the sensor data from the sensor unit 20. This means that the distance traveled by the inspection unit 10 at time t is assigned to the sensor data provided by the sensor unit 20 at time t. In this way, it is possible to determine the location of each structure within the channel. It is advantageous if the distance traveled by the inspection unit 10 in the channel and the movement of the inspection unit 10 within the channel are detected synchronously, i.e., the measurement acquisition by the sensor unit 20 and the displacement sensor 25 occurs simultaneously and at the same frequency.If the measurement data acquisition by the sensor unit 20 and the displacement sensor 25 is not synchronous, it may be advantageous, for example, to assign interpolated values ​​for the distance traveled to the sensor data of the sensor unit 20, which were recorded between two measurement data acquisitions of the displacement sensor 25.

[0078] The combination of sensor data from sensor unit 20 with the distance traveled can be used to document an inspection process. Furthermore, it has been shown that by fusing the detected distances traveled with the detected movements of inspection unit 10, particularly when an inertial measurement unit (IMU) is used as inspection unit 10, fused sensor data can be generated that represent a three-dimensional path of inspection unit 10 in the channel, regardless of the speed at which inspection unit 10 moves within the channel.

[0079] The channel inspection and / or maintenance system 1 according to the invention thus advantageously enables

[0080] - to recognize and classify structures based on the sensor data of sensor unit 20 using the structure recognition module M,

[0081] - To provide information about where which structures are located in the canal, and

[0082] - to reconstruct a three-dimensional path of inspection unit 10 in the channel.

[0083] A further advantage lies in the fact that, with the aid of predetermined, recognized structures, a correction can be made to the distance traveled as recorded by the displacement sensor 25. For example, if the distance traveled is determined based on the revolutions of the drive means 11, it can happen that the measured value for the distance traveled is incorrect if the wheels are slipping. Furthermore, if the location of the couplings in the channel is known (for example, if a channel consists of several pipes, each 2,500 mm long), then a correction can be made to the distance recorded by the displacement sensor 25 for each coupling detected by the structure recognition module M.

[0084] Fig. 2 shows an inspection unit 10 of a channel inspection and / or maintenance system 1 according to the invention in a channel K to illustrate the method according to the invention for detecting and classifying structures in a channel.

[0085] The upper part of Fig. 2 shows the inspection unit 10 arranged in the channel, which has already been moved a certain distance d3 in the channel. The channel K has a socket, which is a structure S within the meaning of the present invention. The channel has no structures before or after the socket S.

[0086] The lower part of Fig. 2 shows the course of the sensor data provided by the sensor unit 20 over the distance, although the course is greatly simplified and only schematically represented here.

[0087] Inspection unit 10 passed over socket S while traveling through channel K, specifically between sections dl and d2. As long as inspection unit 10 has not yet reached socket S, the sensor data shows a largely constant trend. Upon passing over socket S (here with its two front wheels), inspection unit 10 performs a movement (tilting and / or tipping and / or accelerating in one direction) that is detected by sensor unit 20 and manifested in the sensor data as a deflection SA. After passing over socket S, the sensor data again show a largely constant trend.

[0088] The sensor data provided by the sensor unit 20 are continuously fed to the evaluation unit 30 or the structure recognition module M. The structure recognition module M continuously evaluates the sensor data it receives. In the example shown in Fig. 2, the structure recognition module M detects a structure S between the distances dl and d2 traveled. According to the invention, the structure recognition module M can classify the detected structure as a sleeve.

[0089] If the distance traveled is continuously assigned to the sensor data, the structure recognition module M or the evaluation unit 30 can also provide information about where the detected coupling is located in the channel.

[0090] In the example shown in Fig. 2, structure S is a sleeve. Of course, the structure recognition module M can also recognize and classify other structures, such as cracks, deposits, or the like, as will be explained in more detail below with reference to Fig. 3. In one embodiment of the invention, the structure recognition module M can also recognize and classify curves and thus bends or junctions as structures within the meaning of the invention.

[0091] Fig. 3 shows a flowchart with the essential steps of the inventive method for detecting and classifying structures S in a channel K.

[0092] In a first step S1, the sensor unit 20 detects movement of the inspection unit 10 and generates corresponding sensor data. Preferably synchronously, the displacement sensor 25 detects the distance traveled by the inspection unit 10. The sensor data generated by the sensor unit 20 can include the raw data from the individual sensors. Alternatively, fused sensor data can be generated, for example in the form of quatemions.

[0093] In a second step S2, the sensor data and the detected distance are transferred to the structure recognition module M, which performs the structure recognition described in more detail below.

[0094] Steps S1 and S2 are executed continuously; that is, the movement of inspection unit 10 and the distance traveled by inspection unit 10 are continuously recorded at a predetermined frequency by sensor unit 20 and displacement sensor 25, respectively. The sensor data from sensor unit 20 and the detected distance are then continuously transmitted to the structure recognition module M at the respective frequency. The detected distances can be assigned to the corresponding sensor data, allowing the process to later determine where in the channel the respective sensor data was generated. The specified frequency can range from 10 Hz to 2000 Hz. Tests have shown that frequencies between 10 Hz and 1000 Hz, and especially between 20 Hz and 500 Hz, lead to very good results in structure recognition.

[0095] If structure detection is performed outside the channel, for example with a system according to Figure (b) of Fig. 1, and only a limited bandwidth is available for transmitting the sensor data from the inspection unit 10 to the control unit 40 or the evaluation unit 30, then low frequencies, approximately between 10 Hz and 50 Hz, can be used. With low bandwidths, it can also be advantageous to transmit fused sensor data, such as quaternions, to the evaluation unit 30 instead of the raw data from the sensor unit 20. Higher frequencies can also be used if higher bandwidths are available.

[0096] According to one embodiment of the invention, a variable frequency can be provided. For example, the frequency can be adjustable depending on the current speed of the inspection unit 10. Thus, a low frequency can be selected at low speeds of the inspection unit 10, while higher frequencies can be selected at higher speeds. This can prevent, for example, structures traversed by the inspection unit 10 from not being detected or not being detected completely at higher speeds and with too low a frequency.

[0097] According to a further embodiment of the invention, the frequency can be selected depending on whether a structure is detected in the processed block of sensor data (described below). For example, if a structure is detected in a block, the frequency can be increased, resulting in more sensor data being available for structure detection. Conversely, if no structure is detected in a block, the frequency can be reduced again. This allows the sensor data in the area containing structures to be subjected to high-resolution structure detection, while the sensor data in the area without structures can be subjected to low-resolution structure detection. Furthermore, energy consumption can be optimized for both measurement acquisition and structure detection itself.

[0098] The structure recognition module M performs a structure recognition based on the transmitted data (sensor data from sensor unit 20 and the detected distance), as described in more detail with reference to steps S3 to S6.

[0099] In step S3, the sensor data from sensor unit 20, transmitted to the structure recognition module M, is added to a block of sensor data. Sensor data is added to the block until it contains a predetermined number N of sensor data. If the block already contains the predetermined number N of sensor data, the oldest sensor data added to the block is removed. The block thus behaves like a FIFO memory of size N.

[0100] Once the block contains the predetermined number N of sensor data, structure recognition is performed for this block in the subsequent step S4. Structure recognition can be carried out using conventional methods, such as wavelet transformations. However, machine learning methods, especially artificial neural networks, have proven advantageous and are described in more detail below.

[0101] If a structure is detected and classified within the block of sensor data processed in step S4, this block is marked accordingly. The marking can include information about the detected structure (e.g., a sleeve or deposit).

[0102] The process then returns to step S3, in which the next transmitted sensor data is added to the block and the oldest sensor data is removed. This results in a new or updated block of N sensor data. In step S4, a structure detection is performed again on this updated block of sensor data. If the same structure is detected in step S4 for the updated block as for the preceding or previously processed block, the updated block is also marked accordingly. This means that the marking of the current block and the preceding block contain the same information with respect to the detected structure.

[0103] The block markers may include additional information.

[0104] Steps S3 and S4 are repeated for further transmitted sensor data until, in step S4, no structure is detected for the current block of sensor data, or a different structure is detected than in the previous block. The current block is then marked, with the information regarding the detected structure differing from the marking of the previous block. If no structure is detected, the block can also be marked, in which case the marking includes information indicating that the block contains no structure.

[0105] When using an artificial neural network for structure recognition, the network can be trained to recognize multiple or different structures. In this case, the label can include information indicating the confidence level with which each structure was recognized, for example, (socket; 5%), (root; 99%), (deposit; curve; 20%). When checking whether the structure recognized in the current block differs from the structure recognized in the previous block, the structures recognized with the highest confidence can be compared.

[0106] It can also be advantageous to assume a structure has been detected in the current block only if the confidence level for the structure reaches a certain minimum value. For example, a structure in a block is only positively detected if the confidence level for it is, for example, at least 90%. If a different structure is detected in step S4 for the current block of sensor data than in the preceding block, the process continues with step S5. In one embodiment of the process, the process can simultaneously return to step S3.

[0107] At this point, the process is in a state where the detected structure of the current block differs from the structure of the preceding block. Furthermore, the same structure was detected for a number of blocks preceding the previous block.

[0108] Example:

[0109] Let STI, ST2 and ST3 be three different structures that were detected in this order for a number of blocks, and let BSTI, BST2 and BST3 be the blocks with N sensor data each, then at this point, for example, the following temporal sequence of blocks or blocks with the following sequence of detected structures could exist:

[0110] BSTI BSTI BSTI BSTI BST2 BST2 BST2 BST2 BST2 BST2 BST2 BST2 BST3

[0111] Block BST3 is the block currently being processed in step S4. The recognized structure ST3 in the current block differs from the recognized structure ST2 of the preceding block. The same structure, namely structure ST2, was recognized for eight blocks that preceded the current block.

[0112] In step S5, all consecutive blocks for which the same structure was detected and which immediately preceded the current block (in the example above, these would be the eight blocks BST2) are combined to generate or determine an average sensor datum based on these blocks.

[0113] In one embodiment of the method according to the invention, determining the mean sensor data can include determining the exact position of the structure. As explained above, the sensor data can be assigned to the respective detected path, i.e., the path along which the respective sensor data was generated or the respective sensor data was generated. To determine the exact position of the structure, the mean of the paths assigned to the sensor data of the successive blocks for which the same structure was detected can be calculated. In the example above, for the eight successive blocks BST2, the exact position of structure ST2 would be determined by calculating a mean of the paths assigned to these blocks (that is, A*8 paths, since each of the eight blocks has N sensor data, each of which is assigned a path).This average of the distances indicates the exact position of the structure detected with these blocks in the canal.

[0114] Determining the mean of the distances for position determination is therefore advantageous because the speed of the inspection unit 10 is not necessarily constant during the measurement acquisition by the sensor unit 20, i.e. the distance traveled by the inspection unit 10 can vary between each two recorded measurement values.

[0115] According to the invention, the mean sensor datum can also be determined in other ways. For example, the sensor datum with the largest amplitude with respect to a particular attribute can be selected from the successive blocks. Alternatively, the sensor datum located in the middle of the entire set of sensor data can be selected from the successive blocks. Ultimately, how the mean sensor datum is determined depends on its intended purpose. If the precise position of the detected structure is of interest, the mean distance is determined as explained above.

[0116] After step S5, step S6 is executed, in which the average sensor data, along with the classification of the detected features, is provided for further processing. For example, the information that a "sleeve" was detected at position "12.54 m" can be provided. This information can be used, for example, for documenting an inspection process. The above assumed that the same structure was detected in several consecutive blocks. However, the method according to the invention can also be applied if a structure is detected in only one block, i.e., if different structures are detected for the preceding and subsequent blocks. In this case, however, it can be assumed that a false positive detection of a structure has occurred.It can therefore be advantageous to consider a structure as correctly identified only if it has been identified in a predetermined number of consecutive blocks.

[0117] According to the invention, all sensor data can be stored together with the associated paths in order to generate or visualize, for example, a three-dimensional path of the inspection unit 10 through the channel following an inspection process. The three-dimensional path can also be visualized in real time, for example, at the control unit 40, if the data required for this purpose are continuously transmitted to the control unit 40.

[0118] One purpose of determining a classified structure and its corresponding position in the channel might be to trigger an action by inspection unit 10 after the structure is detected. For example, inspection unit 10 could be instructed to retrace a certain distance to take an image or video recording of the structure. If maintenance units, such as a milling cutter, are attached to inspection unit 10, it could, for instance, be instructed to mill away the structure if the detected structure is a deposit.

[0119] Fig. 4 shows a data set for training a pattern recognition system, in particular for training a neural network.

[0120] As stated above with reference to step S4 of the method according to the invention, an artificial neural network can be used for structure recognition and classification. Figure 4 shows sensor data D over the path, which can be provided by the sensor unit 20. As explained above, the raw data from the sensor unit 20, such as the raw data from an inertial measurement unit (IMU), can be used for structure recognition. Alternatively, fused sensor data, such as quatemions from an inertial measurement unit (IMU), can also be used.

[0121] The data set, or sensor data D, was generated during a test drive in a real-world environment. Subsequently, the data set, or sensor data D, was manually labeled; that is, sensor data representing a specific structure, such as a sleeve, was assigned a label. In the example shown in Fig. 4, these structures were labeled "1", while the remaining sections were labeled "0".

[0122] The labeled data set or the labeled sensor data was then used for training the neural network.

[0123] Specifically, for training, consecutive sensor measurements were always grouped together. The labeled dataset was then split into 70% training data and 30% test data, although this ratio could also be chosen differently. Furthermore, the sensor data underwent preprocessing. Specifically, the sensor data was scaled by applying a standard sealer to the dataset to scale the entire dataset to a mean of zero and unit variance. The model or neural network trained with the training data was then tested and verified using the test data.

[0124] Tests have shown that a recognition accuracy of 96% and more can be achieved for a neural network with multiple layers.

[0125] Of course, other models can also be used for structure recognition and trained accordingly, such as support vector machines, decision tree classifiers, random forest classifiers, gradient boosting, or similar approaches. A recognition accuracy of over 92% was achieved with each of the aforementioned models, with gradient boosting, for example, achieving an accuracy of 95%. The accuracy of the models can be improved by using more training data.

[0126] The advantage of the aforementioned models is that they require significantly less training data and are considerably more performant compared to classical image recognition. This allows structure recognition to be performed directly in inspection unit 10, meaning the sensor data does not first need to be transmitted to a control unit 40 outside the channel. This is particularly advantageous when inspection unit 10 is intended to operate autonomously within the channel.

[0127] Fig. 5 shows a block diagram to illustrate structure recognition using a neural network.

[0128] Shown here is a time sequence of sensor data D, where the sensor data includes a so-called sliding window SD that moves along the time axis. This sliding window SD essentially corresponds to the aforementioned block of sensor data; that is, the sliding window SD comprises a number N of specific sensor data points. As the sliding window SD moves along the time axis, a sensor data point is added to the sliding window. Simultaneously, a sensor data point is removed from the sliding window.

[0129] The sensor data from the Sliding Windows SD is fed to the structure recognition module M, which is designed as a neural network. Based on the sensor data from the Sliding Windows SD, the structure recognition module M performs structure recognition, as described with reference to Fig. 3. The structure recognition module M outputs information about the recognized and classified structure as result E. If no structure is recognized, this is also output as a result.

[0130] The sliding window SD is then shifted along the time axis by one sensor datum. The resulting sensor data from the sliding window SD are then used for structure detection. Figure 5 shows a sliding window SD that moves along the time axis, so that a new sensor datum is always added to the sliding window and the oldest sensor datum is removed. With the inventive method, the detection and classification of structures in the channel can therefore also be applied to an existing dataset of sensor data. If the dataset also stores the corresponding distances for each sensor datum, then the position of each detected structure in the channel can also be determined.

[0131] In practice, however, it can be advantageous to add the sensor data directly to the sliding window after acquisition and to feed the updated sliding window to the structure recognition module M, as described with reference to Fig. 3.

[0132] A sensor datum of the aforementioned sensor data can be a one-dimensional sensor datum, i.e., comprising a specific sensor value.

[0133] According to the invention, a sensor datum can also be a multidimensional sensor datum, i.e., comprising several sensor values ​​provided by different sensors.

[0134] The invention makes it possible, for example, to reliably detect sockets in the channel, whereby the invention is not limited to the detection of sockets, but can detect any type of structure that causes a movement of the inspection unit 10.

[0135] The detection of pipe joints, or detected joints, can be used, for example, to correct errors in the distance traveled by inspection unit 10 in the channel. Reference numeral:

[0136] I Canal inspection and / or maintenance system

[0137] 10 inspection units

[0138] II. Propulsion system

[0139] 20 sensor units

[0140] 25 Displacement sensor

[0141] 27 Storage device

[0142] 30 evaluation units

[0143] 40 Control unit

[0144] 50 Transceiver

[0145] D Sensor data

[0146] E Result of structure recognition = recognized and classified pattern K channel

[0147] L Label (Identification)

[0148] M Structure Recognition Module

[0149] S Structure(s) (e.g., socket, deposit, curve, etc.) S1-S6 Steps of the procedure

[0150] SA reading in the sensor data

[0151] SW Block of sensor data (sliding window)

[0152] W Inner wall of the channel W

Claims

February 24, 2026 iPEK International GmbH 156178PC AB Claims 1. Method for detecting and classifying structures (S) in a channel (K), in particular on the inner wall (W) of the channel, wherein the method comprises at least the following steps: Detecting a movement of an inspection unit (10) in the channel by means of a sensor unit (20) assigned to the inspection unit (10), wherein the sensor unit generates sensor data (D) corresponding to the detected movement, and - Transfer of sensor data (D) to a structure recognition module (M) of an evaluation unit (30), wherein the structure recognition module performs structure recognition based on the transmitted sensor data, with which structures in the channel, in particular on the inner wall of the channel, are detected and classified.

2. Method according to the preceding claim, wherein the structure recognition module uses pattern recognition, in particular a neural network (NN), for recognition and classification.

3. Method according to any one of the preceding claims, wherein the movement of the inspection unit (10) is continuously detected by means of the sensor unit (20) and the sensor unit continuously generates corresponding sensor data (D), and The continuously generated sensor data (D) are continuously transmitted to the structure recognition module (M) of the evaluation unit (30), wherein the structure recognition module performs the structure recognition based on a block (SW) of transmitted sensor data, the block of transmitted sensor data comprising the last N transmitted sensor data, where N is a predetermined number of transmitted sensor data.

4. Method according to the preceding claim, wherein a currently transmitted sensor data is added to the block (SW), wherein, upon adding the currently transmitted sensor data to the block (SW), the oldest sensor data is removed from the block, provided that the block comprises N sensor data, thereby forming an updated block of transmitted sensor data, and The structure recognition module performs structure recognition continuously for each updated block.

5. Method according to the preceding claim, wherein for several successive blocks (SW) for which the structure recognition module has recognized a predetermined structure, the sensor data of these blocks are combined to determine an average sensor datum from the sensor data of the several successive blocks.

6. Method according to one of the preceding claims, wherein the sensor data in the form of quatemions are transmitted to the structure recognition module (M) of the evaluation unit (30) and wherein the structure recognition is carried out on the basis of the quatemions.

7. Method according to one of the preceding claims, wherein the distance traveled by the inspection unit (10) in the channel is continuously detected by means of a displacement sensor (25) and wherein each sensor signal (D) is assigned a detected distance traveled.

8. Method according to claim 7 and claim 5, wherein the mean sensor datum is determined based on the detected distances traveled, which are assigned to the respective sensor datums of the several successive blocks.

9. Method according to the preceding claim, wherein the mean sensor datum of the sensor data of the multiple successive blocks (SW) is determined to be that sensor datum whose associated detected distance traveled corresponds to the mean distance of the detected distances traveled assigned to the respective sensor data of the multiple successive blocks.

10. Method according to any one of claims 7 to 9, wherein fused sensor data are generated by fusion of the detected distances traveled with the detected movements, representing a three-dimensional path of the moving inspection unit (10) in the channel.

11. Method according to any one of claims 7 to 10, wherein the distance traveled by the inspection unit (10) in the channel and the movement of the inspection unit (10) in the channel are detected synchronously.

12. Method according to any of the preceding claims, wherein the structures comprise sockets, cracks in the channel wall, deposits, intrusions, bends and / or obstacles.

13. Method according to one of the preceding claims, wherein an inertial measuring unit is used as the sensor unit.

14. Channel inspection and / or maintenance system (1) comprising an inspection unit (10) with drive means (11) and an evaluation unit (30), wherein the inspection unit is movable in a channel (K) by means of the drive means, wherein - wherein the inspection unit has a sensor unit (20) wherein the sensor unit is adapted to detect the movement of the inspection unit in the channel and to generate sensor data (D) corresponding to the detected movement, the sensor unit is operationally coupled with the evaluation unit, wherein the sensor unit is adapted to transmit the generated sensor data to the evaluation unit, and the evaluation unit is adapted to receive the sensor data transmitted by the sensor unit, The evaluation unit includes a structure recognition module (M) which is set up to perform structure recognition based on the received sensor data, with which structures (S) in the channel, in particular on the inner wall (W) of the channel, are detected and classified.

15. Channel inspection and / or maintenance system according to the preceding claim, wherein the structure recognition module (M) uses pattern recognition, in particular a neural network (NN), for detection and classification.

16. Channel inspection and / or maintenance system according to one of the two preceding claims, wherein the latter has a displacement sensor (25) with which the distance traveled by the inspection unit (10) in the channel can be continuously detected.

17. Channel inspection and / or maintenance system according to any one of the preceding claims 14 to 16, wherein the inspection unit (10) comprises a carriage and wherein the propulsion means (11) comprises a number of wheels and / or a number of tracks.

18. Channel inspection and / or maintenance system according to any one of the preceding claims 14 to 17, wherein the sensor unit comprises an inertial measuring unit.

Images