Sewer inspection and / or maintenance system with control unit

Abstract

Sewer inspection and / or maintenance system (10), comprising - a control device (20), - at least one channel inspection and / or maintenance unit (60) operatively coupled to the control device (20), and - an operating unit (B) for a user (U) which is operatively coupled to the control device (20), wherein - the operating unit (B) comprises at least one camera (30) and / or at least one microphone (40), - the operating unit (B) and the control device (20) are designed such that the control device (20) receives camera data from the camera (30) and / or audio data from the microphone (40) from the operating unit (B), and - the control device (20) is adapted, - to recognize at least one gesture of the user (U) in the received camera data and / or to recognize at least one voice command of the user (U) in the received audio data, - to generate at least one control command based on the at least one recognized gesture and / or based on the at least one recognized voice command, and - to transmit the at least one generated control command to the at least one channel inspection and / or maintenance unit (60).

Description

Field of the invention

[0001] The invention relates to a sewer inspection and / or maintenance system with an operating unit. The operating unit can be used to operate or control the sewer inspection and / or maintenance system. Background of the invention

[0002] Sewer inspection and / or maintenance systems typically consist of at least one sewer inspection and / or maintenance unit and an operating unit for the sewer inspection and / or maintenance unit. The term "inspection unit" is used below for a sewer inspection and / or maintenance unit. An inspection unit may, for example, comprise an inspection camera, which may be arranged on a carriage or on a lifting unit of a carriage. Several different inspection units may be arranged on a carriage. Thus, in addition to an inspection camera, a gripper, a milling head, or the like may also be arranged on the carriage. A sewer inspection and / or maintenance system may further comprise accessories, such as a lowering device, with which, for example, a carriage can be lowered into a sewer shaft, or cable drums.

[0003] Control units are provided for operating the inspection units and accessories, with separate control units provided for each inspection unit (e.g., camera or milling head) and each accessory. For example, a milling head requires a different control unit than an inspection camera. For accessories such as the drain device, the control units can be integrated directly on or within the accessory.

[0004] The disadvantage here is that the control units (such as joysticks or control panels) are subject to wear and tear and incur high costs for the required hardware and maintenance. Another disadvantage is that separate personnel are usually required to operate the control units – for example, if a crawler is lowered into the sewer shaft, one person is needed to operate the drainage device and another person to lower the crawler. Such control units can also be extremely complex to handle – for example, a control panel for an inspection camera must provide the option of positioning the camera in the sewer, aligning the camera's optical axis, and controlling the zoom and lighting.

[0005] A further disadvantage is that people with a disability or a physical handicap cannot operate such control units known from the state of the art, or can only operate them to a very limited extent. Object of the invention

[0006] The object of the present invention is therefore to provide solutions with which separate operating units are no longer necessary for different inspection units and accessories, and with which the operation of the various inspection units and accessories is significantly simplified. Inventive solution

[0007] This task is solved by a sewer inspection and / or maintenance system, comprising - a control device, - at least one sewer inspection and / or maintenance unit operatively coupled to the control device, and - a control unit for a user which is operatively coupled to the control device, wherein - the control unit comprises at least one camera and / or at least one microphone, - the operating unit and the control device are designed in such a way that the control device receives camera data from the camera and / or audio data from the microphone from the operating unit, and - the control device is adapted, - to recognize or detect at least one gesture of the user in the received camera data and / or to recognize or detect at least one voice command of the user in the received audio data, - to generate at least one control command based on the at least one recognized or detected gesture and / or based on the at least one recognized or detected voice command, and - to transmit the at least one generated control command to the at least one sewer inspection and / or maintenance unit.

[0008] The camera can include a video camera, a thermal imaging camera, a stereo camera or a time of flight camera (TOF camera).

[0009] According to one embodiment of the invention, different camera types can also be combined with one another, i.e., provided together. For example, the operating unit can comprise a video camera and a thermal imaging camera, or a video camera, a thermal imaging camera, and a TOF camera. The combination of different camera types can be advantageous because different camera data is available that can be combined for the purpose of detecting gestures of the operating personnel. For example, when combining a video camera with a TOF camera, video images (from the video camera) can be combined with depth or distance information (from the TOF camera) in order to, for example, improve the accuracy / recognition rate of gesture detection or reduce misdetections (false positive or false negative results).

[0010] According to the invention, the camera data can therefore include: video images or video signals, temperature data or temperature distributions, distance data, stereoscopic fields, and combinations thereof.

[0011] It may be advantageous if the control device comprises a control unit, wherein the control device further comprises - a video evaluation unit adapted to recognize at least one gesture in the received camera data and to transmit information about the at least one recognized gesture to the control unit, and / or - an audio evaluation unit adapted to recognize the at least one voice command in the received audio data and to transmit information about the at least one recognized voice command to the control unit, wherein the control unit is adapted to generate the at least one control command based on the information about the at least one recognized gesture and / or based on the information about the at least one recognized voice command.

[0012] In one embodiment of the invention, it may be advantageous if - the video evaluation unit comprises an artificial intelligence, in particular an artificial neural network, which is designed to recognize at least one gesture in the received camera data, and / or - the audio evaluation unit comprises an artificial intelligence, in particular an artificial neural network, which is designed to recognize the at least one voice command in the received audio data.

[0013] Furthermore, it may be advantageous if the operating unit further comprises a feedback unit which is operatively coupled to the control device, wherein the control device is adapted - to determine an initial probability for a gesture recognized in the received camera data, which indicates the probability with which the recognized gesture was recognized correctly, and / or - to determine a second probability for a voice command recognized in the received audio data, which indicates the probability with which the recognized voice command was recognized correctly, - to generate warning data when the first probability and / or the second probability is below a predetermined tolerance value and to transmit the generated warning data to the feedback unit for output by the feedback unit.

[0014] This can be advantageous if, for example, the video evaluation unit cannot clearly determine whether the detected gesture is a “thumbs up” gesture or a “pointing finger up” gesture in poor lighting conditions.

[0015] It may be advantageous if the warning data include information about which gesture and / or voice command was recognized with a first and / or second probability that is below the predetermined tolerance value, wherein the feedback unit - a display unit for optical output of the warning data and / or a loudspeaker for acoustic output of the warning data, and - comprises an input unit for receiving user inputs, wherein the gesture and / or voice command recognized below the predetermined tolerance value can be confirmed or rejected with a user input, wherein the feedback unit is adapted to transmit the received user input to the control device, and the control device is adapted to generate the at least one control command only upon confirmation.

[0016] This means that if gestures and / or voice commands are detected below the predetermined tolerance value, the system according to the invention can have them confirmed by the user before corresponding control commands are generated.

[0017] This advantageously provides a safety mechanism that prevents the control unit from generating control commands based on incorrectly recognized gestures. Such "incorrect" control commands can, in the worst case, lead to damage to the inspection unit, for example, if an inspection camera collides with the duct wall due to an incorrect control command.

[0018] In one embodiment of the invention, it may be advantageous if the control device is further adapted to transmit the received user input together with the associated camera data and / or associated audio data to the artificial intelligence of the video evaluation unit and / or to the artificial intelligence of the audio evaluation unit in order to train the respective artificial intelligence.

[0019] This allows the respective artificial intelligence to be continuously trained and improved, so that the error rate is also continuously reduced.

[0020] It may be advantageous if the control device is adapted to automatically assign a gesture and / or a voice command to a specific sewer inspection and / or maintenance unit if the gesture and / or the voice command can be assigned to several sewer inspection and / or maintenance units.

[0021] For example, a "thumbs-up" gesture can be assigned to a lifting arm of a mobile device or to an upward tilt of a camera head. If only a lifting arm is present but no camera head, a corresponding control command is generated for the lifting arm.

[0022] If both the lifting arm and the camera head are present, heuristics can help automatically decide whether the control command should be generated for the lifting arm or the camera head. For example, if the free end of the lifting arm, where the camera head is located, is already close to the inner wall of the sewer, it can be assumed that the lifting arm should not be raised any further and that the camera head should be pivoted upward instead. A control command for the camera head is then generated accordingly.

[0023] Alternatively, or in addition to such heuristics, the feedback unit can be used to query the operator as to which sewer inspection and / or maintenance unit the gesture or voice command should be used for. For example, a button labeled "Lifting Arm" and another button labeled "Camera Head" can be displayed on the input unit. The operator can then press one of the two buttons to select the desired sewer inspection and / or maintenance unit. Alternatively, the selection can be made using voice commands.

[0024] In one embodiment of the invention, the operating unit can be a mobile operating unit, in particular a tablet.

[0025] For a better understanding of the invention, a method for operating a sewer inspection and / or maintenance system is also described below, the method comprising - Generating camera and / or audio data using a camera and / or microphone, - feeding the camera and / or audio data to a control device, wherein the control device - processes the received camera data and / or audio data, - detects at least one gesture of the user in the received camera data and / or detects at least one voice command of the user in the received audio data, - generates at least one control command based on the detected gesture and / or based on the detected voice command, and - transmits the generated control command to an inspection and / or maintenance unit of the sewer inspection and / or maintenance system.

[0026] It is advantageous if the at least one gesture is detected by a video evaluation unit, in particular by an artificial intelligence of the video evaluation unit, and / or the at least one voice command is detected by an audio evaluation unit, in particular by an artificial intelligence of the audio evaluation unit.

[0027] In one embodiment of the method, it may be advantageous if - for a gesture recognized in the received camera data, an initial probability is determined, which indicates the probability with which the recognized gesture was correctly recognized, and / or - for a voice command recognized in the received audio data, a second probability is determined, which indicates the probability with which the recognized voice command was correctly recognized, and - warning data is generated when the first probability and / or the second probability is below a predetermined tolerance value, wherein the generated warning data is transmitted to a feedback unit for output by the feedback unit.

[0028] The warning data may include information about which gesture and / or voice command was recognized with a first and / or second probability that is below the predetermined tolerance value.

[0029] This prevents incorrectly recognized gestures / voice commands from being generated.

[0030] In one embodiment of the method, it may be advantageous if the feedback unit comprises an input unit for receiving user inputs, wherein the gesture and / or voice command recognized below the predetermined tolerance value can be confirmed or rejected with a user input, wherein the feedback unit transmits the received user input to the control device, and the control device only generates the at least one control command upon confirmation.

[0031] In an advantageous embodiment of the method, it can be provided that the received user input together with the associated camera data and / or associated audio data is transmitted to the artificial intelligence of the video evaluation unit (23) and / or to the artificial intelligence of the audio evaluation unit (24) in order to train the respective artificial intelligence.

[0032] It may also be advantageous if the control device automatically assigns a recognized gesture and / or a recognized voice command to a specific sewer inspection and / or maintenance unit if the gesture and / or the voice command can be assigned to several sewer inspection and / or maintenance units. Short description of the characters

[0033] Further details and features of the invention, as well as concrete, particularly advantageous embodiments of the invention, will become apparent from the following description in conjunction with the drawing. It shows: Fig. 1 is a block diagram of a sewer inspection and / or maintenance system according to the invention; Fig. 2 a schematic representation of an embodiment of a sewer inspection and / or maintenance system according to the invention; Fig. 3 an embodiment of an operating unit of a sewer inspection and / or maintenance system according to the invention; Fig. 4 shows a flowchart for a method for operating a sewer inspection and / or maintenance system for understanding the sewer inspection and / or maintenance system according to the invention; and Fig. 5 a schematic representation of another embodiment of a sewer inspection and / or maintenance system according to the invention. Detailed description of the invention

[0034] In the following, a sewer inspection and / or maintenance system is also referred to simply as an "inspection system." A sewer inspection and / or maintenance unit is also referred to simply as an "inspection unit."

[0035] Fig. 1 shows a block diagram of a sewer inspection and / or maintenance system 10 according to the invention.

[0036] The inspection system 10 comprises at least one inspection unit 60, shown here as a cable drum 60a and a carriage with an inspection camera 60n. Additional inspection units may include, for example, manipulators, rinsing nozzles, grippers, and the like.

[0037] The inspection system 10 further includes a control device 20 operatively coupled to the inspection units 60. The control device 20 generates control commands that are transmitted to the respective inspection unit 60 for control. The connection between the control device 20 and the inspection units 60 can be wired and / or wireless. For example, a first inspection unit can be wirelessly coupled to the control device, and a second inspection unit can be wired to the control device.

[0038] The inspection system 10 further comprises an operating unit B that is operatively coupled to the control device 20 and is intended for a user U to operate the inspection units 60. The coupling between the operating unit B and the control device 20 can be wired or wireless.

[0039] According to one embodiment of the invention, the control unit B comprises a camera 30, for example, a video camera, a thermal imaging camera, a stereo camera, or a time-of-flight (TOF) camera. The various camera types can be combined to achieve a better recognition rate when detecting gestures. According to one example, a conventional video camera can be combined with a TOF camera to obtain depth or distance information in addition to the video image from the video camera.

[0040] Below and also in the cases relating to Fig. 2 to Fig. In the exemplary embodiments described in Figure 5, the invention is described using one or more video cameras that provide camera images or video signals in which gestures of the operating personnel are detected. However, the invention is not limited to this. Rather, the respective camera types can provide different camera data in which gestures can be detected. For example, a thermal imaging camera can provide temperature data as camera data, whereby gestures can be detected, for example, based on the temperature distributions contained in the camera data.

[0041] The control device 20 is adapted to detect gestures of the operating personnel U in the images captured by the camera 30 or in the video signal captured by the camera 30. If the control device 20 detects one or more gestures of the operating personnel U in the images or in the video signal, the control device 20 generates one or more corresponding control commands. The control commands are then transmitted to the respective inspection unit 60.

[0042] According to a further embodiment of the invention, the operating unit B comprises a microphone 40. The control device 20 is adapted to detect voice commands from the operating personnel U in the audio data recorded by the microphone 40 or in the audio signal recorded by the microphone 40. If the control device 20 detects one or more voice commands from the operating personnel U in the audio data or in the audio signal, the control device 20 generates one or more corresponding control commands. The control commands are then transmitted to the respective inspection unit 60.

[0043] According to yet another embodiment of the invention, the operating unit B comprises at least one camera 30, such as at least one video camera and / or at least one time-of-flight camera, and at least one microphone. The control device 20 is adapted to detect gestures of the operating personnel in the image or video data and to detect voice commands in the audio data or audio signals. The control device 20 then generates corresponding control commands for the detected gestures and voice commands, which are transmitted to the respective inspection unit 60.

[0044] According to the invention, it is thus possible to control the inspection units 60 of a sewer inspection and / or maintenance system using gestures and / or voice commands. This eliminates the need for costly remote controls, control panels, etc., which are also subject to wear and tear and therefore require regular maintenance. Furthermore, the operation of the inspection units 60 is thus considerably simplified, can be accomplished more intuitively, and operating errors (e.g., incorrect movement of a joystick or pressing the wrong buttons) can be effectively prevented.

[0045] Another advantage is that, especially (but not exclusively) when using voice commands, the hands are free and can be used for other tasks. This eliminates the need for dedicated personnel dedicated exclusively to operating the control panels.

[0046] The control device 20 comprises a control unit 25 and a video evaluation unit 23 and / or an audio evaluation unit 24. If the operating unit B has only one camera 30, it is sufficient if the control device 20 has only the video evaluation unit 23 in addition to the control unit 25. If the operating unit B has only one microphone 40, it is sufficient if the control device 20 has only the audio evaluation unit 24 in addition to the control unit 25. If a camera 30 and a microphone are present, it is of course advantageous if the control device 20 has both the video evaluation unit 23 and the audio evaluation unit 24 in addition to the control unit 25.

[0047] In one embodiment of the invention, multiple cameras 30 can be provided, for example, to record the operating personnel U and their gestures from different angles. The cameras 30 can all be coupled to the video evaluation unit 23.

[0048] The video evaluation unit 23 is adapted to detect gestures of the operating personnel U in the images or video signals recorded by the camera(s). For this purpose, the video evaluation unit 23 can use known gesture recognition algorithms.

[0049] However, it has proven advantageous if the video evaluation unit 23 comprises an artificial intelligence (AI model), in particular an artificial neural network, with which the detection or recognition of gestures is accomplished. The AI ​​model for recognizing gestures can be trained or taught with a number of gestures to be detected. The training data set includes those gestures that are required for controlling the sewer inspection and / or maintenance units 60. Gesture recognition using the AI ​​model must function reliably even under adverse or unfavorable environmental conditions, such as rain, twilight, poor lighting conditions, or different distances between the operating personnel and the camera. It is therefore advantageous if the training data set for each gesture to be detected includes these under different environmental conditions. If the gestures are hand gestures (e.g.For gestures such as a "thumbs up" gesture (e.g., upward hand movement), it is advantageous if the training dataset includes the respective gestures both with and without gloves (especially work gloves). It is advantageous if the training dataset includes gestures from different people.

[0050] If the video evaluation unit 23 detects a gesture in the image data or video signals recorded by the camera, the video evaluation unit 23 generates information about the detected gesture and transmits this information to the control unit 25. Based on this information, the control unit 25 can then generate one or more control commands and transmit them to the respective sewer inspection and / or maintenance unit 60.

[0051] The audio evaluation unit 24 is adapted to detect voice commands of the operating personnel U in the audio data or audio signals recorded by the microphone. For this purpose, the audio evaluation unit 24 can use known speech recognition algorithms.

[0052] However, it has proven advantageous if the audio evaluation unit 24 comprises an artificial intelligence (AI model), in particular an artificial neural network, with which the detection or recognition of voice commands is accomplished. The AI ​​model for recognizing voice commands can be trained or taught with a number of voice commands to be detected. The training data set includes those voice commands required to control the sewer inspection and / or maintenance units 60. Voice command recognition using the AI ​​model must function reliably even under adverse or unfavorable environmental conditions, such as ambient noise (e.g., attempted due to rain) or different distances between the operating personnel and the microphone. It is therefore advantageous if the training data set includes these under different environmental conditions for each voice command to be detected.It is advantageous if the training data set includes voice commands from different people.

[0053] If the audio evaluation unit 24 detects a voice command in the audio data or audio signals recorded by the microphone, the audio evaluation unit 24 generates information about the detected voice command and transmits this information to the control unit 25. Based on this information, the control unit 25 can then generate one or more control commands and transmit them to the respective channel inspection and / or maintenance unit 60.

[0054] The control unit 25 can be adapted to generate the control commands depending on the channel inspection and / or maintenance units 60 coupled to it.

[0055] Example: The control unit 25 is coupled to only one carriage on which a rotatable camera head is mounted. If the control unit 25 receives, for example, information about the voice command "Rotate" (or information about a corresponding gesture – e.g., a rotating hand), the control unit 25 generates the control command "Rotate camera head." However, if the control unit 25 receives information about the voice command "Turn on milling head," this information is discarded by the control unit 25, and no corresponding control command is generated, since no milling head is mounted on the carriage.

[0056] The control unit 25 may further be adapted to assign information about a voice command (or gesture) that would be valid for multiple sewer inspection and / or maintenance units 60 to one of the sewer inspection and / or maintenance units 60 and then generate the corresponding control command for this sewer inspection and / or maintenance unit 60. This assignment may be performed using heuristics or predefined rules.

[0057] Example: The control unit 25 is coupled to a carriage on which a lift is arranged (as for example in Fig. 1). Furthermore, the control unit 25 is coupled to a lowering device with which the trolley can be lowered into a sewer shaft. The trolley is already located in the sewer. If the control unit 25 now receives information about the voice command "Lower" (or information about a corresponding gesture - e.g. a thumbs down), the control unit 25 could generate the control command "Lower" for the lowering device in order to lower the trolley into the sewer shaft, or for the trolley's lifter in order to lower the lifter. Since the trolley is already in the sewer, the control unit 25 generates the control command "Lower" for the trolley's lifter and transmits this to the trolley or to the trolley's lifter.If such an assignment cannot be made unambiguously by the control unit 25 independently, the operating personnel can be asked for which channel inspection and / or maintenance units 60 the voice command (or gesture) should be used (e.g. via the feedback unit described below).

[0058] It is advantageous if the video evaluation unit 23 and / or the audio evaluation unit 24 are adapted to determine a probability for a recognized voice command or a recognized gesture, respectively, which indicates the probability with which the recognized gesture was correctly recognized (by the respective AI model). If the determined probability is above a predetermined tolerance value (e.g., 90%), the gesture or voice command can be considered correctly recognized, and corresponding information can be transferred to the control unit 25.

[0059] If the determined probability is below the predetermined tolerance value (but above a second tolerance value, e.g. 70%), then it can be assumed that the gesture or the voice command was recognized, but the accuracy is not sufficient to generate a control command. In one embodiment of the invention, it is now provided that the video evaluation unit 23 or the audio evaluation unit 24 generates warning data and transmits the generated warning data to a feedback unit 50 for output by the feedback unit 50. The feedback unit can be a loudspeaker or a display or a combination of both. If, for example, a "raising hand" gesture is recognized and the probability of correct recognition is between 70% and 90%, an animated raising hand can be output at the feedback unit (on the display) along with a request to confirm or reject the recognized gesture.If the recognized gesture is correct, the user U can confirm this (for example, by pressing a corresponding button on the display) or reject the gesture (for example, by pressing a corresponding button on the display). The user's input (confirmation or rejection) is then transferred to the video evaluation unit 23 along with recorded video data. The video evaluation unit 23 can then feed this information to the AI ​​model for further training. The same procedure can be applied for voice commands whose probability of correct recognition lies between the two aforementioned tolerance values. This, on the one hand, prevents control commands from being generated for gestures or voice commands that were not recognized well enough (incorrect control of the inspection units 60 is thereby avoided), and on the other hand, the respective AI models can be continuously trained and thus improved.

[0060] If the probability of a correctly recognized gesture or voice command is below the second tolerance value, the gesture or voice command can be rejected.

[0061] Overall, this provides a sewer inspection and / or maintenance system that can be controlled using gestures and / or voice commands. Gesture and / or voice command recognition is achieved using specially trained AI models that can be continuously "retrained" during operation.

[0062] Fig. 2 shows a schematic representation of an embodiment of a sewer inspection and / or maintenance system 10 according to the invention.

[0063] The sewer inspection and / or maintenance system 10 here comprises an inspection vehicle 70, which is located on the surface near the sewer shaft. At least one camera 30, at least one microphone 40, and a feedback unit 50 are arranged on the inspection vehicle 70. A carriage 60 with a lift and an inspection camera arranged thereon is located in the sewer. The carriage is operatively coupled to the control device 20, which is located in the inspection vehicle 70. The at least one camera 30, the at least one microphone 40, and a feedback unit 50 are also coupled to the control device 20. The control device 20 is configured as described with reference to Fig. 1 described.

[0064] The operating personnel U is also located on the surface near the manhole. The operating personnel U is also located in the camera's field of view or within the microphone's detection range. Regarding the microphone, it may be advantageous if it is designed as a directional microphone to capture only a specific area, preferably the area that roughly corresponds to the camera's field of view.

[0065] Using gestures and / or voice commands, the operating personnel U can now control the trolley, the trolley's lift, and the inspection camera mounted on the lift.

[0066] Fig. 3 shows an embodiment of an operating unit of a sewer inspection and / or maintenance system according to the invention.

[0067] In contrast to the Fig. 2, in which the control unit B, i.e. the cameras 30, the microphones 40 and the feedback unit 50, are arranged on the inspection trolley 70, the control unit B is controlled according to the Fig. 3 is formed by a portable computer, such as a tablet computer.

[0068] The portable computer (control unit B) has a camera 30 and a microphone 40. Furthermore, the portable computer has a display unit 51. A symbol serving as a feedback unit can be displayed on the display unit 50. This symbol can be configured, for example, in the form of a traffic light. If a gesture / voice command is correctly recognized, the traffic light can display "green." If a gesture / voice command is correctly recognized with a probability between the two aforementioned tolerance values, the traffic light can display "yellow." In all other cases, the traffic light is "red."

[0069] Two buttons 52 and 53 are also displayed on the display unit. These two buttons are only active when the traffic light is "yellow," i.e., when the user U must confirm or reject the correct recognition of a gesture / voice command. If the user confirms correct recognition, they press button 52; otherwise, they press button 53.

[0070] If the traffic light is “green” or “yellow”, information about the recognized gesture or voice command can also be output on the display unit 51.

[0071] The portable computer is advantageously coupled to the control device 20 via a wireless communication connection K.

[0072] Fig. Figure 4 shows a flowchart for a method for operating a sewer inspection and / or maintenance system. The method described here serves to better understand the sewer inspection and / or maintenance system according to the invention.

[0073] In a first step S1, video data and / or audio data from an operator are recorded. The video data and / or audio data are continuously fed as a video stream and / or audio stream to a video evaluation unit and / or an audio evaluation unit (step S2).

[0074] In step S3, the video stream and / or audio stream are processed and evaluated by the video evaluation unit and / or the audio evaluation unit, with gestures being extracted from the video stream and / or voice commands being extracted from the audio stream in step S4. The extraction can be carried out using appropriately trained AI models, as described with reference to Fig. 1 is described.

[0075] In step S5, it is determined whether an extracted gesture / voice command was correctly recognized. Here, a probability of correct recognition is determined or calculated as described above.

[0076] In the event that the gesture / voice command was correctly recognized and the probability of this is above a predetermined tolerance value, a control command is generated in step S9, which is then transmitted to the corresponding inspection unit in step S10.

[0077] In the event that a gesture / voice command has been correctly recognized, but the probability of correct recognition is below the predetermined tolerance value but above a second tolerance value, information about the recognized gesture / voice command is output to a feedback unit (step S6).

[0078] The operator must confirm or reject the recognized gesture / voice command in step S7. If the confirmation is positive, a control command is generated in step S9, which is then transmitted to the corresponding inspection unit in step S10.

[0079] Both in the case of a confirmation and a rejection of the recognized gesture / voice command, the information on the recognized gesture / voice command together with the associated video data / audio data is transferred to the respective AI model in step S8 in order to train the AI ​​model with this data, preferably continuously.

[0080] In a further embodiment of the invention not shown in the figures, the camera 30 can comprise an infrared camera with infrared illumination (e.g., an IR LED). The operating personnel can be dressed in gloves that at least partially have an infrared-reflecting surface. This allows the operator's hand movements to be detected even more effectively.

[0081] Fig. 5 shows a schematic representation of another embodiment of a sewer inspection and / or maintenance system 10 according to the invention.

[0082] The sewer inspection and / or maintenance system 10 here comprises a cable drum 71, which is located on the surface near the sewer shaft. At least one camera 30, at least one microphone 40, and a feedback unit 50 are arranged on the cable drum 71. A carriage 60 with a lifter and a milling head arranged thereon is located in the sewer. The carriage is operatively coupled to the control device 20, which is located on the cable drum 71. The at least one camera 30, the at least one microphone 40, and a feedback unit 50 are also coupled to the control device 20. The control device 20 is configured as described with reference to Fig. 1 described.

[0083] The operating personnel U is also located on the surface near the manhole. The operating personnel U is also located in the camera's field of view or within the microphone's detection range. Regarding the microphone, it may be advantageous if it is designed as a directional microphone to capture only a specific area, preferably the area that roughly corresponds to the camera's field of view.

[0084] Using gestures and / or voice commands, the operating personnel U can now control the carriage, the carriage's lift, and the milling head mounted on the lift.

[0085] In a further embodiment of the invention, the entire operating unit B including the control device 20 can be designed as a mobile system that can be used for various sewer inspection and / or maintenance units or for various sewer inspection and / or maintenance systems 10.

[0086] The sewer inspection and / or maintenance system 10 may be a sliding system comprising a sliding eel that can be wound up or unwound on a cable drum. For example, an inspection camera may be arranged at the free end of the sliding eel. The operating unit B and the control device 20 may be arranged on the cable drum. Alternatively, the operating unit B may also be designed as a mobile operating unit (e.g., a tablet), as shown in Fig. 3, and the control device 20 can be arranged on the cable drum. The mobile control unit can be connected to the control device 20 wirelessly or by cable.

[0087] As described at the beginning, different camera types can be used or different camera types can be combined with each other. Reference symbol: 10 Sewer inspection and / or maintenance system 20 Control device 23 Video evaluation unit 24 Audio evaluation unit 25 Control unit 30 Camera (e.g. video camera or TOF camera) 40 microphone 50 Feedback unit (e.g. display or speaker) 51 display unit 52 Confirm button 53 Reject button 60 Sewer inspection and / or maintenance unit (60a... 60n) 70 inspection vehicles 71 cable drum 100 channels B Control unit K Communication connection (e.g. wireless) S1... S10 steps of the procedure U Users / Operating personnel

Claims

[1] Sewer inspection and / or maintenance system (10), comprising - a control device (20), - at least one channel inspection and / or maintenance unit (60) operatively coupled to the control device (20), and - an operating unit (B) for a user (U) which is operatively coupled to the control device (20), wherein - the operating unit (B) comprises at least one camera (30) and / or at least one microphone (40), - the operating unit (B) and the control device (20) are designed such that the control device (20) receives camera data from the camera (30) and / or audio data from the microphone (40) from the operating unit (B), and - the control device (20) is adapted, - to recognize at least one gesture of the user (U) in the received camera data and / or to recognize at least one voice command of the user (U) in the received audio data, - to generate at least one control command based on the at least one recognized gesture and / or based on the at least one recognized voice command, and - to transmit the at least one generated control command to the at least one channel inspection and / or maintenance unit (60). [2] Sewer inspection and / or maintenance system according to the preceding claim, wherein the control device (20) comprises a control unit (25) and wherein the control device (20) further comprises - a video evaluation unit (23) which is adapted to recognize the at least one gesture in the received camera data and to transmit information about the at least one recognized gesture to the control unit (25), and / or - an audio evaluation unit (24) which is adapted to recognize the at least one voice command in the received audio data and to transmit information about the at least one recognized voice command to the control unit (25), wherein the control unit (25) is adapted to generate the at least one control command based on the information about the at least one recognized gesture and / or based on the information about the at least one recognized voice command. [3] Sewer inspection and / or maintenance system according to the preceding claim, wherein - the video evaluation unit (23) comprises an artificial intelligence, in particular an artificial neural network, which is designed to recognize the at least one gesture in the received camera data, and / or - the audio evaluation unit (24) comprises an artificial intelligence, in particular an artificial neural network, which is designed to recognize the at least one voice command in the received audio data. [4] Sewer inspection and / or maintenance system according to one of the preceding claims, wherein the operating unit (B) further comprises a feedback unit (50) operatively coupled to the control device (20), wherein the control device (20) is adapted - to determine an initial probability for a gesture recognized in the received camera data, which indicates the probability with which the recognized gesture was recognized correctly, and / or - to determine a second probability for a voice command recognized in the received audio data, which indicates the probability with which the recognized voice command was correctly recognized, and - to generate warning data if the first probability and / or the second probability is below a predetermined tolerance value, and to transmit the generated warning data to the feedback unit (50) for output by the feedback unit (50). [5] Sewer inspection and / or maintenance system according to the preceding claim, wherein the warning data comprise information about which gesture and / or voice command was recognized with a first and / or second probability that is below the predetermined tolerance value, wherein the feedback unit (50) - a display unit for optical output of the warning data and / or a loudspeaker for acoustic output of the warning data, and - comprises an input unit for receiving user inputs, wherein the gesture and / or voice command recognized below the predetermined tolerance value can be confirmed or rejected with a user input, wherein the feedback unit (50) is adapted to transmit the received user input to the control device (20), and the control device (20) is adapted to generate the at least one control command only upon confirmation. [6] Sewer inspection and / or maintenance system according to one of the two preceding claims and according to claim 3, wherein the control device (20) is further adapted to transmit the received user input together with the associated camera data and / or associated audio data to the artificial intelligence of the video evaluation unit (23) and / or to the artificial intelligence of the audio evaluation unit (24) in order to train the respective artificial intelligence. [7] Sewer inspection and / or maintenance system according to one of the preceding claims, wherein the control device is adapted to automatically assign a gesture and / or a voice command to a specific sewer inspection and / or maintenance unit (60) if the gesture and / or the voice command is assignable to a plurality of sewer inspection and / or maintenance units (60). [8] Sewer inspection and / or maintenance system according to one of the preceding claims, wherein the camera (30) comprises a video camera, a thermal imaging camera, a stereo camera or a time of flight camera (TOF camera). [9] Sewer inspection and / or maintenance system according to one of the preceding claims, wherein the operating unit (B) is a mobile operating unit, in particular a tablet.

Images