METHOD AND DEVICE FOR MACHINELY DETERMINING THE FUNCTIONAL STATUS OF CARRYING ROLLERS OF A BELT CONVEYOR SYSTEM

Description

[0001] The invention relates to a method for mechanically determining the functional state of support rollers of a belt conveyor system during operation of the belt conveyor system, wherein at least one unmanned vehicle is provided with at least one imaging sensor system, by means of which the belt conveyor system can be detected at least in sections by sensors in the form of image data, wherein image data of at least a partial area of ​​the belt conveyor system are detected as thermal image data.Furthermore, the invention relates to a method for identifying functionally impaired support rollers of a belt conveyor system on the basis of the aforementioned method and to a device for mechanically determining the functional state of support rollers of a belt conveyor system during operation of the belt conveyor system, the device comprising at least one unmanned vehicle movable along at least a partial area of ​​the belt conveyor system with at least one imaging sensor system, by means of which the belt conveyor system can be detected at least in sections by sensors in the form of image data and which comprises at least one thermal imaging sensor device for detecting thermal image data.

[0002] Belt conveyor systems (belt conveyors, conveyor belt systems, conveyor belt systems, belt conveyor systems, belt conveyor systems) are often used as stationary or semi-stationary continuous conveyors to transport bulk materials such as spoil, ores, fuels, building materials, and the like over a predetermined conveyor route (transport route). In a belt conveyor system, an endless conveyor belt (support belt, conveyor belt) is set in circular motion by at least one drive station (drive drum). The conveyor belt is guided as the upper run between two deflection stations, at which the conveyor belt is deflected by a deflection roller, from one (starting) deflection station to another (ending) deflection station. From the ending deflection station, the deflected conveyor belt is returned as the lower run to the starting deflection station, where it is deflected again and then guided again as the upper run.Typically, the upper run forms the load run (working run, tension run) and the lower run forms the slack run. Along the entire length of the conveyor belt, the upper and lower runs are supported by idlers (track rollers), which serve as support and guide elements.

[0003] For bulk materials, due to their higher load capacity and better material guidance, conveyor belts with a troughed upper run are used in addition to flat conveyor belts (in a flat belt conveyor system) (as a troughed belt in a troughed belt conveyor system, for example with a V-shaped trough for two support rollers each, or with a U-shaped trough, for example with three, four, five or more support rollers) or a rolled upper run (as a hose belt / pipe belt, in each case in a hose belt conveyor system) due to the higher load capacity and better material guidance. Depending on the conveying task, a conveyor belt guided in a troughed or rolled upper run is returned in the lower run in a flat form, in a troughed form, or in some cases even in a rolled form. A U-shaped trough is often found in the more heavily loaded upper run, and a flat form or a V-shaped trough in the returning lower run.

[0004] The endlessly circulating conveyor belt must be supported and guided on the conveying section in the loaded upper run as well as in the returned lower run (there in the unloaded or loaded state). Support is particularly important for the loaded section of the conveyor belt. Support rollers in a support structure, a support roller station (track roller station), are used as load-bearing and guide elements for support and guidance. The support rollers enable rolling contact between the support structure and the conveyed material. Support rollers have a running surface area that is bordered on the front side by two cap areas, the support roller bottoms. The running surface area is usually formed by a cylindrical steel tube section, although other materials and geometries are also possible.The cap areas are usually cast caps or sheet metal caps that are pressed or welded into the front openings of the running surface area. Each cap area contains a bearing support for accommodating an axle passing through the support roller or an axle journal extending into the inner area of ​​the support roller. The bearing supports are roller bearings (which are usually integral components of the cap area), allowing the support rollers to rotate via an internal bearing. Typical designs of support rollers are subject to industrial standards and guidelines (such as DIN 15207, DIN 22112, or VDI 2341).

[0005] In a support roller station, at least one support roller is rotatably mounted and supported. Typically, several support rollers are arranged in a support roller station, especially in support roller stations for troughed or rolled conveyor belts. Support roller stations therefore represent support points for the conveyor belt and are typically designed as a rigid framework structure (support roller frame, idler roller frame, roller frame, roller block) in which the axes of the individual support rollers are arranged either rigidly or as a movable structure (support roller garland, idler roller garland). In such a movable structure, the axes of the individual support rollers are arranged so that they can move relative to one another, for example, by connecting them via chains.Idler roller stations can be individually positioned along the conveyor line, connected to neighboring Idler roller stations as part of a higher-level framework structure, or designed as an integral part of the support structure of the belt conveyor system. The total number of Idler rollers in a belt conveyor system is therefore determined by the number of Idler rollers per Idler roller station and the number of Idler roller stations along the entire conveyor line; this therefore depends on the spacing of the Idler rollers in the conveying direction.

[0006] The conveyor belt of a belt conveyor system typically has an upper cover plate and a lower cover plate, between which a load-bearing element (carcass) is arranged, designed as a ply. This carcass consists of one or more fiber layers embedded in rubber. The fiber layer serves primarily as a longitudinal member (tensile member, belt tensile member), so that fibers are arranged aligned in the longitudinal direction of the conveyor belt. During operation, the tensile forces acting in the conveying direction (in the transport direction, i.e., the direction of movement of the conveyor belt and thus in its longitudinal direction) are transmitted or dissipated via these fibers. The longitudinal members are usually textile fabrics, steel cables, high-strength polymers, or corresponding composite systems. Additional strength members are often provided as transverse reinforcement, which are aligned transversely to the longitudinal direction and reinforce the conveyor belt transversely to the conveying direction.Typically, these additional strength members are steel cords or textile fabrics. The upper cover and lower cover of the conveyor belt are rubber layers, with the upper cover (bearing layer, carrying side) serving to support the bulk material, and the lower cover (running layer, running side) in contact with the deflection rollers, drive pulleys, carrying rollers, and other guide and support elements. Due to the direct contact of the upper cover with the – sometimes sharp-edged – material being conveyed, the upper cover is generally wear-resistant and thicker than the lower cover. The thickness of the lower cover on the carrying roller side is in the range of 3 mm to 5 mm for many conveyor belts.

[0007] Depending on the area of ​​application and the type of material being conveyed, belt conveyor systems are exposed to different loads and environmental conditions. For example, the requirements for belt conveyor systems in open-cast mining differ from those for belt conveyor systems used underground; the same applies to the geographical location, such as subpolar regions or the tropics. The individual operating conditions of a belt conveyor system lead to specific wear of the individual components of such a system, which is why regular monitoring of the functional condition of all wearing components of the belt conveyor system at short intervals is necessary. Monitoring support rollers, and especially their bearings (rolling bearings), is particularly complex, as every belt conveyor system has a large number of support rollers, which – unlike the conveyor belt itself – are stationary and can therefore only be inspected at the respective installation site.Especially with idlers, the ambient climate and the type and quality of the material being conveyed have a significant impact on wear and tear, thus determining the quality requirements of the idlers. If the function of idlers – and especially their bearings – is impaired, the running resistance that the idlers offer to the movement of the conveyor belt in the conveying direction increases. This not only results in higher energy consumption when transporting the material, but also poses the risk of damage to the conveyor belt and therefore a total failure of the entire belt conveyor system. In order to detect and replace impaired idlers in good time before a total failure occurs, critical parameters of the idlers must be monitored at short intervals.

[0008] Increased running resistance of the support rollers results in higher bearing temperatures, so that the temperature of the support rollers and especially the bearings can serve as an indicator of possible functional impairments. For example, the functionality and service life expectancy of support roller bearings can deteriorate the hotter the bearing gets during conveyor operation. The wear that occurs on the bearing depends, among other things, on the lubrication and the quality of the support rollers used (especially the quality of the bearings). Other factors that can lead to greater wear and thus to higher operating temperatures on the bearings include higher speeds (corresponding to high conveyor belt speeds and / or small support roller diameters), higher radial loads acting on the support rollers or greater distances between adjacent support roller stations.To monitor wear, the temperature of the idlers in belt conveyor systems is typically checked at regular intervals to determine their functional status. If a temperature increase is observed, a functional impairment can be detected early, allowing appropriate measures to be taken to minimize the downtime of the belt conveyor system.

[0009] To date, the temperature of the idlers on most belt conveyor systems has been measured by personnel who manually inspect all idlers of the belt conveyor system along the entire length of the conveyor line to ensure their proper functioning. In doing so, they walk, bicycle, or travel the distances between the idlers along the belt conveyor system. Such inspections are performed by visual inspection, acoustic testing, or temperature measurement, for example, using permanently installed temperature probes, a radiation thermometer (infrared thermometer, pyrometer), or a thermal imaging camera.

[0010] Industrial belt conveyor systems, with conveyor speeds of several meters per second and conveying capacities of several kilotons per hour, can have conveyor sections with lengths of 10 km or more. Each idler station can have several idlers, and the distances between adjacent idler stations can be as small as 2 m or less, meaning that a belt conveyor system can contain several thousand idlers. Particularly in the extraction of mineral resources, such as in open-cast mining, the total length of all belt conveyor systems at one site can easily exceed 100 km. Given the resulting large number of idlers, having a service employee walk or drive along all the belt conveyor systems in a regularly repeated sequence involves a lot of work and therefore costs.To reduce costs, increasing the inspection interval is also being considered. However, if the individual idlers are inspected at too long intervals, the risk of failure of the belt conveyor system increases.

[0011] Furthermore, manual functional testing is subject to a certain degree of variability in the specific execution of the inspection, depending on the respective service employee. This negatively impacts the reproducibility of the determined functional status of the idlers – and thus the reliability of the inspection overall. Furthermore, errors by the service employee cannot be ruled out when testing individual idlers (for example, the recorded data on the functional status, quality, and expected service life of the idlers or on the maintenance of the belt conveyor systems may be incomplete or incorrect).Overall, the conventional inspection of the functional status of the support rollers of a belt conveyor system is a process that lacks transparency, which, due to the associated uncertainty, can in turn lead to increased workload (e.g. due to duplicate determinations), which in turn entails increased costs.

[0012] Automated or semi-automated methods are known from the specialist publications "A UAV-Based Framework for Semi-Automated Thermographic Inspection of Belt Conveyors in the Mining Industry" by R. Carvalho et al. [Sensors, Vol. 20, No. 8 (2020), pp. 2243ff.] and "An inspection robot using infrared thermography for belt conveyor" by W. Yang and X. Zheng [submitted in 2019 to the 13th International Conference on Ubiquitous Robots and Ambient Intelligence (URAI) from 2016, pp. 1-6] in which thermal image data, after segmentation and classification, is subjected to data analysis to identify damaged idlers. However, with the current state of technology, the reliability of such an evaluation of thermal image data is not yet ideal. These specialist publications disclose a method according to the preamble of claim 1 and a device according to the preamble of claim 13.

[0013] Accordingly, it is an object of the present invention to provide a method that eliminates the aforementioned disadvantages and, in particular, allows the functional status of support rollers of a belt conveyor system to be determined more reliably and reproducibly than has previously been the case. At the same time, this reduces the required labor and costs, and offers greater transparency regarding the functional status of the support rollers. It also enables the operator and maintenance personnel to document the condition of the support rollers and their maintenance history. This also aims to achieve simple and reliable monitoring of the functional status of support rollers, which can also be carried out at short intervals over a longer period of time without requiring high personnel costs.This monitoring also aims to detect wear on support rollers and the occurrence of any anomalies that could impair function at an early stage, before critical situations arise, in order to proactively address these situations and reduce maintenance costs and downtime compared to conventional methods. Furthermore, it was desirable to provide a method that would allow predictions to be made regarding the future development of the functional status of individual support rollers over time. Based on these predictions, it would be possible to recommend specific maintenance plans that would help optimize plant operation efficiency and minimize downtime. A further objective is to provide a device that offers the aforementioned advantages.

[0014] These objects are achieved by a method for mechanically determining the functional status of support rollers of a belt conveyor system during operation of the belt conveyor system, a method for identifying functionally impaired support rollers of a belt conveyor system, which includes the method for mechanically determining the functional status of support rollers, and a device for mechanically determining the functional status of support rollers of a belt conveyor system during operation of the belt conveyor system, with the features specified in the independent claims. Advantageous further developments emerge from the subclaims, the following description, and the drawings.

[0015] The invention relates to a method for the mechanical determination of the functional status of support rollers of a belt conveyor system during operation of the belt conveyor system, wherein at least one unmanned vehicle is provided with at least one imaging sensor system, by means of which the belt conveyor system can be at least partially sensor-captured in the form of image data, wherein image data of at least a partial area of ​​the belt conveyor system is captured as thermal image data, wherein at least one detection image area position is automatically determined in the captured image data of the belt conveyor system, in which at least a partial area of ​​a support roller is imaged, for each determined detection image area position from the image data, an analysis image area position is automatically determined in the thermal image data, and thermal image data is automatically analyzed in each defined analysis image area position,to automatically determine the functional status of support rollers, wherein the image data captured by the imaging sensor system includes, in addition to thermal image data, also photo image data, wherein image data of at least one partial area of ​​the belt conveyor system is also captured as photo image data, the captured image data of the belt conveyor system, in which at least one detection image area position is automatically determined, are photo image data in which photo image areas are automatically detected as image data areas, and the position of each detected photo image area is automatically provided as a detection image area position. Of particular importance here are partial areas of support rollers that contain their bearings (rolling bearings).

[0016] A method for the mechanical determination of the functional status of support rollers in a belt conveyor system is a process in which the respective functional status of the individual support rollers is determined mechanically, i.e. with the significant involvement of a machine. The functional status is the status of a support roller with regard to the extent to which it can fulfill its defined task, i.e., supporting the conveyor belt in a rolling movement that offers at most slight (ideally negligible) mechanical resistance to a longitudinal movement of the conveyor belt; a support roller that does not fulfill this task or does not fulfill it sufficiently is functionally impaired. In such a method, the functional status of all support rollers is preferably determined.Since the method is specifically limited to determining the functional status of the support rollers, any functional impairments of other components of the belt conveyor system, such as damage to the conveyor belt, cannot be detected. However, it may be possible to supplement the method according to the invention with further measures, thus creating a combined new method in which, in addition to the functional status of the support rollers, functional impairments of other components can also be determined.

[0017] It is important that the method according to the invention for determining the functional status is used during operation of the belt conveyor system, so that the operation of the belt conveyor system does not have to be interrupted for the functional status determination. The present functional status determination is based on the heat development at the support rollers during operation of the belt conveyor system, so the present method is particularly suitable for application during ongoing operation.

[0018] For the method according to the invention, at least one unmanned vehicle with at least one imaging sensor system is provided, and possibly also several unmanned vehicles. A vehicle is understood to be any means of transport or means of operation, regardless of whether it is suitable for transporting persons or not. In principle, this term includes in particular land vehicles (ground-based vehicles), for example road vehicles (such as those with tires), off-road vehicles (such as those with tires or tracks), rail vehicles, aircraft (aircraft) and watercraft, as well as in particular track-bound vehicles (rail-bound vehicles with or without track guidance, such as rail vehicles or cable cars) and trackless vehicles. It is particularly advantageous if, in the present case, a hovering platform (multicopter, in particular a quadrocopter (quadricopter, quadrotor)) is used as the vehicle.Different designs are also possible, such as a gondola hanging from a guide cable and the like. An unmanned vehicle (drone) is a vehicle in or on which no person is or needs to be present while it is moving; in the broadest sense, this term also includes mobile robots. The unmanned vehicle can take any design, for example as a remotely operated vehicle (ROV) or as a semi-autonomous unmanned vehicle, but above all as an autonomous unmanned vehicle. A remotely operated vehicle is a vehicle that is steered and controlled by operators who are not, however, in or on the vehicle; such remote control is often implemented as a radio remote control.The operator can be a human being or an external control system that is not located on the vehicle; such an external control system can be implemented as an external data processing device. A semi-autonomous vehicle is a partially automated vehicle in which an operator must perform individual tasks during operation, which can be done via remote control, for example. In contrast, an autonomous vehicle is a vehicle that moves independently and thus operates independently (i.e., without the direct intervention of a driver or operator) and behaves largely autonomously. In an autonomous vehicle, the vehicle has a control system that is located on the vehicle. Typically, an autonomous vehicle follows a predetermined route, whereby the route can be predetermined, for example, in the form of waypoints to be reached.The vehicle navigates to these waypoints using a satellite-based positioning system, such as a receiver for a navigation satellite system such as GPS, or an optical image acquisition system. An autonomous vehicle typically has additional systems that enable autonomous driving (automated driving, piloted driving), such as a collision avoidance system or an automatic alignment system to ensure optimal alignment at specified waypoints for the best possible acquisition of image data of the target objects.

[0019] The imaging sensor system comprises a sensor arrangement (detector) which, particularly in an imaging process, generates a quantitative two-dimensional or three-dimensional image from measured variables of a real object and its surroundings and outputs this image data. These measured variables and / or information derived therefrom are spatially resolved by the sensor system and encoded using additional parameters such as intensities and / or colors, and output as (particularly digital) image data, i.e., as data sets in which the respective image information of the imaged objects is represented coherently. Two-dimensional images are primarily image representations (images) in which the pixels are arranged two-dimensionally; such two-dimensional images can, for example, be generated and displayed monoscopically or stereoscopically.In contrast, three-dimensional images are primarily image representations in which the pixels are arranged three-dimensionally, for example in the form of point clouds. In addition to pixel representations, two-dimensional images and three-dimensional images can also be composed of elementary two-dimensional or three-dimensional geometric shapes (primitives, basic objects, basic graphic shapes, spatial reference shapes, for example for two-dimensional vector graphics or 3D models) or combinations thereof with pixels. Such a sensor system has at least one sensor device, but can also include additional sensor devices. The measured variables can be, in particular, the wavelength and intensity of the electromagnetic radiation emanating from the object (emitted, reflected, or scattered) within a wavelength range of 100 nm to 1 mm.In this case, the sensor devices can be image sensor devices such as cameras, for example, a still camera for capturing stationary images (single frames) or a motion picture camera for capturing moving images (films, videos), in particular a thermal imaging sensor device, a light image sensor device, or an ultraviolet image sensor device. A thermal imaging sensor device can be, for example, a thermal imaging camera (thermography camera, thermal camera, thermal imaging camera, thermal imaging camera, infrared camera, thermal imaging device), by means of which primarily infrared radiation (IR radiation, thermal radiation) from a wavelength range of approximately 700 nm to 1 mm is detected and provided as thermal image data, in particular from the near infrared range (NIR) from 700 nm to 1,000 nm or from the mid-infrared range (MIR) from 3.5 µm (micrometers) to 15 µm (micrometers).A light image sensor device can be, for example, a light image camera (optical camera, RGB camera), by means of which primarily visible light from a wavelength range of approximately 380 nm to 780 nm (in particular up to 700 nm) is captured and provided as light image data. An ultraviolet image sensor device can be, for example, an ultraviolet camera (UV camera), by means of which primarily ultraviolet radiation (UV radiation) from a wavelength range of approximately 100 nm to 380 nm is captured and provided as ultraviolet image data.In a sensor array, the image in the imaging process can also be generated from other measured variables, for example, from the propagation time or frequency shift of a signal emitted by the sensor array and reflected by the environment. From this signal, spatially resolved distance information is obtained using a scanning process (such as laser scanning, lidar / ladar, sonar, or radar) and provided as range image data. Furthermore, the unmanned vehicle can have additional sensor devices in addition to the imaging sensors to record other parameters from or in the environment.

[0020] According to the invention, it is further provided that the belt conveyor system can be recorded at least in sections by sensors in the form of image data using the sensor system. Image data from at least a partial area of ​​the belt conveyor system is recorded as thermal image data. Such thermal image data often only contain an intensity value as a measured variable, which is correlated with the intensity of the infrared radiation emitted by the surface of the object being imaged and thus represents the spatial profile of the surface temperature of the object. In addition to the thermal image data, other image data is also recorded by sensors using the sensor system, namely light image data. For recording purposes, the unmanned vehicle can be moved along at least a partial area of ​​the belt conveyor system, for example by being moved by its own drive on the vehicle itself or by an external drive.Capturing at least sections of the belt conveyor system can also mean that the entire belt conveyor system can be captured (e.g. in the form of moving images recorded as the unmanned vehicle travels along the entire belt conveyor system). Due to the size of the vehicle, however, the sensors usually only capture a portion of the belt conveyor system at a time using the sensors' devices (and thus by sensors). In this case, it is possible, for example, that no image data is captured for portions of the belt conveyor system that do not contain any support rollers. Irrespective of this, in practice several unmanned vehicles can certainly be used to capture the support rollers along the entire length of the belt conveyor system, or the capture of the entire belt conveyor system can be split up into several individual steps, each of which only captures a portion of the belt conveyor system using sensors.

[0021] According to the invention, it is further provided that at least one recognition image area position is automatically determined in the acquired image data of the belt conveyor system, in which at least a partial area of ​​a support roller is depicted.Automatic means that the system carries out the specific action - in this case the determination of the recognition image area position - as part of an autonomous process according to a defined plan or in relation to defined conditions, without the direct intervention of human operators being required; this does not rule out the possibility that the automatic execution of the specific action can be interrupted by the intervention of human operators, for example in a dangerous situation, or that the intervention of human operators is possible (or even necessary) for steps outside the automatically performed action, or that the action was prepared by human operators before the specific execution, for example when programming the corresponding control or regulation system or when training in connection with machine learning.

[0022] In this case, at least one detection image area position is automatically determined in the image data. At least one detection image area position means that one or more detection image area positions are determined in the image data; if no detection image area position is determined in the image data, it is not necessary to submit this data for further processing. A detection image area position is the position of an image data area (i.e., an area within the image data) in which a specified target object or parts thereof were detected, for example, as a result of an image comparison or pattern recognition. During an image comparison, captured image data is directly compared with specified comparison image data that was recorded at an earlier point in time.During pattern recognition, the acquired image data (if necessary after pre-processing for data reduction, i.e. to reduce the complexity of the image data by removing unwanted or irrelevant image data, for example as part of filter operations, threshold operations, averaging operations or normalization operations) is subjected to feature extraction and feature reduction, followed by a classification of the features into corresponding classes and image recognition. In the case of image data from a belt conveyor system, in addition to the support rollers (including the bearings of these rollers), other components of the system can be considered as corresponding classes for the target objects to be identified, for example scaffolding elements of the support structure, foundations, conveyor belts, deflection stations, shelters, guide plates, cables, hoses and the like, but also the conveyed material or frequently occurring structures in the environment such as rocks.Image areas containing structures that are not associated with support rollers but rather with other target objects can thus be excluded from further analysis. Typically, when classifying objects based on multiple classes, a corresponding probability value is first assigned to each class, and the object is then finally assigned to the class with the highest probability value.

[0023] In this case, the primary objective is to identify support rollers and their parts, which is why classifying individual elements from the image data into the class "support rollers and their parts" might actually be sufficient. Classification is the grouping (dividing, assigning) of objects based on certain characteristics into predefined classes (groups, sets, categories), which together form a classification, i.e., a collection of abstract classes (concepts, types, categories) used for differentiation and organization. However, it is also possible and useful to assign the multitude of individual elements not just to a single class, but to a variety of different classes.Instead, it is also possible to automatically determine the at least one recognition image area position in the recorded image data of the belt conveyor system by automatically recognizing image data areas in the image data in which at least one recognizable object is depicted, automatically recognizing support rollers or partial areas of support rollers among the recognizable objects in the image data areas recognized in this way, and providing the position of the image data area in which the support roller or the partial area of ​​the support roller is depicted as the recognition image area position for each recognized support roller and for each recognized partial area of ​​a support roller.This means that only support rollers or sub-areas of support rollers are recognized among the detectable objects. The detectable objects are therefore not classified into other, equally possible classification classes, such as scaffolding elements of the support structure, foundations, conveyor belts, deflection stations, shelters, cables, hoses, conveyed material, or surrounding structures. Instead, detection would be limited to the detection of support rollers and their sub-areas. By limiting the detection of detectable objects to just one classification class—that of support rollers or their sub-areas—the detection step is greatly simplified.

[0024] If a partial area is determined in the image data in which at least a partial area of ​​a support roller (i.e. an object or several objects selected from the list comprising a support roller or a partial area of ​​a support roller, which also includes a plurality of support rollers or a plurality of their partial areas and combinations thereof) is imaged and recognized, then the position of the recognized partial area within the image data represents its recognition image area position. Typically, a recognition image area is selected such that the support roller or its partial area is arranged centrally therein.The shape of the detection image area, whose position within the image data represents the detection image area position, can be chosen arbitrarily; usually a simple geometric shape is chosen that encloses the detected support roller structure as a frame, for example a rectangle, a circle or an ellipse for two-dimensional image data sets, a cuboid, a sphere or a cylinder for three-dimensional image data sets (of course, more complex shapes are also possible, such as two-dimensional or three-dimensional splines).For two-dimensional image datasets, the respective detection image area position is conveniently specified in the form of two-dimensional coordinates of the detection image area, for example, as Cartesian coordinates or polar coordinates within the image data. For three-dimensional image datasets, it is specified in the form of three-dimensional coordinates of the detection image area, for example, as Cartesian coordinates, spherical coordinates, or cylindrical coordinates within the image data. For a rectangular frame, its position can be specified, for example, by the coordinates of opposite corners (i.e., in the form of two Cartesian coordinate pairs), and for a circular frame, by the center point and radius (i.e., a Cartesian coordinate pair and a distance).

[0025] According to the invention, the method is carried out such that the image data captured by the imaging sensor system also includes photo image data in addition to the thermal image data. Image data from at least one partial area of ​​the belt conveyor system is also captured as photo image data. The captured image data of the belt conveyor system, in which at least one recognition image area position is automatically determined, is photo image data in which photo data areas are automatically recognized as image data areas, and the position of a respectively recognized photo data area is automatically provided as the recognition image area position. Accordingly, the sensor system of the unmanned vehicle also comprises a photo image sensor device in addition to the thermal image sensor device.Image data is acquired by both image sensor devices, with both thermal image data and corresponding light image data being acquired from at least one partial area of ​​the belt conveyor system. The light image data is used to automatically determine the positions of detection image areas (detection image area positions) in which support rollers or parts of support rollers are imaged. Based on the detection image area positions determined from the light image data, the corresponding analysis image area positions are defined in the thermal image data, in which the thermal image data are then analyzed.Light image sensor devices generally have higher image resolutions than thermal image sensor devices. Furthermore, thermal image sensor devices generally only record one measurement variable (the surface temperature of the object, often coded as monochrome intensity or color), whereas light image sensor devices typically record at least two measurement variables (light intensity and light color / wavelength). Therefore, for the detection of target objects such as idlers, for example, through image comparison or pattern recognition, light image data generally provides more information than thermal image data. In addition, thermal image data usually only depicts the shape of the target object, whereas light image data depicts other structural elements such as surface textures and shadows, which are not visible in a thermal image.When using photographs, the higher recognition quality based on the distinctive and characteristic elements of the photographs can be transferred to the less distinctive thermal images. Therefore, recognition based on photographic image data is significantly more reliable than recognition based on thermal image data.

[0026] This method can also be combined with a method in which the captured image data also includes thermal image data - as a result of the parallel use of both types of image data for recognition - photo image data and additional thermal image data - the recognition accuracy can be further increased, which can be particularly advantageous in poor lighting conditions.

[0027] According to one embodiment of the invention, it is further provided that for each detection image area position determined from the image data, an analysis image area position is automatically defined in the thermal image data. The analysis image area position is the position in the thermal image data at which, after definition, an analysis is performed to determine the functional status of the support rollers displayed in the detection image area. The position information for the detection image areas obtained from the image data is thus transferred to the analysis image areas of the thermal image data. If the sensor device with which the image data used to identify the detection image areas were acquired is arranged spatially close to the thermal image sensor device and both sensor devices are aligned identically, the detection image area positions can even be adopted unchanged as corresponding analysis image area positions.If the sensor devices are different and have differences, for example, in terms of their image angles, image field sizes, positions and orientations, static geometric differences can be determined in advance and dynamic differences (for example, due to the arrangement of sensor devices on movable robot arms) can be determined at the respective time, so that the spatially corresponding analysis image area positions can be geometrically determined from the recognition image area positions.

[0028] Finally, according to one embodiment of the invention, thermal image data is automatically analyzed at each defined analysis image area position to automatically determine the functional status of support rollers. Within the scope of the thermal image data analysis, the respective functional status of each support roller depicted in whole or in part in the thermal image data is automatically determined based on the thermal information (temperature information) of points contained in the thermal image data within each analysis image area position (corresponding to the respective detection image area position).The fact that not all of the thermal image data from a thermal image is analyzed, but only a small, component-specific portion of this data, ensures that primarily relevant data is used to determine the functional status, thus enabling a reliable and simultaneously transparent automatic analysis with meaningful, highly significant results. Although the method according to the invention can be applied to determine the functional status of individual idler rollers, it offers particular advantages when determining the status of a large number of idler rollers, in particular when simultaneously determining the functional status of several idler rollers in a idler roller frame or a idler roller garland, as well as when determining the functional status of a idler roller across multiple images.Likewise, this method is particularly useful for troughed conveyor belts or rolled conveyor belts, since in such systems the support rollers are only slightly concealed by other elements of the belt conveyor system, making them particularly easy to access for automated inspection.

[0029] The method preferably comprises several steps in which an automatic evaluation is performed, for example, as part of a recognition or classification. Such an automatic evaluation occurs, for example, during the automatic recognition of image data areas (in particular during the automatic identification of image data areas in which recognizable objects are depicted, as well as during the automatic assignment of a recognizable object to a class, in particular to support rollers or sub-areas of support rollers), during the analysis of thermal image data in the analysis image area position (in particular during the automatic selection of thermal image data in the defined analysis image area positions and during the automatic assignment of determined temperature data to a functional state), during the automatic prediction of support roller service lives and the automatic creation of maintenance plans or image-based position determination.Within the method according to the invention, the individual steps in which such an automatic evaluation is performed can, in principle, be implemented arbitrarily, for example, through a fixed set of rules for evaluation conditions and corresponding limit values. However, it is particularly advantageous if individual or several – possibly even all – of these automatic evaluations are performed within the framework of methods based on machine learning (ML), which is why this embodiment is particularly preferred in each case. Machine learning refers to a method in which learning algorithms develop a complex statistical model from learned sample data (training data).This model can then be applied to data of the same type as the sample data or of a type comparable to the sample data to obtain an automatic evaluation without an evaluation scheme having to be explicitly programmed. Such a statistical model can be obtained, for example, on the basis of artificial neural networks (ANNs, KNNs). These are formed from a large number of nodes arranged as layers in data structures. During the learning phase, weights (weighting factors) at the connections between the nodes are typically varied until results are realized that correspond to the input data. If a criterion or structural element that can serve as the basis for an evaluation is known in advance, this criterion is incorporated into the learning process and communicated to the artificial neural network.Such an artificial neural network can in principle be implemented in the unmanned vehicle itself or separately from it, for example on an external data processing device, such as a server or in a computer cloud.

[0030] According to one embodiment of the invention, the method can be carried out in such a way that image data of at least one partial area of ​​the belt conveyor system is acquired as thermal image data by moving the at least one unmanned vehicle with the at least one imaging sensor system comprising at least one thermal image sensor device for acquiring the thermal image data along at least one partial area of ​​the belt conveyor system, image data of at least one partial area of ​​the belt conveyor system is acquired with the imaging sensor system and the image data comprise at least thermal image data, the at least one recognition image area position is automatically determined in the acquired image data of the belt conveyor system by automatically recognizing image data areas in which at least one partial area of ​​a support roller is imaged,and the position of the respectively detected image data area is automatically provided as the detection image area position of the respective support roller, for each determined detection image area position from the image data, an analysis image area position is automatically defined in the thermal image data, by automatically defining an analysis image area position of the support roller in the thermal image data for each determined detection image area position of a support roller from the image data, which spatially corresponds to the respective detection image area position from the image data, the thermal image data is automatically analyzed in each defined analysis image area position,by automatically determining temperature data of the respective support roller from the thermal image data in the specified analysis image area position of the support roller and automatically assigning the temperature data determined from the analysis image area position to a functional state of the respective support rollers.

[0031] In this case, at least one recognition image area position is automatically determined in the acquired image data of the belt conveyor system by first automatically recognizing image data areas in the acquired image data in which at least a partial area of ​​a support roller is depicted. The position of the respectively recognized image data area is then automatically provided as the recognition image area position of the respectively recognized support roller. Such automatic image data area recognition can be carried out, for example, according to the principle of the pattern recognition generally described above.The sensitivity to changes in the image acquisition conditions (e.g. due to changing lighting conditions, changed recording positions or temporary occlusion) that occurs in other recognition methods such as image comparison with historical comparison image data can thus be reduced, thereby further increasing the reliability and transparency of the determination of the functional status.

[0032] For each determined detection image area position of a support roller from the image data, an analysis image area position of the support roller is automatically defined in the thermal image data, which spatially corresponds to the respective detection image area position from the image data.If the sensor device with which the image data underlying the determination of the detection image areas was acquired is located close to the thermal imaging sensor device and both sensor devices are aligned in a comparable manner, then the detection image area position corresponds spatially to the respective analysis image area position in such a way that both positions are spatially identical within the respective image data; if the representation is identical (e.g. with regard to aperture angle, pixel resolution and display area size of the image data), this means that the detection image area position can be directly adopted as the respective analysis image area position; if there are deviations in the representation, additional corresponding conversions are required for the coordinate transformation.If the two sensor devices are spaced apart and / or the viewing angles differ, further geometric conversions are required to adjust for static and / or dynamic differences, such as parallax compensation, to determine and specify the spatially corresponding analysis image area position in the thermal image data from a detection image area position in the image data. Here, too, refraining from identifying target objects other than support rollers can result in additional efficiency gains and thus improved resource utilization.

[0033] When analyzing the thermal image data, temperature data for the respective support roller is automatically determined from the thermal image data in the defined analysis image area position of the support roller, and the temperature data determined from the analysis image area position is automatically assigned to a functional state of the respective support roller. The temperature data (thermal data) of the support rollers can be, for example, the spatially resolved local surface temperatures of the support rollers and, in particular, their bearings, but also statistically derived characteristic temperature parameters of the respective support roller that represent the current thermal state, such as minimum values, maximum values, mean values, central values ​​(medians), local cluster values, and parameters of the temperature distribution, especially their width, dispersion, and symmetry.Based on the surface temperatures and / or the characteristic temperature parameters of a support roller or a sub-area of ​​a support roller, the functional condition (functional status) of the respective support roller is determined and assigned to this support roller.

[0034] As already explained, individual, several, or all automatic evaluations (recognition steps) can be carried out within the framework of methods based on machine learning. According to a further embodiment of the invention, it has also proven advantageous if the method is carried out in such a way that the recognition quality in the automatic determination of the recognition image area position in the acquired image data, in particular the recognition quality of the automatic recognition of image data areas in which at least one recognizable object is depicted, in particular at least a partial area of ​​a support roller, and / or the recognition quality of the automatic recognition of support rollers or partial areas of support rollers in the recognized image data areas, is improved using a learning method carried out using an artificial neural network, in particular using a single-stage or multi-stage convolutional neural network.Accordingly, during the automatic determination of the recognition image area position in the acquired image data, the recognition quality can be improved using a learning process performed using an artificial neural network. The determination of the recognition image area position in the acquired image data includes at least one recognition step performed using an artificial neural network. This can, in particular, be an automatic recognition of image data areas in which at least one recognizable object is depicted (in particular a support roller or at least a partial area of ​​a support roller), or an automatic recognition of support rollers or partial areas of support rollers in the recognized image data areas.The recognition quality is a statistical measure of the accuracy of the recognition process and represents the proportion of correctly identified detections out of the total number of detections performed: the better the recognition, the higher (and therefore better) the recognition quality. It is particularly advantageous if the learning process is carried out using a single-stage or multi-stage convolutional neural network. A convolutional neural network (CNN) is an artificial neural network capable of processing input data represented as a matrix. For example, it has a filter layer (filter layer, convolutional layer) with one or more filters (typically 16 or 32 filters) for analyzing the data input as a matrix using a discrete convolution, the output of which is also in the form of a matrix.The output data of this filter layer can serve as input data for a subsequent filter layer, if required, followed by an aggregation layer (pooling layer) to pass the most representative signal to the subsequent layers. This structure can be repeated several times if necessary; the network is completed by at least one layer with the structure of a regular neural network (fully connected layer), which is connected to an output layer. The convolutional neural network can be single-stage or multi-stage, i.e., it can have only one layer of trainable weights or multiple layers of trainable weights.

[0035] According to a further embodiment of the invention, the method can be further implemented such that thermal image data is automatically analyzed at each specified analysis image area position by automatically selecting thermal image data that is thermal image data of the support rollers at the specified analysis image area positions, and automatically analyzing the selected thermal data at the specified analysis image area positions. In this case, thermal image data that is thermal image data of the support rollers is automatically selected within the specified analysis image area positions. The selected thermal data is then automatically analyzed at the specified analysis image area positions.This limits the analysis of the thermal image data within the analysis image area positions to those thermal image data that primarily originate from a support roller. It excludes thermal image data that, although located within the analysis image area positions, do not originate from the surface of the support roller. This data is therefore not thermal image data of the support roller and therefore does not provide information on its functional status. The automatic selection of those thermal image data from the analysis image area positions that are attributable to support rollers or their sub-areas can be performed using processes based on a fixed control scheme or on machine learning.In this way, the amount of data to be considered in the analysis can be further reduced, which not only reduces the effort required to determine the functional status, but also improves the quality of the determination of the functional status as well as the transparency of the determination overall.

[0036] According to a further embodiment of the invention, it is possible to automatically select thermal image data that lies in sub-areas within the defined analysis image area positions in which thermal image data are arranged in circular or circle-like contours and / or in which thermal image data correspond to temperatures of the same temperature level. The thermal data in the automatically selected sub-areas represent the thermal data of support rollers and, in particular, of their bearings and are subsequently automatically analyzed. In this case, thermal image data that lie within the defined analysis image area positions (and thus within the respective defined analysis image areas) and in which the thermal image data is arranged in circular or circle-like contours can be automatically selected as sub-areas.The thermal data are arranged in circular or circle-like contours, in particular, if the measured variables in the thermal data (e.g., the intensity values ​​representing surface temperatures) have a spatial distribution that has a shape derived from a circle or a perspectively distorted circle (e.g., an oval, in particular an ellipse), which includes subregions thereof (e.g., an arc of a circle, a partial circle, a circular segment, a ring, a partial ring, a section of a ring, a partial oval, a partial ellipse, and the like); this naturally also includes circular contours. Such circular or circle-like contours can be formed, in particular, concentrically or confocally.The image area (analysis image area) within each analysis image area position is examined for circular contours and a selection is made for those sub-areas that represent the average of all circular or circle-like contours or whose centers or focal points are identical or close to each other (i.e. at a sufficiently small distance from each other).

[0037] Likewise, thermal image data can be automatically selected that lie within the defined analysis image area positions and in which thermal image data correspond to temperatures of the same temperature level. The measured variable underlying the sensory acquisition of the thermal image data is the temperature (surface temperature) of the imaged object. If the thermal image data in sub-areas of the analysis image area correspond to temperatures above or below certain temperature thresholds or temperatures from a temperature interval defined by a pair of temperature thresholds, these thermal image data correspond to temperatures of the same temperature level. In particular, the thermal image data from the same temperature level can also be arranged in spatially connected areas. The respective temperature thresholds are defined in advance.For example, with a typical maximum operating temperature of the surface of support rollers (particularly their bearings) of a belt conveyor system of 70 °C, monitoring of the support roller at short intervals can be provided for temperatures in a range of more than 70 °C up to a maximum of 80 °C, replacement of the support roller can be planned for temperatures in a range of more than 80 °C up to a maximum of 90 °C, and immediate replacement of the support roller can be provided for temperatures in a range of more than 90 °C. If only support rollers in a potentially critical functional condition are to be monitored, then in this constellation temperature intervals can be set such that only temperatures above 70 °C are selected.However, if documentation of temperature data is also required for support rollers that are in proper working order, the temperature data can be determined for all areas and subjected to further analysis only in relevant or potentially interesting temperature ranges. For example, it is possible to only subject the thermal image data from the highest measured temperature interval to further analysis - for example, thermal image data from temperatures above 90°C. If this high thermal image data does not exist, then thermal image data from a temperature interval of more than 80°C up to a maximum of 90°C can be analyzed. If this also does not exist, thermal image data from a temperature interval of more than 70°C up to a maximum of 80°C can be analyzed, and only if this also does not exist.

[0038] According to the two variants described, a selection can be made based on a scheme of fixed rules, preferably using a process based on machine learning. The variants can be used alternatively, but can also be combined to improve detection quality. In this case, for example, in the defined analysis image area positions, the sub-areas in which thermal image data correspond to temperatures of the same temperature level are compared in terms of shape and extent with the sub-areas in which thermal image data are arranged in circular or circle-like contours. The ultimately automatically selected thermal image data is then the thermal image data of an area that results from the different variants as the common intersection (intersection) of the two sub-areas.

[0039] According to a further embodiment of the invention, the recognition quality in the automatic selection of the thermal image data, which are thermal image data of the support rollers, can be improved by means of a learning method carried out by means of an artificial neural network, in particular the recognition quality in the automatic selection of the thermal image data which lie in those sub-areas within the defined analysis image area positions in which thermal image data are arranged in circular or circle-like contours and / or in which thermal image data correspond to the temperatures of a same temperature level.

[0040] Accordingly, the automatic assessments on which the automatic selection of thermal image data is based are the thermal image data of the support rollers, i.e., they are performed within the framework of machine learning-based processes, with the recognition quality of this assessment being improved in learning processes performed using an artificial neural network. In particular, these may be assessments related to the automatic selection of thermal image data arranged in circular or near-circular contours and / or corresponding to temperatures of the same temperature level.

[0041] According to a further embodiment of the invention, the method can be carried out such that the image data of the belt conveyor system are each assigned to a position in the belt conveyor system and / or, for defined analysis image area positions, the thermal image data or evaluation information are each assigned to an individual support roller of the belt conveyor system, wherein the assignment is carried out in particular via radio-based position determination, via a comparison of captured image data with comparison image data, via a path detection of the unmanned vehicle and / or via an orientation detection of the imaging sensors. Such an assignment enables a clear identification of the object depicted in the image data. Accordingly, the image data of the belt conveyor system, i.e. the thermal image data, the photo image data and, if applicable, other image data, can each be assigned to a position as a spatial position in the belt conveyor system.A location is a point in space clearly defined by coordinates, for example, by geographical coordinates, by a relative position within the belt conveyor system, or by the path traveled by the vehicle. The location assigned to the image data can be the location of the objects depicted in the image data, the location of the unmanned vehicle, the location of the image sensor device used to capture the image data, or the location of another specified reference point. Instead or in addition, for specified analysis image area positions, the thermal image data or evaluation information can be assigned to an individual support roller of the belt conveyor system. This assignment is comparable to the relative position specification described above.

[0042] To assign the position, the corresponding spatial position must be determined. From an exact spatial position – possibly taking into account the viewing direction – the respective support roller can usually be clearly identified. The spatial position can be determined in particular using radio-based positioning, by comparing captured image data with comparison image data, by measuring the distance traveled by the unmanned vehicle, and / or by detecting the orientation of the imaging sensors. Radio-based positioning can be carried out, for example, using a navigation satellite system such as GPS or Galileo, a terrestrial radio navigation system such as Decca or Loran, a short-range navigation system, or a short-range transponder system such as RFID transponders at the support roller stations.The route of the unmanned vehicle can be recorded, for example, by precise tracking of the route traveled by the vehicle as well as all changes of direction, if at least one point of the route is known as a partial reference, such as the starting point, the end point or a waypoint in between.The orientation of the imaging sensors can be detected, for example, via an electronic compass (magnetometer) or an inertial measurement unit (IMU) with inertial sensors (inertial sensors) on or in the vehicle (including its sensors), by means of which accelerations and angular rates can be recorded, in conjunction with time recording; instead, time-based continuous image data acquisition is also possible (e.g. as a series image, film, or continuous recording), in which the orientation can be determined based on the time stamp and the recorded changes in direction.

[0043] A comparison of acquired image data with comparison image data can be carried out, for example, on the basis of the thermal image data or photo image data acquired by the sensors (either alone or in combination with one another); these can be recorded monoscopically or stereoscopically as individual images, image sequences, or moving images. Alternatively, such image data can also be acquired by separate devices, for example as distance data from scanning processes such as laser scanning (including laser scanners with thermal imaging cameras), lidar / ladar, sonar, or radar. For comparison, this image data is compared with comparison data, for example with previously acquired comparison image data or a previously created spatial (three-dimensional) digital model of the belt conveyor system and its surroundings, which can be presented, for example, as a three-dimensional point cloud.The comparison can be realized using a fixed set of rules or through machine learning.

[0044] Sometimes the spatial accuracy of individual methods may not be sufficient to allow the unambiguous identification of a depicted support roller. This can be the case, for example, with conventional GPS systems (differential GPS systems, for example, offer higher spatial resolution). In this case, it makes sense to use several methods for determining the position in parallel, for example, using a compass system in addition to a GPS system, for example, based on calibrated inertial sensors, to ensure the required accuracy.

[0045] According to a further embodiment of the invention, the method can further be implemented such that the temperature data determined from the analysis image area position are automatically assigned to a functional state of a support roller by either automatically classifying the determined temperature data of the respective support roller into a functional state from a plurality of predetermined functional states or assigning each of them to a functional state within the framework of a cluster analysis, in particular within the framework of a multivariate cluster analysis. Accordingly, the temperature data determined for a support roller can be automatically classified into a single functional state, wherein this single functional state originates from a plurality of predetermined functional states.The classification classes must be defined in advance according to the requirements of the specific belt conveyor system. In the simplest case, there are two classes: one for idlers that have already experienced functional failure (i.e., a defective idler with complete functional impairment), and one for idlers that have not yet experienced functional failure. Another binary (dual) classification could include a class for idlers that have not yet experienced functional impairment, and a class for idlers that have already experienced functional impairment (without distinguishing whether the functional impairment is merely minor or whether functional failure has already occurred).A more meaningful differentiation is achieved by a ternary classification (e.g., corresponding to a classification of "good," "medium," "poor," or "unworn," "partially worn," and "worn"), which distinguishes between idler rollers that have not yet experienced functional impairment, idler rollers that have already experienced functional impairment but no functional failure, and idler rollers that have already experienced functional failure. Furthermore, the classification can also include additional classes, with a further differentiation of the middle of the three classes appearing particularly useful in order to reflect different wear states and identify different points for replacement, for example, a classification with a total of four classes, five classes, six classes, or more.Instead of classification, assignment can also be carried out using cluster analysis (aggregation analysis), i.e., a method for summarizing (bundling, clustering) data into different homogeneous groups (clusters) based on similarity structures discovered in the data sets during the process. Cluster analysis is a structure-discovery method, so a prior definition of individual classes is not necessary. Optionally, the cluster analysis can be designed as a multivariate cluster analysis and thus depend on more than one variable. It is also useful to use methods based on machine learning for evaluations made as part of this type of assignment, particularly in the context of cluster analysis.

[0046] According to a further embodiment of the invention, it can be advantageous if the method is carried out in such a way that the thermal image data are automatically analyzed in each defined analysis image area position by automatically determining temperature data as characteristic temperature parameters of the respective support roller from the thermal image data in the defined analysis image area position of the support roller, wherein the characteristic temperature parameters of the thermal image data in the defined analysis image area positions are in particular minimum values, maximum values, mean values, central values, local accumulation values ​​and parameters of the temperature distributions, especially their widths, dispersion measures and symmetries.The characteristic temperature parameters are parameters derived from the temperature and characterize the current condition of a support roller; these are typically parameters obtained as part of a statistical analysis of the local temperatures. If the thermal image data does not directly contain the surface temperature, this can first be calculated from the respective thermal image data before the statistical analysis of the temperatures is performed.

[0047] In particular, according to a further embodiment of the invention, it is advantageous if the method is implemented in such a way that the recognition quality in the automatic determination of the functional status of support rollers, in particular the classification of the thermal image data, is improved using a learning process implemented using an artificial neural network. Although the automatic determination of the functional status can in principle be carried out using any suitable method, i.e., even based on a scheme of fixed rules, methods based on machine learning are preferably used for the automatic determination of the functional status.Accordingly, the automatic assessments on which the automatic determination of the functional status of support rollers is based are performed using machine learning processes, with the recognition accuracy of these assessments being improved by learning processes performed using an artificial neural network. These assessments may, in particular, be related to the classification of thermal image data.

[0048] According to a further embodiment of the invention, the method can furthermore be carried out such that the functional status, the image data, and / or, if applicable, the determined temperature data are recorded in a functional status data collection for the respective support roller. This results in a functional status data collection that contains the current functional status of a support roller, preferably the functional statuses of all support rollers of a belt conveyor system. Such a functional status data collection can, for example, be in the form of a database. Such functional status data collections with functional statuses, image data, and / or temperature data (including characteristic temperature parameters) can serve, for example, documentation purposes.

[0049] In addition to current data, such a functional status data collection can also contain historical data representing past functional states. In this form, it can also be used as a basis for creating forecasts regarding the temporal development of the functional status of support rollers. The corresponding data can be stored in a storage device located on the unmanned vehicle or in an external storage device located outside the unmanned vehicle, for example, in a data processing device such as a server.In the case of external storage, the data can be transmitted to the external data device shortly after the image data is captured (e.g., in real time or with a time delay), during the further course of the unmanned vehicle's inspection drive, or only after the inspection drive has ended, for example, from a base station (service station) where the vehicle is located between two inspection drives. For documentation purposes, the photo data in the form of photo images (photos, photographs, RGB images, especially digital photographs) and the thermal image data in the form of thermal images (thermal images, thermography, thermal images, thermal images, especially digital thermal photographs) are stored. Position information and route information are stored separately or can be included as metadata in the corresponding image data.

[0050] The data contained in such a functional status data collection can be accessed by the operator or maintenance personnel, for example, in text form or as an image. This provides an easy overview of various status information, such as a detailed list of all idlers including individual comments and maintenance history, detailed information on the temperature of individual idlers, geoinformation for quickly locating damaged idlers, and overviews for all idlers of wear and functional failure to date, as well as predicted wear and failure behavior. This information on idler condition and maintenance history can be supplemented manually if necessary.

[0051] Furthermore, according to a further embodiment of the invention, it may be expedient if the method is carried out in such a way that, if a detection position for an individual support roller is not detected during the automatic determination of the detection image area position, or if temperature data is determined during the analysis of the thermal image data that does not allow the functional status to be determined, a request command is transmitted to the unmanned vehicle to again capture image data in the area of ​​the belt conveyor system in which the individual support roller is located. In this way, in the event of an error in capture or a faulty capture, the image data can be captured again in order to obtain a complete set of analyzable data for the support rollers. The capture of image data can be limited to those support rollers for which corresponding data is not available.The same procedure can be used in the event of an emergency or incident to investigate the individual causes of irregularities that have occurred in the operational process.

[0052] The invention further includes a method for identifying functionally impaired support rollers of a belt conveyor system during operation of the belt conveyor system. The method comprises the previously described method for automatically determining the functional status of support rollers of a belt conveyor system. Functionally impaired support rollers are identified, a time for replacing the respective support roller is determined based on a comparison with historical data from a functional status database, and the determined time for this support roller is output to a communication interface. In such a method, support rollers of a belt conveyor system are identified that are functionally impaired, i.e., that have already failed or are expected to fail in the foreseeable future.The key step of this process is the previously described method for automatically determining the functional status of support rollers, in which functionally impaired support rollers are identified. The currently recorded data is compared with historical data from a functional status database, and an optimal time for replacing the respective support rollers is determined. To determine the optimal time, a forecast of the expected total failure of a support roller is taken into account, as are system-specific circumstances such as maintenance periods and the like. A forecast can be obtained, for example, by taking historical data into account, for example by extrapolating the historical temperature trend of a support roller to a specified critical maximum temperature (e.g. as part of a regression analysis) at which total failure is to be expected, for example a temperature of 90°C.The replacement of this support roller can then be carried out automatically as part of the regularly scheduled maintenance measure that takes place shortly before the critical maximum temperature is expected to be reached. It is also advisable to use machine learning-based methods for the assessments carried out as part of such a forecast. Based on such analyses, an efficiency-optimized maintenance schedule can be created for the entire belt conveyor system, minimizing downtime and maintenance costs and maximizing replacement intervals.The time determined in this way is output to a communication interface, for example, as visual information on a display device, sent as an email to a predefined email account, sent as an operating note to a system for plant control and monitoring, or inserted as an automatically generated calendar entry in appointment management programs. In addition, further measures can be initiated, such as automatically requesting required service technicians or automatically ordering required spare parts that are not in stock.

[0053] In addition to detecting anomalies and current wear of support rollers during inspection and monitoring, future behavior can also be predicted, for example for predictive maintenance or when creating a digital plant model.

[0054] In the event of an imminent functional failure of a support roller and the associated, urgently required replacement of such a support roller, the output of the maintenance time to the communication interface can also include an alarm signal, possibly combined with an emergency shutdown of the entire belt conveyor system to prevent further damage. Thus, the invention also includes a method for identifying damaged support rollers of a belt conveyor system during operation of the belt conveyor system, the method comprising the aforementioned method for automatically determining the functional status of support rollers of a belt conveyor system in any possible embodiment, wherein damaged support rollers are detected, and if a damaged support roller is detected, a notification signal for this support roller is output to a communication interface.

[0055] Finally, the invention includes a device for mechanically determining the functional state of support rollers of a belt conveyor system during operation of the belt conveyor system, the device comprising at least one unmanned vehicle movable along at least a partial area of ​​the belt conveyor system with at least one imaging sensor system, by means of which the belt conveyor system can be detected at least in sections by sensors in the form of image data and which comprises at least one thermal image sensor device for detecting thermal image data, a data processing device for evaluating the detected image data in order to determine the functional state of support rollers of a belt conveyor system during operation of the belt conveyor system, wherein the data processing device comprises an image area determination module which is configured to automatically determine at least one recognition image area position in detected image data,in which at least a partial area of ​​a support roller is imaged, an image area definition module which is configured to automatically define an analysis image area position in the thermal image data for a determined recognition image area position from the image data, and a state determination module which is configured to automatically analyze thermal image data in a defined analysis image area position in order to automatically determine the functional state of support rollers, wherein the imaging sensor system is configured such that the detectable image data also comprise photo image data in addition to the thermal image data, wherein the at least one imaging sensor system, in addition to the thermal image sensor device, also comprises a photo image sensor device for detecting image data of at least a partial area of ​​the belt conveyor system in the form of photo image data, wherein the image area determination module is configured to use, as image data of the belt conveyor system,in which at least one recognition image area position is automatically determined, to use photographic data in which photographic data areas are automatically recognized as image data areas, and to automatically provide the position of each recognized photographic data area as a recognition image area position.

[0056] According to a further embodiment of the invention, the device can be designed such that the image area determination module has an image area recognition module and an interface module, of which the image area recognition module is configured to automatically recognize image data areas in the acquired image data in which at least a partial area of ​​a support roller is imaged, and of which the interface module is configured to automatically provide the position of the recognized image data area as a recognition image area position of the respective support roller, and wherein the image area definition module is configured to automatically define, for each determined recognition image area position of a support roller from the image data, an analysis image area position of the support roller in the thermal image data, which corresponds spatially to the respective recognition image area position from the image data,and wherein the condition determination module comprises an analysis module and an assignment module, of which the analysis module is configured to automatically determine temperature data of the respective support roller from the thermal image data in the specified analysis image area position of the support roller, and of which the assignment module is configured to automatically assign the temperature data determined from the analysis image area position to a functional state of the respective support rollers.

[0057] According to a further embodiment of the invention, it can be advantageous if the device is designed such that the data processing device (data processing system) has at least one learning-capable artificial neural network, by means of which at least one of the following detections takes place: the detection of support rollers or partial areas of support rollers for the automatic determination of image data areas in the acquired image data in which at least a partial area of ​​a support roller is depicted, the detection of thermal image data of a support roller for the automatic determination of temperature data of the support roller in a defined analysis image area position of the support roller, the detection of functional states of a support roller from the determined temperature data of the support roller for the automatic assignment of the functional state of the support rollers.

[0058] Essential components as well as individual optional components of such a device for mechanically determining the functional status of support rollers of a belt conveyor system have already been described above. The data processing device comprises an image area determination module, an image area definition module, and a status determination module. Furthermore, the image area determination module can comprise an image area recognition module and an interface module, and the status determination module can comprise an analysis module and an assignment module. Depending on the specific design, the data processing device for evaluating the acquired image data can be located on the vehicle itself or at a distance from it, for example, as a server or computer cloud.If the data processing device is located on the vehicle, the data from the sensors can be transmitted to the data processing device for further processing via wireless or cable-based data transmission. If the data processing device is spatially separated from the vehicle, the data can be transmitted to the data processing device wirelessly, for example, via radio transmission, or via cable when the vehicle is in its base station. This means that any artificial neural network can be implemented in the unmanned vehicle itself or separately, for example, on an external data processing device.

[0059] To clarify the invention, representative examples are generally described below with reference to the corresponding figures, wherein individual sub-steps and processes thereof are also explained, which can be linked together in almost any way depending on the respective desired objectives of such a method. These examples reveal further advantages and possible applications, which will be described in more detail below with reference to the attached drawings without limiting the scope of protection of the invention defined by the claims, wherein individual features and steps are also explained, which can be linked together in almost any way depending on the respective desired objectives. The figures each show schematically Fig. 1 a schematic representation of a belt conveyor system with a device for mechanically determining the functional state of the support rollers according to an embodiment of the present invention, Fig. 2 a schematic perspective partial view of a belt conveyor system with a device for mechanically determining the functional state of the support rollers according to a further embodiment of the present invention and Fig. 3 a schematic representation of a method for mechanically determining the functional state of support rollers of a belt conveyor system according to a further embodiment of the present invention.

[0060] In Fig. 1 is a schematic representation of a belt conveyor system 1 with a device for mechanically determining the functional state of the support rollers 13 of the belt conveyor system 1 during operation of the belt conveyor system 1 according to an embodiment of the present invention. Fig. 1 The belt conveyor system 1 shown comprises a conveyor belt 11 which is deflected by two deflection rollers 12, one of which is designed as a drive drum (in the present case, the deflection roller 12 shown on the right). The upper run of the conveyor belt 11 forms the load run, over which the material to be conveyed is moved in the conveying direction (shown by an arrow). In both the upper run and the lower run, the conveyor belt 11 is supported by support rollers 13. In the present case, the conveyor belt 11 is guided flat, so that the support rollers 13 are aligned horizontally in the support roller stations; also possible here is a guide as a troughed conveyor belt or rolled conveyor belt, wherein the support rollers in the support roller stations are then arranged in a U-shape, V-shape or O-shape.

[0061] An unmanned vehicle 2 is arranged on the belt conveyor system 1. This is an autonomously moving vehicle or a remote-controlled vehicle that is remotely controlled by an operator, for example using a radio remote control. The vehicle 2 is a ground-based vehicle, but it can also be designed differently, for example as an aircraft. The unmanned vehicle 2 moves along the entire belt conveyor system 1. Furthermore, several unmanned vehicles can also be assigned to the belt conveyor system 1. For example, it can be provided that each vehicle from a plurality of unmanned vehicles only moves in a partial section of the belt conveyor system 1, so that each vehicle 2 of this plurality is required to cover the entire belt conveyor system 1.

[0062] The unmanned vehicle 2 has an imaging sensor system including a thermal imaging sensor device 21 and a light image sensor device 22. The thermal imaging sensor device 21 and the light image sensor device 22 are arranged spatially close to one another and have the same orientation and a substantially identical angle of view, so that they capture substantially the same object space. In this case, the thermal imaging sensor device 21 is a thermal imaging camera, and the light image sensor device 22 is a light camera (monoscopic or stereoscopic), with both cameras recording the captured image data electronically in digital form. The cameras can record individual images or corresponding moving images (film recordings) as image data.In addition, the unmanned vehicle 2 has a positioning device 23 for detecting the current position of the vehicle 2 in correlation with the respectively acquired image data as position data; in this case, this is a GPS module, with the position information being inserted into the image data as metadata. Instead, however, the light image sensor device 22 can also be used for position determination, possibly even the thermal image sensor device 21. In addition, the sensor system of the vehicle 2 has an orientation detection unit for determining the respective orientation when capturing the image data as orientation information—in this case, an inertial measuring unit.

[0063] The recorded image data, the position data, and the alignment information are transmitted via a data transmission module 24 to a data processing device 3 for evaluation there. In the present case, the data processing device 3 is implemented separately from the vehicle 2, for example, near the control center of the belt conveyor system 1. Alternatively, the data processing device 3 can also be arranged on the vehicle 2.

[0064] In the data processing device 3, the data transmitted by the data transmission module 24 of the unmanned vehicle 2, including the image data—the thermal image data from the thermal imaging camera and the light image data from the light camera—are received by the data transmission module 38 of the data processing device 3. The light image data (and possibly also the thermal image data) are first fed to an image area detection module 31, where they serve as input data for an image area detection module 32 of the image area detection module 31. In the image area detection module 32, image areas in which support rollers 13 or partial areas of support rollers 13 are depicted are automatically detected. The positions of the image areas thus detected (detection image area positions) are provided for further processing via an interface module 33 of the image area detection module 31.The image area definition module 34 obtains the positions of the image areas identified in the photometric data from the interface module 33 and automatically defines spatially corresponding image area positions in the thermal image data as analysis image area positions. The thermal image data with the analysis image area positions thus defined are forwarded to the condition determination module 35. Within the condition determination module 35, the thermal image data is automatically analyzed in an analysis module 36, whereby the analysis is limited here to the correspondingly defined image area positions of the support rollers 13. The results of this analysis are entered as temperature data into the assignment module 37, which automatically assigns a functional state to a support roller 13 based on the temperature data thus obtained from the thermal image data in the respective analysis image area positions.

[0065] The data processing device 3 has several adaptive artificial neural networks (not shown). One of these adaptive artificial neural networks is used to recognize support rollers 13 and sub-regions of support rollers 13 in order to determine image data regions in the acquired image data in which at least a sub-region of a support roller 13 is depicted. Another of these adaptive artificial neural networks is used to recognize thermal image data of a support roller 13 in order to automatically determine the temperature data of the support roller 13 in a defined analysis image area position of the support roller 13. Another of these adaptive artificial neural networks is used to recognize functional states of a support roller 13 from the determined temperature data of the support roller 13 in order to automatically assign a functional state to the respective support rollers 13.Artificial neural networks are also used for the corresponding machine learning processes.

[0066] A schematic perspective partial view of a belt conveyor system 1 with a special device for mechanically determining the functional state of the support rollers 13 of the belt conveyor system 1 during operation of the belt conveyor system 1 according to a special embodiment of the present invention is shown in Fig. 2 shown here. Only the upper run of the conveyor belt 11 is shown as well as an unmanned vehicle 2 of the device for the mechanical determination of the functional status of the support rollers 13 of the belt conveyor system 1. The conveyor belt 11 is guided as a troughed conveyor belt in a U-shape, the guidance is achieved via three support rollers 13 per support roller station, which are arranged in the shape of a trapezoid resting on the shorter base side, with the longer base side being open. In each support roller station, one support roller 13 is arranged horizontally on the shorter base side, while the remaining two support rollers form the legs of the trapezoid. Image data of the belt conveyor system 1 is recorded by the unmanned vehicle 2, which in this case is designed as an autonomous drone in the form of a quadrocopter.The vehicle 2 moves in a suspended (flying) manner parallel to the conveying direction (in the conveying direction or against the conveying direction) next to the upper run of the belt conveyor system 1. In addition to the unmanned vehicle 2, further unmanned vehicles can be provided; these can, for example, cover different sections of the belt conveyor system 1. In its lower region, the vehicle 2 has an imaging sensor system consisting of a thermal imaging sensor device 21 and a light imaging sensor device 22. In addition, the sensor system has inertial sensors and an electronic compass (not shown), with the help of which the orientation of the image sensor devices is recorded as orientation information (after prior calibration). The thermal imaging sensor device 21 and the light imaging sensor device 22, which is aligned parallel to the thermal imaging sensor device 21, record image data from the support rollers 13 of the belt conveyor system 1.The light image sensor device 22 also serves to determine the position by comparing the captured light image data with a three-dimensional point cloud previously created from the terrain and the belt conveyor system 1, thus determining a unique position. Together with the alignment information, this allows a unique assignment of the support rollers 13 captured in the image data. Alternatively, the vehicle 2 can also have a separate positioning device 23, for example, in the form of a GPS module. In the present embodiment, the captured image data and the position data are transmitted from the vehicle 2 as radio signals to a data processing device 3 (not shown), where automatic data evaluation takes place with the involvement of adaptive artificial neural networks.Instead, the data can also be initially stored in a storage device in the vehicle 2 and transferred to the data processing device 3 only after the vehicle 2 returns to a base station, where the secondary batteries of the vehicle 2 are also charged. When using a sufficiently large unmanned vehicle 2, it is also possible to arrange the data processing device 3 together with a storage device on the vehicle 2 itself, so that the data analysis then takes place on the vehicle 2.

[0067] In the following, a computer-aided method for identifying functionally impaired support rollers 13 is described, which is based on the method for mechanically determining the functional status of support rollers 13 of a belt conveyor system 1 during operation of the belt conveyor system 1. This method can be used in particular in connection with Fig. 1 and 2described belt conveyor systems 1, each with a device for mechanically determining the functional status of the support rollers 13. Individual sub-steps of this process are shown schematically in Fig. 3 shown.

[0068] The method involves an unmanned vehicle 2 with at least one imaging sensor system, by means of which the belt conveyor system 1 can be at least partially captured by sensors in the form of image data, wherein image data from at least a partial area of ​​the belt conveyor system 1 is captured as thermal image data (step 200). In the captured image data of the belt conveyor system 1, at least one detection image area position is automatically determined (step 300), in which at least a partial area of ​​a support roller 13 is imaged. For each detection image area position determined from the image data, an analysis image area position is automatically defined in the thermal image data (step 410). In each defined analysis image area position, thermal image data is automatically analyzed (step 400) in order to automatically determine the functional status of support rollers 13.

[0069] The at least one unmanned vehicle 2 is moved along at least a partial area of ​​the belt conveyor system 1 with the at least one imaging sensor system comprising at least one thermal imaging sensor device 21 for capturing the thermal image data. With the aid of the imaging sensor system of the unmanned vehicle 2, which moves along the main extension (conveying direction) of the belt conveyor system 1, photographic image data is captured (step 100) and thermal image data is captured (step 200) from the belt conveyor system 1 in sections. The unmanned vehicle 2 can be an autonomous unmanned vehicle or a remotely controlled unmanned vehicle. Drones or robots are also particularly suitable as autonomous unmanned vehicles. Such a vehicle 2 can in principle be designed as a track-bound or track-free vehicle, in particular as a land vehicle or as an aircraft (for example, a quadcopter).

[0070] In step 100, the imaging sensor system acquires photographic image data of at least a partial area of ​​the belt conveyor system 1, and in step 200, the imaging sensor system acquires thermal image data of at least a partial area of ​​the belt conveyor system 1. The image data acquired by the imaging sensor system of this vehicle 2 depicts, in whole or in part, support rollers 13 of a partial area of ​​the belt conveyor system 1, in particular also the bearings (rolling bearings) of the corresponding support rollers 13. The photographic image data and thermal image data are static monoscopic images, namely digital photographs and digital thermal images, respectively. However, other types of image data can also be used instead, for example, the corresponding moving images (moving images, videos), stereoscopic images, and the like.

[0071] In addition to the spatially resolved image information, the photo data also contains the exact position (spatial position) at which the image was taken (namely, in the form of the position of the vehicle 2), as well as the orientation information indicating the orientation of the photo sensor device 22 at which the image was taken (alternatively or additionally, the corresponding information can also be contained in the thermal image data). This additional information is stored as metadata in the corresponding image files in addition to the pixel data. Alternatively, this information can of course also be stored and transmitted separately from the image data, as long as it can be clearly assigned to the corresponding image data, for example, via a common timestamp. The position is determined by means of a position-determining device 23, which is provided on the vehicle 2 itself or in a data processing device 3.In the present case, the positioning device 23 is a device for radio-based positioning, namely for satellite-based positioning using GPS. Instead, other devices and methods for positioning can also be used, such as positioning based on optical environmental detection, for example, by comparing image data with previously recorded image data or by comparing image data with a previously created three-dimensional terrain model, such as a three-dimensional point cloud, which also maps the corresponding sub-area of ​​the belt conveyor system 1.

[0072] The image data is temporarily stored during or after recording and subsequently transmitted to a data processing device 3—for example, immediately afterward, at fixed time intervals, or after the vehicle 2 returns to a base station. In the data processing device 3, the data is automatically processed and stored together with the results of such data processing as target data, for example, in the form of databases. The target data thus obtained is made available to any users via an interactive communication interface, for example, to the operator's employees, service personnel, maintenance technicians, and the like.

[0073] In step 300, image data regions in which at least a partial region of a support roller 13 is depicted are automatically detected in the acquired image data of the belt conveyor system 1. This step includes automatically detecting image data regions (step 320) and automatically providing detected image data regions as a detection image region position (step 330), as well as improving the detection quality of the automatic detection (step 310). At the end of step 300, the position of the respectively detected image data region is automatically provided as the detection image region position of the respective support roller 13 (step 330).

[0074] In the present case, the image data are photographic data, and the image data areas are photographic data areas. Thus, photographic data areas in which at least one recognizable object is depicted are automatically recognized in the photographic data areas (step 320). In the photographic data areas thus recognized, support rollers 13 or their sub-areas are automatically recognized among the recognizable objects. Thus, the detection and identification of objects, in particular support rollers 13 and the associated bearings (rolling bearings), takes place on the basis of this photographic data. For each recognized support roller 13 and for each recognized sub-area of ​​a support roller 13, the position of the photographic data area in which the support roller 13 or the sub-area of ​​the support roller 13 is depicted is provided for further processing as a recognition image area position (object position and object class) (step 330).

[0075] The automatic detection (step 320) of image data areas represents a central process within step 300. In the present case, the automatic detection 320 of image data areas is based on a machine learning algorithm.The recognition quality in the automatic determination of the recognition image area position in the acquired photo data, in particular the recognition quality of the automatic recognition of photo image data areas in which at least one recognizable object is depicted (in particular at least a partial area of ​​a support roller 13) and / or the recognition quality of the automatic recognition of support rollers 13 or partial areas of support rollers 13 in the recognized photo data areas is improved using a learning process (step 310) which is carried out using an artificial neural network on the basis of training data (training images), for example photo data in which those image areas in which the objects to be learned are depicted - in particular support rollers 13 or partial areas of support rollers 13 - have been manually marked with position and object designation. With the help of the training data, among other things, the recognition quality in the classification is improved.In addition, other objects characteristic of the belt conveyor system 1 can also be marked in the training data, such as sections of scaffolding elements, conveyor belt 11, and the like. In this case, the training data then additionally contains the verified object class assigned to the detected object as an identifier. The artificial neural network is a multi-stage convolutional neural network consisting of two convolutional neural network modules. The first module is used to preselect image data regions in the photographic data in which different objects could be located. The result of this preselect is then passed to the second module to perform an object-based classification in the potentially relevant image data regions thus identified.The result of this classification of image data provides the object class assigned to the detected object (the object name, for example, "support roller," "bearing," "steel cable," or the like), which is assigned a probability value from a range of 0 to 1. The object class with the highest probability value represents a result of this evaluation. The evaluation of the image also provides the corresponding position of the classified object within the image data. In this case, the position of such an object is defined by four numbers, namely two pairs of x / y coordinates, each of which corresponds to point positions in the image. The two points in the image data corresponding to the point positions, as opposite edges of a rectangular selection frame, span a rectangle in which the object is centrally positioned.Depending on the photographic data being examined, several objects can be identified simultaneously in one photograph.

[0076] The image data of the belt conveyor system 1 is each assigned to a position in the belt conveyor system 1. In addition or instead, the thermal image data or evaluation information for defined analysis image area positions is each assigned to an individual support roller 13 of the belt conveyor system 1. The assignment is carried out in particular via radio-based position determination, via a comparison of captured image data with comparison image data, via a path detection of the unmanned vehicle 2 and / or via an orientation detection of the imaging sensors. Such an assignment can be carried out at any time during the process before a comparison with historical data takes place. For this purpose, the belt conveyor system 1 is initially measured and the exact position (e.g. the GPS position) of each individual support roller 13 is determined.For a particularly reliable analysis, a three-dimensional digital model of the belt conveyor system 1 can also be created (this can also serve as an intuitive display of the acquired information). Each photograph and thermal image then contains the position of the vehicle 2 (in the form of GPS coordinates) as metadata, as well as information on the orientation of the image sensor devices during data acquisition (e.g., via an internal electronic compass, an inertial measuring unit, or the swivel angle display of a robot arm), so that the precise assignment of the identified objects from the image recognition (step 320) to a specific support roller 13 of the belt conveyor system 1 is possible. To ensure precise assignment, a GPS module was selected here that offers position detection based on a differential GPS method with a GPS position resolution in the centimeter range.Alternatively, the position can also be determined optically via image comparison. This involves comparing the current (two-dimensional) light camera image (captured using a monoscopic camera) or a point cloud (3D point cloud, captured using a stereoscopic camera, a moving monoscopic camera, a scanning laser, lidar, radar, or the like) with known two-dimensional light images or a georeferenced point cloud. Furthermore, position determination in near-field radio is also possible, for example, via RFID transponders arranged in the support roller stations of the respective support rollers 13 and read by a corresponding reader in the vehicle 2. Position determination using radio-based methods takes place at the time the image data is captured; an image data comparison based on the captured image data can also be performed at a later time.In the present case, the position determination by means of GPS takes place simultaneously with step 100 and / or 200, so that the location information is stored together with the image data and the alignment data and an assignment to specific support rollers 13 then takes place after the automatic recognition of image area data (step 320).

[0077] In step 410, for each determined and provided detection image area position of a support roller 13 from the image data, an analysis image area position of the support roller 13 is automatically defined in the thermal image data, which spatially corresponds to the respective detection image area position from the image data. In the subsequent automatic analysis of thermal image data (step 400), the corresponding temperature data of the support rollers 13 are determined only for the thermal image data in the analysis image area positions and subjected to a thermal analysis in order to then assign the respective functional state (wear state) to the support rollers 13 and their bearings.Since the thermal imaging sensor device 21 and the light image sensor device 22 are closely spaced from one another, are rigidly aligned in the same direction relative to the vehicle 2, and both sensor devices capture the image data approximately simultaneously, the information from the above-described object detection in the light image data (step 300) can be transferred to the thermal image data (due to the known offset between the two image sensor devices, an offset can be calculated out). If an object was detected and marked as support roller 13 in the light image data, in step 410 the rectangular selection frame that encloses support roller 13 in the light image data is defined in the thermal image data as the analysis image area position and is therefore also marked in the thermal image. The subsequent thermal analysis is then limited to thermal image data within this selected analysis image area.

[0078] In step 400, the selected thermal data is automatically analyzed in the specified analysis image area position of the support roller 13. For this purpose, temperature data for the respective support roller 13 is first automatically determined from the thermal image data in the specified analysis image area positions (step 420). The temperature data determined in the analysis image area is then automatically assigned to a functional state of the respective support roller 13 (step 430). Step 400 therefore includes the automatic determination of temperature data (step 420) and the automatic assignment of the temperature data to a functional state (step 430), but not the automatic determination of the analysis image area position (step 410 - however, a process control would also be possible in which the automatic determination of the analysis image area position is part of an automatic analysis of the thermal image data).

[0079] In step 420, thermal image data is automatically analyzed at each specified analysis image area position. To this end, in this step, thermal image data is first automatically selected at the specified analysis image area positions, which are thermal image data of the support rollers 13. The thermal data thus selected is then automatically analyzed at the specified analysis image area positions.

[0080] To ensure that the thermal analysis is performed on the support roller 13 and its bearings and not on other objects such as the conveyor belt 11 or steel cables, which may also be located within the rectangular selection frame defined as the analysis image area, sub-areas are defined within the analysis image area position using two different selection methods. This makes it possible to keep the object region for the subsequent actual data analysis as small as possible. For this purpose, sub-areas of the thermal image data are automatically selected within the defined analysis image area positions in which thermal image data are arranged in circular or circle-like contours. In addition, sub-areas of the thermal image data are automatically selected within the defined analysis image area positions in which the thermal image data correspond to temperatures of the same temperature level.In the first selection method, the selection frame is therefore examined for circular contours. The area for examining the bearing of the support roller 13 is then defined as the average of all circular contours, which are subsequently selected. In the second selection method, based on the temperature distribution in the selection frame, several areas with the same temperature level are selected (the term "temperature level" includes a certain temperature range, the width of which is to be selected according to the specific operating conditions of the belt conveyor system 1). To combine both selection methods, the positions of the sub-areas selected in the first selection method are compared with the positions of the sub-areas selected in the second selection method within each analysis image area position.The sub-area whose thermal image data is subjected to the subsequent actual thermal analysis results from the common intersection (intersection) of the sub-areas selected in the two selection methods. The combination of both selection methods offers a particularly high degree of reliability that the automatically selected thermal image data is truly representative of the support rollers 13 and their bearings.In addition, the recognition quality in the automatic selection of the thermal image data, which are thermal image data of the support rollers 13, in particular in the automatic selection of the thermal image data which lie in such sub-areas within the defined analysis image area positions in which thermal image data are arranged in circular or circle-like contours and / or in which thermal image data correspond to the temperatures of a same temperature level, is improved by means of a learning process carried out by means of an artificial neural network, namely via a multi-stage convolutional neural network which has two "convolutional neural network" modules.

[0081] In step 430, the temperature data determined from the analysis image area position are automatically assigned to a functional state of a support roller 13 by either automatically classifying the determined temperature data of the respective support roller 13 into a functional state from a plurality of predefined functional states or automatically assigning them to a functional state within the framework of a cluster analysis, in particular within the framework of a multivariate cluster analysis. In addition, the recognition quality in the automatic determination of the functional state of support rollers 13, in particular the classification of the thermal image data, is improved using a learning process implemented using an artificial neural network.In this process, characteristic temperature features such as the maximum temperature, the minimum temperature, the mean temperature, the median temperature, as well as characteristic features of the temperature distribution (e.g., its width, symmetry, or any hotspots) are extracted from the selected areas. The automatic assignment to a functional state 430 is performed using a multivariate clustering algorithm, through which the totality of the characteristic features of each support roller 13 and its bearings are assigned to one of several wear states. In the present case, this is an assignment to "replacement required," "replacement required soon," or "replacement not yet foreseeable."This clustering algorithm is continuously trained, for which purpose appropriately verified information on the actual functional state of support rollers 13 is entered as training data, such as that obtained after a manual check during an inspection or maintenance to verify the functional state.

[0082] The acquisition of image data and its analysis, i.e., the method for automatically determining the functional status of support rollers 13 of a belt conveyor system 1 during operation of the belt conveyor system 1 (steps 100 - 400), is carried out at regular intervals. For each support roller 13, the functional status, image data, and determined temperature data are recorded in a functional status data collection, in particular in a functional status database (not shown). In this functional status data collection, each support roller 13 of the belt conveyor system 1 is recorded with its exact GPS position and its respective specifications; for example, for the support roller 13, this includes its size, material, manufacturer, type number, installation date, and the like. For each support roller 13, the storage paths of the associated image data (as raw data and as processed data after processing) as well as the analysis results with a time stamp are also saved in each acquisition cycle.The stored analysis results include the position (support roller 13 and its bearing) and the temperature data, in particular the characteristic temperature parameters such as maximum temperature, minimum temperature, mean temperature, median temperature, and temperature distribution, as well as the determined functional states (as wear condition classes). Furthermore, additional information such as temperature limit values ​​(e.g., 80 °C for a replacement due in the foreseeable future and 90 °C for an immediately required replacement), upcoming maintenance procedures, and the like can be stored for each support roller 13.

[0083] Via an interactive communication interface (local user interface or online user interface, for example via locally installed computer programs or via mobile or web-based applications), the data stored in the functional status data collection can be accessed and retrieved, for example, to visually display the corresponding information. In this way, service employees or maintenance technicians in particular can display markings and lists of all support rollers 13 requiring maintenance and their GPS position, or interactively add corresponding photos and comments for documentation purposes during an inspection and maintenance of support rollers 13, for example, a manual assessment of the wear condition or a list of the specific maintenance work and replacement activities performed.This information is then stored in a database with a timestamp and can be used to improve the quality of the classification as part of a machine learning process. Furthermore, with the help of the functional status data collection, the exact positions of the support rollers 13 can be displayed on a map based on available map material, satellite images, or a three-dimensional digital model of the belt conveyor system 1. For example, a user can interactively select individual support rollers 13 on the map to view additional information about these support rollers 13. In addition, current functional states / wear states as well as selected temperature characteristics of all support rollers 13 can be displayed in an overview diagram.The data on the support rollers 13 can also be called up in list form, whereby individual support rollers 13 can also be selected interactively in this display and their information can be viewed and comments on individual support rollers 13 can be saved, whereby the temperature profile of the respective support roller 13 can be displayed or its thermal image data and the photo image data can be displayed, also as a chronological progression of the historical values ​​including all changes.

[0084] The Fig. 3The method described for identifying functionally impaired support rollers 13 of a belt conveyor system 1 during operation of the belt conveyor system 1 includes, in addition to the previously described method for mechanically determining the functional status of support rollers 13, also the detection of functionally impaired support rollers 13 based on a comparison of the functional status and the temperature data with historical data from a functional status database (step 500). On the basis of this comparison, a time for replacing the respective support roller 13 is determined (step 600). In connection with such predictive maintenance, the method described can therefore be used to monitor the functional status of all support rollers 13 at short intervals during repeated inspection measures without additional personnel expenditure, and to identify critical support rollers 13 at an early stage.In addition, the temporal development of the thermal characteristics can be analyzed by means of extrapolation or regression analysis (step 500), so that a forecast of the expected remaining service life of the support roller 13 can be created and an optimal time for maintenance work can be recommended (step 600). The times determined for the respective support rollers 13 are output to a communication interface, for example in the form of a screen display, as an automatically created calendar entry in an appointment management program, as an automatic order for support rollers or consumables from the manufacturer or dealer, or - in the case of an absolutely necessary immediate replacement - also as a warning signal, if necessary combined with a control intervention to stop the belt conveyor system 1 in order to prevent any consequential damage, such as that which could occur to the conveyor belt 11.

[0085] The previously described method for mechanically determining the functional status of support rollers 13 of a belt conveyor system 1 during operation of the belt conveyor system 1 and, in particular, the previously described method for identifying functionally impaired support rollers 13 of a belt conveyor system 1 during operation of the belt conveyor system 1 are each implemented in the form of a computer program. This computer program is stored on a machine-readable data carrier (computer-readable data carrier, machine-readable storage medium / medium, computer-readable storage medium / medium). List of reference symbols

[0086] 1 Belt conveyor system 11 Conveyor belt 12 Deflection roller 13 Support roller 2 Unmanned vehicle 21 Thermal imaging sensor device 22 Light imaging sensor device 23 Positioning device 24 Data transmission module 3 Data processing device 31 Image area determination module 32 Image area recognition module 33 Interface module 34 Image area definition module 35 State determination module 36 Analysis module 37 Assignment module 38 Data transmission module 100 Acquisition of light image data 200 Acquisition of thermal image data 300 Automatic determination of at least one detection image area position 310 Improvement of the detection quality of the automatic detection 320 Automatic detection of image data areas 330 Automatic provision of detected image data areas as detection image area positions 400 Automatic analysis of thermal image data 410 Automatic definition of the analysis image area position 420automatic determination of temperature data 430automatic assignment to a functional state 500comparison withhistorical data from functional status database 600Determination of the time for an exchange

Claims

1. Method for the machine-based determination of the functional state of support rollers (13) of a belt conveyor system (1) during the operation of the belt conveyor system (1), wherein at least one unmanned vehicle (2) with at least one imaging sensor system is provided, by means of which at least sections of the belt conveyor system (1) can be captured in the form of image data, wherein image data of at least one subregion of the belt conveyor system (1) is captured as thermal image data, in the captured image data of the belt conveyor system (1) at least one identification image region position, in which at least one subregion of a support roller (13) is imaged, is automatically determined, for each determined identification region position from the image data, an analysis image region position is automatically defined in the thermal image data, and in each defined analysis image region position, thermal image data are automatically analysed in order to automatically determine the functional state of support rollers (13), characterised in that the image data that can be captured by the imaging sensor system also comprise photographic image data in addition to the thermal image data, image data of at least one subregion of the belt conveyor system (1) are also captured as photographic image data, the captured image data of the belt conveyor system (1), in which at least one identification image region position is determined automatically, are photographic image data in which photographic image data regions are automatically detected as image data regions, and the position of each detected photographic image data region is automatically provided as the identification image region position.

2. Method according to claim 1, wherein image data of at least one subregion of the belt conveyor system (1) are captured as thermal image data by moving the at least one unmanned vehicle (2) with the at least one imaging sensor system, comprising a thermal image sensor device (21) for capturing the thermal image data and a photographic image sensor device (22) for capturing the photographic image data, along at least one subregion of the belt conveyor system (1), image data from at least one subregion of the belt conveyor system (1) are captured with the imaging sensor system and the image data comprise at least thermal image data and photographic image data, in the captured image data of the belt conveyor system (1) the at least one identification image region position is automatically determined by automatically detecting, in the photographic image data captured by at least one subregion of the belt conveyor system (1), photographic image data regions in which at least one subregion of a support roller (13) is imaged, and the position of the respectively detected photographic image data region is automatically provided as the identification image region position of the respective support roller (13), for each identification image region position determined from the photographic image data, an analysis image region position is automatically defined in the thermal image data by automatically defining, for each identification image region position of a support roller (13) determined from the photographic image data, an analysis image region position of the support roller (13) in the thermal image data, which position corresponds spatially to the respective identification image region position from the photographic image data, the thermal image data in each defined analysis image region position are automatically analysed by automatically determining temperature data of the respective support roller (13) from the thermal image data in the defined analysis image region position of the support roller (13), and automatically assigning the temperature data determined from the analysis image region position to a functional state of the respective support rollers (13).

3. Method according to claim 1 or 2, wherein the at least one identification image region position is automatically determined in the captured photographic image data of the belt conveyor system (1) by automatically detecting, in the photographic image data, photographic image data regions in which at least one detectable object is imaged, automatically detecting support rollers (13) or subregions of support rollers (13) among the detectable objects in the photographic image data regions thus detected, and for each detected support roller (13) as wel as for each detected subregion of a support roller (13), providing the position of the photographic image data region in which the support roller (13) or the subregion of the support roller (13) is imaged as an identification image region position.

4. Method according to any one of claims 1 to 3, wherein the recognition quality in the automatic determination of the identification image region position in the captured photographic image data, in particular the recognition quality of the automatic detection of photographic image data regions in which at least one detectable object is imaged, in particular at least one subregion of a support roller (13), and / or the recognition quality of the automatic detection of support rollers (13) or subregions of support rollers (13) in the detected photographic image data regions is improved through a learning method carried out by means of an artificial neural network, in particular by means of a single-level or multi-level convolutional neural network.

5. Method according to any one of claims 1 to 4, wherein in each defined analysis image region position, thermal image data are automatically analysed by automatically selecting thermal image data in the defined analysis image region positions, which are thermal data of the support rollers (13), and the selected thermal data in the defined analysis image region positions are automatically analysed.

6. Method according to claim 5, wherein thermal image data are automatically selected that lie in those subregions within the defined analysis image region positions in which thermal image data are arranged in circular or near-circular contours and / or in which thermal image data correspond to the temperatures of an equal temperature level.

7. Method according to claim 5 or 6, wherein the recognition quality in the automatic selection of the thermal image data, which are thermal image data of the support rollers (13), is improved through a learning method carried out by means of an artificial neural network, in particular the recognition quality in the automatic selection of the thermal image data which lie in those subregions within the defined analysis image region positions in which thermal image data are arranged in circular or near-circular contours and / or in which thermal image data correspond to the temperatures of an equal temperature level.

8. Method according to any one of claims 1 to 7, wherein the image data of the belt conveyor system (1) are in each case assigned to a position in the belt conveyor system (1) and / or for defined analysis image region positions the thermal image data or evaluation information are assigned to an individual support roller (13) of the belt conveyor system (1), wherein the assignment is performed in particular via a radio-based position determination, by comparing captured image data against reference image data, by recording the route travelled by the unmanned vehicle (2), and / or by detecting the orientation of the imaging sensor system.

9. Method according to claim 2 or any one of claims 3 to 8 referring back to claim 2, wherein the temperature data determined from the analysis image region position are automatically assigned to a functional state of a support roller (13), by either automatically classifying the determined temperature data of the respective support roller (13) to one functional state of a plurality of previously defined functional states or by automatically assigning said data to a functional state as part of a cluster analysis, in particular as part of a multivariate cluster analysis.

10. Method according to any one of claims 1 to 9, wherein the recognition quality in the automatic determination of the functional state of support rollers (13), in particular the classification of the thermal image data, is improved through a learning method carried out by means of an artificial neural network.

11. Method according to any one of claims 1 to 10, wherein the functional state, the image data and / or, if applicable, the determined temperature data for the respective support roller (13) are recorded in a functional state data collection.

12. Method for identifying functionally impaired support rollers (13) of a belt conveyor system (1) during the operation of the belt conveyor system (1), the method comprising the method for the machine-based determination of the functional state of the support rollers (13) of a belt conveyor system (1) according to any one of claims 1 to 11, wherein functionally impaired support rollers (13) are detected, a time for a replacement of the respective support roller (13) is determined based on a comparison with historical data from a functional state database, and for this support roller (13) the determined time is output to a communication interface.

13. Apparatus for the machine-based determination of the functional state of support rollers (13) of a belt conveyor system (1) during the operation of the belt conveyor system (1), the apparatus comprising at least one unmanned vehicle (2) with at least one imaging sensor system that can be moved along at least one subregion of the belt conveyor system (1), by means of which sensor system at least sections of the belt conveyor system (1) can be captured by sensors in the form of image data, and which comprises at least one thermal image sensor device (21) for capturing image data in the form of thermal image data, a data processing device (3) for evaluating the captured image data in order to determine the functional state of support rollers (13) of a belt conveyor system (1) during the operation of the belt conveyor system (1), wherein the data processing device (3) has an image region identification module (31) which is configured to automatically determine in captured image data at least one identification image region position in which at least one subregion of a support roller (13) is imaged, an image region definition module (34) which is configured to automatically define an analysis image region position in the thermal image data for an identification image region position determined from the image data, and a state identification module (35) which is configured to automatically analyse thermal image data at a defined analysis image region position to automatically determine the functional state of support rollers (13), characterised in that the imaging sensor system is configured that the image data that can be captured comprise photographic image data in addition to the thermal image data, wherein the at least one imaging sensor system also comprises, in addition to the thermal image sensor device (21), a photographic image sensor device (22) for capturing image data of at least one subregion of the belt conveyor system (1) in the form of photographic image data, wherein the image region identification module (31) is configured to use, as image data of the belt conveyor system (1) in which at least one identification image region position is automatically determined, photographic image data in which photographic image data regions are automatically detected as image data regions, and to automatically provide the position of a respectively detected photographic image data region as the identification image region position.

14. Apparatus according to claim 13, wherein the image region identification module (31) has an image region detection module (32) and an interface module (33), the image region detection module (32) of which is configured to automatically detect photographic image data regions in the captured photographic image data in which at least one subregion of a support roller (13) is imaged, and the interface module (33) of which is configured to automatically provide the position of the detected photographic image data region as the identification image region position of the respective support roller (13), and wherein the image region definition module (34) is configured to automatically define, for each identification image region position of a support roller (13) determined from the photographic image data, an analysis image region position of the support roller (13) in the thermal image data that corresponds spatially to the respective identification image region position from the photographic image data, and wherein the state identification module (35) has an analysis module (36) and an assignment module (37), the analysis module (36) of which is configured to automatically determine temperature data of the respective support roller (13) in the defined analysis image region position of the support roller (13) from the thermal image data, and the assignment module (37) of which is configured to automatically assign the temperature data determined from the analysis image region position to a functional state of the respective support rollers (13).

15. Apparatus according to claim 14, wherein the data processing device (3) has at least one trainable artificial neural network by means of which at least one of the following detections is carried out: the detection of support rollers (13) or subregions of support rollers (13) for the automatic determination of photographic image data regions in the captured photographic image data, in which at least one subregion of a support roller (13) is imaged, the detection of thermal image data of a support roller (13) for the automatic determination of temperature data of the support roller (13) in a defined analysis image region position of the support roller (13), the detection of functional states of a support roller (13) from the determined temperature data of the support roller (13) for the automatic assignment of the functional state of the support rollers (13).