Methods for operating a technical plant and technical plant

Abstract

A method for operating a technical installation comprising at least one autonomous vehicle is proposed. The autonomous vehicle has at least one laser scanner for acquiring laser scans to detect objects, and it possesses a map of the technical installation. The autonomous vehicle is positioned on a level surface within the installation. Furthermore, the autonomous vehicle has at least one camera for capturing images, and this camera is directed towards the floor of the technical installation. During a reference phase, the camera captures multiple reference images of the floor. The camera's position within the technical installation is determined for each reference image captured.The reference images captured by the camera are each entered into the map at a location corresponding to the camera's position in the technical system at the time the respective reference image was captured. A technical system according to the invention can be operated using the method according to the invention.

Description

[0001] The invention relates to a method for operating a technical system comprising at least one autonomous vehicle, wherein the autonomous vehicle has at least one laser scanner for recording laser scans for detecting objects, and wherein the autonomous vehicle has a map of the technical system. The invention also relates to a technical system comprising at least one autonomous vehicle, which is operable using the method according to the invention.

[0002] The technical facility is primarily an industrial application, such as a production plant, an industrial hall, a paint shop, or a logistics center. Autonomous vehicles are used, for example, to transport materials within the technical facility. The facility also contains other objects, such as walls, columns, production machines, pallets, crates, containers, or transport carts, as well as people and other autonomous vehicles. The autonomous vehicles are equipped with sensors, particularly laser scanners for capturing laser scans, to detect these objects.

[0003] The detected objects in the technical facility, such as walls, columns, production machines, pallets, boxes, containers, or transport trolleys, are assigned to objects that are recorded on the map. When an object in the technical facility is detected by a laser scanner of an autonomous vehicle, the position of the autonomous vehicle on the map can be determined by comparing the detected object with an object recorded on the map of the technical facility.

[0004] German patent application DE 10 2023 001 709 B3 discloses a method for operating a technical system comprising at least one autonomous vehicle equipped with a laser scanner. The laser scanner acquires a laser scan, and the pose of the autonomous vehicle is determined by comparing the laser scan with objects recorded on a map.

[0005] From the document “ZHANG, Linguang; FINKELSTEIN, Adam; RUSINKIEWICZ, Szymon: High-precision localization using ground texture. Preprint. [v3] Wed, 26 Jun 2019, 22:48:48 UTC. 7 pp. https: / / doi.org / 10.48550 / arXiv.1710.10687” a method for optically determining the position of a vehicle using a camera is known.

[0006] An optical localization method for robots is known from the document "SCHMID, Jan Fabian [et al.]: HD ground - a database for ground texture based localization. In: 2022 IEEE International Conference on Robotics and Automation, May 23-27, 2022, Philadelphia, PA, USA. New York: IEEE, 2022. pp. 7628-7634. ​​ISBN 978-1-7281-9681-7. https: / / doi.org / 10.1109 / ICRA46639.2022.9811977".

[0007] A visual SLAM method for robots is known from the document “XU, Kuan [et al.]: Non-iterative SLAM for Warehouse Robots Using Ground Textures. Preprint. [v3] Sat, 18 Nov 2023 16:51:41 UTC. S. 1-15. https: / / doi.org / 10.48550 / arXiv.1710.05502”.

[0008] From DE 10 2016 000 588 A1 a method for the self-localization of a vehicle is known, wherein images are captured successively over time and a current position of the vehicle is determined on the basis of correspondences of image features of currently captured images with reference image features stored in a database and the position data assigned to them.

[0009] A method for determining the position of a robot in a building is known from CN 1 05 806 331 A.

[0010] From US 2020 / 0 210 719 A1 a method for determining the position of a moving object using an image capture device is known.

[0011] From DE 10 2019 123 647 A1, a method for locating a vehicle moving on a road surface is known. In this method, a projection pattern is projected onto the road surface, and the projection pattern is detected by means of an optical sensor arranged on the vehicle.

[0012] From DE 10 2022 211 642 A1, a traffic control device is known that can move autonomously on a floor. The traffic control device comprises a sensor device for detecting a texture of the floor and an evaluation device for determining the current location of the traffic control device based on the detected texture.

[0013] The invention is based on the objective of further developing a method for operating a technical plant as well as a technical plant itself.

[0014] The problem is solved by a method for operating a technical system with the features specified in claim 1. Advantageous embodiments and further developments are the subject of the dependent claims. The problem is also solved by a technical system with the features specified in claim 9.

[0015] A method for operating a technical installation comprising at least one autonomous vehicle is proposed. The autonomous vehicle has at least one laser scanner for acquiring laser scans to detect objects, and it possesses a map of the technical installation. The autonomous vehicle is positioned on a level surface within the installation. Furthermore, the autonomous vehicle has at least one camera for capturing images, and this camera is directed towards the floor of the technical installation. During a reference phase, the camera captures multiple reference images of the floor. The camera's position within the technical installation is determined for each reference image captured.The reference images captured by the camera are entered on the map at a location that corresponds to the respective position of the camera in the technical system when the respective reference image was captured.

[0016] The floor on which the autonomous vehicle is located is a reliable source of information about its current position and orientation within the technical facility. Floors in industrial environments have various macro- and microtextures. These textures include lines and other markings, unevenness, scratches, and the overall surface condition of the floor. The reference images plotted on the map allow for the determination of an autonomous vehicle's position during operation.

[0017] Two-dimensional laser navigation using structural raster maps can fail if the visible environment deviates significantly from the previously recorded raster map. This can occur, for example, if part of the environment is obscured from the laser scanner's field of view, or if the physical structure of the technical installation differs from the recorded map. However, the ground and its appearance are largely immune to such changes, especially along the roadway. Therefore, determining the position of an autonomous vehicle using a visual ground map is feasible with increased robustness and availability.

[0018] According to the invention, before the camera takes the majority of reference images of the ground, the roughness of the ground is increased.

[0019] The roughness of the floor in the technical facility can be increased through various treatments. In particular, treatments such as sandblasting, wire blasting, and brushing are suitable for increasing floor roughness. This increased roughness creates diverse and unique microtextures, macrotextures, and scratches in the floor. As a result, the determination of the position of an autonomous vehicle is further improved and less prone to errors.

[0020] According to an advantageous embodiment of the invention, the camera is arranged on the autonomous vehicle such that an optical axis of the camera runs in a vertical direction. The vertical direction is perpendicular to the ground.

[0021] According to an advantageous embodiment of the invention, a position of the autonomous vehicle in the technical system is determined each time a reference image is taken, and the respective position of the camera is calculated from the position of the autonomous vehicle and the arrangement of the camera on the autonomous vehicle.

[0022] According to an advantageous embodiment of the invention, the respective position of the autonomous vehicle in the technical system is determined when a reference image is taken by taking a laser scan of a part of the technical system, which has at least one object, with the at least one laser scanner of the autonomous vehicle, and comparing the laser scan with at least one object recorded in the map, and determining the respective position of the autonomous vehicle by comparing the laser scan with the at least one object recorded in the map.

[0023] According to an advantageous embodiment of the invention, the autonomous vehicle has an odometric system for determining its position. The respective position of the autonomous vehicle within the technical system is determined by means of the odometric system when a reference image is captured.

[0024] According to an advantageous embodiment of the invention, the autonomous vehicle has a GPS sensor for determining its position. The respective position of the autonomous vehicle within the technical system is determined by the GPS sensor when a reference image is captured.

[0025] According to an advantageous embodiment of the invention, the reference images of the ground taken by the camera are combined to form an overall image of the technical system.

[0026] According to an advantageous embodiment of the invention, the technical system comprises at least one further autonomous vehicle located on the level floor within the system. This vehicle has a map of the system in which reference images of the floor are plotted, and it includes at least one laser scanner for acquiring laser scans and at least one camera for capturing images. During an operational phase, the camera of the further autonomous vehicle captures at least one operational image of the floor, and this image is compared with the reference images plotted on the map. The position of the further autonomous vehicle within the system is determined by comparing the captured operational image with the reference images plotted on the map.

[0027] A technical system according to the invention comprises at least one autonomous vehicle, wherein the autonomous vehicle has at least one laser scanner for recording laser scans for detecting objects, and wherein the autonomous vehicle has a map of the technical system. The technical system is operable using the method according to the invention.

[0028] In the technical system according to the invention, determining the position of an autonomous vehicle during an operational phase using a visual ground map is possible with increased robustness and availability compared to traditional 2D laser navigation with structural raster maps. The determination of the position of an autonomous vehicle is thus safe and precise.

[0029] The invention is not limited to the combination of features in the claims. For a person skilled in the art, further meaningful combinations of claims and / or individual claim features and / or features of the description and / or the figures will arise, in particular from the problem statement and / or the problem arising from a comparison with the prior art.

[0030] The invention will now be explained in more detail with reference to the illustrations. The invention is not limited to the embodiments shown in the illustrations. The illustrations only depict the subject matter of the invention schematically. They show: Fig. 1: a schematic representation of an autonomous vehicle in a technical facility and Fig. 2: a schematic representation of a map of a technical facility.

[0031] Fig. Figure 1 shows a schematic representation of an autonomous vehicle 1 in a technical facility. The technical facility is an industrial application, for example a production plant, an industrial hall, a paint shop, or a logistics center.

[0032] The technical system comprises several autonomous vehicles 1. Only one such autonomous vehicle 1 is shown in the present illustration. The autonomous vehicles 1 serve primarily for transporting material within the technical system. The autonomous vehicle 1 shown here is located on a flat surface 5 of the technical system. A vertical direction Z runs perpendicular to the surface 5.

[0033] The autonomous vehicle 1 has a drive system, an electrical energy storage device for supplying the drive system, and a control unit for controlling the drive system. The drive system includes, for example, an electric motor, a transmission, and drive wheels. The electrical energy storage device is, in particular, a rechargeable battery.

[0034] Furthermore, the autonomous vehicle 1 has a communication device for wireless communication with other autonomous vehicles 1 and with other participants, such as a central server, within the technical system. The communication device of the autonomous vehicle 1 is designed, for example, for data transmission via WLAN, Bluetooth, or light.

[0035] The autonomous vehicle 1 is equipped with two laser scanners 2. The laser scanners 2 are used to record laser scans in a horizontal scan plane for the detection of objects within the technical system. The horizontal scan plane runs parallel to, or at least approximately parallel to, the floor 5 on which the autonomous vehicle 1 is located. The scan plane is positioned a scan height away from the floor 5. For example, the scan height is 20 centimeters.

[0036] When an object is detected, the laser scanners 2 each record the distance to the object and the direction in which the object is located. The laser scanners 2 are mounted at opposite corners of the autonomous vehicle 1 and each detect objects within an angular range of approximately 270°.

[0037] The technical facility contains objects such as walls, columns, production machines, pallets, crates, and containers. These objects are primarily located on floor 5 of the facility. Autonomous vehicle 1 has a map of the technical facility. The objects in question are marked on this map.

[0038] The technical system includes a central server. This server also maintains a map of the system. Among other things, the server is used for offline updates of the map. When necessary, detected changes from autonomous vehicle 1 are transmitted to the server. The server then updates the map by recording the detected changes. Finally, the server transmits the updated map to autonomous vehicle 1 as needed.

[0039] The server has a communication module for wireless communication with the autonomous vehicles 1 and with other participants in the technical system. In particular, maps of the technical system are transmitted between the communication device of the autonomous vehicle 1 and the communication module of the server 20.

[0040] The autonomous vehicle 1 also includes a receiver unit, not shown here. The receiver unit is located on the underside of the autonomous vehicle 1. Energy can be inductively transferred to the receiver unit from a current-carrying primary conductor. The energy inductively transferred from the primary conductor to the receiver unit is used, for example, to charge the electrical energy storage device of the autonomous vehicle 1. The receiver unit includes a transformer head, which has a coil.

[0041] The autonomous vehicle 1 has a camera 16 for capturing images. The camera 16 is arranged on the autonomous vehicle 1 such that one optical axis of the camera 16 runs in the vertical direction Z. The camera 16 is therefore directed towards the floor 5 of the technical installation and is specifically designed to capture images of the floor 5.

[0042] The autonomous vehicle 1 has an odometric system for determining its position. The odometric system includes sensors for detecting steering movements and wheel rotations, particularly of the drive wheels, of the autonomous vehicle 1. The autonomous vehicle 1 also has a GPS sensor for determining its position.

[0043] The autonomous vehicle 1 is equipped with a navigation computer. The navigation computer is used to navigate the autonomous vehicle 1 within the technical facility. The navigation computer plans routes to be driven while avoiding collisions with objects that are recorded on the map of the technical facility.

[0044] Fig. Figure 2 shows a schematic representation of a map of a technical facility. The objects recorded on the map are not shown. The map of the technical facility will be expanded during a reference phase.

[0045] During the reference phase, the autonomous vehicle 1 travels along a trajectory 85 on the floor 5 through the technical facility. The respective position of the autonomous vehicle 1 is determined, for example, by means of the odometric system, the GPS sensor, or by comparing laser scans with the objects recorded on the map. The trajectory 85 is chosen in such a way as to avoid a collision with the objects recorded on the map.

[0046] During travel along trajectory 85, the camera 16 of the autonomous vehicle 1 captures a plurality of reference images 81 of the ground 5. Simultaneously with the capture of each such reference image 81, the position of the camera 16 within the technical system is determined.

[0047] First, the position of the autonomous vehicle 1 within the technical system is determined when a reference image 81 is captured. The position of the autonomous vehicle 1 corresponds, for example, to the position of its center of gravity or the center point of a laser scanner 2. The respective position of the camera 16 is then calculated from the position of the autonomous vehicle 1 and the arrangement of the camera 16 on the autonomous vehicle 1.

[0048] The respective position of the autonomous vehicle 1 within the technical installation when a reference image 81 is acquired is determined, for example, by having the laser scanners 2 of the autonomous vehicle 1 acquire laser scans of a part of the technical installation which contains at least one object, and by comparing the laser scans with the objects recorded on the map. The respective position of the autonomous vehicle 1 is then determined by comparing the laser scans with the objects recorded on the map.

[0049] Alternatively or additionally, the respective position of the autonomous vehicle 1 in the technical system is determined by means of the odometric system when a reference image 81 is taken. Alternatively or additionally, the respective position of the autonomous vehicle 1 in the technical system is determined by means of the GPS sensor when a reference image 81 is taken.

[0050] The reference images 81 taken by camera 16 are each entered on the map at a location corresponding to the respective position of camera 16 within the technical installation at the time the respective reference image 81 was taken. The reference images 81 taken by camera 16 of the ground 5 are then combined to form a complete image of the technical installation.

[0051] The map of the technical facility is thus expanded during the reference phase to include the recorded reference images 81 as well as the overall image. The reference images 81 contain, in particular, detailed information on the surface properties of the ground 5, especially microtextures, macrotextures, scratches, and lines. This information from the reference images 81 is later used to determine the positions of the autonomous vehicles 1.

[0052] The extended map of the technical system is then transmitted from autonomous vehicle 1 to the server. The server transmits the extended map of the technical system to all autonomous vehicles 1. Thus, the autonomous vehicles 1 are able to determine their respective position within the technical system using the extended map and the reference images 81 it contains.

[0053] During an operational phase, to determine the position of the autonomous vehicle 1 within the technical installation, the camera 16 of the autonomous vehicle 1 captures an operational image of the ground 5. The captured operational image is compared with the reference images 81 plotted on the map. The position of the autonomous vehicle 1 within the technical installation is then determined by comparing the captured operational image with the reference images 81 plotted on the map. Reference symbol list 1 autonomous vehicle 2 laser scanners 5 Floor 16 cameras 81 Reference image 85 Trajectory Z Vertical direction

Claims

[1] Method for operating a technical plant which comprising at least one autonomous vehicle (1) wherein the autonomous vehicle (1) has a map of the technical installation, and wherein the autonomous vehicle (1) is located on a level floor (5) in the technical installation, and wherein the autonomous vehicle (1) has at least one camera (16) for taking pictures, and wherein the camera (16) is directed towards the floor (5) of the technical installation, and wherein during a reference phase, a plurality of reference images (81) of the ground (5) are taken by the camera (16), and wherein in each case a position of the camera (16) in the technical system is determined when taking a reference image (81), and wherein The reference images (81) taken by the camera (16) are entered in the map at a location which corresponds to the respective position of the camera (16) in the technical system when the respective reference image (81) was taken. characterized by , that the autonomous vehicle (1) has at least one laser scanner (2) for recording laser scans for detecting objects, and that before the majority of reference images (81) of the ground (5) are taken by the camera (16), the roughness of the ground (5) is increased. [2] Method according to claim 1, characterized by , that the camera (16) is arranged on the autonomous vehicle (1) such that an optical axis of the camera (16) extends in a vertical direction (Z), and that the vertical direction (Z) is perpendicular to the ground (5). [3] Method according to any of the preceding claims, characterized by, that a position of the autonomous vehicle (1) in the technical system is determined when a reference image (81) is taken, and that the respective position of the camera (16) is calculated from the position of the autonomous vehicle (1) and the arrangement of the camera (16) on the autonomous vehicle (1). [4] Method according to claim 3, characterized by , that the respective position of the autonomous vehicle (1) in the technical system is determined when a reference image (81) is taken, by a laser scan of a part of the technical system which has at least one object is taken by at least one laser scanner (2) of the autonomous vehicle (1), and the laser scan is compared with at least one object recorded on the map, and the respective position of the autonomous vehicle (1) is determined by comparing the laser scan with the at least one object recorded on the map. [5] Method according to claim 3, characterized by , that the autonomous vehicle (1) has an odometric system for determining the position of the autonomous vehicle (1), and that the respective position of the autonomous vehicle (1) in the technical system is determined when a reference image (81) is taken using the odometric system. [6] Method according to claim 3, characterized by , that the autonomous vehicle (1) has a GPS sensor for determining the position of the autonomous vehicle (1), and that the respective position of the autonomous vehicle (1) in the technical system is determined when a reference image (81) is taken using the GPS sensor. [7] Method according to any of the preceding claims, characterized by , that the reference images (81) taken by the camera (16) of the ground (5) are combined to form an overall image of the technical installation. [8] Method according to any of the preceding claims, characterized by , that the technical installation includes at least one further autonomous vehicle (1) which is located on the level ground (5) in the technical installation, and which has a map of the technical installation in which reference images (81) of the ground (5) are entered, and which at least one laser scanner (2) for recording laser scans and at least one camera (16) for taking pictures, and that during an operating phase at least one operating image of the ground (5) is taken by the camera (16) of the further autonomous vehicle (1), and that the taken operating image is compared with the reference images (81) entered in the map, and that the position of the other autonomous vehicle (1) in the technical installation is determined by comparing the recorded operating image with the reference images (81) entered in the map. [9] Technical installation, comprising at least one autonomous vehicle (1) wherein the autonomous vehicle (1) has at least one laser scanner (2) for recording laser scans for detecting objects, and wherein the autonomous vehicle (1) has a map of the technical system, and wherein the technical system is operated using the method according to one of the preceding claims.

Images