Multimodal fiducial markers, heterogeneous perception devices, and multimodal systems incorporating both

Abstract

This application relates to a multimodal system for accurately detecting a target location for a vehicle and navigating the vehicle to the target location. The system includes at least one multimodal fiducial marker (4) and a heterogeneous perception device (8) coupled to an unmanned or manned vehicle (9). The marker (4) can be detected by analyzing visual, thermal, and point cloud data. The heterogeneous perception device (8) uses cameras and range sensors to collect both photometric and radiometric data. This multimodal system improves the situational awareness of the vehicle (9) and increases its detection and navigation capabilities. As a result, the system (4, 8) operates in a multimodal manner under challenging environmental conditions, providing the vehicle (9) with the ability to perform robust, redundant, and reliable detection of the vehicle's target location (10).

Description

[Technical Field]

[0001] The present invention relates generally to detection systems and methods, and more particularly to systems, methods, and apparatus for assisting unmanned or manned vehicles in detecting target locations. [Background technology]

[0002] Unmanned vehicles are autonomous robots that have been used in several applications due to their ability to maintain very stable navigation and approach places that are not easily accessible, resulting in the collection of high-quality data. These vehicles can be water-based, ground-based, or air-based, and their main application areas include several industries such as film, military, agriculture, and surveillance. Furthermore, some inspection tasks are performed by robotic vehicles, meaning a reduction in cost and total time consumption. A typical mission cycle for a robotic vehicle involves departure from a base, performing its task, and returning to the base or other target location.

[0003] A key feature that enables autonomous operation is the vehicle's ability to accurately land / berth at a predetermined target location. A critical task to enable a vehicle to successfully land / berth at a desired target location is the vehicle's ability to autonomously detect and recognize the target location in real time. If the vehicle loses track of the target location, the vehicle may jeopardize mission continuity and may damage the vehicle's equipment.

[0004] For unmanned aerial vehicles such as drones, the 1-3 meter landing accuracy of conventional GPS (Global Positioning System)-based methods does not meet the high accuracy required for some applications that require centimeter-level precision. Furthermore, the performance of conventional visual methods does not feature the required robustness and precision to operate in challenging weather and lighting conditions. And in most extreme scenarios, manned vehicles experience these difficulties equally.

[0005] Achieving precise docking / landing of unmanned or manned vehicles requires accurate readings of the relative pose between the vehicle and a detected target location. Some robotic applications use artificial markers to label locations of interest and to help the unmanned vehicle, or the autopilot module of a manned vehicle, read its relative position. The most common types of artificial markers used include visual markers, infrared (IR) beacons, thermal markers, and retroreflective markers. However, independent of the marker type and the sensor used, estimating the relative pose requires knowledge of the marker's pose within a fixed coordinate frame. Knowledge of the marker's actual size is usually also required for a scale factor. This information, combined with the correlation between the detected marker and its actual pose, allows the relative distance from the vehicle to the target location to be estimated.

[0006] When using a calibrated camera sensor, the task of calculating the camera sensor's position (3D (three dimensional) reference point) from the marker and its projection (2D (two dimensional) projection point) onto the image plane is known as the Perspective-n-Point (PnP) problem. There are different solutions that can solve this problem, and their performance depends on the type of marker used. Each of these methods allows for estimating the change from the camera to the marker frame. Then, knowing the change from the camera to the vehicle frame and from the marker to the target position frame, the vehicle pose is calculated. Unlike when using a camera sensor, the use of a ranging sensor such as a 3D LiDAR (Light Detection And Ranging) can directly perceive the depth of the surrounding environment. This property allows for easy calculation of the relative pose between the vehicle and the marker, which can be detected by analyzing the LiDAR point cloud.

[0007] Numerous systems use visual information to identify visual markers for a wide variety of tasks, including for detecting berthing / landing areas. Several studies have been performed in the field of computer vision, which can increase the accuracy of berthing / landing. The most common methods use black and white markers placed on berthing / landing targets. Each marker presents a layout that encodes a unique identification through a binary code. These markers are generally based on regular geometric shapes (such as squares), which make any of them distinguishable from any other. In robotics, some of the most common fiducial markers are AR (Augmented Reality) tags, April tags, and ArUco (Augmented Reality Universidad de Cordoba). The rectangular shape of these markers allows for the extraction of the camera pose from their four corners, as long as the camera is properly calibrated. In the literature, these markers are detected by first extracting the edges of images collected by a visual camera, then filtering the contours that form polygons with four vertices, and finally extracting the binary code for each of the contours. Circular markers, such as Configurable and Combinable (CC) tags and S tags, are also used in some robotics applications. While optimization techniques exist to improve the detectability of these markers, their main drawback is that they depend on lighting and environmental conditions and only work efficiently indoors or during clear daylight hours.

[0008] Another type of marker used to detect the target location area is the use of light-emitting diodes (LEDs). Therefore, the vehicle must be equipped with a sensor capable of capturing the light emitted by the LEDs. The most common method proposes the use of beacons with IR LEDs and IR cameras for target location detection. The main drawback of these IR marker methods is that they are only suitable for indoor operation, given their sensitivity to sunlight. Other drawbacks relate to the short range of the beam and the limited operating range implied by the angle of emission, given that adjusting the angle of the camera introduces significant localization errors.

[0009] As a result, conventional solutions consist of unimodal methods, which reduce generalizability: these methods are limited to controlled scenarios with favorable conditions.

[0010] In summary, providing vehicles with precise target location capabilities is essential for them to operate autonomously and effectively. An unmanned or manned vehicle must be able to detect a target location and perform the necessary maneuvers to reach it without compromising its own safety or the integrity of its surrounding environment.

[0011] However, current solutions do not provide the robustness and reliability required for accurate detection of and navigation to a target location due to their lack of ability to function in highly demanding scenarios, particularly under challenging lighting and weather conditions, including daytime and nighttime operation.

[0012] These facts are disclosed to illustrate the technical problem that the present invention addresses. Summary of the Invention [Means for solving the problem]

[0013] This document discloses a multimodal fiducial marker for relative pose estimation, the multimodal fiducial marker comprising a first component and a second component; and a heat source, the first component and the second component configured to provide a surface, the surface including a first portion of the first component and a second portion of the second component; the heat source configured to heat the surface by thermal conduction through the first and second components; the first portion having a reflectivity different from a reflectivity of the second portion; the first component having a thermal conductivity different from a thermal conductivity coefficient of the second component; and the first and second portions arranged in a pattern of geometric shapes that encode data.

[0014] In one preferred embodiment, the pattern of geometric shapes is a binary code, in particular a thermal optical retroreflective binary identification pattern.

[0015] In one preferred example, the reflectance is a reflectance for visible light.

[0016] In one embodiment, the surface is substantially non-reflective in the infrared light spectrum.

[0017] The following drawings provide preferred embodiments to illustrate the present invention and should not be seen as limiting the present invention. [Brief explanation of the drawings]

[0018] [Figure 1] 1 is a representation of one embodiment of a multimodal fiducial marker of the present application, where the reference numerals indicate: 1 - first portion of the surface of the marker; 2 - second portion of the surface of the marker; 3 - heat source; 4 - multimodal fiducial marker. [Figure 2] 1 is a representation of one embodiment of a heterogeneous perception device of the multimodal system described herein, where reference numbers represent: 5 - 3D - LIDAR unit; 6 - visual camera unit; 7 - thermal camera unit; 8 - heterogeneous perception device. [Figure 3]Illustrative example of a multimodal system described herein, where reference numbers represent: 4—multimodal fiducial marker; 8—heterogeneous perception device; 9—unmanned vehicle (airborne); 10—target location. [Figure 4] 1 is a flowchart illustrating a position estimation procedure performed by a heterogeneous perception device in one embodiment of the multimodal system described herein, where reference numbers represent: 5—3D LIDAR unit; 6—visual camera unit; 7—thermal camera unit; 8—heterogeneous perception device; 8.1—processor-based device; 9—unmanned vehicle; 10—target position. [Figure 5] 5A and 5B show thermal imaging tests comparing acrylic markers with multimodal fiducial markers, where the reference numbers represent: 4—multimodal fiducial marker, 503—acrylic marker, and 505—hot spot. [Figure 6] FIG. 6 shows a thermal imaging test for testing IR reflection, where the reference numeral :601-IR reflection. [Figure 7] FIG. 1 illustrates exemplary acrylic markers and multimodal fiducial markers, where reference numbers represent: 4—multimodal fiducial marker, and 503—acrylic marker. DETAILED DESCRIPTION OF THE INVENTION

[0019] Detailed Description It is therefore an object of the present application to provide a multi-modal system that assists unmanned or manned vehicles in accurately detecting and navigating to a target location.

[0020] Such a system therefore provides the ability to operate under challenging environmental and lighting conditions (at different heights, in intense sunlight, in no light or dark environments, etc.) and in a multimodal manner using photometric and radiometric data (thus including operation in rain and fog) to perform robust, redundant and reliable detection of vehicle target positions.

[0021] To this end, the system includes at least one multimodal fiducial marker and a heterogeneous perception device coupled to the vehicle. The multimodal fiducial marker can be detected and located by analyzing visual, thermal, and point cloud data. It is an active marker that can improve the relative localization of the vehicle, which is particularly relevant for navigation operations in unmanned vehicles. In its operation, the heterogeneous perception device collects both photometric and radiometric data using cameras and distance sensors. The data are fused and combined together using specific methods, which are also described herein.

[0022] Thus, in an advantageous configuration, the present multimodal system comprises: at least one multimodal fiducial marker as described herein; each marker positioned at a target location; and At least one heterogeneous sensory device as described herein It consists of:

[0023] The present multimodal fiducial markers are configured to generate a unique thermally retroreflective binary identification pattern. To this end, advantageous configurations of the markers include: a surface of predetermined geometric shape and a heat source; The surface has a first portion of a first component and a second portion of a second component arranged to form a maker's layout configured to encode a unique identification pattern in a binary code, with each of the first and second components being of a different secondary color. The first component has a reflectance that is different from the reflectance of the second component; The heat source is configured to heat a surface of the marker and has a different thermal conductivity coefficient than the first and second components.

[0024] The heterogeneous perception device of the present application is configured to be coupled to a vehicle and configured to detect multimodal fiducial markers as described herein positioned at target locations on the vehicle. In an advantageous configuration of the device, the device comprises: a visual camera unit; a thermal camera unit; a 3D lidar unit; and a processor-based device; the visual camera unit is configured to collect image data within the visible light spectrum of the target location area to estimate the pose of the marker relative to a coordinate system of the visual camera unit; the thermal camera unit is configured to collect both thermal and radiometric data of the target location area to estimate the pose of the marker relative to a coordinate system of the thermal camera unit; the 3D lidar unit is configured to collect range and radiometric data from the target location area to estimate the pose of the marker relative to a coordinate system of the 3D lidar unit; the processor-based device is programmed to process the marker pose estimation data acquired by the visual camera unit, the thermal camera unit, and the 3D lidar unit to determine a relative pose estimate between the device and the marker to determine a position of the marker; the relative positioning of the visual camera unit, the thermal camera unit, and the 3D lidar unit relative to one another is known; The processor-based device is operable to transmit this location information to the vehicle.

[0025] A method for detecting multimodal fiducial markers using a heterogeneous perception device coupled to a vehicle is also an object of this application, the method comprising the steps of: scanning a target location area with a heterogeneous perception device to obtain target location data; identifying a marker from the target location data; collecting image data, thermal data, radiometric data, and range data from the marker using a sensor portion of the device; estimating the orientation of the marker relative to the coordinate system of each sensor unit of the device; determining a relative pose estimate between the device and the marker based on the estimated pose of the marker obtained in the estimating step, thereby determining a position of the corresponding marker; Transmitting the marker's location information to the vehicle.

[0026] FIG. 1 shows a representation of one embodiment of the present multimodal fiducial marker.

[0027] For the purposes of this application, a multimodal fiducial marker (4) for vehicle relative pose estimation is described, which is operable for any type of vehicle (9), whether manned or unmanned, water-based, land-based, or air-based.

[0028] The marker has a surface of a predetermined geometric shape, i.e. a geometric shape that is predetermined and recognizable by the perceptual device and by the relative pose detection and estimation algorithm. As an example, the marker (4) has a rectangular shape with dimensions of 0.22 x 0.22 x 0.02 meters and can be detected by first extracting the edges of the image collected by the visual camera of the device, followed by filtering the corners to form a polygon with four vertices.

[0029] The surface of the marker has a first portion (1) of a first component and at least a second portion of a second component, and these portions and each component are of a different secondary color and arranged in a manner to form a particular layout of the marker that can encode a unique identification pattern in a binary code. In addition, the first component has a reflectivity that differs from the reflectivity of the second component.

[0030] The marker (4) also comprises a heat source (3) operable to heat a surface of the marker, the first component and the second component having different thermal conductivity coefficients.

[0031] Considered within this set of technical features, the markers (4) are active markers, configured to generate a unique thermal retroreflective binary identification pattern, which can be detectable and locatable by analysis of visual, thermal, and point cloud data, and which improve the relative localization of a vehicle (9) equipped with the heterogeneous perception device (8) of the present application, particularly for precision landing or mooring operations (depending on the type of vehicle (9)).

[0032] In an alternative embodiment of the marker (4) of the present application, the surface geometry of the marker (4) is a planar geometry. More specifically, the first portion (1) and at least the second portion (2) are arranged on a two-dimensional surface of the marker (4). As a result, the marker (4) is configured to generate a two-dimensional thermally retroreflective binary identification pattern.

[0033] Instead, the marker geometry is a spatial geometry. More specifically, the first and at least second portions (1, 2) are arranged as overlapping, distinct shapes. As a result, the marker (4) is configured to generate a three-dimensional, thermally retroreflective binary identification pattern.

[0034] In other alternative embodiments of the marker (4), the binary code used to encode the unique identification pattern of the marker can be any of a library of binary codes such as ArUco or April tag or AR tag codes.

[0035] In other alternative embodiments of the marker (4), the first component is white and the second component is black. Optionally, the first component is blue and the second component is red. Other combinations of colors for the first and second components are provided as examples: green / blue, yellow / brown, or light gray / dark gray.

[0036] In another embodiment of the marker (4), the first component has a reflectivity of at least 70% and the second component is non-retroreflective. Alternatively, the first component is comprised of a layer of retroreflective material having a reflectivity of at least 70%, which is applied on top of at least the first component material. The first component material has a thermal conductivity of at least 88 W / mK. The second component is comprised of at least one non-retroreflective material, which has a maximum thermal conductivity of 0.38 W / mK. Optionally, the first material is aluminum and the second material is cork.

[0037] Finally, in another embodiment of the marker (4), the heat source (3) is operable to heat the surface of the marker to a temperature of at least 100° C. The heat source (3) may be an electrically heated bed, powered by a mains plug or by a battery device.

[0038] In this way, all the technical features related to binary coding, brought about by different color components and retroreflective and thermal properties, can be combined to achieve a synergistic effect that allows such a single marker (4) to be detected and located in adverse environments, representing a compact and easily used solution for multiple applications.

[0039] For the purposes of this application, we describe a heterogeneous perception device (8) coupled to a vehicle (9) and configured to detect the multimodal fiducial markers (4) previously described.

[0040] The device (8) is configured to perceive multimodal information, enabling the vehicle (9) to which it is coupled to successfully land / berth in hostile environments. Furthermore, the device is designed for harsh offshore environments and acquires both photometric and radiometric data, such as visual, thermal, and point cloud information.

[0041] The device comprises a visual camera unit (6); a thermal camera unit (7); a 3D lidar unit (5); and a processor-based device (8.1), a visual camera unit configured to collect image data of a target location area within the visible light spectrum to estimate a pose of the marker relative to a coordinate system of the visual camera unit; the thermal camera unit is configured to collect both thermal and radiometric data of the target location area to estimate the pose of the marker relative to a coordinate system of the thermal camera unit; the 3D lidar unit is configured to collect range data and radiometric data from the target location area to estimate an attitude of the marker relative to a coordinate system of the 3D lidar unit; The processor-based device is programmed to process the marker pose estimation data acquired by the visual camera unit, the thermal camera unit, and the 3D lidar unit (6, 7, 5) to determine a relative pose estimate between the device (8) and the marker (4) to determine a position (10) of the marker (4); the relative positioning of the visual camera unit, the thermal camera unit (6, 7), and the 3D lidar unit (5) to one another is known; The processor-based device is operable to transmit this location information to the vehicle (9).

[0042] The visual camera (6) collects images within the visible light spectrum, while the thermal camera (7) collects both thermal and radiometric information of the scene, representing a more robust sensory method independent of lighting conditions. On the other hand, the 3D lidar (5) uses a beam to directly acquire distance data from the surrounding environment, which is represented as a point cloud. While the camera sensor (6) collects denser data for short-range procedures, the 3D lidar (5) has a larger field of view and distance range suitable for long-range operations. Thus, the device (8) not only allows for the collection of multimodal and complementary information about target positions, but also, when coupled to a vehicle (9), plays an important and meaningful role in navigation operations, increasing situational awareness of the operating scenario and contributing to safer operation of the vehicle (9).

[0043] The pose estimates provided by the sensors (5, 6, 7) must be combined to output a single, redundant localization of the detected marker. To this end, a weighted average is applied to ensure short processing times and increase computational efficiency in embedded systems, ensuring real-time detection and relative pose estimation. In particular, and in other embodiments of the device (8), a processor-based device (8.1) is configured to generate a relative pose estimate between the device (8) and the marker (4). (outside 1) JPEG2025537290000002.jpg1413 is programmed to determine the following method:

number

number

[0044] For the purposes of this application, a method is described for detecting a multimodal fiducial marker (4) using a heterogeneous perception device (8) coupled to a vehicle (9), the marker (4) being positioned at a target location (10). The method comprises: Scanning a target location area using a heterogeneous perception device (8) to obtain target location data; identifying a marker (4) from the target location data; collecting image data, thermal data, radiometric data, and distance data from the marker (4) using the sensor portion (5, 6, 7) of the device (8); estimating the orientation of the marker relative to the coordinate system of each sensor unit (5, 6, 7) of the device (8); determining a relative pose estimate between the device (8) and the marker (4) based on the estimated pose of the marker (4) obtained in the previous step, and determining a corresponding position (10) of the marker (4); transmitting the position information of the marker (4) to the vehicle (9); Includes.

[0045] The present application also describes a multimodal system, the multimodal system comprising: at least one multimodal fiducial marker (4) as described herein; at least one heterogeneous perception device as described herein; Each marker (4) is positioned at a target location (10).

[0046] More specifically, the system includes one or more vehicles (9), each vehicle (9) coupled with a heterogeneous perception device, the system configured to operate according to the method for detecting multimodal fiducial markers described herein. The vehicles can be unmanned or manned, and can be water-based, land-based, or air-based. Optionally, the vehicle (9) is a drone, a watercraft, or an automated guided vehicle.

[0047] While variables such as altitude, lighting conditions, and marker occlusion caused by environmental conditions reduce the detection rate of current technology systems, especially in some complex maneuvering scenarios, the complementarity of the device (8) and marker (4) increases robustness and redundancy in target location detection and landing / berthing operations compared to other standard, limited systems.

[0048] Based on the technical description given, below we present, as examples, some scenarios of the application of the system, where multimodal fiducial markers (4) and heterogeneous perception devices (8) are used for relative pose estimation and manipulation of unmanned or manned vehicles (9).

[0049] Drone Landing:

[0050] Detecting and locating the landing area using markers (4) and devices (8) allows for safe, precise, and reliable landings. This autonomous capability is useful for several rotary-wing drones (9) coupled with devices (8), both manned and unmanned (fully autonomous and remotely controlled).

[0051] Package Delivery (Air):

[0052] The drone (9) has a device (8) coupled to its structure that allows it to detect a marker placed at a specific target location (10), land and deliver a package. Another possibility is to detect the target (10) and drop a package into the air (with or without a parachute).

[0053] Wind Farm Inspection:

[0054] The precision landing capability afforded by the use of the above system allows drones (9) to be stationed at wind farms (both onshore and offshore) with devices (8) coupled to the structure, allowing for more frequent and extensive inspections. In this case, markers (4) are placed on the turbine structure itself, or on a suitable platform for the drone to land on.

[0055] Vessel and / or ground vehicle berthing:

[0056] It enables precise relative localization of berthing stations (10) for both surface ships and ground vehicles (9), such as rovers and AGVs (Automated Guided Vehicles), to assist in berthing operations.

[0057] Tests were performed comparing a 4 mm thick plate acrylic marker with the disclosed multimodal fiducial markers with the same dimensions, code, and heated bed as the heat source.

[0058] After 5 hours and 10 minutes, the inside of the acrylic marker was hotter but this did not appear to adversely affect detection; after approximately 5 hours and 35 minutes, the corners of the acrylic marker were foaming / deforming due to the increased temperature; after 6 hours and 30 minutes, the acrylic marker was observed to visibly foam / deform in a very rapid / easily deformable manner.

[0059] A thermal radiation reflection test was performed using a soldering iron as the hot body (~450°C), and the soldering iron was moved above the marker so that the camera picked up only indirect / reflected radiation.

[0060] Note that the effect of reflections from the hot object can be seen and the detection speed deteriorates significantly over time. The larger the hot object, the more significant the effect will be.

[0061] If the same reflection test is performed on the disclosed multimodal fiducial marker, the reflection of a soldering iron on the floor of a room can be seen in the thermal image with no effect.

[0062] In summary, over time, the acrylic marker heats up and the thermal contrast required to detect the code is no longer present; the acrylic marker has thermal radiation reflective properties that allow "hot" artifacts to adversely affect the detection of the code; and the acrylic marker is not perfectly flat, which reduces the accuracy of estimating (and even detecting) the marker's location.

[0063] More generally, acrylic fiducial markers present several problems, namely: they are not robust to heating because they can flex, and they are not robust to atmospheric conditions in outdoor environments; while acrylic acts primarily as a filter, it is prone to continuous heating, particularly in the center of the acrylic marker, and thus the temperature contrast tends to deteriorate significantly over time; furthermore, acrylic is not a visual marker and is not radiometric at lidar frequencies.

[0064] Whenever "comprising" is used in this document, it contemplates the presence of stated features, values, steps or components, but does not exclude the presence or addition of one or more other features, values, steps, components or groups thereof.

[0065] The present invention should in no way be seen as limited to the described embodiments, as those of ordinary skill in the art will envision many possible modifications thereof. The above-described embodiments can be combined.

[0066] The following claims further detail particular embodiments of the invention.

Claims

1. A multimodal fiducial marker (4) for relative pose estimation, a first component and a second component; a heat source (3), the first component and the second component are configured to provide a surface, the surface including a first portion (1) of the first component and a second portion (2) of the second component; the heat source is configured to heat the surface by thermal conduction through the first component and the second component; the first portion has a reflectivity different from the reflectivity of the second portion; the first component has a thermal conductivity coefficient different from the thermal conductivity coefficient of the second component; The multimodal fiducial marker (4) wherein the first and second portions are arranged in a pattern of geometric shapes that encode data.

2. 2. The multimodal fiducial marker (4) according to claim 1, wherein the geometric pattern is a binary code, in particular a thermal optical retroreflective binary identification pattern.

3. 3. The multimodal fiducial marker (4) of claim 1 or 2, wherein the reflectance is a reflectance of visible light.

4. A multimodal fiducial marker (4) according to any one of claims 1 to 3, wherein the surface is substantially non-reflective in the infrared light spectrum.

5. the surface geometry is a planar geometry, the first portion (1) and at least the second portion (2) are arranged on a two-dimensional surface of the multimodal fiducial marker (4), and the multimodal fiducial marker (4) is configured to generate a two-dimensional thermal retroreflective binary identification pattern; or 2. The multimodal fiducial marker (4) of claim 1, wherein the geometry of the multimodal fiducial marker is a spatial geometry, the first portion and at least the second portion (1, 2) are arranged as overlapping of different shapes, and the multimodal fiducial marker (4) is configured to generate a three-dimensional thermal retroreflective binary identification pattern.

6. 6. A multimodal fiducial marker (4) according to any one of claims 1 to 5, wherein the binary code used to encode the unique identification pattern of the multimodal fiducial marker is an ArUco or AprilTag or ARTag code.

7. 7. A multimodal fiducial marker (4) according to any one of claims 1 to 6, wherein the first component is white and the second component is black, and optionally the first component is blue and the second component is red.

8. A multimodal fiducial marker (4) according to any one of claims 1 to 7, wherein the first component has a reflectivity of at least 70% and the second component is non-retroreflective.

9. the first component comprises a layer of retroreflective material having a reflectivity of at least 70%, the layer of retroreflective material being applied at least on top of a substrate of the first component; A multimodal fiducial marker (4) according to any one of claims 1 to 4, wherein the second component is made of at least one material that is non-retroreflective.

10. 7. The multimodal fiducial marker (4) of claim 6, wherein the material of the first component has a thermal conductivity of at least 88 W / mK and the material of the second component has a maximum thermal conductivity of 0.38 W / mK, and optionally, the material of the first component is aluminum and the material of the second component is cork.

11. 11. A multimodal fiducial marker (4) according to any of claims 1 to 10, wherein the heat source (3) is operable to heat the surface of the multimodal fiducial marker to a temperature of at least 100°C, preferably the heat source (3) being an electric heating bed, the electric heating bed being powered by a mains plug or a battery device.

12. A heterogeneous perception device (8) configured to be coupled to a vehicle (9) and configured to detect a multimodal fiducial marker (4) according to any one of claims 1 to 8, wherein the multimodal fiducial marker (4) is positioned at a target location (10), and the heterogeneous perception device (8) comprises: A visual camera unit (6); Thermal camera unit (7), 3D Rider Club (5) and a processor-based device (8.1), the visual camera unit is configured to collect image data of a target location area in the visible light spectrum to estimate a pose of the multimodal fiducial marker relative to a coordinate system of the visual camera unit; the thermal camera unit is configured to collect both thermal and radiometric data of the target location area to estimate a pose of the multimodal fiducial marker relative to a coordinate system of the thermal camera unit; the 3D LIDAR unit is configured to collect range and radiometric data from the target location area to estimate a pose of the multimodal fiducial marker relative to a coordinate system of the 3D LIDAR unit; the processor-based device is programmed to process data acquired by the visual camera unit, the thermal camera unit, and the 3D lidar unit (6, 7, 5) that estimate the pose of the multimodal marker to determine a relative pose estimate between the heterogeneous perception device (8) and the multimodal reference marker (4) to determine the position of the multimodal reference marker (4); The relative positions of the visual camera unit, the thermal camera unit (6, 7), and the 3D lidar unit (5) are known; The processor-based device is operable to transmit the location information to the vehicle (9). Heterogeneous Perception Devices (8).

13. The processor-based device (8.1) generates a relative pose estimate between the heterogeneous perception device (8) and the multimodal reference marker (4). (Outside 1) 【number】 is programmed to determine based on the method of the following formula: [Equation 1] Here, (Outside 2) 【number】 is the position estimate of the heterogeneous perception device (8) relative to the multimodal fiducial marker (4), X i = (x i , y i , z i ) is an estimate of the relative position of the multimodal fiducial marker (4) provided by the visual camera unit (V), the thermal camera unit (T), and the 3D lidar unit (L), λ i is a Boolean variable that is equal to 1 if a sensor corresponding to said heterogeneous perception device detects said multimodal fiducial marker (4), and is equal to 0 otherwise; w i is the dynamic weight, calculated as follows: [Equation 2] Here, (Outer 3) 【number】 The heterogeneous perception device according to claim 9 , wherein σ represents the average error and σ represents the standard deviation.

14. A method for detecting a multimodal fiducial marker (4) according to any one of claims 1 to 8 using a heterogeneous perception device (8) according to claim 9 or 10, wherein the multimodal fiducial marker (4) is positioned at a target location (10) and the heterogeneous perception device (8) is coupled to a vehicle (9), comprising: scanning a target location area using the heterogeneous perception device (8) to obtain target location data; identifying the multimodal fiducial marker (4) from the target position data; collecting the image data, the thermal data, the radiometric data, and the range data from the multimodal fiducial marker (4) using the visual camera portion, the thermal camera portion, and the 3D LIDAR portion (5, 6, 7) of the heterogeneous perception device (8); estimating the pose of the multimodal fiducial marker relative to the coordinate systems of the visual camera unit, the thermal camera unit, and the 3D lidar unit (5, 6, 7) of the heterogeneous perception device (8); determining a relative pose estimate between the heterogeneous perception device (8) and the multimodal reference marker (4) based on the estimated pose of the multimodal reference marker (4) and determining a corresponding position (10) of the multimodal reference marker (4); transmitting information about the location of the multimodal fiducial marker (4) to the vehicle (9); method.

15. At least one multimodal fiducial marker (4) according to any one of claims 1 to 8; and at least one heterogeneous perception device (8) according to claim 9 or 10, A multimodal system in which each of the multimodal fiducial markers (4) is positioned at a target location (10).

16. 13. The multimodal system of claim 12, further comprising one or more vehicles (9), each of said vehicles (9) coupled to one of said heterogeneous perception devices (8), said multimodal system being configured to operate according to the method of claim 11.

17. 14. The multimodal system of claim 13, wherein the vehicle (9) is unmanned or manned, water-based, land-based, or air-based, and optionally the vehicle (9) is a drone, a watercraft, or an automated guided vehicle.

18. Use of a multimodal fiducial marker (4) according to any one of claims 1 to 8 and a heterogeneous perception device (8) according to claim 9 or 10 for relative pose estimation and operation of an unmanned or manned vehicle (9).

Images