Multi-modal fiducial marker, heterogeneous perception device and multi-modal system including same

Abstract

The application relates to a multi-modal system for accurate detection and navigation of a vehicle to a target position. The system comprises at least one multi-modal fiducial marker (4) and a heterogeneous perception device (8) coupled to an unmanned or manned vehicle (9). The marker (4) can be detected by analyzing visual, thermal and point cloud data. The heterogeneous perception device (8) employs cameras and ranging sensors to collect both photometric and radiometric data. The multi-mode system improves the situation awareness of the carrier (9), and improves the detection capability and navigation capability of the carrier (9). Thus, the system (4, 8) provides the vehicle (9) with the ability to operate under harsh environmental conditions via a multi-modal means in order to perform robust, redundant and reliable detection of the vehicle target position (10).

Description

Technical Field

[0001] The present disclosure generally relates to detection systems and methods. More specifically, the present application relates to systems, methods, and devices for assisting an unmanned or manned vehicle in detecting a target position. Background Art

[0002] An unmanned vehicle is an autonomous robot that has been used in several applications due to its ability to maintain very stable navigation and approach hard-to-reach areas to collect high-quality data. The vehicle can be of the water-based, land-based, or air-based type, and its main application areas include several industries such as cinematography, military, agriculture, and surveillance. In addition, the robotic vehicle is performing several inspection tasks, thereby reducing costs and total time consumption. The typical cycle of a robotic vehicle task includes leaving a base station, performing its task, and returning to the base station or another target position.

[0003] An important feature that allows autonomous operation is the vehicle's ability to precisely land / dock at a predetermined target position. The key task that enables the vehicle to successfully land / dock at the desired target position is its ability to autonomously detect and identify the target position in real time. If the vehicle misses the target position, this may endanger the continuity of the mission and damage the vehicle equipment.

[0004] In the case of an unmanned aerial vehicle, such as a drone, the landing accuracy of traditional GPS-based methods of 1 - 3 m cannot meet the high precision required for several applications that require centimeter-level accuracy. In addition, the performance of traditional vision methods does not have the robustness and accuracy required to operate under challenging weather and lighting conditions. In the most extreme cases, manned vehicles also experience these difficulties.

[0005] To achieve precise docking / landing of an unmanned or manned vehicle, it is necessary to accurately retrieve the relative pose between the vehicle and the detected target position. Several robotic applications use artificial markers to mark positions of interest and assist the automatic control module of an unmanned or manned vehicle in retrieving its relative position. The most commonly used types of artificial markers include visual markers, infrared (IR) beacons, thermal markers, and retroreflective markers. However, regardless of the type of marker and the sensor used, relative pose estimation requires knowledge of the pose of the marker in a fixed coordinate system. Generally, it is also necessary to know the actual size of the marker to determine the scale factor. With this information, along with the correct association between the detected marker and its true pose, it is possible to estimate the relative distance from the vehicle to the target position.

[0006] When using a calibrated camera sensor, the task of calculating the position of a marker (3D reference point) and its projection onto the image plane (2D projection point) is known as the Perspective-n-Point (PnP) problem. There are different solutions to solve this problem, and their performance depends on the type of marker used. Each of these methods allows estimating the transformation from the camera to the marker frame. Then, the vehicle pose is calculated by knowing the transformation from the camera to the vehicle frame and from the marker to the target position frame. Different from when using a camera sensor, using a range sensor, such as 3D LiDAR (Light Detection and Ranging), can directly sense the depth of the surrounding environment. This feature enables the relative pose between the vehicle and the marker to be easily calculated, which can be detected by analyzing the LiDAR point cloud.

[0007] Many systems rely on visual information to identify visual markers for various tasks, including docking / landing area detection. Several studies have been conducted in the field of computer vision to improve docking / landing accuracy. The most common approach uses monochromatic markers placed on docking / landing targets. Each marker presents a layout encoding a unique identifier through a binary code. They are usually based on regular geometric shapes (e.g., squares), making any one of them distinguishable from any other marker area. In robotics, some of the most popular fiducial markers are ARTag, AprilTag, and ArUco. As long as the camera is correctly calibrated, the quadrilateral shape of these markers allows the camera pose to be extracted from their four corners. In short, these markers are detected by first extracting the edges of the image collected by the visual camera, then filtering the contour lines that form a polygon with four vertices, and finally extracting their corresponding binary code. Circular markers, such as CCTag and STag, are also used in several robotic applications. Although there are optimization techniques to improve the detectability of these markers, their main drawback is that they are dependent on lighting and environmental conditions and can only operate efficiently indoors or on sunny days.

[0008] Other types of markers used to detect the target position area include the use of light-emitting diodes (LEDs). Therefore, the vehicle must be equipped with a sensor capable of capturing the light emitted by the said LED. The most common approach recommends using beacons with IR LEDs and IR cameras for target position detection. The main drawback of these IR marker methods is that, given their sensitivity to sunlight, they are only suitable for indoor operations. Another drawback is related to the limitation of the operating range implied by the short range and radiation angle of the light beam, as the tilt of the camera produces high positioning errors.

[0009] Therefore, traditional solutions consist of unimodal methods with low generalization potential. These means are limited to controlled scenarios with favorable conditions.

[0010] In summary, in order for vehicles to operate autonomously and effectively, they must be given precise target localization capabilities. Unmanned or manned vehicles must be able to detect the target position and perform the required maneuvers to reach there without compromising their own safety and the integrity of the surrounding environment.

[0011] However, current solutions do not provide the robustness and reliability required to accurately detect and navigate to a target position in high-demand scenarios, especially since it is not sufficient to perform accordingly under challenging lighting and weather conditions, including day and night operations.

[0012] Disclosing these facts is to illustrate the technical problems that the present disclosure is committed to solving. Summary of the Invention

[0013] This document discloses a multimodal fiducial marker for relative pose estimation, including: a first component and a second component, arranged to provide a surface including a first part of the first component and a second part of the second component; a heat source, arranged to heat the surface by heat conduction through the components; the reflectivity of the first part being different from the reflectivity of the second part; and the thermal conductivity of the first component being different from the thermal conductivity of the second component; wherein the parts are arranged to encode a geometric pattern of data.

[0014] In one embodiment, the marker according to the preceding claim, wherein the geometric pattern is a binary code, in particular a thermo-optic retroreflective binary identification pattern.

[0015] In one embodiment, the marker according to any one of the preceding claims, wherein the reflectivity is visible light reflectivity.

[0016] In one embodiment, the marker according to any one of the preceding claims, wherein the surface is substantially non-reflective in the infrared spectrum. Brief Description of the Drawings

[0017] The following drawings provide illustrations for the preferred embodiments of the present disclosure and should not be regarded as limiting the scope of the invention.

[0018] Figure 1 : Illustration of an embodiment of the multimodal fiducial marker of the present application, where the numerical reference symbols represent: 1 - the first part of the surface of the marker; 2 - the second part of the surface of the marker; 3 - the heat source; 4 - the multimodal fiducial marker.

[0019] Figure 2:Illustration of an embodiment of the heterogeneous sensing device of the multimodal system described in this application, where the numerical reference symbols denote: 5–3D-LiDAR unit; 6–visual camera unit; 7–thermal camera unit; 8–heterogeneous sensing device.

[0020] Figure 3 :Illustrative example of the multimodal system described in this application, where the numerical reference symbols denote: 4–multimodal fiducial marker; 8–heterogeneous sensing device; 9–unmanned vehicle (airborne type); 10–target location.

[0021] Figure 4 :Flowchart depicting the position estimation routine performed by the heterogeneous sensing device in an embodiment of the multimodal system described in this application, where the numerical reference symbols denote: 5–3D-LiDAR unit; 6–visual camera unit; 7–thermal camera unit; 8–heterogeneous sensing device; 8.1–processor-based device; 9–unmanned aerial vehicle (airborne type); 10–target location.

[0022] Figure 5A 、 5B :Thermal image test comparing acrylic markers and multimodal fiducial markers, where the numerical reference symbols denote: 4–multimodal fiducial marker, 503–acrylic marker, 505–hot spot.

[0023] Figure 6 :Thermal image test for testing IR reflection, where the numerical reference symbol denotes: 601–IR reflection.

[0024] Figure 7 :Illustrative acrylic markers and multimodal fiducial markers, where the numerical reference symbols denote: 4–multimodal fiducial marker, and 503–acrylic marker. Detailed Description

[0025] Thus, an object of this application is a multimodal system that assists an unmanned or manned vehicle in accurately detecting and navigating to a target location.

[0026] Thus, such a system provides the ability to operate in harsh environmental and lighting conditions (such as at different altitudes, in strong sunlight, in lightless or dark environments; thus, it includes operation affected by rain and fog) via a multimodal approach that utilizes photometric and radiometric data, in order to perform a robust, redundant, and reliable detection of the target location of the vehicle.

[0027] For this purpose, the system includes at least one multimodal fiducial marker and a heterogeneous sensing device coupled to the vehicle. The multimodal fiducial marker is capable of detecting and localizing by analyzing visual, thermal, and point cloud data. This is an active marker that can improve the relative positioning of the vehicle, which is highly relevant to navigation operations, especially in autonomous vehicles. In turn, the heterogeneous sensing device utilizes cameras and range sensors to collect both photometric and radiometric data. The data is fused and combined together by a specific method also described in this application.

[0028] Thus, the multimodal system of this application improves the vehicle's situational awareness and enhances its detection and navigation capabilities. In an advantageous configuration, the system consists of:

[0029] At least one multimodal fiducial marker as described in this application; each marker is located at a target position; and

[0030] At least one heterogeneous sensing device as described in this application.

[0031] The multimodal fiducial marker of this application is adapted to generate a unique thermoreflective binary identification pattern. For this purpose, in an advantageous configuration of the marker, it includes:

[0032] A surface with a defined geometry that has a first part of a first component and a second part of at least a second component. These parts are arranged to form the layout of the marker, which is configured to encode a unique identification pattern in a binary code, and each of the first and second components has a different binary color.

[0033] The reflectivity of the first component is different from that of the second component; and

[0034] A heat source that is operable to heat the surface of the marker; and wherein the first and second components have different thermal conductivities.

[0035] The heterogeneous sensing device of this application is adapted to be coupled to the vehicle and is configured to detect the multimodal fiducial marker described in this application located at the target position of the vehicle. In an advantageous configuration of the device, it includes:

[0036] A visual camera unit configured to collect image data in the visible spectrum of the target position area and estimate the pose of the marker relative to the coordinate system of the visual camera unit.

[0037] A thermal camera unit configured to collect both thermal and radiometric data of the target position area and estimate the pose of the marker relative to the coordinate system of the thermal camera unit.

[0038] A 3D-LiDAR unit, configured to collect ranging and radiometric measurement data from the target location area and estimate the pose of the marker relative to the 3D-LiDAR unit coordinate system.

[0039] A processor-based device:

[0040] Programmed to process the pose estimation data of the marker obtained by the vision, thermal, and 3D-LiDAR units to determine the relative pose estimation between the device and the marker and determine the position of the marker; the relative positioning between the vision and thermal camera units and the 3D-LiDAR unit is known; and

[0041] Operable to send the position information to the vehicle.

[0042] Another object of the present application is a method for detecting a multi-modal fiducial marker using a heterogeneous sensing device coupled to a vehicle. The method includes the following steps:

[0043] Scanning a target location area using the heterogeneous sensing device to obtain target location data;

[0044] Identifying a marker from the target location data;

[0045] Collecting image, thermal, radiometric measurement, and ranging data from the marker using the sensor units of the device;

[0046] Estimating the pose of the marker relative to the coordinate system of each sensor unit of the device;

[0047] Determining the relative pose estimation between the device and the marker based on the estimated pose of the marker obtained in iv. and determining the corresponding position of the marker;

[0048] Sending the position information of the marker to the vehicle.

[0049] Figure 1 A diagram showing an embodiment of the multi-modal fiducial marker of the present application.

[0050] In one object of the present application, a multi-modal fiducial marker (4) for relative pose estimation of a vehicle is described. The marker (4) is operable for any type of vehicle (9), whether manned or unmanned, water-based, land-based, or air-based type.

[0051] The marker includes a surface defining a geometry, i.e., a geometry that is predefined and recognizable by the sensing device and the relative pose detection and estimation algorithm. For example, the marker (4) has a quadrilateral shape with dimensions of 0.22×0.22×0.02 meters and can be detected by first extracting the edges of the image collected by the visual camera of the device and then filtering the contour lines to form a polygon with four vertices. Circular markers can also be used.

[0052] The surface of the marker has a first part (1) of a first component and at least a second part (2) of a second component. Such parts and the corresponding components have different binary colors and are arranged in a manner that forms a specific layout of the marker, and the specific layout can encode a unique identification pattern in a binary code. In addition, the reflectivity of the first component is different from that of the second component.

[0053] The marker (4) further includes a heat source (3) that is operable to heat the surface of the marker, and wherein the first and second components have different thermal conductivities.

[0054] In view of being included in this set of technical features, the marker (4) is an active marker suitable for generating a unique thermo - retro - reflective binary identification pattern, allowing it to be detected and located by analyzing visual, thermal, and point - cloud data. This active marker improves the relative positioning of the vehicle (9) equipped with the heterogeneous sensing device (8) of this application, especially for precise landing or docking maneuvers (depending on the type of vehicle (9)).

[0055] In an alternative embodiment of the marker (4) of this application, the geometry of its surface is a planar geometry. More specifically, the first part (1) and at least the second part (2) are arranged on the two - dimensional surface of the marker (4). Therefore, the marker (4) is suitable for generating a two - dimensional thermo - retro - reflective binary identification pattern.

[0056] Alternatively, the geometry of the marker is a spatial geometry. More specifically, the first and at least the second parts (1, 2) are arranged in a variety of different shapes. Therefore, the marker (4) is suitable for generating a three - dimensional thermo - retro - reflective binary identification pattern.

[0057] In another alternative embodiment of the marker (4), the binary code used to encode the unique identification pattern of the marker can be any one in the binary code library, such as ArUco, AprilTAg, or ARTag code.

[0058] In another alternative embodiment of the marker (4), the first component is white and the second component is black. Optionally, the first component is blue and the second component is red. Other color combinations of the first and second components are presented by way of example: green / blue, yellow / brown, or light gray / dark gray.

[0059] In another embodiment of the marker (4), the reflectivity of the first component is at least 70%, and the second component is non-retroreflective. Alternatively, the first component consists of a layer of retroreflective material with a reflectivity of at least 70%, and the layer of retroreflective material is applied on top of at least the first component material. The material of the first component may have a thermal conductivity of at least 88 W / m -1 K -1 . With respect to the second component, it consists of at least one non-retroreflective material, and the maximum thermal conductivity of the material is 0.38 W / m -1 K -1 . Optionally, the first material is aluminum and the second material is cork.

[0060] Finally, in another embodiment of the marker (4), the heat source (3) is operable to heat the surface of the marker to a temperature of at least 100 °C. The heat source (3) can be an electric heating bed powered by a power plug or a battery unit.

[0061] In this way, by combining all the technical features related to binary coding provided by different color components, as well as the retroreflective performance and thermal performance, a synergistic effect can be achieved, enabling this single marker (4) to be detected and located in adverse environments, representing a compact and easy-to-use solution for various applications.

[0062] In one object of the present application, a heterogeneous sensing device (8) is described, which is coupled to a vehicle (9) and is configured to detect the multi-modal fiducial marker (4) that has been described, and the marker (4) is located at a target position (10) of the vehicle.

[0063] The device (8) is adapted to sense multi-modal information to allow the vehicle (9) coupled thereto to successfully land / dock in adverse environments. In addition, the device is designed for harsh marine environments and obtains both photometric and radiometric measurement data, such as visual, thermal, and point cloud information.

[0064] The device includes:

[0065] A visual camera unit (6), configured to collect image data in the visible spectrum of the target position area and estimate the pose of the marker relative to the coordinate system of the visual camera unit;

[0066] A thermal camera unit (7), configured to collect both thermal and radiation measurement data of the target position area and estimate the pose of the marker relative to the coordinate system of the thermal camera unit;

[0067] A 3D-LiDAR unit (5), configured to collect range and radiation measurement data from the target position area and estimate the pose of the marker relative to the 3D-LiDAR unit coordinate system;

[0068] A processor-based device (8.1):

[0069] Programmed to process the pose estimation data of the marker obtained by the vision, thermal, and 3D-LiDAR units (6, 7, 5) to determine the relative pose estimation between the device (8) and the marker (4), and to determine the position (10) of the marker (4); the relative positioning between the vision and thermal camera units (6, 7) and the 3D-LiDAR unit (5) is known; and

[0070] Operable to send the position information to the vehicle (9).

[0071] Since the vision camera (6) collects images in the visible spectrum, while the thermal camera (7) collects thermal and radiation measurement information of the scene, it represents a more robust sensing method that is independent of lighting conditions. On the other hand, the 3D LiDAR (5) uses laser beams to directly obtain range data represented in a point cloud from the surrounding environment. Although the camera sensor (6) collects denser data for close-range procedures, the 3D LiDAR (5) has a larger field of view and range, suitable for long-range operations. Therefore, the device (8) not only allows the collection of multimodal and complementary information about the target position, but also plays an important and significant role in navigation control after being coupled with the vehicle (9), improving the situation awareness of the operation scenario and contributing to the safer operation of the vehicle (9).

[0072] The pose estimations given by each sensor (5, 6, 7) need to be fused to output a single redundant positioning of the detected marker. For this purpose, weighted averaging is applied to ensure short processing time and improve the computational efficiency of the embedded system, thus guaranteeing real-time detection and relative pose estimation. In particular, and in another embodiment of the device (8), the processor-based device (8.1) is programmed to determine the relative pose estimation between the device (8) and the marker (4) based on the following method

[0073]

[0074]

[0075]

[0076] wherein

[0077] is the position estimate of the device (8) relative to the marker (4);

[0078] X i =(x i , y i , z i ) is the estimate of the relative position of the marker (4) given by the vision camera unit (V), the thermal camera unit (T), and the 3D-LiDAR unit (L);

[0079] λ i is a Boolean variable that equals 1 when its corresponding sensor detects the marker (4) and 0 otherwise; and

[0080] w i is the dynamic weight calculated according to the following expression:

[0081]

[0082] where

[0083] represents the average error, and σ represents the standard deviation.

[0084] In one object of the present application, a method for detecting a multi-modal fiducial marker (4) using a heterogeneous sensing device (8) coupled to a vehicle (9) is described, and the marker (4) is located at a target position (10). The method includes:

[0085] scanning the target position area using the heterogeneous sensing device (8) to obtain target position data;

[0086] identifying the marker (4) from the target position data;

[0087] collecting image, thermal, radiation measurement, and ranging data from the marker (4) using the sensor units (5, 6, 7) of the device (8);

[0088] estimating the pose of the marker relative to the coordinate system of each sensor unit (5, 6, 7) of the device (8);

[0089] based on the estimated pose of the marker (4) obtained in iv., determining the relative pose estimate between the device (8) and the marker (4), and determining the corresponding position (10) of the marker (4);

[0090] sending the position information of the marker (4) to the vehicle (9).

[0091] The present application also describes a multimodal system, comprising:

[0092] at least one multimodal fiducial marker (4) as described in the present application; each marker (4) being positioned at a target location (10); and

[0093] at least one heterogeneous sensing device as described in the present application.

[0094] More specifically, the system includes one or more vehicles (9), each vehicle (9) being coupled with a heterogeneous sensing device, and the system being configured to operate according to the method for detecting multimodal fiducial markers described in the present application. The vehicle (9) can be of the unmanned or manned type and can be of the waterborne, land-based or airborne type. Optionally, the vehicle (9) is a drone, a ship or an automated guided vehicle.

[0095] Compared with other standard and limited systems, the complementarity of the device (8) and the marker (4) improves the robustness and redundancy in target location detection and precise landing / docking tasks, especially in several complex operation scenarios where variables such as altitude, lighting conditions and marker occlusion caused by environmental conditions reduce the detection rate of existing systems.

[0096] Based on the technical description made, several application scenarios of the system are presented below by way of example, where the multimodal fiducial marker (4) and the heterogeneous sensing device (8) are used for relative pose estimation and control of unmanned or manned vehicles (9).

[0097] Drone landing:

[0098] Using the marker (4) and the device (8) to detect and locate the landing area allows for a safe, accurate and reliable landing. This autonomous skill is applicable to several rotary-wing drones (9) coupled with the device (8), including manned and unmanned (fully autonomous and remotely operated) ones.

[0099] Parcel delivery (by air):

[0100] The drone (9) has a device (8) coupled to its structure such that it can detect the marker (4) placed at a specific target location (10), land and deliver the parcel. Another possibility includes detecting the target (10) and dropping the parcel in the air (with or without a parachute).

[0101] Wind farm inspection:

[0102] Using the precise landing capabilities provided by the system, a drone (9) with a device (8) structurally coupled thereto is enabled to remain in a wind farm (onshore and offshore), thereby allowing for more frequent and extensive inspections. In this case, markers (4) are placed on the turbine structure itself or on a platform suitable for drone landing.

[0103] Docking of ships and / or ground vehicles:

[0104] The use of the system enables precise relative positioning of docking stations (10) for both surface ships and ground vehicles (9) (such as rovers and AGVs) to assist in docking maneuvers.

[0105] Tests were conducted comparing an acrylic marker of a 4mm thick plate having the same dimensions, code, and a heated bed as the heat source with the disclosed multimodal fiducial marker.

[0106] It was observed that after 5:10 minutes, the interior of the acrylic marker became hot, but this did not seem to affect detection; at approximately 5:35 minutes, the corners of the acrylic marker bubbled / deformed as the temperature increased; at 6:30 minutes, the acrylic marker was visually bubbled / deformed, i.e., deformed quickly / easily.

[0107] Thermal radiation reflection tests were conducted using a soldering iron as the heat body (~450 °C), moving the soldering iron above the marker so that the camera only picked up the indirect / reflected radiation.

[0108] The effect of the reflection from the heat body could be observed, noting that the detection rate deteriorated quite significantly over time. This would be even more pronounced if the heat body were larger.

[0109] For the same reflection test on the disclosed multimodal fiducial marker, the reflection of the soldering iron on the room floor could be seen in the thermal image, with no effect.

[0110] In summary, over time, the acrylic marker gets hot and no longer requires a thermal contrast to detect the code; and the acrylic marker has thermal radiation reflection characteristics that allow for the generation of "thermal" artifacts, which have a negative impact on code detection; the acrylic marker is not completely flat, which reduces the accuracy of estimating the marker position (and even detection).

[0111] More generally, there are several problems with acrylic fiducial markers, namely: they are not mechanically robust to heating or to atmospheric conditions in the outdoor environment as they warp; although the acrylic initially acts as a filter, it continuously heats up, especially at the center of the acrylic marker, so the temperature contrast tends to deteriorate significantly over time; in addition, acrylic is not a visual marker and is not radiometric at the frequencies of LiDAR.

[0112] As used in this document, the term "comprising" is intended to mean the presence of the stated features, integers, steps, components, but does not preclude the presence or addition of one or more other features, integers, steps, components or groups thereof.

[0113] The present disclosure should not in any way be construed as limited to the described embodiments, and many possibilities of modification thereof will be foreseen by those of ordinary skill in the art. The above embodiments are combinable.

[0114] The following claims further define specific embodiments of the present disclosure.

Claims

1. A multi-modal fiducial marker (4) for relative pose estimation, comprising: A first component and a second component, arranged to provide a surface comprising a first part (1) of the first component and a second part (2) of the second component; A heat source (3), arranged to heat the surface by heat conduction through the components; The first part has a reflectance different from that of the second part; And, The first component has a thermal conductivity different from that of the second component; Wherein the parts are arranged to encode a geometric pattern of data.

2. The marker (4) according to the preceding claim, wherein the geometric pattern is a binary code, in particular a thermo-optic retroreflective binary identification pattern.

3. The marker (4) according to any one of the preceding claims, wherein the reflectance is a visible light reflectance.

4. The marker (4) according to any one of the preceding claims, wherein the surface is substantially non-reflective in the infrared spectrum.

5. The marker (4) according to claim 1, wherein The geometry of the surface is a planar geometry; and wherein the first part (1) and at least the second part (2) are arranged on the two-dimensional surface of the marker (4); the marker (4) is adapted to produce a two-dimensional thermoreflective binary identification pattern; Or Wherein the geometry of the surface is a spatial geometry; and wherein the first part and at least the second part (1, 2) are arranged in a plurality of different shapes; The marker (4) is adapted to produce a three-dimensional thermo-optic retroreflective binary identification pattern.

6. The marker (4) according to any one of the preceding claims, wherein the binary code for encoding the unique identification pattern of the marker is an ArUco or AprilTAg or ARTag code.

7. The marker (4) according to any one of the preceding claims, wherein the first component is white and the second component is black; optionally, the first component is blue and the second component is red.

8. The marker (4) according to any one of the preceding claims, wherein the first component has a reflectance of at least 70% and the second component is non-retroreflective.

9. The marker (4) according to any one of claims 1 to 4 of the preceding claims, wherein the first component comprises a retroreflective material layer having a reflectance of at least 70%, the retroreflective material layer being applied on top of at least the substrate of the first component; and The second component is composed of at least one non-retroreflective material.

10. The marker (4) according to claim 6, wherein the material of the first component has a thermal conductivity of at least 88 W / m -1 K -1 and the material of the second component has a maximum thermal conductivity of 0.38 W / m -1 K -1 ; optionally, the first material is aluminum and the second material is cork.

11. The marker (4) according to any one of the preceding claims, wherein the heat source (3) is operable to heat the surface of the marker to a temperature of at least 100 °C; preferably, the heat source (3) is an electric heating bed; the heating bed is powered by a power plug or a battery unit.

12. A heterogeneous sensing device (8) adapted to be coupled to a vehicle (9) and configured to detect a multimodal fiducial marker (4) according to any one of claims 1 to 8; The marker (4) is positioned at a target location (10); The device (8) comprises: A visual camera unit (6), configured to collect image data in the visible spectrum of the target location area and estimate the pose of the marker relative to the coordinate system of the visual camera unit; A thermal camera unit (7), configured to collect both thermal and radiometric measurement data of the target location area and estimate the pose of the marker relative to the coordinate system of the thermal camera unit; A 3D-LiDAR unit (5), configured to collect ranging and radiometric measurement data from the target location area and estimate the pose of the marker relative to the coordinate system of the 3D-LiDAR unit; Processor-based device (8.1): Programmed to process pose estimation data of markers obtained by the vision, thermal, and 3D-LiDAR units (6, 7, 5) to determine a relative pose estimate between the device (8) and the marker (4) and to determine the position (10) of the marker (4); the relative positioning between the vision and thermal camera units (6, 7) and the 3D-LiDAR unit (5) is known; and Operable to send the position information to the vehicle (9).

13. The apparatus according to claim 9, wherein the processor-based means (8.1) is programmed to determine a relative pose estimate between the apparatus (8) and the marker (4) based on the following method λ i ∈ {0, 1}, w i,j ∈[0,1], Wherein is the position estimate of the device (8) relative to the marker (4); X i = (x i , y i , z i ) is the estimation of the relative position of the marker (4) given by the visual camera unit (V), the thermal camera unit (T), and the 3D-LiDAR unit (L); λ i is a Boolean variable that is equal to 1 when the corresponding sensor detects the marker (4) and 0 otherwise; and w i is a dynamic weight calculated according to the following expression: Wherein denotes the mean error, and σ denotes the standard deviation.

14. A method of detecting a multimodal fiducial marker (4) according to any one of claims 1 to 8 using a heterogeneous sensing device (8) according to claim 9 or 10, the marker (4) being located in a target position (10), the device (8) being coupled to a vehicle (9); the method comprising: Scanning a target position area using the heterogeneous sensing device (8) to obtain target position data; Identifying the marker (4) from the target position data; Collecting image, thermal, radiometric, and ranging data from the marker (4) using sensor units (5, 6, 7) of the device (8); Estimating the pose of the marker relative to the coordinate system of each sensor unit (5, 6, 7) of the device (8); Determining a relative pose estimate between the device (8) and the marker (4) based on the estimated pose of the marker (4) and determining the corresponding position (10) of the marker (4); Sending the position information of the marker (4) to the vehicle (9).

15. A multimodal system comprising: At least one multimodal fiducial marker (4) according to any one of claims 1 to 8; each marker (4) being located at a target position (10); At least one heterogeneous sensing device (8) according to claim 9 or 10.

16. The system according to claim 12, further comprising one or more vehicles (9); each vehicle (9) being coupled with a heterogeneous sensing device (8); the system being configured to operate according to the method of claim 11.

17. The system according to claim 13, wherein the vehicle (9) is of an unmanned or manned type and is of an aquatic, land-based, or airborne type; optionally, the vehicle (9) is a drone, a ship, or an automated guided vehicle.

18. Use of a multimodal fiducial marker (4) according to any one of claims 1 to 8 and a heterogeneous sensing device (8) according to claim 9 or 10 for relative pose estimation and manipulation of an unmanned or manned vehicle (9).

Images