Laser radar and RGB-IR camera fused multi-mode environment sensing equipment

Abstract

The utility model belongs to the technical field of environment sensing equipment, and discloses laser radar and RGB-IR camera fused multi-mode environment sensing equipment, which comprises an equipment host and a suspension mechanism capable of suspending and connecting the equipment host to an unmanned aerial vehicle platform, the equipment host comprises a shell, and a laser radar module and an RGB-IR camera module which are mounted on the shell; the suspension mechanism comprises a suspension arm and a multi-shaft suspension platform, the end of the suspension arm is provided with a fastener, the multi-shaft suspension platform is connected between the suspension arm and the shell, and the suspension mechanism is clamped on an unmanned aerial vehicle platform through the fastener; and the multi-shaft suspension platform comprises at least one driving motor capable of driving the equipment host to execute relative rotation. According to the utility model, the laser radar module and the RGB-IR camera are integrated to carry out sampling control, and synchronous dual control of the laser radar module and the RGB-IR camera is completed, so that fusion of laser radar data and RGB-IR image data is effectively realized.

Description

Technical Field

[0001] This utility model belongs to the field of environmental sensing equipment technology, specifically relating to a multimodal environmental sensing device that integrates lidar and RGB-IR camera. Background Technology

[0002] With the rapid development of remote sensing technology and intelligent sensing systems, the fusion of multi-source heterogeneous sensors has gradually become an important research direction in fields such as mobile mapping, smart agriculture, and smart cities. Traditional single-vision sensing systems are often limited by issues such as illumination, occlusion, and color consistency, making it difficult to provide stable and reliable spatial and semantic information. Therefore, fusing lidar with image sensors has become an important path to improve the accuracy and robustness of environmental perception.

[0003] LiDAR is used to provide spatial perception. LiDAR has high-precision ranging and 3D spatial modeling capabilities, and can be used to obtain 3D spatial structure information of a scene. However, it lacks semantic information such as color, texture and spectrum, making it difficult to identify features such as object category, growth state and surface material, and it is difficult to independently complete high-level environmental understanding.

[0004] Image perception is provided using RGB-IR cameras. RGB-IR cameras introduce an infrared (IR) channel on top of traditional red, green, and blue imaging, enabling them to acquire visible light images as well as near-infrared information under complex lighting conditions such as low light and nighttime, thus possessing a certain near-infrared spectral sensing capability. However, RGB-IR cameras lack direct depth sensing capability, making it difficult to accurately estimate 3D geometry. Traditional vision systems, mostly based on RGB cameras, suffer from performance degradation at night or under complex lighting conditions.

[0005] In summary, if LiDAR and RGB-IR cameras can be precisely fused in space and time to form a perception system that complements spatial structure and image, it is possible not only to achieve 3D modeling of targets, but also to acquire infrared / visible light image information of targets, thereby significantly improving the robustness and recognition capability of the environmental perception system. To this end, this application proposes a compact, unified-viewpoint LiDAR + image sensor integrated device that supports low-light perception. Utility Model Content

[0006] In view of this, in order to solve the problems mentioned in the background art, the purpose of this utility model is to provide a multimodal environmental perception device that integrates lidar and RGB-IR camera, which is suitable for the dual task scenario of UAV platform to perform three-dimensional structural perception and image acquisition of terrain, target or environment, and is applicable to environmental mapping and target image recognition in scenarios such as agricultural monitoring, power surveying, security inspection, disaster assessment, and smart cities.

[0007] To achieve the above objectives, this utility model provides the following technical solution:

[0008] A multimodal environmental perception device that integrates lidar and RGB-IR camera includes a device host and a suspension mechanism capable of suspending the device host on a drone platform;

[0009] The main unit of the device includes a housing and a lidar module and an RGB-IR camera module mounted on the housing;

[0010] The suspension mechanism includes a suspension arm with a buckle at the end, and a multi-axis suspension platform connected between the suspension arm and the housing. The suspension mechanism is fastened to the UAV platform by the buckle.

[0011] The multi-axis suspension platform includes at least one drive motor capable of driving the main unit of the equipment to perform relative rotation.

[0012] Preferably, the housing is a hollow structure, and the main bodies of the lidar module and the RGB-IR camera module are both disposed inside the housing. The housing has an opening for exposing the window of the lidar module sensor and the lens of the RGB-IR camera module to the outside of the housing.

[0013] Preferably, the device host further includes a main control module installed inside the housing, and the main control module is used to perform sampling control, data storage and transmission control on the lidar module and the RGB-IR camera module.

[0014] Preferably, the at least one drive motor includes an R-axis motor, and the R-axis motor is capable of driving the main unit of the device to perform a roll angle rotation.

[0015] Preferably, the at least one drive motor includes a P-axis motor, and the P-axis motor is capable of driving the main unit of the device to perform pitch angle rotation.

[0016] Preferably, the at least one drive motor includes an R-axis motor and a P-axis motor, a drive arm is connected between the R-axis motor and the P-axis motor, and the drive arm is capable of performing roll angle rotation under the drive of the R-axis motor, and the main unit of the device is capable of performing pitch angle rotation under the drive of the P-axis motor.

[0017] Preferably, the motor shafts of the R-axis motor and the P-axis motor are perpendicular to each other.

[0018] Preferably, the suspension arm is connected to the drive arm via an R-axis motor, and the drive arm is connected to the housing via a P-axis motor.

[0019] Preferably, both the suspension arm and the drive arm have an L-shaped structure.

[0020] Preferably, the multi-axis suspension platform further includes a platform controller, the platform controller having a built-in first IMU sensor and the device host having a second IMU sensor installed. The first IMU sensor and the second IMU sensor work together to detect the current attitude data of the device host, and the platform controller performs feedback drive control on the at least one drive motor based on the detected current attitude data.

[0021] Compared with the prior art, this utility model has the following advantages:

[0022] (1) The multimodal environmental perception device of this utility model integrates the lidar module and the RGB-IR camera into one sampling control, and completes the synchronous dual control of the lidar module and the RGB-IR camera, thereby effectively realizing the fusion of lidar data and RGB-IR image data. Moreover, its RGB-IR camera supports infrared imaging, which can effectively enhance the image perception capability under complex lighting conditions such as night or low light.

[0023] (2) The lidar module and the RGB-IR camera share a suspension mechanism to achieve synchronous dynamic adjustment of roll angle and pitch angle.

[0024] (3) By adapting the suspension mechanism to the UAV platform, the UAV platform can be used to achieve multimodal perception of the target or ground surface in flight, both structural and image dimensions, which significantly improves the equipment's environmental perception capability and accuracy. Attached Figure Description

[0025] Figure 1 This is a perspective view of the present utility model;

[0026] Figure 2 This is a perspective view of the main unit of the device in this utility model;

[0027] Figure 3 This is a perspective view of the multi-axis suspension platform in this utility model;

[0028] In the diagram: Main unit - 1; Housing - 11; LiDAR module - 12; RGB-IR camera module - 13; Suspension mechanism - 2; Fastener - 21; Multi-axis suspension platform - 22; Drive motor - 23; R-axis motor - 231; P-axis motor - 232; Suspension arm - 24; Drive arm - 25. Detailed Implementation

[0029] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present utility model.

[0030] This invention provides a multimodal environmental perception device that integrates LiDAR and RGB-IR camera. By integrating RGB-IR camera and LiDAR and applying them to a UAV platform, it enables multimodal perception of targets or ground surfaces in flight, encompassing both structural and image dimensions, thus providing support for intelligent analysis across multiple scenarios.

[0031] like Figure 1 As shown, the multimodal environmental perception device that integrates lidar and RGB-IR camera includes a device host 1 and a suspension mechanism 2 that can suspend the device host 1 on a drone platform.

[0032] Continue to refer to Figure 2 As shown, the device host 1 includes a hollow housing 11, and a lidar module 12, an RGB-IR camera module 13, and a main control module (not shown) are installed inside the housing 11. An opening is provided on the housing 11 to expose the sensor window of the lidar module 12 and the lens of the RGB-IR camera module 13 to the outside of the housing 11, ensuring that the lidar module 12 and the RGB-IR camera module 13 can respectively acquire data images of the target through their sensors. The main control module is mainly used for sampling control, data storage (implemented by a storage module built into the main control module, such as a solid-state drive, not shown), and data transmission (implemented by a communication module built into the main control module, not shown) of the lidar module 12 and the RGB-IR camera module 13.

[0033] Continue to refer to Figure 3 As shown, the suspension mechanism 2 includes a suspension arm 24 with a buckle 21 at the end, and a multi-axis suspension platform 22 connected between the suspension arm 24 and the housing 11. The suspension mechanism 2 is snapped onto the UAV platform by the buckle 21. The multi-axis suspension platform 22 includes at least one drive motor 23 capable of driving the device host 1 to perform relative rotation.

[0034] Specifically:

[0035] In one embodiment of the present invention, the at least one drive motor 23 includes an R-axis motor 231, and the R-axis motor 231 is capable of driving the device host 1 to perform a roll angle rotation.

[0036] In another embodiment of the present invention, the at least one drive motor 23 includes a P-axis motor 232, and the P-axis motor 232 is capable of driving the host device 1 to perform pitch angle rotation.

[0037] In another detailed embodiment of this utility model, the at least one drive motor 23 includes an R-axis motor 231 and a P-axis motor 232. A drive arm 25 is connected between the R-axis motor 231 and the P-axis motor 232. The drive arm 25 can perform roll angle rotation under the drive of the R-axis motor 231, and the main unit 1 of the device can perform pitch angle rotation under the drive of the P-axis motor 232.

[0038] Based on this detailed embodiment, the motor shafts of the R-axis motor 231 and the P-axis motor 232 are perpendicular to each other, and both the suspension arm 24 and the drive arm 25 have an L-shaped structure.

[0039] When connecting the UAV platform and the device host 1 through the suspension mechanism 2, the housing 11 and the drive arm 25 are connected through the P-axis motor 232, and the drive arm 25 and the suspension arm 24 are connected through the R-axis motor 231. The buckle 21 is fixed to the top of the suspension arm 24, thereby making the whole device form an integrated structure, and the whole device can be quickly snapped onto the UAV platform through the buckle 21.

[0040] In addition, the multi-axis suspension platform 22 also includes a platform controller (not shown in the figure). The platform controller has a built-in first IMU sensor (not shown in the figure). The first IMU sensor is used to detect the current attitude data of the drive arm 25. The platform controller outputs a first control signal based on the comparison result between the current attitude data detected by the first IMU sensor and the target attitude data. The first control signal acts on the R-axis motor 231 to drive the R-axis motor 231 to perform a corresponding roll angle rotation. A second IMU sensor (not shown in the figure) is installed in the device host 1. The second IMU sensor is used to detect the current attitude data of the device host 1. The platform controller outputs a second control signal based on the comparison result between the current attitude data detected by the second IMU sensor and the target attitude data. The second control signal acts on the P-axis motor 232 to drive the P-axis motor 232 to perform a corresponding pitch angle rotation.

[0041] A power interface electrically connected to the platform controller is provided on the suspension arm 24 of the suspension mechanism 2. Based on this, the power supply of the whole device can be realized by connecting a lithium battery or a drone platform power supply system through the power interface and power cord (located on the R-axis suspension arm 24 in the figure). In addition, a power switch is also provided on the suspension arm 24 to control the power on or off of the whole device.

[0042] From the above, the main functions of each component of the equipment are as follows:

[0043] LiDAR module 12: Used to acquire three-dimensional point cloud data of the target or environment.

[0044] RGB-IR camera module 13: Used to acquire image data containing infrared information.

[0045] Main control module: Used for sampling control, data storage, and data transmission control of the lidar module 12 and the RGB-IR camera module 13. Both the lidar module 12 and the RGB-IR camera module 13 are electrically connected to the main control module.

[0046] Fastener 21: Enables the connection between the overall structure and the UAV platform.

[0047] R-axis motor 231 and P-axis motor 232: respectively drive the drive arm 25 and the main unit 1 to rotate relative to each other, so as to realize the angle adjustment in the pitch and roll directions.

[0048] IMU sensor: Real-time sensing of the pitch and roll angles of the host device 1, and used to dynamically feed back to the platform controller to achieve self-stabilization.

[0049] Platform controller: This controller outputs a control signal based on the comparison between the current attitude data detected by the IMU sensor and the target attitude data. The control signal acts on at least one drive motor 23 to execute corresponding rotations, ensuring the main unit 1 stably maintains its current attitude. The main control module, R-axis motor 231, P-axis motor 232, and IMU sensor are all electrically connected to the platform controller.

[0050] In summary, when performing measurements using the equipment of this utility model:

[0051] The device is quickly connected to the drone platform via the snap fastener 21; the power interface of the drone platform power supply system is connected to the device power interface (not shown in the figure) via the power cord, and the device is powered on and started by pressing and holding the power switch.

[0052] The main control module receives acquisition control commands sent from the PC, including sampling commands and control commands: The sampling commands are transmitted to the LiDAR module 12 and the RGB-IR camera module 13 for sampling. The control commands are transmitted to the platform controller. Based on the control commands and feedback from the IMU sensor, the platform controller controls the R-axis motor 231 and P-axis motor 232 to perform relative rotation drive on the drive arm 25 and the main unit 1, thereby adjusting the roll and pitch angles of the main unit 1. During the adjustment process, the IMU sensor senses the current pitch and roll spatial Euler angles in real time and feeds them back to the platform controller. The platform controller further implements feedback control of the R-axis motor 231 and P-axis motor 232, thereby enabling the main unit 1 to maintain the set pitch and roll angles, achieving self-stabilization.

[0053] During the data acquisition process, the main control module stores the data collected by the LiDAR module 12 and the RGB-IR camera module 13, and also transmits the data to the PC. In addition, the main control module has a built-in GPS module, which allows for real-time GPS positioning of the entire device while transmitting data.

[0054] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "" and / or "" indicate that either one or both can be selected. Furthermore, the terms "includes," "contains," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the statement "includes a..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0055] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A multimodal environmental sensing device that integrates lidar and RGB-IR camera, characterized in that: Includes a device host (1) and a suspension mechanism (2) capable of suspending and connecting the device host (1) to the drone platform; The device host (1) includes a housing (11) and a lidar module (12) and an RGB-IR camera module (13) mounted on the housing (11); The suspension mechanism (2) includes a suspension arm (24) with a buckle (21) at the end, and a multi-axis suspension platform (22) connected between the suspension arm (24) and the housing (11). The suspension mechanism (2) is snapped onto the UAV platform by the buckle (21). The multi-axis suspension platform (22) includes at least one drive motor (23) capable of driving the main unit of the equipment (1) to perform relative rotation.

2. The multimodal environmental perception device fusion of lidar and RGB-IR camera according to claim 1, characterized in that: The housing (11) is a hollow structure. The main bodies of the lidar module (12) and the RGB-IR camera module (13) are both located inside the housing (11). An opening is provided on the housing (11) to expose the window of the lidar module (12) sensor and the lens of the RGB-IR camera module (13) to the outside of the housing (11).

3. The multimodal environmental perception device that fuses lidar and RGB-IR camera according to claim 1, characterized in that: The device host (1) also includes a main control module installed inside the housing (11), and the main control module is used to perform sampling control, data storage and transmission control on the lidar module (12) and the RGB-IR camera module (13).

4. The multimodal environmental perception device that fuses lidar and RGB-IR camera according to claim 1, characterized in that: The at least one drive motor (23) includes an R-axis motor (231), and the R-axis motor (231) is capable of driving the main unit of the device (1) to perform a roll angle rotation.

5. The multimodal environmental perception device fusion of lidar and RGB-IR camera according to claim 1, characterized in that: The at least one drive motor (23) includes a P-axis motor (232), and the P-axis motor (232) is capable of driving the host device (1) to perform pitch angle rotation.

6. The multimodal environmental sensing device that fuses lidar and RGB-IR camera according to claim 1, characterized in that: The at least one drive motor (23) includes an R-axis motor (231) and a P-axis motor (232). A drive arm (25) is connected between the R-axis motor (231) and the P-axis motor (232). The drive arm (25) can perform roll angle rotation under the drive of the R-axis motor (231). The main unit of the device (1) can perform pitch angle rotation under the drive of the P-axis motor (232).

7. A multimodal environmental perception device fusion of lidar and RGB-IR camera according to claim 6, characterized in that: The motor shafts of the R-axis motor (231) and the P-axis motor (232) are perpendicular to each other.

8. A multimodal environmental perception device fusion of lidar and RGB-IR camera according to claim 6 or 7, characterized in that: The suspension arm (24) is connected to the drive arm (25) via an R-axis motor (231), and the drive arm (25) is connected to the housing (11) via a P-axis motor (232).

9. A multimodal environmental sensing device fusion of lidar and RGB-IR camera according to claim 4, 5 or 6, characterized in that: Both the suspension arm (24) and the drive arm (25) have an L-shaped structure.

10. A multimodal environmental perception device fusion of lidar and RGB-IR camera according to claim 1, characterized in that: The multi-axis suspension platform (22) also includes a platform controller. The platform controller has a built-in first IMU sensor and a second IMU sensor installed in the device host (1). The first IMU sensor and the second IMU sensor work together to detect the current attitude data of the device host (1). The platform controller performs feedback drive control on the at least one drive motor (23) based on the detected current attitude data.

Images