Hyperspectral camera and RGB-IR binocular camera combined multichannel visual perception device

Abstract

The utility model belongs to the technical field of image perception and multi-mode visual systems, and discloses a multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera, which comprises a visual perception mechanism and a rotary table mechanism capable of rotatably adjusting the yaw observation angle of the visual perception mechanism. The visual perception mechanism comprises a hyperspectral camera and two RGB-IR cameras, the hyperspectral camera and the RGB-IR cameras share one camera base, and the two RGB-IR cameras are distributed on the two sides of the hyperspectral camera; the visual perception mechanism is connected with the rotary table mechanism through a camera seat, and when the rotary table mechanism drives the visual perception mechanism to rotate and adjust, a rotating shaft of the rotary table mechanism is perpendicular to a connecting line of the two RGB-IR cameras. According to the utility model, the hyperspectral camera and the RGB-IR camera are creatively integrated into one control system, so as to conveniently and synchronously obtain the spatial shape, spectral characteristics, visible light, infrared information and other multi-dimensional image information of a target.

Description

A multi-channel visual sensing device combining a hyperspectral camera and an RGB-IR binocular camera Technical Field

[0001] This utility model belongs to the field of image perception and multimodal vision system technology, specifically relating to a multi-channel vision perception device that combines a hyperspectral camera and an RGB-IR binocular camera. Background Technology

[0002] Traditional image acquisition devices often rely on a single camera, such as an RGB camera or an infrared camera, which can only provide partial visible light or infrared information and cannot fully reflect the material, composition or physiological state of an object, making it difficult to meet the needs of multi-dimensional image information acquisition and fusion.

[0003] Hyperspectral cameras can simultaneously acquire spatial and continuous spectral information at each pixel, taking both "photographs" and "spectral" images. The fusion and analysis of spatial and spectral information makes them suitable for applications such as classification, identification, remote sensing, and agricultural pest and disease monitoring. However, hyperspectral cameras are typically line-scan cameras, which suffer from narrow fields of view and high requirements for scanning stability. RGB-IR cameras, on the other hand, possess both visible light and infrared sensing capabilities. They can acquire visible light images and near-infrared information under complex lighting conditions such as low light and nighttime. They also feature high frame rates, stereo vision, and infrared enhancement capabilities.

[0004] In summary, if a hyperspectral camera and an RGB-IR camera can be scientifically integrated into a synchronous system, spatial depth, visible texture, infrared features, and hyperspectral features can be obtained simultaneously. This has extremely high application value for multidimensional information fusion and intelligent analysis. Based on this, this application proposes an image acquisition device that can integrate a hyperspectral camera and an RGB-IR camera. Summary of the Invention

[0005] In view of this, in order to solve the problems mentioned in the background art, the purpose of this utility model is to provide a multi-channel visual perception device that combines a hyperspectral camera and an RGB-IR binocular camera.

[0006] To achieve the above objectives, this utility model provides the following technical solution:

[0007] A multi-channel visual sensing device combining a hyperspectral camera and an RGB-IR binocular camera includes a visual sensing mechanism and a turntable mechanism capable of rotating and adjusting the yaw observation angle of the visual sensing mechanism.

[0008] The visual perception mechanism includes a hyperspectral camera and two RGB-IR cameras. The hyperspectral camera and the RGB-IR cameras share a camera mount, and the two RGB-IR cameras are distributed on both sides of the hyperspectral camera.

[0009] The visual sensing mechanism is connected to the turntable mechanism via a camera mount, and when the turntable mechanism drives the visual sensing mechanism to rotate and adjust, its rotation axis is perpendicular to the line connecting the two RGB-IR cameras.

[0010] Preferably, the visual perception mechanism further includes a core processing module, which is used for data acquisition control, image data storage, and image data transmission of the hyperspectral camera and the RGB-IR camera.

[0011] Preferably, the turntable mechanism includes a housing and a servo motor fixedly mounted on the housing, and the visual perception mechanism is driven to rotate by the servo motor.

[0012] Preferably, a hollow turntable is connected between the servo motor and the camera mount. The hollow turntable is rotatably mounted on the housing and is driven to rotate by the servo motor.

[0013] Preferably, a power control module is also installed inside the housing, and the power control module is used to control the power supply start-up or power outage shutdown of the vision sensing mechanism and the turntable mechanism.

[0014] Preferably, the camera mount is further equipped with a turntable control module. The turntable control module has a built-in IMU sensor. The IMU sensor is used to detect the rotation angle of the camera mount. The turntable control module is used to control the rotation drive of the hollow turntable by the servo motor based on the rotation angle detected by the IMU sensor.

[0015] Preferably, the multi-channel visual sensing device further includes a support frame for supporting the turntable mechanism, and the turntable mechanism further includes a connector that fixes the support frame to the housing.

[0016] Preferably, the support frame includes a tripod, and the top of the tripod is provided with a socket for inserting and locking the connector.

[0017] Preferably, the insertion platform is fixed with a rotating ball by a connecting rod, and a damping support is fixed on the top of the tripod. The top of the damping support has a cavity that can accommodate and fit the rotating ball, so that the rotating ball can rotate freely within the cavity.

[0018] Compared with the prior art, this utility model has the following advantages:

[0019] (1) The multi-channel visual perception device of this utility model creatively integrates a hyperspectral camera and an RGB-IR camera into a control system to facilitate the synchronous acquisition of multi-dimensional image information such as the spatial shape, spectral characteristics and visible light and infrared information of the target. It has extremely high application value for multi-dimensional information fusion and intelligent analysis.

[0020] (2) A binocular camera system is formed by using two RGB-IR cameras distributed on both sides of the hyperspectral camera, thereby forming binocular vision. The left and right paths have parallax, enabling the overall device to realize functions such as depth map calculation, spatial positioning, and obstacle detection. Attached Figure Description

[0021] Figure 1 is a perspective view of this utility model;

[0022] Figure 2 is a perspective view of the visual perception mechanism in this utility model;

[0023] Figure 3 is a perspective view of the transfer platform mechanism of this utility model;

[0024] Figure 4 is a perspective view of the support frame in this utility model;

[0025] In the diagram: Visual perception mechanism-1; Hyperspectral camera-11; RGB-IR camera-12; Camera mount-13; Turntable mechanism-2; Housing-21; Servo motor-22; Hollow turntable-23; Connector-24; Support frame-3; Tripod-31; Insertion platform-32; Rotating ball-33; Damping support-34. Detailed Implementation

[0026] The technical solutions of the present utility model will be clearly and completely described below with reference to the accompanying drawings of the embodiments. Obviously, the described embodiments are only some embodiments of the present utility model, and not all embodiments. Based on the embodiments of the present utility model, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the protection scope of the present utility model.

[0027] As shown in Figure 1, a multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera includes a support frame 3, a turntable mechanism 2, and a visual perception mechanism 1 connected sequentially from bottom to top.

[0028] Referring again to Figure 4, the support frame 3 includes a tripod 31, a damping support 34, and a mounting plate 32. The damping support 34 has a recessed cavity at its top. A rotating ball 33 is fixed to the bottom of the mounting plate 32 via a connecting rod, and the rotating ball 33 is embedded in the recessed cavity at the top of the damping support 34. A switch is provided inside the damping support 34. When the switch is turned on, and the mounting plate 32 is subjected to an external force (which should be greater than the sum of the weights of the turntable mechanism 2 and the visual perception mechanism 1), the rotating ball 33 can rotate freely within the recessed cavity.

[0029] Referring again to Figure 3, the turntable mechanism 2 includes a housing 21, a servo motor 22 fixed to the housing 21, and a connector 24. The connector 24 can be inserted into and locked into the insertion platform 32, thereby achieving a fixed connection between the overall support frame 3 and the turntable mechanism 2 through the cooperation of the connector 24 and the insertion platform 32. A hollow turntable 23 is rotatably mounted on the top of the housing 21, and the visual sensing mechanism 1 is fixedly mounted on the hollow turntable 23. The hollow turntable 23 is driven to rotate by the servo motor 22, thereby allowing the yaw observation angle of the visual sensing mechanism 1 to be adjusted by the rotation driven by the servo motor 22. In addition, a power control module (not shown in the figure) is also installed inside the housing 21. The power control module is used to control the power supply start-up or power-off shutdown of the vision sensing mechanism 1 and the turntable mechanism 2. A turntable control module (not shown in the figure) is also installed inside the camera mount 13. The turntable control module has a built-in IMU sensor (not shown in the figure). The IMU sensor is used to detect the real-time rotation angle of the camera mount 13. The turntable control module is used to output a control signal based on the comparison result between the real-time rotation angle detected by the IMU sensor and the target rotation angle. The control signal acts on the servo motor 22, so that the servo motor 22 drives the hollow turntable 23 to rotate accordingly, thereby enabling the vision sensing mechanism 1 to achieve 0-360° yaw angle measurement.

[0030] Referring again to Figure 2, the visual perception mechanism 1 includes a hyperspectral camera 11 and two RGB-IR cameras 12. The hyperspectral camera 11 and the RGB-IR cameras 12 share a camera mount 13, and the two RGB-IR cameras 12 are distributed on both sides of the hyperspectral camera 11. When the turntable mechanism 2 drives the visual perception mechanism 1 to rotate and adjust, its rotation axis is perpendicular to the line connecting the two RGB-IR cameras 12. A core processing module (not shown in the figure) is also provided inside the camera mount 13, and this core processing module is used for data acquisition control, image data storage, and image data transmission of the hyperspectral camera 11 and the RGB-IR cameras 12. Specifically, the hyperspectral camera 11 is used to perform hyperspectral scanning imaging of a large field of view ground area and automatically perform target detection and recognition, generating a three-dimensional data cube containing spatial and spectral information; the RGB-IR camera 12 is used to acquire image data that simultaneously possesses visible light and infrared information. The two RGB-IR cameras 12 are respectively installed on both sides of the hyperspectral camera 11 to form a dual-view perspective, and the left and right views have parallax. Based on this, the core processing module can calculate the depth map through the image data acquired by the two RGB-IR cameras 12 to achieve spatial positioning, obstacle detection, etc.

[0031] In summary, and in conjunction with the structure shown in Figure 3, when performing measurements using the device of this invention:

[0032] The housing 21 of the turntable mechanism 2 is provided with a power switch, a power interface, etc. Based on this, when performing data acquisition of the whole equipment, the power cord is plugged into the power interface and then the power switch is turned on (short press) to power on the power control module. The power control module is electrically connected to the servo motor, the turntable control module and the core processing module, thereby realizing the power supply and start-up of the whole equipment.

[0033] After startup, the core processing module can acquire acquisition and control commands from the PC via wireless communication (or a network interface can be set on the housing 21 to achieve a wired connection between the device and the PC). These commands include data acquisition commands and angle control commands. The turntable control module compares and calculates the angle control commands processed by the core processing module with the real-time angle detected by the IMU sensor, and outputs a control signal to control the servo motor 22 to rotate the hollow turntable 23 accordingly. This allows the hollow turntable 23 to be positioned to a preset yaw angle under the drive of the servo motor 22. The hyperspectral camera 11 and the RGB-IR camera 12 perform data acquisition based on the data acquisition signals processed by the core processing module, and the acquired data can be stored by the core processing module. When the core processing module transmits information to the PC, in addition to the acquired image data, it may also include device status information such as voltage, current, temperature, time, and GPS information.

[0034] Finally, it should be noted that in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. "" and / or "" indicate that either one or both can be selected. Furthermore, the terms "includes," "contains," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or terminal device that includes a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or terminal device. Without further limitations, an element defined by the statement "includes a..." does not exclude the presence of other identical elements in the process, method, article, or terminal device that includes the element.

[0035] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the embodiments of the present invention, and are not intended to limit them. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention. Any changes or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera, characterized in that: The system includes a visual sensing mechanism (1) and a turntable mechanism (2) capable of rotating and adjusting the yaw observation angle of the visual sensing mechanism (1). The visual sensing mechanism (1) includes a hyperspectral camera (11) and two RGB-IR cameras (12). The hyperspectral camera (11) and the RGB-IR cameras (12) share a camera mount (13), and the two RGB-IR cameras (12) are distributed on both sides of the hyperspectral camera (11). The visual sensing mechanism (1) is connected to the turntable mechanism (2) through the camera mount (13), and when the turntable mechanism (2) drives the visual sensing mechanism (1) to rotate and adjust, its rotation axis is perpendicular to the line connecting the two RGB-IR cameras (12).

2. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 1, characterized in that: The visual perception mechanism (1) also includes a core processing module, which is used to perform data acquisition control, image data storage and image data transmission for the hyperspectral camera (11) and the RGB-IR camera (12).

3. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 1, characterized in that: The turntable mechanism (2) includes a housing (21) and a servo motor (22) fixedly mounted on the housing (21), and the visual perception mechanism (1) is driven to rotate by the servo motor (22).

4. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 3, characterized in that: A hollow turntable (23) is connected between the servo motor (22) and the camera mount (13). The hollow turntable (23) is rotatably mounted on the housing (21), and the hollow turntable (23) is driven to rotate by the servo motor (22).

5. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 3, characterized in that: The housing (21) is also equipped with a power control module, which is used to control the power supply start-up or power outage shutdown of the visual perception mechanism (1) and the turntable mechanism (2).

6. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 4, characterized in that: The camera mount (13) is also equipped with a turntable control module. The turntable control module has a built-in IMU sensor. The IMU sensor is used to detect the rotation angle of the camera mount (13). The turntable control module is used to control the rotation of the servo motor (22) to the hollow turntable (23) according to the rotation angle detected by the IMU sensor.

7. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 3, characterized in that: The multi-channel visual perception device also includes a support frame (3) for supporting the turntable mechanism (2), and the turntable mechanism (2) also includes a connector (24) for fixing the support frame (3) to the housing (21).

8. The multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 7, characterized in that: The support frame (3) includes a tripod (31), and the top of the tripod (31) is provided with a socket (32) that can be inserted into and locked into the connector (24).

9. A multi-channel visual perception device combining a hyperspectral camera and an RGB-IR binocular camera according to claim 8, characterized in that: The insertion platform (32) is fixed with a rotating ball (33) by a connecting rod. The tripod (31) is fixed with a damping support (34) on the top. The damping support (34) has a cavity on the top that can accommodate and fit the rotating ball (33) so that the rotating ball (33) can rotate freely in the cavity.

Images