RGB-IR 3D camera
By adopting pairs of adjustable FOI structured light sources and lighting sources in RGB-IR 3D cameras, combined with a compact cross-shaped layout and a translucent material shell, the acquisition of depth information under different light conditions in the prior art is solved. Difficult problems achieve high-quality long-term and short-distance depth detection and compactness, and are suitable for a variety of industrial-grade applications.
Patent Information
- Application Number
- CN202323583967.0
- Authority / Receiving Office
- CN · China
- Patent Type
- Utility models(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-27
- Publication Date
- 2024-11-22
- Estimated Expiration
- 2033-12-27
AI Technical Summary
There is a lack of binocular stereoscopic depth cameras based on RGB-IR sensors on the market, especially in different light conditions, it is difficult to achieve stable imaging and high-quality depth information acquisition at long and near distances. The existing 3D cameras exist in volume, cost and power. limit.
An RGB-IR 3D camera is designed, using a paired structured light source and lighting source, with adjustable FOI size, combining lens and RGB-IR sensors to achieve a large Field of View and a wide detection range, and through a compact ten Font layout and light-transmitting material housing ensure the camera compactness and low power consumption.
It realizes the provision of clear images and depth information under different lighting conditions day and night, and is suitable for safety monitoring, autonomous driving, robotics and medical imaging, covering long-distance target detection, and meeting the compactness and cost requirements of industrial-grade applications.
Smart Images

Figure CN222050691U_ABST
Abstract
Description
Technical Field
[0001] The utility model relates to the field of 3D cameras (or depth cameras), in particular to an RGB-IR 3D camera. Background Art
[0002] As of now, there are many types of depth / 3D cameras on the market, each based on different technical principles and suitable for different application scenarios. The following are some of the main types of depth cameras:
[0003] 1. Structured light depth camera:
[0004] Principle: Use a specific light pattern (such as a stripe pattern) to illuminate the object, and then calculate the depth information of the object by analyzing the changes in the reflected light pattern.
[0005] Applications: Facial recognition, augmented reality (AR) and virtual reality (VR).
[0006] 2. Time of Flight (ToF) Depth Camera:
[0007] Principle: Determine the distance of an object by measuring the time it takes for light to be emitted and reflected back to the camera.
[0008] Applications: Indoor navigation, robotics, and some types of 3D modeling.
[0009] 3. Stereoscopic Depth Camera:
[0010] Principle: Use two or more cameras to capture the same scene from different angles, and then calculate depth information by analyzing the parallax between the images.
[0011] Applications: Self-driving cars, robotic vision, 3D reconstruction.
[0012] 4. Laser scanning (Lidar) depth camera:
[0013] How it works: Send out a laser pulse and measure the time it takes for it to reflect back to the camera, thereby calculating the position and distance of an object.
[0014] Applications: Autonomous driving, terrain mapping, environmental monitoring.
[0015] There are many binocular stereo depth cameras on the market. These cameras use two lenses to simulate human binocular vision to capture depth information. The application areas of binocular stereo depth cameras are also expanding, including autonomous driving, robotics, augmented reality, virtual reality and other industries. Binocular stereo depth cameras with integrated RGB and IR functions are relatively rare. The following are some examples of binocular stereo depth cameras with RGB and IR functions. Intel's RealSense D435i model not only has an RGB camera, but also is equipped with an IR transmitter and receiver for depth perception. Microsoft Kinect combines an RGB camera and an IR sensor to capture depth information and color information. OrbbecAstra Pro provides RGB and IR functions. However, the RGB camera and IR camera on these depth cameras are separated. The two need to be synchronized and accurately aligned with RGBD to establish a one-to-one correspondence and obtain depth information. However, due to the influence of assembly accuracy, calibration accuracy and usage process, it is often difficult to achieve a one-to-one correspondence between RGB image pixels and depth maps or 3D point clouds at different distances.
[0016] The RGB-IR sensor is a relatively new type of highly integrated image sensor that combines traditional RGB (red, green, and blue) pixels with IR (infrared) pixels. This sensor can capture information from both the visible light and infrared spectra, providing richer and more detailed image data. Monocular cameras based on RGB-IR sensors are already available on the market.
[0017] However, there is no binocular stereo vision camera with structured light based on RGB-IR sensors on the market, which can be applied to various applications that need to work under different light conditions, such as security monitoring, biometrics, and other scenarios that require stable imaging under day-night conditions, including autonomous driving, robots, etc. This type of new 3D camera needs to solve the following industrial application challenges: (1) Horizontal Field of View (HFOV)>90 degrees; (2) long detection distance, greater than or equal to 6 meters; (3) compact camera structure and small size, easy to install on machines such as AMR (autonomous mobile robot); (4) dustproof, waterproof, and shockproof; (5) low power, limited by laser safety requirements; (6) reasonable cost. Summary of the invention
[0018] The present application provides an RGB-IR 3D camera, which is a binocular stereo vision camera with structured light based on an RGB-IR sensor. The light sources are also arranged in pairs and have FOIs (Field of Illumination) of different sizes. Light sources of different FOIs can be enabled according to the distance of the object.
[0019] The RGB-IR 3D camera provided in the present application includes a lens, an RGB-IR sensor, an IR structured light source, and an IR illumination light source. The RGB-IR sensor is connected to a sensor control board, and the IR structured light source and the IR illumination light source are connected to the light source control board. The lens, the RGB-IR sensor, the IR structured light source, and the IR illumination light source are all arranged in pairs, and the IR structured light source and the IR illumination light source are located between a pair of lenses, wherein the FOI sizes of the two IR structured light sources are different, and the FOI sizes of the two IR illumination light sources are also different.
[0020] Preferably, the two IR structured light sources and the two IR illumination light sources are arranged in a cross shape, wherein the two IR structured light sources and the two IR illumination light sources are symmetrically arranged.
[0021] Preferably, the two IR structured light sources are arranged left and right, and the two IR illumination light sources are arranged up and down.
[0022] Preferably, at the IR structured light source and the IR illumination light source, the camera housing is made of a light-transmitting material.
[0023] Preferably, it further comprises a processor module, and the light source control board and the sensor control board are connected to the processor module.
[0024] Preferably, it also includes a communication module connected to the processor module.
[0025] The highly integrated binocular stereo vision camera with structured light based on RGB-IR sensor provided by the present application fills the gap in the current 3D camera industrial application market and technology. The structured light source and the illumination light source are arranged in pairs, and the FOI size of the two structured light sources and the two illumination light sources achieves a large FOV (Field of View) and a wide detection range. The present application fills the gap in the 3D camera market and technology field, especially in the following fields: (1) Security and monitoring: it can provide clear images and depth information under different lighting conditions during the day and night. (2) Automobile: night driving assistance system and autonomous driving technology; (3) Robots working day and night; (4) Medical imaging: specific infrared wavelengths can be used to observe blood flow and tissue health; (5) Scientific research: ecology and wildlife research, especially under night conditions. The present application covers both long-range and short-range target detection. BRIEF DESCRIPTION OF THE DRAWINGS
[0026] Figure 1 This is the block diagram of this application;
[0027] Figure 2 This is the light source layout diagram for this application;
[0028] Figure 3This is the control flow chart of the imaging of near and far targets in this application.
[0029] In the figure: 1-first RGB-IR sensor; 2-second RGB-IR sensor; 3-first lens; 4-second lens; 5-first IR structured light source; 6-second IR structured light source; 7-first IR illumination light source; 8-second IR illumination light source; 10-light source control board; 11-sensor control board; 13-communication module; 20-processor / AI module; 30-subsequent processing system. DETAILED DESCRIPTION
[0030] like Figure 1 , 2 As shown, the RGB-IR 3D camera provided by the present application includes a housing, a first lens 3 and a second lens 4. The first RGB-IR sensor 1 and the second RGB-IR sensor 2 are respectively arranged behind the first lens 3 and the second lens 4. The first RGB-IR sensor 1 and the second RGB-IR sensor 2 are installed on the sensor control board 11 in the housing, connected to the sensor control board, and controlled by the sensor control board. The sensor control board is a printed circuit board. The first lens 3 and the second lens 4 can be installed on the housing, or fixedly installed on the sensor control board. The housing is also provided with a first IR structured light source 5, a second IR structured light source 6, a first IR illumination light source 7 and a second IR illumination light source 8. The light sources are all located between the first lens 3 and the second lens 4, and are preferably arranged in the middle position between the first lens 3 and the second lens 4. The first IR structured light source 5, the second IR structured light source 6, the first IR illumination light source 7 and the second IR illumination light source 8 can all emit infrared light sources, are installed on the light source control board 10, are connected to the light source control board 10, and are controlled by the light source control board. The FOI sizes of the first IR structured light source 5 and the second IR structured light source 6 are different, one of which is larger and the other is smaller. The FOI sizes of the first IR illumination light source 7 and the second IR illumination light source 8 are also different, one is larger and the other is smaller. A processor / AI module 20 and a communication module 13 (such as GMSL, FPDLINK, USB, ETHERNET, MIPI, optical fiber, etc.) are also arranged in the housing. The communication module 13, the light source control board 10, and the sensor control board 11 are all connected to the processor / AI module. The 3D camera is connected to a subsequent processing system 30 outside the camera through the communication module 13.
[0031] like Figure 2As shown, the first IR structured light source 5, the second IR structured light source 6, the first IR illumination light source 7 and the second IR illumination light source 8 are arranged in a cross shape around the left and right centers of the light source control board 10, and the light source control board is installed close to the front panel of the shell: the first IR structured light source 5 and the second IR structured light source 6 are arranged on the left and right sides, and the first IR illumination light source 7 and the second IR illumination light source 8 are arranged on the upper and lower sides. When the camera is working, the first IR structured light source 5 and the second IR structured light source 6 will not emit at the same time, and the first IR illumination light source 7 and the second IR illumination light source 8 will not emit at the same time. Therefore, a small installation spacing can be maintained between the first IR structured light source 5 and the second IR structured light source 6, and between the first IR illumination light source 7 and the second IR illumination light source 8. The first IR structured light source 5, the second IR structured light source 6, the first IR illumination light source 7 and the second IR illumination light source 8 all need to emit infrared light outward, and the front cover of the shell needs a window for light transmission, which is made of light-transmitting material. If this window is wider and higher, it will cause the entire shell to be higher and thicker. However, since the present application adopts a compact cross-shaped layout, the window has a smaller length and width. Therefore, although the camera provided by the present application has a larger FOV and a longer detection range, the whole device can still maintain a small height and thickness, a compact structure, a small volume, a simple and beautiful appearance, and an appearance that is easy to integrate and install. The first IR illumination light source and the second IR illumination light source are implemented using an infrared speckle projector (IR Dot Pattern Projector).
[0032] The working principle of this application is as follows: The RGB-IR 3D camera of this application can work in multiple modes according to different application scenarios, including but not limited to: RGB (2D) Only, IR (2D) Only, RGB-D, IR-D, RGBIR-D. The RGB-IR 3D camera can also work in a mixed mode (Combo Mode), including but not limited to: Stereo + structured light, Stereo + TOF (TOF module is added to the camera).
[0033] like Figure 3As shown, when the RGB-IR 3D camera needs to use infrared light (for example, when the light is dark or insufficient, or when infrared information is needed) to detect a target at a longer distance, the first IR structured light source 5 (IRI-W) and the first IR illumination light source 7 (DPP-W) of the wider FOI are turned off, and the second IR structured light source 6 (IRI-N) and the second IR illumination light source 8 (DPP-N) of the narrower FOI are started and projected to a longer distance. The camera generates an RGB-IR image through the first lens 3 and the second lens 4, the first RGB-IR sensor 1 and the second RGB-IR sensor 2, and the sensor control board 11, which is processed by the processor / AI module (this module is optional) and transmitted to the subsequent processing system 30 outside the camera through the communication module 13 for further analysis and application.
[0034] When the camera needs to use infrared light to detect a target at a relatively close distance, the second IR structured light source 6 (IRI-N) and the second IR illumination source 8 (DPP-N) of the narrower FOI are turned off, and the first IR structured light source 5 (IRI-W) and the first IR illumination source 7 (DPP-W) of the wider FOI are activated to project to a relatively close distance and a relatively wide range. The camera generates an RGB-IR image through the first lens 3 and the second lens 4, the first RGB-IR sensor 1 and the second RGB-IR sensor 2, and the sensor control board 11, and processes it through a microprocessor or an AI module, and transmits it to the subsequent processing system 30 outside the camera through the communication module 13 for further analysis and application. High-quality RGB-IR imaging and depth detection at close and long distances are achieved.
Claims
1. An RGB-IR 3D camera, comprising a lens, an RGB-IR sensor, an IR structured light source, and an IR illumination light source, wherein the RGB-IR sensor is connected to a sensor control board, and the IR structured light source and the IR illumination light source are connected to a light source control board, characterized in that: The lens, RGB-IR sensor, IR structured light source, and IR illumination light source are all arranged in pairs, and the IR structured light source and IR illumination light source are located between a pair of lenses, wherein the FOI sizes of the two IR structured light sources are different, and the FOI sizes of the two IR illumination light sources are also different.
2. The RGB-IR 3D camera of claim 1, wherein: The two IR structured light sources and the two IR illumination light sources are arranged in a cross shape, wherein the two IR structured light sources and the two IR illumination light sources are symmetrically arranged.
3. The RGB-IR 3D camera of claim 2, wherein: Two IR structured light sources are arranged left and right, and two IR illumination light sources are arranged up and down.
4. The RGB-IR 3D camera of claim 1, 2 or 3, wherein: At the IR structured light source and the IR illumination light source, the camera housing is made of a light-transmitting material.
5. The RGB-IR 3D camera of claim 1, 2 or 3, characterized in that: It also includes a processor module, and the light source control board and the sensor control board are connected to the processor module.
6. The RGB-IR 3D camera of claim 5, wherein: Also included is a communication module connected to the processor module.
