Three-dimensional sensing device and method based on transition camera
By using transition cameras and parallel light tube calibration devices in the three-dimensional perception system, the lidar and camera's visual axis are aligned, which solves the problem of insufficient data synchronization and registration accuracy, and achieves higher image accuracy and system stability , suitable for complex environment perception.
Patent Information
- Application Number
- CN202411010430.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-26
- Publication Date
- 2024-11-29
AI Technical Summary
The existing perception technology cannot effectively solve the problem of insufficient data synchronization and registration accuracy during the fusion process of lidar and cameras, resulting in low accuracy and clarity of the fusion visual image, and the calibration parameters change over time affect performance and reliability. .
Using a three-dimensional sensing device and method based on transition camera, the parallel light tube calibration device ensures the coaxial design of the visual axis of the transition camera and the ranging laser, and the beam adjustment is performed using rotating double prisms to achieve accurate alignment and calibration of the lidar and the camera. Ensure the collimation of the optical path.
Improve image accuracy and clarity after the fusion of lidar and camera, enhance the stability and reliability of the system, realize seamless docking and multi-view imaging between large field of view color cameras and ranging laser point clouds, suitable for Complex environment awareness and target recognition tasks.
Smart Images

Figure CN119044992A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of three-dimensional imaging, and in particular to a three-dimensional perception device and method based on a transition camera. Background Art
[0002] The working principle of TOF laser radar (also called single-line laser radar) is based on the time of flight method (TOF). The laser transmitter emits a beam of laser, which is diffusely reflected after hitting an object in space and returns to the laser receiver. The radar module calculates the distance between the transmitter and the object based on the time difference between the transmitted and received signals. TOF laser radar has the advantages of simple structure, small size, low power consumption, small data volume and low cost. The mobile robot is equipped with laser radar, which can not only provide the perception module with information about surrounding objects, but also realize the complete expression of three-dimensional information in combination with the camera.
[0003] With the increasing demand for applications such as autonomous driving, robot navigation, and environmental monitoring, the requirements for perception systems are also getting higher and higher. LiDAR data points are sparse and visual cameras have high resolution. The joint calibration of the two is the basis for their fusion. Traditional target calibration-based methods usually use a specific form of calibration plate, but the final calibration accuracy is heavily dependent on 3D and 2D feature points and requires manual intervention. Targetless calibration methods mostly solve external parameters from continuous motion data or multi-view data. In practical applications, the relative position of fixed sensors will change due to mechanical vibrations, and the calibration parameters will become inaccurate over time. This change will seriously affect the performance and reliability of the fusion technology.
[0004] In summary, existing perception technologies cannot overcome the various challenges encountered in the fusion process of lidar and cameras. The synchronization and registration accuracy of data is insufficient, and the accuracy and clarity of the fused visual images are not high. Summary of the invention
[0005] The purpose of the present invention is to provide a three-dimensional perception device and method based on a transition camera in order to improve the accuracy and clarity of the image obtained by the fusion of laser radar and camera. The optical path design of the transition camera's visual axis and the ranging laser is coaxial to ensure that the transition camera and the ranging point cloud have a precisely aligned matching relationship. At the same time, the use of a parallel light tube for joint calibration of the camera's visual axis and the ranging laser can ensure the coaxial optical path design, thereby ensuring the collimation of the optical path, avoiding beam deviation, and providing clearer and more accurate imaging results.
[0006] The purpose of the present invention can be achieved by the following technical solutions:
[0007] The present invention proposes a three-dimensional sensing device based on a transition camera, wherein the three-dimensional sensing device integrates laser ranging and camera imaging, and is calibrated using a collimator calibration device, wherein the three-dimensional sensing device comprises a laser transmitter and a laser receiver, wherein a first reflector is provided on one side of the laser transmitter, wherein a small hole is provided in the first reflector, wherein a beam splitter is provided on one side of the first reflector, wherein a transition camera and a laser receiver are provided on both sides of the beam splitter, wherein a visible light bandpass filter is provided between the transition camera and the beam splitter, wherein a rotating double prism is provided on the other side of the first reflector, wherein a target object in space is provided on one side of the rotating double prism, and wherein the three-dimensional sensing device further comprises a large field of view camera facing the target object in space;
[0008] The collimator calibration device comprises a second reflecting mirror, a half-reflecting half-mirror, a beacon laser, a beam analyzer, a collimating lens group and a corner cube prism.
[0009] Furthermore, the device is arranged in a three-dimensional coordinate system, wherein the large field of view camera is arranged parallel to the Z axis of the three-dimensional coordinate system, and the light of the object reflected by the first reflector is emitted along the positive direction of the X axis.
[0010] Furthermore, the first reflector is placed at an angle of 45°.
[0011] Furthermore, the center of the rotating double prism is coated with a laser anti-reflection film.
[0012] The present invention also proposes a three-dimensional perception method based on a transition camera. Based on the above three-dimensional perception device, the method includes the following steps:
[0013] The three-dimensional sensing device is calibrated based on the collimator, and the visual axis of the ranging laser and the transition camera is controlled to be coaxial with the collimator, thereby making the visual axis of the transition camera coaxial with the ranging laser;
[0014] After calibration, the laser transmitter is controlled to emit a ranging laser, which passes through a small hole in the first reflector and irradiates the surface of the rotating double prism. The ranging laser irradiates the target object for diffuse reflection to generate a reflected laser, which is deflected by the rotating double prism and irradiates the mirror surface of the first reflector. After being reflected by the beam splitter, it is absorbed by the laser receiver to obtain a ranging point cloud. At the same time, the visible light generated by the space object passes through the same optical path as the reflected laser, and is received by the transition camera through the beam splitter and the visible light bandpass filter to generate a transition camera image. The transition camera image has a definite matching relationship with the ranging point cloud.
[0015] The camera with a large field of view captures the target object and obtains a color image;
[0016] The camera image registration algorithm is used to fuse the transition camera image of the matching ranging point cloud with the color image. Finally, a three-dimensional image containing color texture and depth information is generated based on the transition camera.
[0017] Furthermore, the specific steps of calibrating the three-dimensional sensing device based on the collimator calibration device are:
[0018] The beacon laser is emitted through a collimator with a focal length of f, and is reflected by a half-reflecting mirror and a second reflector, and then becomes parallel light through a collimating lens group, wherein the wavelength of the beacon laser matches the visible light bandpass filter;
[0019] The parallel light enters the corner cube prism and is reversed. The reversed light enters the beam analyzer through a half-reflecting half-mirror. The position of the light spot on the beam analyzer is recorded. The light spot is the coaxial point (X1, Y1) of the beacon light.
[0020] The parallel light also enters the transition camera after passing through the rotating double prism, the first reflector, the beam splitter and the visible light bandpass filter, and generates a beacon light spot on the transition camera. The installation posture of the transition camera is adjusted so that the beacon light spot is imaged at the center of the transition camera, and the visual axis of the transition camera is coaxial with the parallel light.
[0021] Control the laser transmitter to emit the ranging laser, the ranging laser enters the collimator calibration device and irradiates the beam analyzer, adjust the emission intensity of the laser transmitter and record the centroid position (X2, Y2) of the ranging laser spot on the beam analyzer, adjust the azimuth and elevation angles of the laser transmitter installation, control the centroid position (X2, Y2) and the beacon light's transmitting and receiving coaxial point (X1, Y1) to be the same, then the ranging laser is coaxial with the collimator light.
[0022] Furthermore, the visual axis of the transition camera is coaxial with the ranging laser:
[0023] The light of the ranging laser after passing through the small hole in the first reflector is coaxial with the visual axis of the transition camera after being reflected by the first reflector, and the pointing is controlled together by rotating the double prism.
[0024] Furthermore, the light of the ranging laser after passing through the small hole in the first reflector and the visual axis of the transition camera after being reflected by the first reflector are coaxial specifically:
[0025] The coaxiality between the transition camera's visual axis and the ranging laser is within a certain range.
[0026] Furthermore, the coaxiality between the transition camera visual axis and the ranging laser is:
[0027] ((X1-X2) / f,(Y1-Y2) / f).
[0028] Furthermore, the rotating biprism moves according to a certain scanning rule.
[0029] Compared with the prior art, the present invention has the following beneficial effects:
[0030] (1) The device system provided by the present invention is not only capable of large-scale macroscopic observation, but also capable of small-scale microscopic scanning, and possesses multi-sensing three-dimensional perception capabilities. At the same time, it makes full use of the mature algorithm of camera image registration to accurately register and fuse the images taken by the large-field color camera and the transition camera. Ultimately, based on a simple transition camera, the laser ranging data and the high-resolution color image are seamlessly connected, and the matching and fusion of the large-field color camera and the ranging laser point cloud are achieved, thereby establishing a bridge between the laser radar and the color camera. Users can more conveniently obtain comprehensive and detailed three-dimensional imaging data, which is suitable for various complex environmental perception and target recognition tasks.
[0031] (2) The present invention ensures that the transition camera and the ranging point cloud have a precisely aligned matching relationship by designing the optical path of the transition camera's visual axis and the ranging laser coaxially. At the same time, the collimator is used to perform joint calibration of the camera's visual axis and the ranging laser, which can ensure the coaxial optical path design, thereby ensuring the collimation of the optical path, avoiding beam deviation, and providing clearer and more accurate imaging results. In addition, after the optical path is coaxially aligned, the stability of the optical system is improved, which is conducive to long-term stable operation and maintenance.
[0032] (3) The present invention uses a rotating dual prism to uniformly control the coaxial visual axis and laser to ensure that the ranging target point is determined and known in the camera image. The prism moves along a certain scanning trajectory, and scanning points can be added to points of interest in space, while simultaneously increasing the shooting angle and point cloud density, thereby achieving multi-view imaging and dense point cloud.
[0033] (4) The present invention fully integrates the laser radar point cloud and visible light multi-band imaging, has the ability to adapt to penetrate different imaging media and resist environmental interference, and integrates multiple data to ensure imaging quality. At the same time, the rotating dual prism used has a large reduction ratio, which can reduce the impact of mechanical vibration on beam pointing, reduce the changes in camera and radar calibration parameters caused by environmental or mechanical vibration, and thus ensure the fusion effect and reliability of the two. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a structural diagram of the sensing device of the present invention;
[0035] Figure 2 It is a structural diagram of the distance measurement part in the sensing device of the present invention;
[0036] Figure 3 This is a structural diagram of the present invention during calibration;
[0037] In the figure, there are laser transmitter 1, laser receiver 2, beam splitter 3, first reflector 4, rotating biprism 5, target object 6, visible light bandpass filter 7, transition camera 8, large field of view camera 9, second reflector 10, half-reflective half-mirror 11, beacon laser 12, beam analyzer 13, collimating lens group 14, and corner cube prism 15. DETAILED DESCRIPTION
[0038] The present invention is described in detail below in conjunction with the accompanying drawings and specific embodiments. This embodiment is implemented based on the technical solution of the present invention, and provides a detailed implementation method and specific operation process, but the protection scope of the present invention is not limited to the following embodiments.
[0039] The present invention proposes a three-dimensional perception device based on a transition camera, the structure of which is shown in FIG. Figure 1 As shown, and the collimator calibration device is used for calibration as shown Figure 3 As shown. The three-dimensional sensing device includes a laser transmitter 1 and a laser receiver 2. A first reflector 4 is provided on one side of the laser transmitter 1. A small hole is provided in the first reflector 4. A beam splitter 3 is provided on one side of the first reflector 4. A transition camera 8 and a laser receiver 2 are provided on both sides of the beam splitter 3. A visible light bandpass filter 7 is provided between the transition camera 8 and the beam splitter 3. A rotating double prism 5 is provided on the other side of the first reflector 4. A target object 6 in space is provided on one side of the rotating double prism 5. The three-dimensional sensing device also includes a large field of view camera 9 facing the target object 6 in space.
[0040] The collimator calibration device comprises a second reflector 10 , a semi-reflective mirror 11 , a beacon laser 12 , a beam analyzer 13 , a collimating lens group 14 and a corner cube prism 15 .
[0041] The present invention also proposes a three-dimensional perception method based on the above-mentioned device. Based on the above-mentioned three-dimensional perception device, the method comprises the following steps:
[0042] The three-dimensional sensing device is calibrated based on the collimator calibration device so that the visual axis of the transition camera 8 is coaxial with the ranging laser;
[0043] After calibration, the laser transmitter 1 is controlled to emit a ranging laser, which passes through a small hole in the first reflector 4 and irradiates the surface of the rotating double prism 5. The ranging laser irradiates the target object 6 for diffuse reflection to generate a reflected laser. After the reflected laser is deflected by the rotating double prism 5, the laser irradiates the mirror surface of the first reflector 4, passes through the beam splitter 3, is absorbed by the laser receiver 2, and obtains a ranging point cloud. At the same time, the visible light of the space object 6 passes through the same optical path, passes through the beam splitter 3 and the visible light bandpass filter 7, and is received by the transition camera 8 to generate a transition camera image, which has a definite matching relationship with the ranging point cloud.
[0044] The large-field-of-view camera 9 photographs the target object 6 to obtain a color image;
[0045] The camera image registration algorithm is used to fuse the transition camera image that matches the ranging point cloud with the color image. Finally, based on the transition camera, the 3D perception device achieves the matching and fusion of the large field of view color camera and the ranging laser point cloud, generating a 3D image containing color texture and depth information.
[0046] In the device of the present invention, the laser transmitter 1, the laser receiver 2 and the spectroscope 3 form a TOF radar ranging module. The spectroscope 3 is used to reflect a large amount of laser light generated by diffuse reflection of the object so that the main echo laser light can be smoothly received by the laser receiver. By measuring the time required for the laser beam to be emitted from the device and reflected and received by the object, the distance between the object and the device is calculated. The structure diagram of the ranging part in the sensing device is shown in FIG. Figure 2 The structural diagram during calibration is shown in Figure 3 shown.
[0047] The transition camera 8 with a small field of view, the beam splitter 3, and the visible light bandpass filter 7 constitute a transition camera imaging module, which uses a high-resolution small field of view camera to capture object images. The beam splitter allows visible light to pass through to reduce the influence of lasers. The visible light bandpass filter is located in front of the transition camera and only allows visible light within a certain wavelength range to pass through to reduce imaging dispersion caused by the rotating prism.
[0048] The first reflector 4 and the rotating double prism 5 form a beam adjustment and control module. The reflector with a small hole is used to ensure that the emitted laser passes through and adjust the direction of the camera's visual axis so that it remains coaxial with the ranging laser. Specifically, the laser passes through the small hole along the Z axis, and the camera's visual axis is incident along the negative direction of the X axis. After being reflected by the reflector, it remains coaxial with the laser. During the working process, the laser and the visual axis are jointly pointed and controlled by the rotating double prism, and move along a certain scanning trajectory in the conical target space.
[0049] The transition camera 8, the large field of view camera 9 and the fusion algorithm constitute the internal and external camera matching module. The visual axis of the transition camera is coaxial with the laser, and the image is fused with the ranging point cloud, but the filter in front of it will reduce the color accuracy and contrast of the image. Accordingly, a large field of view color camera is used to obtain a clearer image with rich color texture. The two images are feature extracted and matched to achieve the registration and fusion of the target image. Therefore, based on the transition camera, the three-dimensional fusion of the large field of view color camera and the laser radar data point cloud can be achieved.
[0050] The TOF radar ranging module uses dTOF-direct measurement of the time of flight method to directly measure the time interval between the emission and reception of the light pulse. The TOF distance refers to the distance between the scanning instrument and the target point, excluding the offset distance of the light beam in the device, that is, t 光脉冲发射与接收 =t 光束通过装置+Δt TOF In addition, to avoid multi-path interference caused by optical components, the receiving window of the reflected laser needs to be set at t 光束通过装置 Finally, the reflected laser in the window can be used for effective ranging.
[0051] The beam splitter in the transition camera imaging module can realize laser reflection and visible light transmission, ensuring that the main laser generated by the diffuse reflection of the object is irradiated to the TOF radar ranging module and is received by the laser receiver as much as possible, ensuring a certain laser intensity while avoiding the influence of the laser on the imaging; the visible light bandpass filter only allows visible light in the 520-540nm band to pass, which ensures that the strongest wave of visible light can pass and reduces the imaging dispersion caused by the visible light passing through the prism.
[0052] In the beam adjustment and control module, the aperture of the reflector with a small hole is determined according to the emission radius of the ranging laser to ensure that the emitted laser passes smoothly without being blocked; the center of the rotating double prism is coated within a certain radius to increase the transparency of the ranging band laser and reduce reflection. The rotating prism jointly controls the deflection of the ranging laser and the camera's visual axis. Before work, a collimator is used for joint calibration to ensure that the ranging laser point is aligned with the imaging center point, that is, the image information and depth information of the center position point of the three-dimensional imaging picture are known. During work, the double prism rotates continuously, and the three-dimensional imaging point moves in space according to a certain law, and finally completes the complete scanning and imaging of the target, realizing the three-dimensional information fusion of the transition camera and the ranging radar.
[0053] Since the transition camera can only receive visible light in a certain band, the color and light flux of the resulting image are lost, which reduces the image resolution and color accuracy to a certain extent. To this end, a large field of view camera is placed in the direction of the parallel device optical axis to shoot the same target space together with the transition camera. The two cameras extract key feature points through feature detection algorithms, and then use matching algorithms to find corresponding feature pairs, and use transformation matrices and image transformations to achieve registration, thereby realizing the fusion of pixel and color information of the two images, and obtaining a three-dimensional target image with rich color textures and precise position depth.
[0054] During the working process of the present invention, first, calibration is required, and the specific steps of calibration are:
[0055] 1) The beacon light 12 is emitted through a collimator with a focal length of f. After being emitted, the beacon laser 12 is reflected by a half-reflecting mirror 11, and then reflected by a total reflection mirror, that is, the second reflection mirror 10, and becomes parallel light after being collimated by the lens of the collimating lens group 14. The intensity of the beacon light is adjusted to be moderate, and the wavelength of the beacon light matches the visible light bandpass filter;
[0056] 2) Use the corner cube prism 15 to reverse the parallel light. The parallel light generated by the collimator is self-collimated and reversed through the corner cube prism 15, and returns to the beam analyzer 13 through the half-reflecting half-mirror 11. The beam analyzer 13 performs background elimination processing and records the position of the light spot on the beam analyzer 13 at this time. The light spot is the coaxial point (X1, Y1) of the beacon light.
[0057] 3) Capture the beacon light at the receiving end of the optical path to be measured. Place the optical path system to be measured at the rear end of the collimator, and adjust the rotating biprism 5 in the optical path to be measured so that the parallel light generated by the collimator can be irradiated to the receiving end of the optical path to be measured; use the transition camera 8 in the optical path to be measured to capture the beacon light spot, and the beacon laser 12 is imaged in the camera field of view. Adjust the installation posture of the transition camera 8 so that the parallel beacon light is coaxial with the camera visual axis;
[0058] 5) The optical path to be measured emits a ranging laser. Turn on the laser transmitter 1 at the measuring end with a wavelength of 1064nm, adjust the azimuth and elevation angles of the laser transmitter 1, so that the laser enters the collimator and irradiates the beam analyzer 13 at the focal plane. Adjust the intensity of the laser so that the peak value of the light spot on the beam analyzer approaches saturation, and record the size and image of the light spot; record the centroid position (X2, Y2) of the laser spot on the beam analyzer at this time, and make the centroid after tracking the same as the coaxial point of the beacon light receiving and transmitting, then the coaxiality between the ranging laser and the camera visual axis is ((X1-X2) / f, (Y1-Y2) / f).
[0059] In the above steps, if necessary, fine-tuning can be performed to ensure that the laser transmitter, the parallel optical axis and the visual axis of the camera are completely coaxial, and the imaging matching is accurate and stable. After calibration, the optical path behind the rotating double prism in the device to be tested, that is, the laser transmitter 1, the laser receiver 2, the beam splitter 3, the first reflector 4, the visible light bandpass filter 7, and the transition camera 8 are fastened as a whole after calibration, so as to ensure that the ranging laser and the visual axis of the transition camera are always coaxial, and the optical axis / visual axis direction is adjusted by the rotating double prism to strictly ensure the matching relationship between the laser beam pointing to the target feature and the center pixel point of the camera field of view, thereby simplifying the matching relationship between the feature point cloud and the target image feature.
[0060] In the above calibration process, the visual axis of the transition camera is first adjusted to be coaxial with the parallel light, and then the ranging laser is adjusted to be coaxial with the parallel light, thereby achieving the coaxiality of the camera and the ranging laser.
[0061] After calibration, the laser transmitter 1 emits a beam of ranging laser through the aperture of the first reflector 4 with a small hole, and irradiates the surface of the rotating double prism 5. As the prism rotates, the laser can point to the target at any position. When the laser irradiates the surface of the target object 6 and diffusely reflects, it follows Lambert's cosine law:
[0062] I(θ)=I n*cosθ(0.1)
[0063] I is the light intensity received by the observer, I n is the light intensity in the normal direction of the object surface, and θ is the angle between the light and the normal. The diffusely reflected laser with the largest light intensity is taken as the main laser.
[0064] The reflected laser beam is deflected multiple times by the rotating biprism 5 and returns to the surface of the first reflector 4. At this time, most of the echo laser is irradiated on the mirror surface of the first reflector 4, reflected by the reflector placed at a 45° angle, and emitted along the positive direction of the X-axis. The direction is further adjusted by the beam splitter 3, and the reflected laser is received by the laser receiver 2. According to the time difference between the laser emission and the reception of the reflected laser, the host computer can calculate the distance from the device to the target point on the surface of the object. In order to reduce the reflection of the laser by each optical element, a laser anti-reflection film is coated within a certain radius at the center of the rotating biprism 5 to reduce laser reflection. It should be noted that the TOF distance measurement is the distance from the device to the target point on the surface of the object, and the time distance of the laser in each element and device needs to be eliminated, that is, t 光脉冲发射与接收 =t 光束通过装置 +Δt TOF , t 光束通过装置 exist Figure 2 The lower right corner is marked as t offset , which can be obtained by using the device to measure multiple fixed known distances and then analyzing and evaluating them.
[0065] The present invention uses a laser with a wavelength of 1064nm, the spectroscope 3 allows visible light in the 400-700nm band to pass through and reflects the 1064nm laser, and the visible light bandpass filter 7 only allows visible light in the 520-540nm band to pass through.
[0066] At the same time, the visible light of the space object passes through the same optical path, and is received by the lens of the transition camera 8 through the beam splitter and the visible light bandpass filter. This process is the passive imaging of the transition camera. According to the reversible law of the optical path, the camera has a central visual axis pointing to the target space object. Due to the action of the beam adjustment device and the pointing control module, the camera visual axis is coaxial with the ranging laser, and the two are jointly regulated by rotating the double prism 5. The double prism moves according to a certain scanning law, and the ranging laser and the imaging visual axis can actively observe the area of interest on a large scale and depict the fine scale. Since the ranging laser point and the imaging center point are aligned, the image after the complete scan is accurately aligned and matched with the point cloud, and no feature extraction and fusion post-processing are required. The filter in front of the transition camera causes the image to lose most of its color and part of its resolution. The device is also equipped with a large field of view camera 9, which is arranged parallel to the Z axis to shoot the space target consistent with the transition camera 8. By making full use of the mature algorithm of camera image registration, it is easy to achieve deep fusion of the images of the large field of view camera 9 and the transition camera 8. The transition camera 8 and the ranging point cloud have a precisely aligned matching relationship, so that the transition camera 8 realizes the fusion of the large field of view color camera and the TOF ranging radar, and obtains a complete three-dimensional image containing color texture and depth information. The device of the present invention integrates two-dimensional imaging and depth ranging functions, does not need to move fixed cameras and lasers, has few moving structures, a compact system structure, and good robustness. Optical axis alignment and synchronous control reduce the operational complexity of three-dimensional measurement, and achieves higher flexibility and better three-dimensional imaging results based on the transition camera.
[0067] The preferred specific embodiments of the present invention are described in detail above. It should be understood that a person skilled in the art can make many modifications and changes based on the concept of the present invention without creative work. Therefore, any technical solution that can be obtained by a person skilled in the art through logical analysis, reasoning or limited experiments based on the concept of the present invention on the basis of the prior art should be within the scope of protection determined by the claims.
Claims
1. A three-dimensional perception device based on a transition camera, characterized in that: The three-dimensional sensing device integrates laser ranging and camera imaging, and is calibrated using a collimator calibration device. The three-dimensional sensing device comprises a laser transmitter (1) and a laser receiver (2). A first reflector (4) is provided on one side of the laser transmitter (1), wherein a small hole is provided in the first reflector (4). A beam splitter (3) is provided on one side of the first reflector (4), and a transition camera (8) and a laser receiver (2) are provided on both sides of the beam splitter (3). A visible light bandpass filter (7) is provided between the transition camera (8) and the beam splitter (3). A rotating double prism (5) is provided on the other side of the first reflector (4), and a target object (6) in space is provided on one side of the rotating double prism (5). The three-dimensional sensing device further comprises a large field of view camera (9) facing the target object (6) in space. The collimator calibration device comprises a second reflector (10), a semi-reflective and semi-transmissive mirror (11), a beacon laser (12), a beam analyzer (13), a collimating lens group (14) and a corner cube prism (15).
2. A three-dimensional perception device based on a transition camera according to claim 1, characterized in that: The device is arranged in a three-dimensional coordinate system, wherein the large-field-of-view camera (9) is arranged parallel to the Z axis of the three-dimensional coordinate system, and the first reflector (4) reflects light of the object and emits it along the positive direction of the X axis.
3. A three-dimensional perception device based on a transition camera according to claim 2, characterized in that: The first reflector (4) is placed at an angle of 45°.
4. The three-dimensional perception device based on a transition camera according to claim 3, characterized in that: The center of the rotating double prism (5) is coated with a laser anti-reflection film.
5. A three-dimensional perception method based on a transition camera, characterized in that: Based on the three-dimensional sensing device according to any one of claims 1 to 4, the method comprises the following steps: Calibrate the three-dimensional sensing device based on the collimator calibration device, control the ranging laser, the transition camera visual axis and the collimator light to be coaxial, and thus make the visual axis of the transition camera (8) coaxial with the ranging laser; After calibration, the laser transmitter (1) is controlled to emit a distance-measuring laser. The distance-measuring laser passes through a small hole in a first reflector (4) and is irradiated onto the surface of a rotating double prism (5). The distance-measuring laser is irradiated onto a target object (6) and diffusely reflected to generate a reflected laser. The reflected laser is deflected by the rotating double prism (5) and irradiated onto the mirror surface of the first reflector (4). After being reflected by a beam splitter (3), it is absorbed by a laser receiver (2) to obtain a distance-measuring point cloud. At the same time, the visible light generated by the space object (6) passes through the same optical path as the reflected laser, and is received by a transition camera (8) through the beam splitter (3) and a visible light bandpass filter (7). A transition camera image is generated. The transition camera image has a certain matching relationship with the distance-measuring point cloud. A large-field-of-view camera (9) photographs the target object (6) to obtain a color image; The camera image registration algorithm is used to fuse the transition camera image of the matching ranging point cloud with the color image. Finally, a three-dimensional image containing color texture and depth information is generated based on the transition camera.
6. The three-dimensional perception method based on a transition camera according to claim 5, characterized in that: The specific steps of calibrating the three-dimensional sensing device based on the collimator calibration device are: A beacon laser (12) is emitted through a collimator calibration device with a focal length of f, the beacon laser (12) is reflected by a half-reflecting mirror (11) and a second reflector (10), and then passes through a collimating lens group (14) to become parallel light, and the wavelength of the beacon laser (12) matches the visible light bandpass filter (7); The parallel light enters the corner cube prism (15) and is reversed. The reversed light enters the beam analyzer (13) of the collimator calibration device through the half-reflecting half-mirror (11), and the light spot position on the beam analyzer (13) is recorded. The light spot is the coaxial point (X1, Y1) of the beacon light for transmitting and receiving. The parallel light also passes through a rotating double prism (5), a first reflector (4), a beam splitter (3) and a visible light bandpass filter (7) before entering a transition camera (8), generating a beacon light spot on the transition camera (8). The installation position of the transition camera (8) is adjusted so that the beacon light spot is imaged at the center of the transition camera (8), and the visual axis of the transition camera (8) is coaxial with the parallel light. The laser transmitter (1) is controlled to emit a distance measuring laser, and the distance measuring laser enters a collimator calibration device and is irradiated onto a beam analyzer (13). The emission intensity of the laser transmitter (1) is adjusted and the centroid position (X2, Y2) of the light spot of the distance measuring laser on the beam analyzer (13) is recorded. The azimuth and elevation angles of the laser transmitter (1) are adjusted to control the centroid position (X2, Y2) and the coaxial point (X1, Y1) of the beacon light to be the same, so that the distance measuring laser is coaxial with the collimator light.
7. The three-dimensional perception method based on a transition camera according to claim 6, characterized in that: The visual axis of the transition camera (8) is coaxial with the ranging laser: The light of the ranging laser after passing through the small hole in the first reflector (4) and the visual axis of the transition camera (8) after being reflected by the first reflector (4) are coaxial, and the direction is controlled together by the rotating double prism (5).
8. The three-dimensional perception method based on a transition camera according to claim 7, characterized in that: The light of the ranging laser after passing through the small hole in the first reflector (4) and the visual axis of the transition camera (8) after being reflected by the first reflector (4) are specifically coaxial: The coaxiality between the visual axis of the transition camera (8) and the ranging laser is within a certain range.
9. The three-dimensional perception method based on a transition camera according to claim 8, characterized in that: The coaxiality between the visual axis of the transition camera (8) and the ranging laser is: ((X1-X2) / f,(Y1-Y2) / f).
10. The three-dimensional perception method based on a transition camera according to claim 5, characterized in that: The rotating double prism (5) moves according to a certain scanning rule.
Citation Information
Cited By
Spatial non-cooperative dynamic target approaching visual detection method
CN119850888A
Correlation light measuring device based on inclined plane mirror and control method
CN120871079A
A slope-mirror-based correlation light measuring device and control method
CN120871079B
