Small polarization three-dimensional imaging method and device in complex illumination environment

Abstract

The invention discloses a small-sized polarization three-dimensional imaging method and device in a complex illumination environment, and the method comprises the steps: constructing a small-sized polarization three-dimensional imaging device and a measurement model in the complex illumination environment, enabling a visible light camera and a polarization camera to be coaxially designed, and synchronously achieving the spatial single polarization and stereoscopic vision, namely, two types of new three-dimensional imaging modes; through polarization three-dimensional dense reconstruction and stereoscopic vision reconstruction data, the azimuth ambiguity in the polarization dense reconstruction process is removed and the zenith angle is corrected through re-projection error minimization, so that the polarization three-dimensional reconstruction precision is improved; the polarized three-dimensional dense point cloud is converted to a camera coordinate system based on stereoscopic vision reconstruction three-dimensional point cloud, and accurate data is provided for three-dimensional reconstruction or pose measurement and other applications. The method is suitable for realizing small-sized polarization three-dimensional imaging in a complex illumination environment, has the advantages of interference resistance, high precision, high speed and the like, and has important significance in ensuring three-dimensional dense imaging, positioning / attitude determination and the like in the fields of aerospace and the like in the complex illumination environment.

Description

Technical Field

[0001] The present invention relates to the technical field of optical sensing detection, and particularly relates to a small-sized polarization three-dimensional imaging method and device under a complex illumination environment. Background Art

[0002] With the rapid development of artificial intelligence, computer vision, and robotics technologies, three-dimensional imaging technology has become a key support in fields such as industrial inspection, autonomous driving, aerospace, and medical diagnosis. Traditional three-dimensional imaging methods mainly rely on stereo vision, structured light projection, lidar (LiDAR), and time-of-flight (ToF), etc. For example, stereo vision restores depth information through multi-view image matching, structured light achieves high-precision reconstruction by projecting coded light spots, while LiDAR and ToF directly measure distances by actively emitting laser pulses. However, there are significant limitations in complex illumination environments (such as strong direct light, low illuminance, high dynamic range, specular reflection, etc.): Stereo vision relies on the robustness of feature matching, but is prone to false matching in weak texture regions or when the illumination is uneven; Structured light is easily interfered by ambient light, especially in strong light or reflective scenes where the projected pattern will be submerged;

[0003] Although lidar has strong anti-interference ability, it has low resolution, high cost, and is difficult to obtain surface material information;

[0004] ToF technology is easily affected by the multipath effect, and the error is significant on the surface of transparent or highly reflective objects. At present, the challenges of complex illumination environments for three-dimensional imaging are mainly reflected in two aspects: physical interference (such as strong light noise, shadows, highlights) and algorithm robustness (such as feature extraction and matching failure). For example, in the satellite on-orbit maintenance mission, the surface of the spacecraft forms a very high contrast due to direct sunlight and the deep space background, and it is difficult for traditional cameras to simultaneously capture the details of the shadow area and the texture of the highlighted area; in the autonomous driving scenario, the specular reflection of the vehicle paint or glass will cause holes in the LiDAR point cloud. Therefore, there is an urgent need for a new three-dimensional imaging technology that can work stably under complex illumination, and has both high precision and anti-interference ability.

[0005] Polarization is the spatial distribution characteristic of the vibration direction of light waves. The polarization state of the light reflected or scattered by the object surface is closely related to its material, roughness, and geometric shape. In recent years, due to its sensitivity to surface physical properties, polarization imaging technology has gradually become a research hotspot in the field of three-dimensional reconstruction. By analyzing the light intensity information in different polarization directions, the zenith angle and azimuth angle of the surface normal vector can be deduced, and then the three-dimensional morphology of the object can be reconstructed. Compared with traditional methods, polarization three-dimensional imaging has the following advantages: (1) Resistance to light interference. Polarization information is not sensitive to changes in light intensity and can maintain stability in high-dynamic-range scenarios; (2) Strong detail resolution ability. It can capture micro-surface structure features and is suitable for weakly textured or textureless objects; (3) Multi-physical information fusion. Geometric shape and material properties (such as metal, plastic, glass) can be obtained simultaneously. However, there are still two major bottleneck problems in polarization three-dimensional reconstruction: (1) Azimuth ambiguity: There is a periodic ambiguity between the polarization angle and the azimuth angle of the surface normal vector, resulting in an uncertain direction of the normal vector; (2) Noise sensitivity: Polarization information is easily affected by sensor noise, multi-path reflection, and non-ideal polarization effects, and it is difficult for the reconstruction accuracy to meet the engineering requirements. Existing research mostly solves the ambiguity problem through multi-view polarization image fusion or introducing prior geometric models, but the algorithm complexity is high and it depends on scene assumptions, making it difficult to be applied in real time in dynamic environments. Summary of the Invention

[0006] Based on the above background, the main purpose of the present invention is to provide a small-scale polarization three-dimensional imaging method and device in a complex lighting environment, which has the characteristics of simple structural composition, high imaging efficiency, strong versatility, etc., and can perform three-dimensional dense imaging and target positioning / orientation in a complex lighting environment.

[0007] To achieve the above purpose, the technical solution of the present invention includes:

[0008] A small-scale polarization three-dimensional imaging method in a complex lighting environment, and the implementation steps are as follows:

[0009] Step S110: Build a small-scale polarization three-dimensional imaging device in a complex lighting environment, construct a measurement model of the small-scale polarization three-dimensional imaging device in a complex lighting environment. The small-scale polarization three-dimensional imaging device includes a virtual visible light camera, a polarization camera, a beam splitter, a reflector, and a visible light camera. The visible light camera is used to collect visible light images of the spatial target and form a coaxial stereo vision system with the polarization camera. The polarization camera is used to collect the polarization images of the spatial target and perform target polarization three-dimensional imaging. The beam splitter and the reflector are used to achieve the coaxial design of the visible light camera and the polarization camera. The virtual visible light camera is used to describe the spatial distribution of the visible light camera and the polarization camera in the optical system, and form a coaxial stereo imaging model;

[0010] Step S120: Design the visible light camera and the polarization camera in combination with a semi-transparent and semi-reflective mirror and a reflector to achieve their coaxial design;

[0011] Step S130: Synchronously collect visible light images and polarization images in the same-source scene through a visible light camera and a polarization camera. By performing dense feature point matching on the visible light images and polarization images, complete three-dimensional dense reconstruction data based on the polarization camera, and achieve stereo vision reconstruction data based on the visible light camera and the polarization camera;

[0012] Step S140: Based on the stereo vision reconstruction data of the visible light camera and the polarization camera in Step S130, minimize the back-projection error of image feature points to eliminate the azimuth ambiguity in the three-dimensional dense reconstruction data completed by the polarization camera and correct the zenith angle deviation;

[0013] Step S150: Using the stereo vision reconstruction data achieved by the visible light camera and the polarization camera as a constraint, convert the relative data obtained by three-dimensional reconstruction of the polarization camera, that is, the three-dimensional dense reconstruction data, into absolute scale data, namely obtain the dense three-dimensional point cloud data relative to the coordinate system of the polarization camera;

[0014] Step S160: Perform part segmentation according to the characteristics of the space target, construct the coordinate system of the space target itself, convert the dense three-dimensional point cloud data obtained in Step S150 into the camera coordinate system, and then obtain the pose parameters of the space target in the coordinate system of the polarization camera.

[0015] A small polarization three-dimensional imaging device under a complex lighting environment, used to execute the small polarization three-dimensional imaging method under a complex lighting environment, includes:

[0016] A virtual visible light camera, used to describe the spatial distribution of the visible light camera and the polarization camera in the optical system, and form a coaxial stereo imaging model;

[0017] A polarization camera, used to collect polarization images of the space target and for target polarization three-dimensional imaging;

[0018] A beam splitter, used to divide the space light into two parts, 50% passes through the beam splitter and enters the polarization camera, and 50% of the light is reflected by the beam splitter and reaches the reflector;

[0019] A reflector, used to reflect the light reflected by the beam splitter and enter the visible light camera;

[0020] A visible light camera, used to collect polarization images of the space target and for target polarization three-dimensional imaging;

[0021] Among them, the visible light camera and the polarization camera are coaxial designed through the combined design of the beam splitter and the reflector;

[0022] The device further includes:

[0023] A processor, used to execute the following steps:

[0024] Taking the stereoscopic vision reconstruction data achieved by a visible light camera and a polarization camera as a constraint, the relative data obtained by three-dimensional reconstruction of the polarization camera, namely the three-dimensional dense reconstruction data, is converted into absolute scale data, that is, the dense three-dimensional point cloud data relative to the coordinate system of the polarization camera is obtained;

[0025] According to the characteristics of the space target, part segmentation is carried out, the coordinate system of the space target itself is constructed, the dense three-dimensional point cloud data is converted into the camera coordinate system, and then the pose parameters of the space target in the coordinate system of the polarization camera are obtained.

[0026] The beneficial effects of the present invention compared with the prior art are as follows:

[0027] The present invention provides a novel small-sized polarization three-dimensional imaging method and device under a complex illumination environment, including: constructing a small-sized polarization three-dimensional imaging device and a measurement model under a complex illumination environment, coaxially designing a visible light camera and a polarization camera, and synchronously realizing spatial single polarization and stereoscopic vision, namely two new three-dimensional imaging modes; through polarization three-dimensional dense reconstruction and stereoscopic vision reconstruction data, minimizing the reprojection error to remove the azimuth ambiguity in the polarization dense reconstruction process and correct the zenith angle deviation, so as to improve the polarization three-dimensional reconstruction accuracy; based on the stereoscopic vision reconstruction three-dimensional point cloud, converting the polarization three-dimensional dense point cloud into the camera coordinate system to provide accurate data for applications such as three-dimensional reconstruction or pose measurement.

[0028] Through the coaxial polarization-stereoscopic vision system design and the tightly coupled reconstruction algorithm, the present invention first integrally combines a visible light camera and a polarization camera coaxially, and uses a beam splitter to synchronously capture RGB images and multi-angle polarization images, which not only ensures the field of view consistency but also avoids the calibration error caused by mechanical displacement; secondly, taking the three-dimensional point cloud reconstructed by stereoscopic vision as a constraint, dynamically corrects the polarization azimuth ambiguity and corrects the zenith angle deviation by minimizing the reprojection error, and at the same time uses the polarization dense point cloud to fill the sparse area of the stereo matching; realizes real-time reconstruction through embedded GPU acceleration, and the volume and power consumption of the device can meet the stringent requirements of aerospace payloads. The polarization information provides dense surface normal vectors, while the stereoscopic vision calculates the absolute depth through parallax. The combination of the two can simultaneously solve the polarization ambiguity and correct the zenith angle deviation and the sparsity of stereo matching problems; the polarization camera and the visible light camera share the optical path, which can realize hardware miniaturization and synchronous data acquisition, and has the characteristics of small size and low power consumption.

[0029] With the rapid development of space exploration and intelligent equipment, this technology can be applied in fields such as aerospace, deep space exploration, national defense and military, etc. For example, in on-orbit satellite services, the manipulator of the space station needs to identify the attitude and deformation of the target spacecraft in real time under the alternation of strong sunlight and the earth's shadow; in lunar / Mars exploration, the rover needs to quickly reconstruct the terrain under extreme sunlight conditions (such as lunar dust scattering, rock specular reflection) to avoid travel risks; in UAV autonomous navigation, in urban canyons or forest environments, the UAV needs to complete obstacle avoidance and positioning under the interference of strong backlight, glass curtain wall reflection, etc. The solution proposed by the present invention can provide data support for realizing high-precision and high-reliability space exploration and intelligent perception, and has important engineering application value.

[0030] The present invention is suitable for realizing small-sized polarization three-dimensional imaging in complex illumination environments, and has the advantages of anti-interference, high precision, high speed, etc., which is of great significance for ensuring three-dimensional dense imaging and positioning / attitude determination in fields such as aerospace under complex illumination environments. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is a flowchart of a method for small-sized polarization three-dimensional imaging in a complex illumination environment according to the present invention;

[0032] Figure 2 is a schematic diagram of a device for small-sized polarization three-dimensional imaging in a complex illumination environment according to the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0033] The basic idea of the present invention is: to construct a small-sized polarization three-dimensional imaging device and a measurement model in a complex illumination environment, co-axially design a visible light camera and a polarization camera, and synchronously realize spatial single polarization and stereo vision, that is, two new three-dimensional imaging modes; through polarization three-dimensional dense reconstruction and stereo vision reconstruction data, minimize the reprojection error to remove the azimuth ambiguity in the polarization dense reconstruction process and correct the zenith angle deviation, so as to improve the accuracy of polarization three-dimensional reconstruction; based on the stereo vision reconstruction three-dimensional point cloud, convert the polarization three-dimensional dense point cloud to the camera coordinate system to provide accurate data for applications such as three-dimensional reconstruction or pose measurement.

[0034] The following takes a method and device for small-sized polarization three-dimensional imaging in a complex illumination environment as an example to further elaborate on the present invention.

[0035] As Figure 1 shown, a method for small-sized polarization three-dimensional imaging in a complex illumination environment according to the present invention mainly includes the following steps:

[0036] Step S110: Set up a small-scale polarization three-dimensional imaging device in a complex lighting environment, and construct a measurement model for the small-scale polarization three-dimensional imaging device in a complex lighting environment. The small-scale polarization three-dimensional imaging device includes a virtual visible light camera, a polarization camera, a beam splitter, a reflector, and a visible light camera. The visible light camera is used to collect visible light images of spatial targets and form a coaxial stereo vision system with the polarization camera. The polarization camera is used to collect polarization images of spatial targets and perform three-dimensional polarization imaging of the targets. The beam splitter is used to divide the spatial light into two parts, with 50% passing through the beam splitter and entering the polarization camera, and 50% of the light being reflected by the beam splitter and reaching the reflector. The reflector is used to reflect the light reflected by the beam splitter and enter the visible light camera. The virtual visible light camera is used to describe the spatial distribution of the visible light camera and the polarization camera in the optical system, forming a coaxial stereo imaging model. The measurement model means that by synchronously collecting visible light images and polarization images of the same source scene through the visible light camera and the polarization camera, performing dense feature point matching on the visible light images and the polarization images, realizing stereo vision reconstruction data based on the visible light camera and the polarization camera, completing three-dimensional dense reconstruction data based on the polarization camera, and using the stereo vision reconstruction data to perform scale transformation on the dense reconstruction data of the polarization camera to obtain the dense three-dimensional point cloud of the spatial target.

[0037] Among them, the visible light camera and the polarization camera are coaxially designed to establish a spatial single polarization and stereo vision synchronous imaging model, supplemented by a power supply, a camera trigger, and a data acquisition and processing device to achieve three-dimensional imaging of space. The following step S120 details the coaxial design method.

[0038] Step S120: The visible light camera and the polarization camera are combined and designed through a semi-transparent and semi-reflective mirror and a reflector to achieve coaxial design, that is, synchronous acquisition of homologous images of spatial targets can be realized. The coaxial design method through the beam splitter and the reflector includes that after the spatial light passes through the beam splitter, the light is divided into two parts, where 50% of the light directly enters the polarization camera for collecting polarization images of spatial targets, and the other 50% of the light is incident on the reflector and enters the visible light camera after being reflected by the reflector. The visible light camera is located at the position of the virtual visible light camera in the space, Figure 2 forming a coaxial stereo design with the polarization camera. Under the above configuration conditions, it can be ensured that the light of the same spatial target passes through the optical system and is simultaneously imaged on both the polarization camera and the visible light camera, forming a coaxial stereo imaging mode in space.

[0039] This step also includes:

[0040] Step S120-1: The visible light camera is configured with a short focal length lens to achieve large-angle and low-resolution imaging of space targets; the polarization camera is configured with a long focal length lens to achieve small-angle and high-resolution imaging of the targets. The focal length ratio of the two is less than 1:3, that is, it is ensured that the lens of the visible light camera is smaller than that of the polarization camera. For example, the lens of the visible light camera is 8 mm, and the lens configured for the polarization camera is not less than 24 mm, ensuring synchronous imaging of the two and enabling the construction of a coaxial stereo vision sensor;

[0041] Step S120-2: Based on the coaxial design of the visible light camera and the polarization camera, calibration of the internal parameters and external parameters of the visible light camera and the polarization camera is achieved to remove the reconstruction error introduced by optical distortion. In the implementation, a planar checkerboard target is used to synchronously calibrate the internal parameters and external parameters of the above two cameras, providing a basis for subsequent three-dimensional imaging;

[0042] Step S130: The visible light camera and the polarization camera are used to synchronously collect visible light images and polarization images in the same-source scene. By performing dense feature point matching on the visible light images and polarization images, three-dimensional dense reconstruction data is completed based on the polarization camera, and stereo vision reconstruction data is achieved based on the visible light camera and the polarization camera.

[0043] Among them, the coaxial-designed visible light camera and polarization camera can construct a stereo vision sensor, and thus three-dimensional point cloud reconstruction can be performed.

[0044] Step S130 further includes:

[0045] In the pinhole imaging model, the coaxial vision system is equivalent to the movement of the optical center along the optical axis, and the change in the distance between the camera optical center and the object is equivalent to the change in the focal length. In this imaging system, the focal lengths of the visible light camera and the polarization camera are different, and the above coaxial stereo imaging model can be constructed. Therefore, the depth information of the object can be obtained by imaging the same target with the above cameras with different focal lengths, which is specifically expressed as follows:

[0046] (1)

[0047] Among them, , are the focal lengths of the polarization camera and the visible light camera respectively, satisfying , , are the radial radii of a pair of matching points in the coaxial stereo vision system, which can be determined by dense feature point matching.

[0048] The coaxial stereo vision system refers to the synchronous imaging mode of the polarized camera and the visible light camera for spatial targets constructed through a beam splitter and a reflector, realizing the coaxial stereo vision system of the polarized camera and the visible light camera in the optical system. After obtaining the object depth information based on the above coaxial stereo vision system, it is then substituted into the visible light camera imaging model and converted into spatial three-dimensional coordinates , which can be expressed as,

[0049] (2)

[0050] Among them, is the coordinate of the matching point in the visible light camera image coordinate system.

[0051] Here, the dense matching method of image feature points can be adopted to complete the dense feature point matching. Since the images collected by the visible light camera and the polarized camera are homologous images, and there is only a scale difference between the two images, their matching efficiency, quality, and accuracy are all higher than those of the conventional image feature point matching accuracy;

[0052] Step S130-2, the polarized camera performs a single exposure imaging, and can synchronously collect the 0°, 45°, 90°, and 135° polarization gray values of each pixel. Using the obtained polarization gray values , , , , the polarization angle and the degree of polarization can be calculated in real time; The calculation of the Stockes components is as follows:

[0053] (3)

[0054] The method for obtaining the degree of polarization and the polarization angle is: , .

[0055] The following is the relationship between the degree of polarization and the zenith angle:

[0056] Let the surface of the target in space be represented by the Cartesian surface equation, , where , , are the spatial three-dimensional point coordinates. Let be the normal vector of any three-dimensional point, and the zenith angle and the azimuth angle can be represented. Their expressions are:

[0057] (4)

[0058] When the change of the polarization component when light is reflected and refracted on the target surface satisfies the Fresnel formula, when specular reflection occurs on the target surface, if the incident light is unpolarized light, the main axis direction of specular reflection is perpendicular to the surface normal vector and the outgoing light ray, and the degree of polarization The expression is:

[0059] (5)

[0060] Among them, is the specular polarization degree, is the refractive index of the target surface, is the outgoing angle. When diffuse reflection occurs on the target surface, in fact, the light first enters the target through refraction, and after depolarization by scattering, it becomes polarized light again through refraction. The main axis direction of the outgoing light is in the plane of the surface normal vector and the outgoing light ray. The degree of polarization of diffuse reflection is expressed as:

[0061] (6)

[0062] Among them, is the diffuse reflection polarization degree.

[0063] According to the change curves of the polarization degree of the outgoing light in the cases of specular reflection and diffuse reflection, it can be seen that if the polarization degree of the outgoing light is known, there are usually two different solutions for the outgoing angle in the case of specular reflection, while at most one solution can be obtained in the case of diffuse reflection, and the obtained angle depends on the specific value of the refractive index. For most non-metallic materials, their refractive index is generally between 1.3 and 1.6.

[0064] Suppose the known target mask , the gradient field to be integrated , and the depth field reconstruction result is . The cost function is constructed according to the error of the gradient field in the area where the target is located as follows:

[0065] (7)

[0066] Among them, is the cost function constructed by the error of the gradient field, and represent the gradient operators in the x and y directions. Integrating using the Frankot-Chellappa algorithm can obtain the three-dimensional spatial point cloud information, realizing the acquisition of the three-dimensional dense point cloud in space by a single-polarization camera.

[0067] Step S140: Based on the stereo vision reconstruction data of the visible light camera and the polarization camera in step S130, eliminate the azimuth ambiguity in the three-dimensional dense reconstruction data completed by the polarization camera and correct the zenith angle deviation.

[0068] In step S130, three-dimensional dense reconstruction data can be obtained based on the polarization camera, but the scale information is lacking. Based on the stereo vision reconstruction data obtained from the three-dimensional reconstruction of the visible light and polarization cameras in step S130, and minimizing the reprojection error of the image feature points to remove the azimuth ambiguity and correct the zenith angle deviation in the reconstruction process of the polarization camera, so as to improve the three-dimensional point cloud reconstruction accuracy of the polarization camera.

[0069] Due to the problems of azimuth ambiguity and zenith angle deviation in the three-dimensional reconstruction by a single polarization camera, in the present invention, based on the three-dimensional data obtained from the stereo reconstruction of the visible light camera and the polarization camera, and combining the image feature points in the reconstruction process of the polarization camera, an objective function is established by minimizing the back-projection error of the image feature points. Specifically, step S140 includes:

[0070] The three-dimensional point perspective projection imaging model in the coordinate system of the polarization camera is as follows:

[0071] (8)

[0072] Among them, represents the image feature point coordinates, , are the horizontal and vertical indices of the image point coordinates. is the internal parameter matrix of the polarization camera, is the three-dimensional point coordinates in the polarization coordinate system corresponding to the image point coordinates.

[0073] An objective function is established by minimizing the back-projection error of the image feature points, as follows,

[0074] (9)

[0075] Among them, is the feature point coordinates obtained by back-projecting the three-dimensional point coordinates to the image, is the back-projection error of the image feature points.

[0076] By minimizing the above back-projection error, the azimuth ambiguity in the reconstruction process of the polarization camera can be eliminated and the zenith angle deviation can be corrected, improving the reconstruction accuracy. Realize the elimination of azimuth ambiguity and the correction of zenith angle deviation in the reconstruction of the polarization camera, complete the dense reconstruction of the spatial three-dimensional point cloud, and provide a data basis for subsequent target detection and perception;

[0077] Step S150: Using the stereo vision reconstruction data obtained from the stereo three-dimensional reconstruction of the visible light camera and the polarization camera as a constraint, convert the relative data obtained from the three-dimensional reconstruction of the polarization camera, i.e., the three-dimensional dense reconstruction data, into absolute scale data, that is, obtain the dense three-dimensional point cloud data relative to the coordinate system of the polarization camera, such as Figure 2 the three-dimensional point cloud information of the aircraft in

[0078] Among them, the method of converting the three-dimensional dense reconstruction data into absolute scale data includes: (1) Using the polarization camera and the visible light camera to collect images, and performing image feature point matching, and reconstructing based on the coaxial stereo vision principle to obtain the three-dimensional coordinates of the spatial sparse feature points ; (2) Performing scale normalization processing on the spatial three-dimensional data reconstructed in step 8 . Take any two different three-dimensional points in the point cloud , and calculate their ratios in , , the three-component scale factors , and the specific calculation formula is as follows,

[0079] (10)

[0080] Based on the above scale factors, perform ratio conversion on to obtain the converted point cloud ,

[0081] (11)

[0082] Step S160: Perform part segmentation according to the characteristics of the spatial target, construct the coordinate system of the spatial target itself, convert the dense three-dimensional point cloud data obtained in step S150 into the coordinate system of the polarization camera, and then obtain the pose parameters of the spatial target in the coordinate system of the polarization camera.

[0083] Taking an aircraft as an example, taking the center of gravity of the aircraft as the origin of the aircraft coordinate system , the direction of the aircraft nose as the axis of the aircraft coordinate system, the right wing as the Y axis, and the X axis perpendicular to the OZY plane, the pose parameters of the aircraft in the coordinate system of the polarization camera can be obtained, which can be applied to the optical detection and intelligent perception of targets in complex spatial environments.

[0084] According to the embodiments of the present invention, there is also provided a small polarization three-dimensional imaging device in a complex illumination environment, including:

[0085] A virtual visible light camera, used to describe the spatial distribution of the visible light camera and the polarization camera in the optical system, and form a coaxial stereo imaging model;

[0086] A polarization camera, which is used to collect polarization images of space targets and is used for three-dimensional polarization imaging of targets;

[0087] A beam splitter, which is used to divide the space light into two parts, 50% of the light passes through the beam splitter and enters the polarization camera, and 50% of the light is reflected by the beam splitter and reaches the reflector;

[0088] A reflector, which is used to reflect the light reflected by the beam splitter and enter the visible light camera;

[0089] A visible light camera, which is used to collect visible light images of space targets and forms a coaxial stereo vision system with the polarization camera;

[0090] Among them, the visible light camera and the polarization camera are coaxially designed through the combined design of the beam splitter and the reflector;

[0091] The device further includes:

[0092] A processor, which is used to perform the following steps:

[0093] Taking the stereo vision reconstruction data achieved by the visible light camera and the polarization camera as a constraint, converting the relative data obtained by three-dimensional reconstruction of the polarization camera, i.e., the three-dimensional dense reconstruction data, into absolute scale data, that is, obtaining the dense three-dimensional point cloud data relative to the coordinate system of the polarization camera;

[0094] According to the characteristics of the space target, performing part segmentation, constructing the coordinate system of the space target itself, converting the dense three-dimensional point cloud data into the camera coordinate system, and further obtaining the pose parameters of the space target in the coordinate system of the polarization camera.

[0095] Although the present invention has been described above according to a limited number of embodiments, those skilled in the art in this technical field will understand that within the scope of the present invention thus described, other embodiments can be envisioned. In addition, it should be noted that the language used in this specification is mainly selected for the purpose of readability and teaching, rather than for the purpose of explaining or limiting the subject matter of the present invention.

Claims

1. A small-scale polarization three-dimensional imaging method under complex lighting environment, characterized in that: The implementation steps are as follows: Step S110: building a small polarization 3D imaging device under a complex lighting environment, constructing a measurement model of the small polarization 3D imaging device under a complex lighting environment, wherein the small polarization 3D imaging device comprises a virtual visible light camera, a polarization camera, a beam splitter, a reflector, and a visible light camera, wherein the visible light camera is used to collect visible light images of space targets and to form a coaxial stereo vision system with the polarization camera, wherein the polarization camera is used to collect polarization images of space targets and to perform polarization 3D imaging of the targets, wherein the beam splitter and the reflector are used to realize the coaxial design of the visible light camera and the polarization camera, and wherein the virtual visible light camera is used to describe the spatial distribution of the visible light camera and the polarization camera in the small polarization 3D imaging device to form a coaxial stereo imaging model; Step S120: The visible light camera and the polarization camera are combined and designed through a beam splitter and a reflector to achieve a coaxial design of the two; Step S130: synchronously collecting visible light images and polarization images in the same source scene through a visible light camera and a polarization camera, completing three-dimensional dense reconstruction data based on the polarization camera by performing dense feature point matching on the visible light image and the polarization image, and realizing stereoscopic vision reconstruction data based on the visible light camera and the polarization camera; Step S140: based on the stereoscopic vision reconstruction data of the visible light camera and the polarization camera in step S130, the back projection error of the image feature points is minimized to eliminate the azimuth ambiguity in the three-dimensional dense reconstruction data completed by the polarization camera and correct the zenith angle deviation; Step S150: using the stereo vision reconstruction data realized by the visible light camera and the polarization camera as constraints, converting the relative data obtained by the polarization camera for three-dimensional reconstruction, i.e., three-dimensional dense reconstruction data, into absolute scale data, i.e., obtaining dense three-dimensional point cloud data relative to the polarization camera coordinate system; Step S160: perform part segmentation according to the characteristics of the space target, construct the space target's own coordinate system, convert the dense three-dimensional point cloud data obtained in step S150 into the polarization camera coordinate system, and then obtain the position and posture parameters of the space target in the polarization camera coordinate system.

2. The small-scale polarization three-dimensional imaging method under complex lighting environment according to claim 1, characterized in that: In step S120, the method of coaxial design through the combination of a beam splitter and a reflector includes: after the space target reflects the light and enters the small polarization three-dimensional imaging device, the beam splitting operation is performed through the beam splitter to divide the light into two parts, 50% of the light directly passes through the beam splitter and reaches the polarization camera, and the other 50% of the light passes through the beam splitter and is reflected to the reflector. After being reflected by the reflector, the light reaches the visible light camera.

3. The small-scale polarization 3D imaging method under complex illumination environment according to claim 1, characterized in that: Step S120 also includes: Step S120-1, designing a visible light camera and a polarization camera as follows: the visible light camera is configured with a short-focus lens to achieve low-resolution imaging of a space target at a large viewing angle; the polarization camera is configured with a long-focus lens to achieve high-resolution imaging of a target at a small viewing angle, and the focal length ratio of the two is less than 1:3; Step S120 - 2 , based on the coaxial design of the visible light camera and the polarization camera, calibration of the intrinsic parameters and extrinsic parameters of the visible light camera and the polarization camera is achieved.

4. The small-scale polarization 3D imaging method under complex lighting environment according to claim 3, characterized in that: In step S120 - 2 , a planar checkerboard target is used to synchronously calibrate the intrinsic and extrinsic parameters of the visible light camera and the polarization camera.

5. The small-scale polarization 3D imaging method under complex illumination environment according to claim 3, characterized in that: Step S130 also includes: Step S130-1, imaging the same target by using a visible light camera and a polarization camera with different focal lengths to obtain depth information of the object: (1) in, , are the focal lengths of the polarization camera and the visible light camera, respectively, satisfying , , It is the radial radius of a pair of matching points in the coaxial stereo vision system. After obtaining the depth information of the object based on the above coaxial stereo vision system, it is substituted into the visible light camera imaging model and converted into spatial three-dimensional coordinates. , expressed as: (2) in, is the coordinate of the image matching point in the visible light camera image coordinate system, is the horizontal axis component, is the ordinate component; Step S130-2: The polarization camera is used to expose the image once, and the polarization grayscale values ​​of 0°, 45°, 90°, and 135° of each pixel are collected synchronously to calculate the polarization angle in real time. and degree of polarization .

6. The small-scale polarization three-dimensional imaging method under complex lighting environment according to claim 5, characterized in that: Step S130-2 also includes: When light is reflected and refracted on the target surface, the change in polarization component satisfies the Fresnel formula. When light is specularly reflected on the target surface, if the incident light is non-polarized light, the direction of the specular reflection axis is perpendicular to the surface normal vector and the outgoing light, and the degree of polarization is The expression is: (5) in, is the specular reflection polarization degree, is the refractive index of the target surface, is the exit angle.

7. The small-scale polarization 3D imaging method under complex illumination environment according to claim 5, characterized in that: Step S130-2 also includes: When light is diffusely reflected on the surface of a target, it actually enters the target through refraction first, is depolarized by scattering, and then refracted again to become polarized light. The main axis direction of the outgoing light is in the plane where the surface normal vector and the outgoing light are located. The polarization degree of diffuse reflection is It is expressed as: (6) in, is the diffuse reflection polarization degree.

8. The small-scale polarization three-dimensional imaging method under complex lighting environment according to claim 1, characterized in that: Step S140 includes: The three-dimensional point perspective projection imaging model in the polarization camera coordinate system is as follows: (8) in, represents the coordinates of the image feature points, , is the horizontal and vertical index of the image point coordinates, is the polarization camera intrinsic parameter matrix, is the three-dimensional point coordinate in the polarization camera coordinate system corresponding to the image point coordinate; The objective function is established by minimizing the back-projection error of image feature points, as follows: (9) in, is the coordinate of the feature point of the image back-projected from the three-dimensional point coordinates, is the back-projection error of the image feature points; By minimizing the above back-projection error, the azimuth ambiguity of the polarization camera during the reconstruction process is eliminated and the zenith angle deviation is corrected.

9. The small-scale polarization three-dimensional imaging method under complex lighting environment according to claim 8, characterized in that: Ways to convert 3D dense reconstruction data into absolute scale data include: The polarization camera and the visible light camera are used to collect images, and the image feature points are matched. Based on the principle of coaxial stereo vision, the images are reconstructed to obtain the three-dimensional coordinates of the spatial sparse feature points. ; The three-dimensional point coordinates in the polarization camera coordinate system corresponding to the image point coordinates Perform scale normalization and obtain the three-dimensional coordinates of spatial sparse feature points For any two different 3D points in , , Three-component scale factor , the specific calculation formula is as follows, (10) Based on the above scaling factors, Perform scale conversion to obtain the converted point cloud , (11)。 10. A small polarization 3D imaging device under complex lighting environment, used to perform the small polarization 3D imaging method under complex lighting environment described in any one of claims 1 to 9, characterized in that: include: A virtual visible light camera is used to describe the position of the visible light camera at an ideal position in space, so as to form a coaxial stereo vision system with a polarization camera; Polarization camera, used to collect polarization images of space targets and to perform three-dimensional polarization imaging of targets; The beam splitter is used to split the light reflected by the space target. 50% of the light is transmitted into the polarization camera, and 50% of the light is reflected to the reflector. A reflector, used to reflect the light reflected from the beam splitter and enter the visible light camera; Visible light camera, used to collect visible light images of space targets and form a coaxial stereo vision system with the polarization camera; Among them, the visible light camera and the polarization camera are designed in a coaxial manner through a combination of a beam splitter and a reflector; The device also includes: A processor for performing the following steps: Using the stereo vision reconstruction data realized by the visible light camera and the polarization camera as constraints, the relative data obtained by the polarization camera for 3D reconstruction, i.e., the 3D dense reconstruction data, is converted into absolute proportional data, i.e., dense 3D point cloud data relative to the polarization camera coordinate system is obtained; The parts are segmented according to the characteristics of the space target, the space target's own coordinate system is constructed, the dense three-dimensional point cloud data is converted to the camera coordinate system, and then the pose parameters of the space target in the polarization camera coordinate system are obtained.

Images