Dynamically adjustable polarization camera array calibration device and layered imaging method and system

Abstract

The invention discloses a dynamic adjustable polarization camera array calibration device and a layered imaging method and system, and the device comprises an adjustable polarization camera array module which is composed of M * N polarization cameras, and the distance between the cameras is dynamically adjusted through a preset guide rail; the mobile calibration device is composed of a high-precision checkerboard calibration plate and a laser ranging unit, and the precision of the high-precision checkerboard calibration plate is 0.1 mm; and the control module comprises a motion control unit and a screw rod device, and the control module is used for realizing the movement of the high-precision checkerboard calibration plate in the depth direction at the step length of 40mm. According to the invention, through the dynamically adjustable polarization camera array and the depth-parallax model, the problems of fixed base line, complex calibration and insufficient polarization information utilization of the traditional camera array are solved, and the target sensing and three-dimensional reconstruction capabilities in a complex environment are significantly improved.

Description

Technical Field

[0001] The present invention relates to the field of computer vision and optical measurement technology, and more specifically, to a dynamically adjustable polarization camera array calibration device, a layered imaging method, and a system. Background Art

[0002] Perceptual imaging and target detection have always been hot topics in the field of computer vision, and researchers have conducted extensive research on these topics. The demand for perceptual imaging and recognition in challenging and complex environments, such as occlusion, camouflage, fog, and smoke, has been steadily increasing. Traditional imaging methods are often affected by factors such as illumination changes, scattering, shadows, noise, and occlusion. To address these issues, scientists have proposed polarization imaging methods to improve the recognizability of objects in complex environments. Polarization imaging excels in object recognition and penetration capabilities, and has a wide range of applications in defogging, underwater imaging, object shape recovery, material recognition, and more.

[0003] Camera arrays were originally designed to capture light field information, sampling the light field formed by all radiation in the scene from multiple perspectives and representing it with images from different perspectives. Camera arrays based on light field descriptions usually consist of dozens of cameras, forming a wide baseline "synthetic aperture" to achieve focused computational imaging at different depths. These camera arrays play a key role in computational imaging, target tracking, video stitching, and security monitoring. However, fixed-position camera array systems face the challenges of adapting to changing application scenarios, capturing polarization information at different depths, and achieving imaging of polarization layers at arbitrary depths.

[0004] Traditional camera array systems have the following technical bottlenecks: (1) Fixed baseline limitation: Existing planar arrays (e.g., 5×15 layout) use fixed camera spacing, which makes it difficult to adapt to the parallax changes of targets at different depths, resulting in a decrease in the accuracy of layered imaging; (2) The calibration complexity is high. Traditional calibration methods (such as Zhang’s calibration method) assume that the parallax and depth have a linear relationship and ignore the nonlinear distortion in the actual optical system. (3) Insufficient utilization of polarization information. The existing system does not fully integrate multi-angle polarization information (0°, 45°, 90°, 135°), resulting in limited material recognition and surface detail reconstruction effects. Summary of the Invention

[0005] The purpose of the present invention is to provide a dynamically adjustable polarization camera array calibration device, layered imaging method and system, which are suitable for target perception and three-dimensional reconstruction in complex environments (such as scenes with occlusion, smoke, camouflage, etc.), and are used to address the shortcomings of traditional camera arrays in the prior art, such as fixed baselines, complex calibration, and insufficient utilization of polarization information.

[0006] A first aspect of the present invention provides a dynamically adjustable polarization camera array calibration device, comprising: Adjustable polarization camera array module, consisting of several polarization cameras, where the camera spacing is dynamically adjusted via preset guide rails; Mobile calibration device, consisting of a high-precision checkerboard calibration plate and a laser ranging unit; The control module includes a motion control unit and a screw device, and the control module is used to realize the movement of the high-precision chessboard calibration plate in the depth direction with a preset step length.

[0007] A second aspect of the present invention provides a dynamically adjustable polarization camera array layered imaging method, which is applied to the above-mentioned dynamically adjustable polarization camera array calibration device. The method comprises the following steps: Acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; Extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image; Vector calculation is performed on the polarization-corrected image set to perform depth layer separation to obtain a layered depth map.

[0008] In this solution, the acquisition of target images specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

[0009] In this solution, extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

[0010] In this solution, the target parameters are obtained by performing model fitting after feature point detection and disparity calculation based on the calibration data, specifically including: Perform feature point detection on the calibration images in the calibration image set, where checkerboard corner points are extracted from each calibration image to match corresponding points between different cameras to obtain matching point pairs; Calculate the horizontal disparity of the matching point pair ; Define horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportionality coefficient, is the power index, For paranoid items, for depth; Using the preset algorithm Optimize and get the basic formula of parallax: ; in, is parallax, As the baseline, For depth, is the focal length, is the pixel size; The depth-disparity model parameters are obtained based on various parameters in the nonlinear relationship, and the camera parameters are obtained based on various parameters in the basic disparity formula.

[0011] In this solution, performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map specifically includes: Performing Stokes calculation on each pixel point of the polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle; Using a threshold value to distinguish metal from non-metal based on the linear polarization degree and the linear polarization angle; The depth of each pixel is inferred based on the depth-disparity model to perform depth layer separation to obtain the layered depth map.

[0012] A third aspect of the present invention further provides a dynamically adjustable polarization camera array layered imaging system, comprising a memory and a processor, wherein the memory includes a dynamically adjustable polarization camera array layered imaging method program, and when the dynamically adjustable polarization camera array layered imaging method program is executed by the processor, the following steps are implemented: Acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; Extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image; Vector calculation is performed on the polarization-corrected image set to perform depth layer separation to obtain a layered depth map.

[0013] In this solution, the acquisition of target images specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

[0014] In this solution, extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

[0015] In this solution, the target parameters are obtained by performing model fitting after feature point detection and disparity calculation based on the calibration data, specifically including: Perform feature point detection on the calibration images in the calibration image set, where checkerboard corner points are extracted from each calibration image to match corresponding points between different cameras to obtain matching point pairs; Calculate the horizontal disparity of the matching point pair ; Define horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportionality coefficient, is the power index, For paranoid items, for depth; Using the preset algorithm Optimize and get the basic formula of parallax: ; in, is parallax, As the baseline, For depth, is the focal length, is the pixel size; The depth-disparity model parameters are obtained based on various parameters in the nonlinear relationship, and the camera parameters are obtained based on various parameters in the basic disparity formula.

[0016] In this solution, performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map specifically includes: Performing Stokes calculation on each pixel point of the polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle; Using a threshold value to distinguish metal from non-metal based on the linear polarization degree and the linear polarization angle; The depth of each pixel is inferred based on the depth-disparity model to perform depth layer separation to obtain the layered depth map.

[0017] A fourth aspect of the present invention provides a computer-readable storage medium, which includes a program for a dynamically adjustable polarization camera array layered imaging method for a machine. When the program for the dynamically adjustable polarization camera array layered imaging method is executed by a processor, the steps of a dynamically adjustable polarization camera array layered imaging method as described in any one of the above items are implemented.

[0018] The present invention discloses a dynamically adjustable polarization camera array calibration device, layered imaging method, and system. By utilizing a dynamically adjustable polarization camera array and a depth-parallax model, these devices address the issues of fixed baselines, complex calibration, and insufficient utilization of polarization information in traditional camera arrays. These improvements significantly enhance target perception and 3D reconstruction capabilities in complex environments. Specific beneficial effects are as follows: 1. Improved accuracy: Compared with traditional methods, the depth reconstruction accuracy is improved by more than 3 times; 2. Enhanced adaptability, supporting dynamic switching of multiple array layouts from 5×5 to 5×15; 3. Function expansion, adding polarized material recognition capability, with metal / non-metal classification accuracy exceeding 90%. BRIEF DESCRIPTION OF THE DRAWINGS

[0019] Figure 1 A schematic structural diagram of a dynamically adjustable polarization camera array calibration device according to the present invention is shown; Figure 2 A schematic diagram showing the steps of a dynamically adjustable polarization camera array layered imaging method of the present invention is shown; Figure 3 A diagram showing a depth-parallax relationship model verification diagram of a dynamically adjustable polarization camera array layered imaging method of the present invention is shown; Figure 4 A three-dimensional surface diagram of a generalized array model of a dynamically adjustable polarization camera array layered imaging method according to the present invention is shown; Figure 5A block diagram of a dynamically adjustable polarization camera array layered imaging system of the present invention is shown. DETAILED DESCRIPTION

[0020] In order to more clearly understand the above-mentioned objects, features and advantages of the present invention, the present invention is further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be noted that, in the absence of conflict, the embodiments of the present application and the features therein can be combined with each other.

[0021] In the following description, many specific details are set forth to facilitate a full understanding of the present invention. However, the present invention may also be implemented in other ways different from those described herein. Therefore, the scope of protection of the present invention is not limited to the specific embodiments disclosed below.

[0022] To overcome the limitations of the above-mentioned background technology, the present invention proposes an innovative solution, namely, through a flexible polarization camera array with a mobile calibration device and a polarization camera array calibration and layered imaging method based on a depth-parallax model, which can adapt to different scenarios and meet the needs of obtaining polarization information and array layer imaging in various situations, thus breaking away from the limitations brought by traditional fixed arrays.

[0023] Figure 1 A schematic structural diagram of a dynamically adjustable polarization camera array calibration device of the present application is shown.

[0024] like Figure 1 As shown, the present application discloses a dynamically adjustable polarization camera array calibration device, comprising: Adjustable polarization camera array module, consisting of several polarization cameras, where the camera spacing is dynamically adjusted via preset guide rails; Mobile calibration device, consisting of a high-precision checkerboard calibration plate and a laser ranging unit; The control module includes a motion control unit and a screw device, and the control module is used to realize the movement of the high-precision chessboard calibration plate in the depth direction with a preset step length.

[0025] It should be noted that, in this embodiment, the adjustable polarization camera array module described in this application is specifically composed of M×N polarization cameras, wherein the camera spacing can be dynamically adjusted through a guide rail (adjustment range 60-1000mm), and the mobile calibration device integrates a high-precision checkerboard calibration plate (specifications are 120×120cm, accuracy is 0.1mm) and a laser ranging unit, as well as a control module. Specifically, through a motion control unit and a screw device, the calibration plate is moved in the depth direction with a preset step size. In one embodiment, when the depth direction is "6-30m", it can move in steps of 40mm.

[0026] Figure 2A flow chart of a dynamically adjustable polarization camera array layered imaging method of the present application is shown.

[0027] like Figure 2 As shown, the present application discloses a dynamically adjustable polarization camera array layered imaging method, comprising the following steps: S202, acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; S204, extracting target parameters based on the fitted depth-disparity model and performing disparity correction on the target image; S206 , performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map.

[0028] It should be noted that in this embodiment, before performing actual measurement, it is necessary to fit the depth-disparity model first, wherein the process of fitting the target parameters will be described in detail in the subsequent specification. When applied, the target image is first collected, that is, the four-directional polarization image corresponding to the actual scene is collected, and then the target parameters are extracted based on the fitted depth-disparity model to perform parallax correction on the target image, and finally vector calculation is performed on the corrected polarization image set to perform depth layer separation to obtain a layered depth map.

[0029] According to an embodiment of the present invention, acquiring a target image specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

[0030] It should be noted that in this embodiment, when layering polarization images, the target image must first be acquired, that is, the polarization images in four directions are acquired. First, the perspective alignment is performed based on the central camera perspective as a reference for image acquisition. The homography matrix H is used to achieve perspective alignment during parallax correction: ; ; in, is the polarization image, is the polarization image after view alignment, is the parallax depth on the horizontal axis, is the disparity depth on the vertical axis, the homography matrix It is a "3×3" projection transformation matrix used to describe the mapping relationship between two planes. In the camera array, it maps the image coordinates (xi, yi) of the non-central view to the coordinates (xc, yc) of the central view.

[0031] Furthermore, the four directions mentioned in the above description correspond to 0°, 45°, 90° and 135°, so the polarization image finally obtained corresponds to the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction ,in, Corresponding to the image Image.

[0032] According to an embodiment of the present invention, extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

[0033] It should be noted that in this embodiment, the camera spacing (60-1000mm) is adjusted by the guide rail according to the target depth range (such as near view / far view), and the control module drives the calibration plate to move in the depth direction with a step length of 40mm, synchronously triggering all cameras to collect the calibration plate image. Among them, each movement records the checkerboard image (including polarization directions 0°, 45°, 90° and 135°) taken by each camera and the actual depth value returned by the laser rangefinder Get the calibration image set (Each image corresponds to a depth ) and a sequence of depth values .

[0034] Then, based on the calibration data, feature point detection and disparity calculation are performed, and then model fitting is performed to obtain target parameters. For details, please refer to the subsequent instructions for detailed description.

[0035] According to an embodiment of the present invention, performing feature point detection and disparity calculation based on the calibration data and then performing model fitting to obtain target parameters specifically includes: Perform feature point detection on the calibration images in the calibration image set, where checkerboard corner points are extracted from each calibration image to match corresponding points between different cameras to obtain matching point pairs; Calculate the horizontal disparity of the matching point pair ; Define horizontal parallax and depth Nonlinear relationship Get, among them, It is a nonlinear relationship. is the proportionality coefficient, is the power index, For paranoid items, for depth; Using the preset algorithm Optimize and get the basic formula of parallax: ; in, is parallax, As the baseline, For depth, is the focal length, is the pixel size; The depth-disparity model parameters are obtained based on various parameters in the nonlinear relationship, and the camera parameters are obtained based on various parameters in the basic disparity formula.

[0036] It should be noted that, in this embodiment, the nonlinear optimization includes the following steps: (1) Feature point detection; (2) Disparity calculation; (3) Model fitting; (4) Model verification; Specifically, feature point detection is performed on the calibration images in the calibration image set, wherein the checkerboard corner points are extracted from each calibration image to match the corresponding points between different cameras to obtain matching point pairs, and then the horizontal disparity of the matching point pairs is calculated. , and define the horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportional coefficient (related to the geometric parameters of the calibration device), is the power exponent (usually [0.8,1.2]), is the bias term (compensation for system error or baseline offset, compensation for mechanical error, unit: pixel), is the depth, and finally the preset algorithm (LM algorithm can be used) is used to Optimize and obtain the basic formula of parallax of generalized array model: ;in, is parallax, As the baseline, For depth, is the focal length, is the pixel size, and finally the model is verified, such as Figure 3 As shown, it is a depth-disparity relationship model verification diagram, where the LM (Levenberg-Marquardt) algorithm is a nonlinear least squares optimization algorithm.

[0037] Specifically, the basic disparity formula is extended to the three-dimensional spatial relationship to establish the three-dimensional relationship between depth, disparity and baseline: ;in, 、 、 are array layout parameters, For depth, As the baseline, the final depth-disparity model is obtained by nonlinear least squares fitting, such as Figure 4 As shown, it is a three-dimensional surface plot of the generalized array model.

[0038] According to an embodiment of the present invention, performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map specifically includes: Performing Stokes calculation on each pixel point of the polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle; distinguishing metal from sub-metal based on the linear polarization degree and the linear polarization angle threshold; The depth of each pixel is inferred based on the depth-disparity model to perform depth layer separation to obtain the layered depth map.

[0039] It should be noted that, in this embodiment, Stokes calculation is performed on the pixel points of each polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle. The calculation formula of the fused image is as follows: ; Among them, F is the fused image, I is the polarization image, DOLP is the degree of linear polarization (DOLP), and AOLP is the angle of linear polarization (AOLP). They are two core parameters in polarization imaging, which respectively characterize the polarization characteristics of light. Based on these two parameters, a threshold can be used to distinguish between metal masks and non-metal masks. Finally, the depth of each pixel is inferred based on the depth-parallax model to perform depth layer separation to obtain the layered depth map.

[0040] Figure 5 A block diagram of a dynamically adjustable polarization camera array layered imaging system of the present invention is shown.

[0041] like Figure 5As shown, the present invention discloses a dynamically adjustable polarization camera array layered imaging system, including a memory and a processor. The memory includes a dynamically adjustable polarization camera array layered imaging method program. When the dynamically adjustable polarization camera array layered imaging method program is executed by the processor, the following steps are implemented: Acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; Extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image; Vector calculation is performed on the polarization-corrected image set to perform depth layer separation to obtain a layered depth map.

[0042] It should be noted that in this embodiment, before performing actual measurement, it is necessary to fit the depth-disparity model first, wherein the process of fitting the target parameters will be described in detail in the subsequent specification. When applied, the target image is first collected, that is, the four-directional polarization image corresponding to the actual scene is collected, and then the target parameters are extracted based on the fitted depth-disparity model to perform parallax correction on the target image, and finally vector calculation is performed on the corrected polarization image set to perform depth layer separation to obtain a layered depth map.

[0043] According to an embodiment of the present invention, acquiring a target image specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

[0044] It should be noted that in this embodiment, when layering polarization images, the target image must first be acquired, that is, the polarization images in four directions are acquired. First, the perspective alignment is performed based on the central camera perspective as a reference for image acquisition. The homography matrix H is used to achieve perspective alignment during parallax correction: ; ; in, is the polarization image, is the polarization image after view alignment, is the parallax depth on the horizontal axis, is the disparity depth on the vertical axis, the homography matrix It is a "3×3" projection transformation matrix used to describe the mapping relationship between two planes. In the camera array, it maps the image coordinates (xi, yi) of the non-central view to the coordinates (xc, yc) of the central view.

[0045] Furthermore, the four directions mentioned in the above description correspond to 0°, 45°, 90° and 135°, so the polarization image finally obtained corresponds to the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction ,in, Corresponding to the image Image.

[0046] According to an embodiment of the present invention, extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

[0047] It should be noted that in this embodiment, the camera spacing (60-1000mm) is adjusted by the guide rail according to the target depth range (such as near view / far view), and the control module drives the calibration plate to move in the depth direction with a step length of 40mm, synchronously triggering all cameras to collect the calibration plate image. Among them, each movement records the checkerboard image (including polarization directions 0°, 45°, 90° and 135°) taken by each camera and the actual depth value returned by the laser rangefinder Get the calibration image set (Each image corresponds to a depth ) and a sequence of depth values .

[0048] Then, based on the calibration data, feature point detection and disparity calculation are performed, and then model fitting is performed to obtain target parameters. For details, please refer to the subsequent instructions for detailed description.

[0049] According to an embodiment of the present invention, performing feature point detection and disparity calculation based on the calibration data and then performing model fitting to obtain target parameters specifically includes: Perform feature point detection on the calibration images in the calibration image set, where checkerboard corner points are extracted from each calibration image to match corresponding points between different cameras to obtain matching point pairs; Calculate the horizontal disparity of the matching point pair ; Define horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportionality coefficient, is the power index, For paranoid items, for depth; Using the preset algorithm Optimize and get the basic formula of parallax: ; in, is parallax, As the baseline, For depth, is the focal length, is the pixel size; The depth-disparity model parameters are obtained based on various parameters in the nonlinear relationship, and the camera parameters are obtained based on various parameters in the basic disparity formula.

[0050] It should be noted that, in this embodiment, the nonlinear optimization includes the following steps: (1) Feature point detection; (2) Disparity calculation; (3) Model fitting; (4) Model verification; Specifically, feature point detection is performed on the calibration images in the calibration image set, wherein the checkerboard corner points are extracted from each calibration image to match the corresponding points between different cameras to obtain matching point pairs, and then the horizontal disparity of the matching point pairs is calculated. , and define the horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportional coefficient (related to the geometric parameters of the calibration device), is the power exponent (usually [0.8,1.2]), is the bias term (compensation for system error or baseline offset, compensation for mechanical error, unit: pixel), is the depth, and finally the preset algorithm (LM algorithm can be used) is used to Optimize and obtain the basic formula of parallax of generalized array model: ;in, is parallax, As the baseline, For depth, is the focal length, is the pixel size, and finally the model is verified, such as Figure 3 As shown, it is a depth-disparity relationship model verification diagram, where the LM (Levenberg-Marquardt) algorithm is a nonlinear least squares optimization algorithm.

[0051] Specifically, the basic disparity formula is extended to the three-dimensional spatial relationship to establish the three-dimensional relationship between depth, disparity and baseline: ;in, 、 、 are array layout parameters, For depth, As the baseline, the final depth-disparity model is obtained by nonlinear least squares fitting, such as Figure 4 As shown, it is a three-dimensional surface plot of the generalized array model.

[0052] According to an embodiment of the present invention, performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map specifically includes: Performing Stokes calculation on each pixel point of the polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle; distinguishing metal from sub-metal based on the linear polarization degree and the linear polarization angle threshold; The depth of each pixel is inferred based on the depth-disparity model to perform depth layer separation to obtain the layered depth map.

[0053] It should be noted that, in this embodiment, Stokes calculation is performed on the pixel points of each polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle. The calculation formula of the fused image is as follows: ; Among them, F is the fused image, I is the polarization image, DOLP is the degree of linear polarization (DOLP), and AOLP is the angle of linear polarization (AOLP). They are two core parameters in polarization imaging, which respectively characterize the polarization characteristics of light. Based on these two parameters, a threshold can be used to distinguish between metal masks and non-metal masks. Finally, the depth of each pixel is inferred based on the depth-parallax model to perform depth layer separation to obtain the layered depth map.

[0054] A fourth aspect of the present invention provides a computer-readable storage medium, which includes a dynamically adjustable polarization camera array layered imaging method program. When the dynamically adjustable polarization camera array layered imaging method program is executed by a processor, the steps of a dynamically adjustable polarization camera array layered imaging method as described in any one of the above items are implemented.

[0055] The present invention discloses a dynamically adjustable polarization camera array calibration device, layered imaging method and system. By using a dynamically adjustable polarization camera array and a depth-parallax model, the device solves the problems of fixed baselines, complex calibration and insufficient utilization of polarization information in traditional camera arrays, significantly improving target perception and three-dimensional reconstruction capabilities in complex environments.

[0056] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. The device embodiments described above are merely schematic. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as: multiple units or components can be combined, or can be integrated into another system, or some features can be ignored or not executed. In addition, the coupling, direct coupling, or communication connection between the components shown or discussed can be through some interfaces, and the indirect coupling or communication connection of the devices or units can be electrical, mechanical or other forms.

[0057] The units described above as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units; they may be located in one place or distributed across multiple network units; some or all of the units may be selected according to actual needs to achieve the purpose of the scheme of this embodiment.

[0058] In addition, all functional units in the embodiments of the present invention may be integrated into one processing unit, or each unit may be separately used as a unit, or two or more units may be integrated into one unit; the above-mentioned integrated units may be implemented in the form of hardware or in the form of hardware plus software functional units.

[0059] Those skilled in the art will appreciate that all or part of the steps of the above-mentioned method embodiments may be implemented by hardware associated with program instructions, and the aforementioned program may be stored in a computer-readable storage medium. When the program is executed, the program executes the steps of the above-mentioned method embodiments. The aforementioned storage medium includes various media that can store program codes, such as mobile storage devices, read-only memories (ROMs), random access memories (RAMs), magnetic disks, or optical disks.

[0060] Alternatively, if the integrated units described above are implemented as software modules and sold or used as standalone products, they can also be stored on a computer-readable storage medium. Based on this understanding, the technical solutions of the embodiments of the present invention, or the portion that contributes to the prior art, can be embodied in the form of a software product. This computer software product, stored on a storage medium, includes instructions for enabling a computer device (such as a personal computer, server, or network device) to execute all or part of the methods described in various embodiments of the present invention. The aforementioned storage media include various media capable of storing program code, such as removable storage devices, ROM, RAM, magnetic disks, or optical disks.

Claims

1. A dynamically adjustable polarization camera array calibration device, characterized in that: include: Adjustable polarization camera array module, consisting of several polarization cameras, where the camera spacing is dynamically adjusted via preset guide rails; Mobile calibration device, consisting of a high-precision checkerboard calibration plate and a laser ranging unit; The control module includes a motion control unit and a screw device, and the control module is used to realize the movement of the high-precision chessboard calibration plate in the depth direction with a preset step length.

2. A dynamically adjustable polarization camera array layered imaging method, characterized in that: The method applied to the dynamically adjustable polarization camera array calibration device according to claim 1 comprises the following steps: Acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; Extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image; Vector calculation is performed on the polarization-corrected image set to perform depth layer separation to obtain a layered depth map.

3. The method for layered imaging using a dynamically adjustable polarization camera array according to claim 2, wherein: The acquisition of the target image specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

4. The method for layered imaging using a dynamically adjustable polarization camera array according to claim 3, wherein: The step of extracting target parameters based on the fitted depth-disparity model and performing disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

5. The method for layered imaging using a dynamically adjustable polarization camera array according to claim 4, wherein: The method of performing feature point detection and disparity calculation based on the calibration data and then performing model fitting to obtain target parameters specifically includes: Perform feature point detection on the calibration images in the calibration image set, where checkerboard corner points are extracted from each calibration image to match corresponding points between different cameras to obtain matching point pairs; Calculate the horizontal disparity of the matching point pair ; Define horizontal parallax and depth The nonlinear relationship is obtained ,in, It is a nonlinear relationship. is the proportionality coefficient, is the power index, For paranoid items, for depth; Using the preset algorithm Optimize and get the basic formula of parallax: ； in, is parallax, As the baseline, For depth, is the focal length, is the pixel size; The depth-disparity model parameters are obtained based on various parameters in the nonlinear relationship, and the camera parameters are obtained based on various parameters in the basic disparity formula.

6. The method for layered imaging using a dynamically adjustable polarization camera array according to claim 5, wherein: The performing vector calculation on the corrected polarization image set to perform depth layer separation to obtain a layered depth map specifically includes: Performing Stokes calculation on each pixel point of the polarization image in the corrected polarization image set to obtain the linear polarization degree and linear polarization angle; Using a threshold value to distinguish metal from non-metal based on the linear polarization degree and the linear polarization angle; The depth of each pixel is inferred based on the depth-disparity model to perform depth layer separation to obtain the layered depth map.

7. A dynamically adjustable polarization camera array layered imaging system, characterized in that: The system comprises a memory and a processor, wherein the memory comprises a program for a dynamically adjustable polarization camera array layered imaging method, and when the program is executed by the processor, the following steps are implemented: Acquiring a target image, wherein the target image includes a four-directional polarization image of an actual scene; Extracting target parameters based on the fitted depth-disparity model to perform disparity correction on the target image; Vector calculation is performed on the polarization-corrected image set to perform depth layer separation to obtain a layered depth map.

8. The dynamically adjustable polarization camera array layered imaging system according to claim 7, characterized in that: The acquisition of the target image specifically includes: Image acquisition is performed based on the perspective alignment of the central camera perspective, where the perspective alignment is to map the coordinates of other camera images to the central perspective through the homography matrix; Polarization images are obtained based on the four directions of the actual scene, which correspond to the 0° direction, 45° direction, 90° direction and 135° direction. The polarization images obtained specifically include the polarization image corresponding to the 0° direction. , polarization image corresponding to the 45° direction , polarization image corresponding to the 90° direction Polarization image corresponding to the 135° direction .

9. The dynamically adjustable polarization camera array layered imaging system according to claim 8, characterized in that: The step of extracting target parameters based on the fitted depth-disparity model and performing disparity correction on the target image specifically includes: Initializing a calibration device and collecting calibration data, wherein the calibration data includes a calibration image set and a depth value sequence; Feature point detection and disparity calculation are performed based on the calibration data, and then model fitting is performed to obtain target parameters, where the target parameters include depth-disparity model parameters and camera parameters.

10. A computer-readable storage medium, characterized in that The computer-readable storage medium includes a dynamically adjustable polarization camera array layered imaging method program. When the dynamically adjustable polarization camera array layered imaging method program is executed by a processor, the steps of a dynamically adjustable polarization camera array layered imaging method as described in any one of claims 2 to 6 are implemented.

Images