A depth estimation method combining image correction and disparity estimation

Abstract

The present invention discloses a depth estimation method integrating image correction and parallax estimation, comprising the following steps: arranging a camera array on one side of a target to be measured, obtaining a rotation matrix and a translation vector between each camera and a central camera; segmenting the target to be measured in an image acquired by each camera through target recognition, and processing the background to be black; then performing a preprocessing operation on the image of the target to be measured; using a SuperGlue matching algorithm to complete feature point matching work between an image under the viewing angle of a central camera and images under the viewing angles of other cameras; performing coordinate transformation on each group of feature matching points; performing a refocusing operation on each group of feature matching points according to a triangular geometric relationship between parallax and depth and in combination with a translation vector; obtaining the size of a reliability factor of each group of feature matching points according to a calculation formula of the reliability factor, and evaluating the depth estimation quality of each group of feature matching points in turn according to the size of the reliability factor, thereby improving the accuracy of depth estimation.

Description

Technical Field

[0001] The present invention relates to the fields of image processing, computer vision and light field computational imaging, and in particular to a depth estimation method that integrates image correction and disparity estimation based on multi-view stereo matching of a camera array. Background Art

[0002] Light field technology can not only obtain position information, but also angle information, so it has important research significance in the field of depth estimation. The acquisition of light field information mainly depends on microlens array cameras or camera arrays. Camera arrays have certain advantages in depth estimation above the meter level due to their large baseline distance. In recent years, depth estimation based on camera arrays has been widely used in the fields of airborne remote sensing imaging, online detection of optical defects, etc. There is parallax between corresponding points in images from different perspectives. Refocusing the obtained target matching points and combining parallax clues can obtain the depth information of the target. Patent CN113129350A proposes a depth estimation method based on a camera array, but it does not consider issues such as image correction, and cannot generate confidence in target points.

[0003] The method of using a camera array to obtain light field information for depth estimation has high requirements for hardware assembly accuracy. There is a slight translation or rotation between the actual position of the camera array and the ideal position, which affects the final depth estimation accuracy, and the traditional binocular correction method cannot be directly applied to the correction of the camera array image. In addition, if the collected image is directly corrected, the amount of calculation will increase and the calculation efficiency will be reduced. Summary of the invention

[0004] In order to overcome the aforementioned deficiencies of the prior art and solve the problems of large amount of computation in light field image processing and inability to judge self-occlusion, the present invention integrates image correction into the calculation process of the parallax-based depth estimation method, evaluates the quality of depth estimation by setting a reliability factor, and judges the self-occlusion situation, and proposes a more efficient depth estimation method that integrates image correction and parallax estimation.

[0005] The objective of the present invention is achieved through the following technical solutions:

[0006] A depth estimation method integrating image correction and disparity estimation, comprising:

[0007] S1. Set a camera array on one side of the target to be measured, use the camera array to capture the image of the target to be measured, select one camera in the camera array as the central camera, obtain the relative position relationship between each camera and the central camera through binocular positioning, and obtain the rotation matrix and translation vector between the cameras;

[0008] S2. Segment the target object in the image acquired by each camera through target recognition, and process the area except the target object in the image into black; then perform preprocessing operations on the image of the target object; the preprocessing operations include median filtering and super-resolution processing, through which the processing range of the image is narrowed and a clearer image is obtained.

[0009] S3. Use SuperGlue matching algorithm to complete the feature point matching of the image under the perspective of the central camera and the image under the perspective of other cameras to obtain feature point pairs. Feature point pairs refer to a pair of feature points representing the same object point in the central camera and other cameras. Set the feature point pairs corresponding to the same object point in the target object to be measured obtained by all cameras as a group of feature matching points, record the viewpoint number (i, j) and pixel coordinates p (u, v) of each group of feature matching points, and save the confidence value α of each feature point pair in each group of feature matching points obtained during the matching process i ;

[0010] S4. The pixel coordinates p of each set of feature matching points obtained by the array camera are converted into the coordinates P in the camera coordinate system through the intrinsic parameter matrix. c , based on the rotation matrix of the coordinate P c Perform a rotation transformation to obtain the coordinate P c ′, and then use the internal parameter matrix to transform the coordinates P c ' is converted back into pixel coordinates p';

[0011] S5. performing a refocusing operation on each set of feature matching points according to the triangular geometric relationship between disparity and depth and in combination with the translation vector;

[0012] S6. The reliability factor of each group of feature matching points is obtained according to the calculation formula of the reliability factor, and the depth estimation quality of each group of feature matching points is evaluated in turn according to the size of the reliability factor to improve the depth estimation accuracy.

[0013] Further, in step S1,

[0014] P 00 =R (ij,00) (P ij -T (ij,00) ) (1)

[0015] Where P 00 represents the coordinates of point P in the (0,0) camera coordinate system, i.e., the central camera coordinate system. ij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), T (ij,00) is the translation vector between camera (i,j) and the center camera.

[0016] Further, in step S4,

[0017] P cij '=R (ij,00) P cij (2)

[0018] T (ij,00) '=R (ij,00) T (ij,00) (3)

[0019] In the formula, P′ cij is the coordinate of point P in the (i, j) camera coordinate system after transformation, P cij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), T (ij,00) is the translation vector between camera (i, j) and the central camera; T′ (ij,00) =[ΔX (ij,00) ′,ΔY (ij,00) ′,ΔZ (ij,00) ′], is the translation vector between each camera coordinate system and the central camera coordinate system after transformation, ΔX (ij,00) ′、ΔY (ij,00) ′, ΔZ (ij,00) ′ represents the translation amount of each camera coordinate system and the central camera coordinate system in the x, y, and z directions after transformation.

[0020] Further, in step S5,

[0021] u ij ”(s n )=u ij '-s n f xij ΔX (ij,00) ' (4)

[0022] v ij ”(s n )=v ij '-s n f yij ΔY (ij,00) ' (5)

[0023] In the formula, s n Indicates the nth focusing degree, u″ ij (s n ) and v″ ij (s n ) represent the focusing degree s in the refocusing process n The horizontal and vertical coordinates of the pixel at time, u′ ij (s n ) and v′ij (s n ) pixel coordinate translation magnitude and s n Proportional to f xij and f yij Respectively represent the equivalent focal length of camera (i, j) in the x and y directions in pixels, u ij ′ and v ij ′ respectively represent the pixel coordinates of the transformed feature matching points, ΔX (ij,00) and ΔY (ij,00) ′ respectively represents the translation of each camera coordinate system and the central camera coordinate system in the x and y directions after transformation; determine the minimum variance value of the pixel coordinates of the feature matching point under the set focus, and obtain the focus s0 when the pixel coordinate variance value is the smallest; then the depth value Z at the object point corresponding to the feature matching point is

[0024]

[0025] Furthermore, in step S6, the calculation formula of the reliability factor is:

[0026]

[0027] In the formula, Q is the reliability factor, α i (i=1,2,…,m-1) represents the confidence level of each feature point pair in each set of feature matching points. When each set of feature matching points has m cameras successfully matched, m-1 feature point pairs are generated, k m is the confidence probability when the number of matches is m, V(u,v,s0) is the minimum variance value in the refocusing process of each set of feature matching points, and k m The size is related to the number m of object points that are successfully matched under M viewing angles corresponding to each set of feature matching points.

[0028] The present invention also provides an application of a depth estimation method that integrates image correction and parallax estimation, which is applied to light field image processing, airborne remote sensing imaging, and online detection of optical defects.

[0029] Furthermore, the present invention also provides a depth estimation device integrating image correction and disparity estimation, comprising:

[0030] A camera array unit, used for capturing an image of a target object to be measured;

[0031] A calibration unit is used to obtain the relative position relationship between each camera in the camera array and the central camera, and obtain the rotation matrix and translation vector between the cameras;

[0032] A processing unit, used for segmenting the target to be measured in the image acquired by each camera, processing the background black of the target to be measured; and performing preprocessing operations on the image of the target to be measured;

[0033] The feature matching unit is used to match the feature matching points of the image under the central camera's perspective with the image under the other camera's perspective, and record the perspective label (i, j) and pixel coordinates p (u, v) of each set of feature matching points, and save the confidence value α obtained by each set of feature matching points during the matching process. i ;

[0034] A feature matching point correction unit, used to perform coordinate transformation on the feature matching points, correct the rotation matrix error between the cameras, and obtain a new translation vector for the refocusing unit;

[0035] A refocusing unit, used to perform a refocusing operation on each set of feature matching points and obtain a depth estimation result with the minimum variance value during the refocusing process;

[0036] The evaluation unit is used to evaluate the depth estimation quality of each group of feature matching points according to the reliability factor, so as to improve the depth estimation accuracy.

[0037] Furthermore, the present invention also provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the depth estimation method integrating image correction and disparity estimation as described in any one of claims 1 to 6 are implemented.

[0038] Furthermore, the present invention also provides a computer-readable storage medium having a computer program stored thereon, characterized in that when the computer program is executed by a processor, the steps of the depth estimation method integrating image correction and disparity estimation as described in any one of claims 1 to 6 are implemented.

[0039] Compared with the prior art, the technical solution of the present invention has the following beneficial effects:

[0040] 1. The present invention evaluates the depth estimation effect of feature matching points in multi-view images by setting a reliability factor, comprehensively considering the influence of the number of cameras, the confidence of feature matching points and the minimum variance of each group of feature matching points, thereby improving the robustness of the algorithm.

[0041] 2. The present invention can realize self-occlusion judgment for the situation that the surface of an object may be occluded by itself, and realizes more accurate depth estimation of the edge of complex three-dimensional surface by setting evaluation factors for the judgment of self-occlusion.

[0042] 3. The method of the present invention compensates for the influence of the position relationship errors between cameras through the rotation transformation of the feature matching point coordinates and the coefficients during refocusing, which improves the measurement accuracy compared to directly applying the binocular correction algorithm to the camera array.

[0043] 4. The present invention uses a camera array to collect multi-view light field images, and in the refocusing process, combines the calibrated inter-camera translation vector and rotation matrix to correct only the feature matching points of the target object to be measured, which can reduce redundant information and reduce the amount of calculation. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 is a flowchart of depth estimation in an embodiment of the present invention;

[0045] Figure 2 The schematic diagram of camera array stereo imaging is shown in Figure 2. The distance between the optical centers of adjacent cameras, i.e. the baseline, is b, and the focal length of each camera is f. w (X w ,Y w ,Z w ) The image point coordinates on each camera are set to p ij (x ij ,y ij );

[0046] Figure 3 Schematic diagram of the matching results of feature matching points at each viewing angle under self-occlusion. DETAILED DESCRIPTION

[0047] The present invention will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not used to limit the present invention.

[0048] like Figure 1 As shown, this embodiment further illustrates the depth estimation method that integrates image correction and disparity estimation. This embodiment uses a 4×4 camera array to capture light field information to verify the depth estimation method. After acquiring the image, the camera array first segments the target objects to be measured from each perspective through target recognition, and processes the background area in the image except the target object into black, and then performs preprocessing operations on the image of the target object to be measured. The present invention first performs feature point matching on a set of segmented images, and then performs depth estimation on the disparity conditions at different perspectives based on the feature matching points. The image preprocessing link enables subsequent feature point matching to obtain more feature matching points. Taking into account the manufacturing errors of the camera array, it is inevitable to correct the image before depth estimation. The binocular correction algorithm is implemented in two camera coordinate systems and cannot be directly applied to the case of a camera array. To this end, this embodiment integrates image correction into the calculation process of the parallax-based depth estimation method. Specifically:

[0049] S1. A 4×4 camera array is set in a vertical plane on one side of the target to be measured, and the image of the target to be measured is captured by the camera array. One camera in the camera array is selected as the central camera, and the relative position relationship between each camera and the central camera is obtained through binocular positioning, and the rotation matrix and translation vector between the cameras are obtained;

[0050] P 00 =R (ij,00) (P ij -T (ij,00) ) (1)

[0051] Where P 00 Represents the coordinates of point P in the camera coordinate system (0,0), P ij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), where T (ij,00) =[ΔX (ij,00) ,ΔY (ij,00) ,ΔZ (ij,00) ] T , is the translation vector between camera (i,j) and the central camera.

[0052] S2. Segment the target objects in the image acquired by each camera through target recognition, and process the areas in the image except the target objects in black to reduce the processing range of the image; then perform preprocessing operations on the image of the target object, including median filtering and super-resolution processing, to obtain a clearer image for better depth estimation.

[0053] S3. After image preprocessing, before using parallax relationship to measure distance, it is necessary to match the feature points of the captured multi-view images. Considering the impact of the number and confidence of feature matching point pairs on the distance measurement range and distance measurement accuracy in the scene, it is necessary to select a matching method that can overcome the difficulties such as lack of texture, complex three-dimensional geometric structure of the scene, illumination changes and occlusion. Therefore, this embodiment uses the SuperGlue matching algorithm to complete the feature point matching work of the image under the perspective of the central camera and the image under the perspective of other cameras to obtain feature point pairs. The feature point pair refers to a pair of feature points representing the same object point in the central camera and other cameras. The feature point pairs corresponding to the same object point in the target object to be measured obtained by all cameras are set as a group of feature matching points, and the viewpoint label (i, j) and pixel coordinates p (u, v) of each group of feature matching points are recorded, and the confidence size α of each feature point pair in each group of feature matching points obtained during the matching process is saved. i ;

[0054] S4. After obtaining the pixel coordinates of each set of feature matching points, the pixel coordinates p of each set of feature matching points obtained by the array camera are converted through the intrinsic parameter matrix to obtain the coordinates P in the camera coordinate system. c , based on the rotation matrix of the coordinate P c Perform a rotation transformation to obtain the coordinate P c ′, and then use the internal parameter matrix to transform the coordinates P c ' is converted back into pixel coordinates p';

[0055] like Figure 2 As shown, select Cam 00 When the number of cameras is used as the central camera, the rotation matrix and translation vector between each camera and the central camera can be obtained through binocular calibration. It should be noted that the transformation used in binocular calibration is no longer applicable to camera array calibration. Therefore, in the rough calibration process, after obtaining the pixel coordinates of the feature matching points of an object point in space at each camera, the pixel coordinates p of the feature matching points obtained by the array camera are first converted to P in the camera coordinate system through the intrinsic parameter matrix. c , and perform rotation transformation to eliminate the influence caused by the non-parallel coordinate systems of each camera. At this time, there is also a translation relationship between the cameras. After the coordinate transformation, there is

[0056] P cij '=R (ij,00) P cij (2)

[0057] T (ij,00) '=R (ij,00) T (ij,00) (3)

[0058] In the formula, P′ cij is the coordinate of point P in the (i, j) camera coordinate system after transformation, P cij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), T (ij,00) is the translation vector between camera (i, j) and the central camera; T (ij,00) ′=[ΔX (ij,00) ′,ΔY (ij,00) ′,ΔZ (ij,00) ′] T , is the translation vector between each camera coordinate system and the central camera coordinate system after transformation, ΔX (ij,00) ′、ΔY (ij,00) ′, ΔZ (ij,00) ′ respectively represents the translation of each camera coordinate system and the central camera coordinate system in the x, y, and z directions. Finally, the transformed camera coordinates are converted back into pixel coordinates p′(u′, v′) through the intrinsic parameter matrix.

[0059] S5. performing a refocusing operation on each set of feature matching points according to the triangular geometric relationship between disparity and depth and in combination with the translation vector;

[0060] By using the different projection positions of object points in the same space on each imaging plane, its depth information can be obtained. Since there is an inverse relationship between parallax and depth, the spatial object point P can be deduced based on the triangular geometric relationship. w The depth information of is as follows:

[0061]

[0062] Where, d ij is the coordinate difference between the feature matching point of the same spatial object point on each camera and the corresponding feature matching point on the central camera, B ij is the baseline distance between each camera and the central camera, ΔX (ij,00) ′ and ΔY (ij,00) ′ respectively represents the displacement between each camera and the central camera in the x direction and the y direction. The image point coordinates (x′) of the spatial object point P on each camera ij ,y′ ij ) and its image pixel coordinates (u′ ij ,v′ ij ) has a proportional relationship, x′ ij =u′ ij ·d x , y′ ij =v′ ij ·d y , d x and d y Let f be the physical size of each pixel in the x and y directions. x =f / d x 、f y =f / d y , f x and f y Represents the equivalent focal length in pixels in the x-direction and y-direction respectively. The above formula can be further converted into

[0063]

[0064] f xij and f yij Respectively represent the equivalent focal length of camera (i, j) in the x-direction and y-direction in pixels; it can be seen that when the camera parameters and relative positions are determined, it is only necessary to obtain the image pixel pair position of the feature matching point of a certain object point in the imaging plane of each camera to obtain the depth information of the point. Based on this model, the refocusing process of the array image can be expressed as

[0065] u ij ”(s n )=u ij '-s n f xij ΔX (ij,00) ' (6)

[0066] v ij ”(s n )=v ij '-s n f yij ΔY (ij,00) ' (7)

[0067] In the formula, s n Indicates the nth focusing degree, u″ ij (s n ) and v″ ij (s n ) indicates that the focusing degree during the refocusing process is s n The pixel coordinate at time, u′ ij (s n ) and v′ ij (s n ) pixel coordinate translation magnitude and s n Formulas 6 and 7 take into account the influence of the translation relationship between camera coordinate systems left after the image is roughly corrected, and compensate for the position error of the coordinates of the feature matching points under different viewing angles.

[0068] When the focusing degree is s n When , the variance of the sub-image feature matching points after focusing in the angle direction is

[0069]

[0070] The smaller the variance, the better the focus state. By calculating the minimum variance value of the pixel coordinates of the feature matching point under the set focus, the depth value at the corresponding point of the pixel can be estimated.

[0071]

[0072] S6. The reliability factor of each group of feature matching points is obtained according to the calculation formula of the reliability factor, and the depth estimation quality of each group of feature matching points is evaluated in turn according to the size of the reliability factor to improve the depth estimation accuracy.

[0073] Since feature point matching is based on operations under two perspectives, the probability that a feature point that is successfully matched by an object point appears in M ​​images is p = m / M, 2≤m≤M, where m represents the number of successful matches of an object point under M perspectives. wIf it can be matched in m cameras, then it is only necessary to translate the m feature matching points of the feature point and obtain the object point P w Depth information. Suppose each set of feature matching points has a confidence value α during the matching process i , represents the probability that each set of feature matching points corresponds to the same object point. In this embodiment, the reliability factor calculation formula of the specified depth information is as follows:

[0074]

[0075] In the formula, Q is the reliability factor; α i (i=1,2,…,m-1) represents the confidence level of each feature point pair in each set of feature matching points. When each set of feature matching points has m cameras successfully matched, m-1 feature point pairs are generated; V(u,v,s0) is the minimum variance value in the refocusing process of each set of feature matching points; k m is the confidence probability when the number of matches is m. Its size is related to the number m of successful matches of the object point under M viewing angles. The specific values ​​are as follows:

[0076] m 2 3 4 5 6 7 8 9 <![CDATA[k m ]]> 0.3183 0.3536 0.3676 0.3750 0.3750 0.3827 0.3850 0.3867 m 10 11 12 13 14 15 16 <![CDATA[k m ]]> 0.3880 0.3891 0.3900 0.3907 0.3914 0.3919 0.3924

[0077] For objects with complex shapes, self-occlusion must also be considered during feature point matching. Due to angle reasons, the side of the object may be occluded by itself in some viewing angles. Figure 3 As shown, two self-occlusion situations are shown, and the shaded part represents the camera that can match the target object. Let the position of the camera that can match the target object be 1, and the position that cannot be matched be 0, and generate a 4×4 array. Calculate the sum of the values ​​of the adjacent elements of each element in the array. The adjacent elements include the eight surrounding elements (except edge elements). Calculate the sum of the number of occurrences when the sum of the values ​​of the adjacent elements is greater than 2, let it be i, and set the self-occlusion evaluation factor p

[0078] p=-0.125|i-8|+1 (11)

[0079] Where p is a number between 0 and 1. The larger p is, the more likely it is self-occluded. When p>0.75, k m Should be appropriately increased.

[0080]

[0081] The depth estimation quality of feature points in multi-view images is evaluated according to the reliability factor level, and the depth estimation results with low reliability factor levels are deleted, and the outliers in the calculation results are eliminated. According to calculations, when Q>0.0025, the reliability of the depth estimation result is high, and the relative error of the depth estimation result can be controlled within 1%.

[0082] The present invention is not limited to the embodiments described above. The above description of the specific embodiments is intended to describe and illustrate the technical solution of the present invention. The above specific embodiments are merely illustrative and not restrictive. Without departing from the scope of the present invention and the scope of protection of the claims, a person of ordinary skill in the art can also make many forms of specific changes under the guidance of the present invention, which all fall within the scope of protection of the present invention.

Claims

1. A depth estimation method combining image correction and disparity estimation, characterized in that: include: S1. Set a camera array on one side of the target to be measured, use the camera array to capture the image of the target to be measured, select one camera in the camera array as the central camera, obtain the relative position relationship between each camera and the central camera through binocular positioning, and obtain the rotation matrix and translation vector between the cameras; S2. Segment the target object in the image acquired by each camera through target recognition, and process the area of ​​the image except the target object to be measured into black; then perform preprocessing operation on the image of the target object to be measured; S3. Use SuperGlue matching algorithm to complete the feature point matching of the image under the perspective of the central camera and the image under the perspective of other cameras to obtain feature point pairs. Feature point pairs refer to a pair of feature points representing the same object point in the central camera and other cameras. Set the feature point pairs corresponding to the same object point in the target object to be measured obtained by all cameras as a group of feature matching points, record the viewpoint number (i, j) and pixel coordinates p (u, v) of each group of feature matching points, and save the confidence value α of each feature point pair in each group of feature matching points obtained during the matching process i ; S4. The pixel coordinates p of each set of feature matching points obtained by the array camera are converted into the coordinates P in the camera coordinate system through the intrinsic parameter matrix. c , based on the rotation matrix of the coordinate P c Perform a rotation transformation to obtain the coordinate P c ′, and then use the internal parameter matrix to transform the coordinates P c ' is converted back into pixel coordinates p'; S5. performing a refocusing operation on each set of feature matching points according to the triangular geometric relationship between disparity and depth and in combination with the translation vector; S6. The reliability factor of each set of feature matching points is obtained according to the calculation formula of the reliability factor, and the depth estimation quality of each set of feature matching points is evaluated in turn according to the size of the reliability factor to improve the depth estimation accuracy; the calculation formula of the reliability factor is: In the formula, Q is the reliability factor, α i (i=1,2,…,m-1) represents the confidence level of each feature point pair in each set of feature matching points. When each set of feature matching points has m cameras successfully matched, m-1 feature point pairs are generated. V(u,v,s0) is the minimum variance value in the refocusing process of each set of feature matching points. k m is the confidence probability when the number of matches is m, k m The size of is related to the number m of object points that are successfully matched under M viewing angles corresponding to each set of feature matching points.

2. The depth estimation method combining image correction and disparity estimation according to claim 1, characterized in that: In step S1, P 00 =R (ij,00) (P ij -T (ij,00) ) (1) Where P 00 represents the coordinates of point P in the (0,0) camera, i.e., the center camera coordinate system. ij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), T (ij,00) is the translation vector between camera (i,j) and the center camera.

3. The depth estimation method combining image correction and disparity estimation according to claim 1, characterized in that: In step S4, P cij '=R (ij,00) P cij (2) T (ij,00) '=R (ij,00) T (ij,00) (3) In the formula, P′ cij is the coordinate of point P in the (i, j) camera coordinate system after transformation, P cij represents the coordinates of point P in the camera coordinate system (i, j), R (ij,00) is the rotation matrix between camera (i,j) and the center camera (0,0), T (ij,00) is the translation vector between camera (i, j) and the central camera; T (ij,00) ′=[ΔY (ij,00) ′,ΔY (ij,00) ′,ΔZ (ij,00) ′], is the translation vector between each camera coordinate system and the central camera coordinate system after transformation, ΔX (ij,00) ′、ΔY (ij,00) ′, ΔZ (ij,00) ′ represents the translation amount of each camera coordinate system and the central camera coordinate system in the x, y, and z directions after transformation.

4. The depth estimation method combining image correction and disparity estimation according to claim 1, characterized in that: In step S5, u ij ”(s n )=u ij '-s n f xij ΔX (ij,00) ' (4) v ij "(S n )=v ij '-S n f yij ΔY (ij,00) ' (5) In the formula, s n Indicates the nth focusing degree, u″ ij (s n ) and v″ ij (s n ) represent the focusing degree s in the refocusing process n The horizontal and vertical coordinates of the pixel at time, u′ ij (s n ) and v′ ij (s n ) pixel coordinate translation magnitude and s n Proportional, f xij and f yij Respectively represent the equivalent focal length of camera (i, j) in the x and y directions in pixels, u ij ′ and v ij ′ respectively represent the pixel coordinates of the transformed feature matching points, ΔX (ij,00) ′ and ΔY (ij,00) ′ represents the translation of each camera coordinate system and the central camera coordinate system in the x and y directions respectively; Determine the minimum variance value of the pixel coordinates of the feature matching point under the set focus, and find the focus s0 when the pixel coordinate variance value is the smallest; then the depth value Z at the object point corresponding to the feature matching point is 5. The depth estimation method combining image correction and disparity estimation according to claim 1, characterized in that: In step S2, the preprocessing operation includes median filtering and super-resolution processing. Step S2 is used to reduce the processing range of the image and obtain a clearer image.

6. A depth estimation device integrating image correction and disparity estimation, characterized in that: include: A camera array unit, used for capturing an image of a target object to be measured; A calibration unit is used to obtain the relative position relationship between each camera in the camera array and the central camera, and obtain the rotation matrix and translation vector between the cameras; A processing unit, used for segmenting the target object to be measured in the image acquired by each camera, and processing the background black of the target object to be measured; And perform preprocessing operations on the image of the target object to be measured; The feature matching unit is used to match the feature matching points of the image under the central camera's perspective with the image under the other camera's perspective, and record the perspective label (i, j) and pixel coordinates p (u, v) of each set of feature matching points, and save the confidence value α obtained by each set of feature matching points during the matching process. i ; A feature matching point correction unit is used to transform the coordinates of the feature matching points and correct the rotation matrix errors between the cameras to obtain a new translation vector for the refocusing unit; A refocusing unit, used to perform a refocusing operation on each set of feature matching points and obtain a depth estimation result with the minimum variance value during the refocusing process; The evaluation unit is used to evaluate the depth estimation quality of each set of feature matching points according to the reliability factor to improve the depth estimation accuracy; the calculation formula of the reliability factor is: In the formula, Q is the reliability factor, α i (i=1,2,…,m-1) represents the confidence level of each feature point pair in each set of feature matching points. When each set of feature matching points has m cameras successfully matched, m-1 feature point pairs are generated. V(u,v,s0) is the minimum variance value in the refocusing process of each set of feature matching points. k m is the confidence probability when the number of matches is m, k m The size of is related to the number m of object points that are successfully matched under M viewing angles corresponding to each set of feature matching points.

7. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the program, the steps of the depth estimation method integrating image correction and disparity estimation as described in any one of claims 1 to 5 are implemented.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the depth estimation method combining image correction and disparity estimation as described in any one of claims 1 to 5 are implemented.

Images