3D image processing method and system with low calculation requirement

Abstract

The invention provides a low-calculation-requirement 3D image processing method and system. The method specifically comprises the steps that collected left-eye and right-eye images and image display and calibration information actually to be obtained are obtained; constructing a coordinate transformation matrix, and performing translation zooming processing on the left eye image and the right eye image; constructing a common internal reference matrix of the left-eye camera and the right-eye camera, performing epipolar correction on coordinates of the left-eye image and the right-eye image after translation and zooming, and performing distortion correction on the coordinates of the left-eye image and the right-eye image after epipolar correction; performing first interpolation to obtain displayable left and right eye images; calculating left and right disparity maps based on the interpolated left and right eye images, and calculating a depth map based on the left and right disparity maps to obtain depth information; reconstructing a 3D point cloud based on the depth information and the left and right eye image coordinates after distortion correction; and rotating the 3D point cloud, regenerating left-eye and right-eye 2D image coordinates, performing second interpolation, and finally outputting the image. The method has the advantages of high precision and low delay.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and specifically to a low-computational-requirement 3D image processing method and system. Background Technology

[0002] In fields such as minimally invasive surgical navigation and endoscopic 3D imaging, the real-time performance of images directly impacts surgical safety and operational smoothness. In existing 3D imaging systems, to correctly display 3D images, it's necessary to eliminate lens distortion and image distortion caused by the epipolar lines of binocular cameras. This typically requires two independent steps: distortion correction and epipolar correction. Distortion correction first transforms the image coordinates to a normalized camera coordinate system. Distortion coefficients (such as radial coefficients k1, k2, k3 and tangential coefficients p1, p2) are used to calculate the coordinates of distorted points. Then, the image is mapped back to the original coordinates using an intrinsic parameter matrix, and a corrected image is generated through interpolation, mapping the original image pixels to an ideal, distortion-free coordinate system. Epipolar correction, based on the camera's intrinsic and extrinsic parameters obtained from camera calibration, establishes a projection transformation model. The original image is projected onto a distortion-free and strictly row-aligned image plane to obtain a binocular image with strictly row-aligned pixels. Finally, invalid pixels generated during the correction process are interpolated to restore the image, ensuring the left and right images meet the coplanar row alignment requirement. This creates the following bottlenecks in the image processing workflow of binocular endoscope systems:

[0003] 1. Poor real-time performance: In traditional algorithms, distortion correction and epipolar correction are independent algorithm processes. Distortion correction (such as radial distortion correction of fisheye lens) and epipolar correction (to achieve row alignment of left and right eye images) need to be executed independently step by step. Each correction step also requires image interpolation, resulting in superimposed delays in the processing flow. The cumulative delay caused by the two-stage interpolation operation significantly affects the real-time performance of the images displayed by the 3D endoscope system.

[0004] During surgery, camera shake and undesirable rotations can cause dizziness in the surgeon due to the real-time 3D image. To counteract this shake or rotation, additional image rotation is required. However, simple image rotation can disrupt the established left-right epipolar relationship, leading to decreased computational accuracy, reduced visual comfort, and even malfunction of the stereo matching algorithm. To eliminate the effects of image rotation and meet the requirements for magnified observation of lesion details within the endoscopic field of view, additional algorithmic processing and two image interpolation steps are necessary (at least including rotation + interpolation + additional algorithm + interpolation), further increasing image latency.

[0005] 2. Hardware resource conflict: Multiple interpolations require separate calls to the GPU texture unit, which consumes several times the DSP block resources in the endoscope FPGA processing chip, making it difficult for the system to run stably at high resolution and high frame rate.

[0006] 3. Dynamic scene distortion: Step-by-step correction leads to inconsistent image deformation between frames of moving organs (such as a beating heart), affecting the stability of the surgical field of view. Summary of the Invention

[0007] In order to overcome the shortcomings of the prior art, the purpose of this invention is to provide a 3D image processing method and system with low computational requirements.

[0008] To achieve the above-mentioned objectives of this invention, this invention provides a low-computational-requirement 3D image processing method, comprising the following steps:

[0009] Acquire the left and right eye images and display and calibrate the actual images to be acquired;

[0010] Determine the scaling parameters and the image translation amount, construct a coordinate transformation matrix, and perform translation and scaling processing on the left and right eye images based on the coordinate transformation matrix to obtain the translation and scaling coordinates of the left and right eye images;

[0011] Construct a common intrinsic parameter matrix for the left and right cameras. Use the common intrinsic parameter matrix to perform epipolar correction on the coordinates of the translated and scaled left and right camera images to obtain the coordinates of the left and right camera images. Perform distortion correction on the epipolar corrected coordinates of the left and right camera images.

[0012] The coordinates of the left and right eye images after epipolar correction and distortion correction are interpolated for the first time to obtain displayable left and right eye images;

[0013] The left and right disparity maps are calculated based on the interpolated left and right eye images, and the depth map is calculated based on the left and right disparity maps to obtain depth information;

[0014] 3D point cloud reconstruction based on depth information and the coordinates of the left and right eye images after distortion correction;

[0015] The 3D point cloud is rotated by a predefined rotation angle;

[0016] The camera intrinsics are used to regenerate the coordinates of the left and right eye 2D images from the rotated 3D point cloud.

[0017] Perform a second interpolation on the regenerated left and right eye 2D image coordinates, and output the image.

[0018] This method replaces traditional distortion correction, epipolar correction, and interpolation in these two stages with a single composite coordinate mapping and single interpolation, eliminating the need for independent step-by-step execution and significantly reducing computation time. It also avoids dynamic scene distortion and inconsistent image frame deformation caused by step-by-step correction, particularly when applied to minimally invasive surgical navigation and endoscopic 3D imaging, improving the stability of the surgical field of view. Furthermore, this method performs only two interpolations, requiring only two interpolator units, saving DSP resources. This method can maintain epipolar geometric constraints and stereoscopic vision while rotating binocular stereo images, achieving efficient real-time processing of 3D images.

[0019] Optionally, the mean values ​​of the intrinsic parameters of the left and right cameras can be used as a common intrinsic parameter.

[0020] During epipolar correction, the coordinates of the translated and scaled left and right eye images are normalized using the common intrinsic parameters, and the normalized coordinates are rotated to align the angles of the left and right eye images.

[0021] Optionally, when rotating the normalized coordinates of the left and right eye images, the rotation matrix is ​​multiplied by the normalized coordinates of either of the left and right cameras; the transpose of the rotation matrix is ​​multiplied by the normalized coordinates of the other of the left and right cameras.

[0022] Epipolar correction eliminates the need for image interpolation, further avoiding processing delays and improving the real-time performance of the displayed image.

[0023] Optionally, during distortion correction, the normalized distance of each pixel to the optical center is calculated based on the left and right eye image coordinates after epipolar correction; distortion correction is performed on the left and right eye images based on this normalized distance, and then the left and right eye image coordinates are obtained by inverse normalization.

[0024] Distortion correction eliminates the need for image interpolation, further avoiding processing delays and improving the real-time performance of the displayed image.

[0025] Optionally, during interpolation, it is determined whether the Euclidean distance between the current interpolation point and the center point of the image is greater than a preset radius threshold. If so, the current interpolation point is determined to be an edge region, and nearest neighbor interpolation is performed; otherwise, the current interpolation point is determined to be a center region, and bilinear interpolation is performed.

[0026] This alternative solution employs a hybrid interpolation strategy, prioritizing the interpolation accuracy of the central field of view while reducing computational burden to ensure the clarity of key areas. This balances the accuracy and speed of interpolation, thereby improving the overall speed and accuracy of 3D image processing.

[0027] Optionally, when calculating the left and right disparity maps,

[0028] Divide the left and right eye images into a reference image and a target image. Traverse all pixels in the reference image and perform the following operations on each pixel:

[0029] Take a pixel in the reference image and the surrounding n*n range as the reference region; perform an iterative search on each pixel in the target image corresponding to the same row of the reference region to find the region in the target image with the smallest cost value compared to the reference point in the reference region as the target region. The reference region and the target region are the same size, and the minimum cost value is used as the disparity value of the center point of the current reference region.

[0030] All disparity values ​​constitute the left and right disparity maps.

[0031] Optionally, the cost value is calculated according to the following formula:

[0032]

[0033] Where n is the size of the reference region or search region, i and j are the region position subscripts, abs is the absolute value function, R refers to the reference region, and T is the target region.

[0034] Optionally, the depth map can be calculated based on the following formula:

[0035]

[0036] Where Zmap is the depth map, f is the camera focal length, B is the baseline length, and Emap is the disparity map.

[0037] Optionally, the 3D point cloud can be reconstructed according to the following formula:

[0038]

[0039] Where Z represents depth information, X1 and Y1 are the coordinates of the distorted planar image, and X... c ,Y c Z c For the calculated 3D point cloud coordinates, This indicates finding the inverse of the intrinsic parameter matrix of the left camera.

[0040] The present invention also proposes a 3D image processing system, including a data receiving unit, a data processing unit, and a display unit; the data receiving unit is used to acquire left and right eye image information, and the data receiving unit and the data processing unit are connected and communicate with each other; the data processing unit processes the left and right eye image information according to the above-mentioned low-computation-requirement 3D image processing method, and displays the processing results through the display unit.

[0041] The beneficial effects of this invention are:

[0042] This invention employs a preset coordinate mapping, using a coordinate backward calculation method to reverse-engineer the final display image back to the initial image coordinates. This replaces traditional distortion correction, epipolar correction, and interpolation in these two stages with a single composite coordinate mapping and single interpolation, saving the computational and caching requirements of interpolation, greatly reducing the amount of computation, significantly improving computational efficiency, meeting the real-time requirements of 3D image output, and avoiding the dynamic scene distortion problem caused by step-by-step correction, as well as the problem of inconsistent deformation between image frames.

[0043] This invention features low latency and supports high frame rate image processing.

[0044] This invention reduces the need for interpolation units, reducing the traditional four sets of interpolation units to only two sets of parallel interpolators, which significantly reduces hardware resource consumption, saves DSP computing resources, reduces chip area and power consumption, and optimizes system integration.

[0045] This invention features zero intermediate buffering throughout the entire calculation process, directly outputting a distortion-free image with coplanar row alignment, eliminating the need to store intermediate correction results, thus significantly saving buffer space, and is particularly suitable for endoscopes.

[0046] This invention eliminates the accumulation of step-by-step errors in traditional solutions and avoids image distortion caused by multiple independent interpolations in traditional methods, thereby improving image accuracy. In particular, it enhances the stability of surgery when applied to minimally invasive surgical navigation and endoscopic 3D imaging.

[0047] The hybrid interpolation strategy of this invention prioritizes optimization of the central field of view, performs high-precision interpolation on the central field of view, and uses neighboring point sampling for the edge regions. This reduces the computational burden while ensuring the clarity of key areas, making it particularly suitable for application in minimally invasive surgical navigation and endoscopic 3D imaging.

[0048] This invention maintains epipolar geometric constraints, preserves stereoscopic visual effects, and achieves efficient real-time processing while rotating binocular stereo images, ensuring the geometric accuracy of depth reconstruction and 3D display.

[0049] This invention has high clinical application value:

[0050] High surgical safety: When applied in surgery, especially in minimally invasive surgical navigation and endoscopic 3D imaging, the low latency and high precision of this invention can reduce surgeon error, lower the risk of minimally invasive surgery, and improve surgical safety.

[0051] High compatibility: Supports mainstream endoscope optical parameters, compatible with existing equipment, and can be upgraded without replacing hardware.

[0052] Low system cost: Reduced DSP and memory requirements of FPGAs enable high-performance 3D endoscope systems to run on more economical hardware platforms.

[0053] Additional aspects and advantages of the invention will be set forth in part in the description which follows, and in part will be obvious from the description, or may be learned by practice of the invention. Attached Figure Description

[0054] The above and / or additional aspects and advantages of the present invention will become apparent and readily understood from the description of the embodiments taken in conjunction with the following drawings, in which:

[0055] Figure 1 This is a flowchart illustrating the present invention;

[0056] Figure 2 These are the original left and right eye images;

[0057] Figure 3 These are the left and right eye images after the first interpolation;

[0058] Figure 4 This is the final image displayed. Detailed Implementation

[0059] Embodiments of the present invention are described in detail below. Examples of these embodiments are shown in the accompanying drawings, wherein the same or similar reference numerals denote the same or similar elements or elements having the same or similar functions throughout. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and should not be construed as limiting the present invention.

[0060] In the description of this invention, unless otherwise specified and limited, it should be noted that the terms "installation", "connection" and "linking" should be interpreted broadly. For example, they can refer to mechanical or electrical connections, or internal connections between two components. They can be direct connections or indirect connections through an intermediate medium. Those skilled in the art can understand the specific meaning of the above terms according to the specific circumstances.

[0061] Example 1

[0062] like Figure 1 As shown, this invention provides an embodiment of a low-computational-requirement 3D image processing method, primarily applied in 3D imaging systems. The method specifically includes the following steps:

[0063] Step 1: Obtain the acquired left and right eye images and display and calibration information of the actual desired image. In this embodiment, the acquired left and right eye images are as follows: Figure 2As shown. Image display and calibration information includes rotation angle, image translation coordinates, intrinsic parameters of the left and right cameras, rotation matrices of the left and right cameras, distortion parameters of the left and right cameras, actual imaging center point, actual imaging area size, and magnified / cropped area size information. In this embodiment, a hybrid mapping model in polar coordinates is constructed to integrate image display and calibration information into the hybrid mapping model. During construction, the center point of the polar coordinates is the image center point, which is a constant value, and the radius of the center area is a pre-input value.

[0064] Step 2: Determine the scaling parameters and image translation amount, construct a coordinate transformation matrix, and perform translation and scaling processing on the left and right eye images based on the coordinate transformation matrix to obtain the translated and scaled coordinates of the left and right eye images. Specifically,

[0065] The method for constructing the coordinate transformation matrix in this embodiment is as follows:

[0066]

[0067] in, Here, is the coordinate transformation matrix, and s is the scaling parameter. A represents the magnification or reduction factor of the image, and t x , t y This represents the amount of image translation.

[0068] The coordinates of the left and right eyes after translation and scaling using the coordinate transformation matrix are:

[0069]

[0070] Where x and y are the original coordinates of the image. Taking 4K resolution as an example, x is an integer from 1 to 3840 (inclusive), y is an integer from 1 to 2160 (inclusive), and x' and y' are the coordinates of the image after translation and scaling.

[0071] Step 3: Construct a common intrinsic parameter matrix for the left and right cameras, use the common intrinsic parameter matrix to perform epipolar correction on the coordinates of the translated and scaled left and right camera images, and then obtain the coordinates of the left and right camera images.

[0072] The steps for constructing the common intrinsic parameter matrix of the left and right cameras are as follows:

[0073] First, obtain the pre-calibrated intrinsic parameter matrices of the left and right cameras:

[0074]

[0075]

[0076] In the formula, For the intrinsic parameter matrix of the left eye camera, This is the intrinsic parameter matrix of the right eye camera. Let x be the focal length of the left-eye camera in the x-direction. Let be the focal length of the left-eye camera in the y-direction. The optical center point of the left eye camera in the x-direction. Let be the optical center point of the left eye camera in the y-direction. Let x be the focal length of the right eye camera in the x-direction. Let be the focal length of the right eye camera in the y-direction. The optical center point of the right eye camera in the x-direction. The point is the optical center of the right eye camera in the y-direction. All units are pixels.

[0077] Then, the mean values ​​of the intrinsic parameters of the left and right cameras are used as common intrinsic parameters, resulting in the following common intrinsic parameter matrix:

[0078]

[0079] During epipolar correction, common intrinsic parameters are used to transform the pixel coordinates of the left and right eye images into normalized coordinates. The normalized coordinates of the left eye image are:

[0080]

[0081]

[0082] The normalized coordinates of the right eye image are:

[0083]

[0084]

[0085] Among them, X 左 and Y 左 Let x' and y' be the coordinates of the left eye camera after translation and scaling in step two. 右 and Y 右 The coordinates of the right camera are x' and y' after translation and scaling in step two. The common optical center point in the x-direction. The common optical center point in the y-direction. Let be the common focal length in the x-direction. Let be the common focal length in the y-direction.

[0086] Then, the normalized coordinates of the left and right eye images are rotated to align their angles. Different rotation strategies are used when rotating the normalized coordinates of the left and right eye images: using a rotation matrix multiplied by the normalized coordinates of either camera; or using the transpose of the rotation matrix multiplied by the normalized coordinates of the other camera. In this embodiment, the rotation matrix is ​​multiplied by the normalized coordinates of the left eye image:

[0087] Left:

[0088] Multiply the normalized coordinates of the right eye image by the transpose of the rotation matrix:

[0089] right:

[0090] Then divide the normalized coordinates of the rotated left and right eye images by the third term:

[0091] Left:

[0092] right:

[0093] in, This is a rotation matrix, which is one type of calibration information from step one. , , The coordinates of the rotated left eye image. , , The coordinates of the rotated right eye image. , The coordinates of the aligned left eye image. , The coordinates of the right eye image after alignment.

[0094] Step 4: Perform distortion correction on the coordinates of the left and right eye images after epipolar correction. Details are as follows:

[0095] First, based on the epipolar-corrected coordinates of the left and right eye images, calculate the normalized distance of each pixel to the optical center. For the left eye image, the normalized distance of each pixel to the optical center is:

[0096] ;

[0097] For the right eye image, the normalized distance of each pixel to the optical center is:

[0098]

[0099] Then, distortion correction is performed on the left and right eye images.

[0100] For the left eye image, its distortion-corrected coordinates are:

[0101]

[0102]

[0103] in, , , These are the radial distortion parameters of the left eye image. , These are the tangential distortion parameters of the left eye image.

[0104] For the right eye image, its distortion-corrected coordinates are:

[0105]

[0106]

[0107] in, , , These are the radial distortion parameters of the right eye image. , These are the tangential distortion parameters of the right eye image.

[0108] Then, the coordinates of the distortion-corrected images are inversely normalized to obtain the coordinates of the left and right eyes:

[0109] For the left eye image, the inverse normalized coordinates are:

[0110]

[0111]

[0112] For the right eye image, the inverse normalized coordinates are:

[0113]

[0114]

[0115] Step 5: Perform the first interpolation on the coordinates of the left and right eye images after epipolar correction and distortion correction to obtain displayable left and right eye images, as shown in the image. Figure 3 As shown.

[0116] Interpolation methods can be one or more of the following: bilinear interpolation, nearest neighbor interpolation, cubic spline interpolation, and cubic interpolation. This embodiment uses a hybrid interpolation method to balance interpolation accuracy and speed. The same interpolation method is used for both the left and right eye images, as detailed below:

[0117] First, determine the center region: calculate the distance from the current interpolation point (X1, Y1) to the image center (x1, Y1). c , y c (Taking 4K resolution as an example, x) c For 1920, y x The Euclidean distance d (where d is 1080) is:

[0118]

[0119] The Euclidean distance *d* from the current interpolation point to the image center is compared with a preset radius threshold *Rthreshold*. The preset radius threshold *Rthreshold* can be determined experimentally and ranges from 0% to 50% of the diagonal length in the image. The diagonal length is the diagonal length of the face image in pixel coordinates; for example, the diagonal length of a 4K resolution image is... 4406, at this time the preset radius threshold Rthreshold can be 0~2203, usually 2203.

[0120] If d ≤ preset radius threshold Rthreshold, then the current interpolation point is determined to be the central region, and bilinear interpolation is performed on this point.

[0121] If d > the preset radius threshold Rthreshold, then the current interpolation point is determined to be an edge region, and the nearest neighbor interpolation method is used to interpolate the point.

[0122] Then interpolation is performed. When performing bilinear interpolation on the central region, the specific method is as follows: obtain the four adjacent integer pixels based on the coordinates of the point where bilinear interpolation is needed.

[0123]

[0124]

[0125]

[0126]

[0127] In this context, for the current point requiring dual-line interpolation, Q11 is its upper left integer pixel, Q12 is its upper right integer pixel, Q21 is its lower left integer pixel, Q22 is its lower right integer pixel, floor is the floor function, ceil is the floor function, I is the original image pixel value, and X1 and Y1 are the horizontal and vertical coordinates of the current point to be interpolated.

[0128] Then calculate the weighting coefficients of the current interpolation point in the horizontal and vertical directions. and

[0129]

[0130]

[0131] Then, a bilinear weighted calculation is performed:

[0132]

[0133] In the above formula, x and y are the original image coordinates before coordinate transformation, and P is the interpolated pixel value. This refers to the pixel value after interpolation of the original image coordinates x, y. This pixel value is then used to interpolate the current point that needs interpolation.

[0134] When performing nearest neighbor interpolation on edge regions, the specific method is as follows:

[0135] The original image coordinates (X1, Y1) of the interpolation point that needs nearest neighbor interpolation are rounded to the nearest integer:

[0136]

[0137]

[0138] Then directly retrieve the corresponding pixel value: The pixel value is used to interpolate the current point that needs interpolation.

[0139] Step 6: Calculate the left and right disparity maps based on the interpolated left and right eye images, and calculate the depth map based on the left and right disparity maps to obtain depth information.

[0140] In this embodiment, the calculation of the left and right disparity maps uses a stereo matching algorithm. Specifically, it includes the following steps:

[0141] The left and right images are divided into a reference image and a target image. For example, the left view is the reference image and the right view is the target image. A pixel in the first row and first column of the reference image and the n*n area around this pixel are taken as the reference area R, where n is a positive integer and the value range is preferably, but not limited to, 0-1000.

[0142] Iteratively search for each pixel in the same row as the reference region R on the target image to find the closest region as the target region T. Specifically,

[0143] Define the search area, which is the image on the target map at the same location as the reference area.

[0144] In this embodiment, the closest pixel in the target image to the reference region R is evaluated by calculating the cost value. The lower the cost value, the closer the pixel is to the reference region R. The cost value is calculated according to the following formula:

[0145]

[0146] Where n is the size of the reference region or search region, i and j are the region position indices, and abs is the absolute value function.

[0147] Then increment the x-coordinate of the search area by one, and repeat the above steps until you obtain the cost value (E1 to Em) for each pixel in the same row as the reference area R on the target image. m is the maximum value of the search x-coordinate range; for example, in 4K resolution, m ranges from 0 to 3840.

[0148] Sort the values ​​from E1 to Em and take the minimum value as the disparity value of the current reference area center point.

[0149] Then, iterate through all pixels in the reference image, divide the reference region R, and calculate the disparity value corresponding to all pixels in the reference image. Finally, all disparity values ​​constitute the left and right disparity maps.

[0150] Calculate the depth map based on the disparity map:

[0151]

[0152] Where Zmap is the depth map, which contains the depth information corresponding to each pixel on the reference image, f is the camera focal length, B is the baseline length, and Emap is the disparity map.

[0153] Step 7: Reconstruct the 3D point cloud based on depth information and the distortion-corrected left and right eye images. In this embodiment, the 3D point cloud is reconstructed according to the following formula:

[0154]

[0155] Where Z represents depth information, X1 and Y1 are the coordinates of the distorted planar image, and X... c ,Y c Z c For the calculated 3D point cloud coordinates, This indicates finding the inverse of the intrinsic parameter matrix of the left camera.

[0156] Step 8: Rotate the 3D point cloud by a predefined rotation angle. Specifically,

[0157] Align the obtained 3D point cloud coordinates at a certain angle. The rotation, the resulting coordinate matrix is:

[0158]

[0159] in:

[0160]

[0161]

[0162] In the above formula, Let X be the defined rotation angle matrix. w Y w Z w These are the rotated 3D coordinates.

[0163] Step 9: Use camera intrinsics to regenerate the left and right eye 2D image coordinates from the rotated 3D point cloud. The regenerated left eye 2D image coordinate matrix is ​​as follows:

[0164]

[0165] The regenerated coordinate matrix of the right eye 2D image is:

[0166]

[0167] Where X2 and Y2 are the coordinates projected onto the 2D image.

[0168] Step 10: Perform a second interpolation on the regenerated left and right eye 2D image coordinates. The specific steps are the same as the first interpolation in Step 5.

[0169] Step 11: Output the image, as shown below. Figure 4 As shown.

[0170] Example 2

[0171] The present invention also proposes an embodiment of a 3D image processing system. This 3D image processing system includes a data receiving unit, a data processing unit, and a display unit; the data receiving unit is used to acquire left and right eye image information, and the data receiving unit and the data processing unit are connected and communicate with each other; the data processing unit processes the left and right eye image information according to the low-computational-requirement 3D image processing method described in Embodiment 1, and displays the processing results through the display unit.

[0172] In the description of this specification, references to terms such as "one embodiment," "some embodiments," "example," "specific example," or "some examples," etc., indicate that a specific feature, structure, material, or characteristic described in connection with that embodiment or example is included in at least one embodiment or example of the invention. In this specification, the illustrative expressions of the above terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in one or more embodiments or examples.

[0173] Although embodiments of the invention have been shown and described, those skilled in the art will understand that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the claims and their equivalents.

Claims

1. A low-computational-requirement 3D image processing method, characterized in that, Includes the following steps: Acquire the left and right eye images and display and calibrate the actual images to be acquired; Determine the scaling parameters and the image translation amount, construct a coordinate transformation matrix, and perform translation and scaling processing on the left and right eye images based on the coordinate transformation matrix to obtain the translation and scaling coordinates of the left and right eye images; Construct a common intrinsic parameter matrix for the left and right cameras. Use the common intrinsic parameter matrix to perform epipolar correction on the coordinates of the translated and scaled left and right camera images to obtain the coordinates of the left and right camera images. Perform distortion correction on the epipolar corrected coordinates of the left and right camera images. The coordinates of the left and right eye images after epipolar correction and distortion correction are interpolated for the first time to obtain displayable left and right eye images; The left and right disparity maps are calculated based on the interpolated left and right eye images, and the depth map is calculated based on the left and right disparity maps to obtain depth information; 3D point cloud reconstruction based on depth information and the coordinates of the left and right eye images after distortion correction; The 3D point cloud is rotated by a predefined rotation angle; The camera intrinsics are used to regenerate the coordinates of the left and right eye 2D images from the rotated 3D point cloud. Perform a second interpolation on the regenerated left and right eye 2D image coordinates, and output the image.

2. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, The mean values ​​of the intrinsic parameters of the left and right cameras are used as a common intrinsic parameter. During epipolar correction, the coordinates of the translated and scaled left and right eye images are normalized using the common intrinsic parameters, and the normalized coordinates are rotated to align the angles of the left and right eye images.

3. The low-computational-requirement 3D image processing method according to claim 2, characterized in that, When rotating the normalized coordinates of the left and right eye images, the rotation matrix is ​​multiplied by the normalized coordinates of either of the left and right cameras; the transpose of the rotation matrix is ​​multiplied by the normalized coordinates of the other of the left and right cameras.

4. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, During distortion correction, the normalized distance of each pixel to the optical center is calculated based on the left and right eye image coordinates after epipolar correction. The distortion correction is then performed on the left and right eye images based on this normalized distance, and then the left and right eye image coordinates are obtained by inverse normalization.

5. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, During interpolation, it is determined whether the Euclidean distance between the current interpolation point and the center point of the image is greater than a preset radius threshold. If so, the current interpolation point is determined to be an edge region, and nearest neighbor interpolation is performed; otherwise, the current interpolation point is determined to be a center region, and bilinear interpolation is performed.

6. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, When calculating the left and right disparity maps, Divide the left and right eye images into a reference image and a target image. Traverse all pixels in the reference image and perform the following operations on each pixel: Take a pixel in the reference image and the surrounding n*n range as the reference region; perform an iterative search on each pixel in the target image corresponding to the same row of the reference region to find the region in the target image with the smallest cost value compared to the reference point in the reference region as the target region. The reference region and the target region are the same size, and the minimum cost value is used as the disparity value of the center point of the current reference region. All disparity values ​​constitute the left and right disparity maps.

7. The low-computational-requirement 3D image processing method according to claim 6, characterized in that, The cost is calculated using the following formula: Where n is the size of the reference region or search region, i and j are the region position subscripts, abs is the absolute value function, R refers to the reference region, and T is the target region.

8. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, The depth map is calculated based on the following formula: Where Zmap is the depth map, f is the camera focal length, B is the baseline length, and Emap is the disparity map.

9. The low-computational-requirement 3D image processing method according to claim 1, characterized in that, Reconstruct the 3D point cloud using the following formula: Where Z represents depth information, X1 and Y1 are the distortion-corrected planar image coordinates, and X... c ,Y c Z c For the calculated 3D point cloud coordinates, This indicates finding the inverse of the intrinsic parameter matrix of the left camera.

10. A 3D image processing system, characterized in that, It includes a data receiving unit, a data processing unit, and a display unit; the data receiving unit is used to acquire left and right eye image information, and the data receiving unit and the data processing unit are connected and communicate with each other; the data processing unit processes the left and right eye image information according to any one of claims 1-9 using the low-computational-requirement 3D image processing method, and displays the processing result through the display unit.

Images