Image distortion correction method and device, endoscope equipment and storage medium

Abstract

The invention discloses an image distortion correction method and device, endoscope equipment and a storage medium, and relates to the technical field of endoscope imaging, and the method comprises the steps: obtaining a camera internal reference matrix, a distortion coefficient and an original distortion image of an endoscope imaging device; identifying an effective view field area in the original distorted image, and adaptively generating a corresponding view field mask according to the effective view field area; establishing a virtual spherical space according to the optical center of the camera, and establishing a pixel reverse mapping relation between the corrected image and the original distorted image; obtaining source pixel point coordinates of the corrected image relative to the original distorted image according to the camera internal reference matrix, the distortion coefficient and the pixel reverse mapping relation; pixel sampling is carried out in the original distortion image according to the source pixel point coordinates, a pixel sampling result is obtained, and a corrected image is generated in combination with the view field mask. The wide-angle field of view can be reserved while geometric distortion is eliminated, and the technical problem that the field of view is lost in the endoscope image distortion correction process is solved.

Description

Technical Field

[0001] This application relates to the field of endoscopic imaging technology, and in particular to an image distortion correction method, device, endoscopic equipment, and storage medium. Background Technology

[0002] In modern minimally invasive surgery, the endoscope is the surgeon's "eye." To obtain the widest possible field of view and reduce the number of times the endoscope needs to be moved, wide-angle lenses have become the mainstream design. However, an inherent physical characteristic of wide-angle lens design is that it introduces severe radial distortion (i.e., "barrel distortion"), causing the image to be severely compressed in the peripheral regions. This distortion is not simply an aesthetic issue, but directly leads to a series of serious clinical problems. For example, distortion causes straight instruments (such as puncture needles and surgical scissors) to appear curved on the screen, making it difficult for surgeons to determine their true orientation. Simultaneously, the shape and size of lesions (such as polyps or tumors) can be misperceived due to their different positions on the screen, severely interfering with diagnosis and surgical planning. Furthermore, distortion disrupts the perspective of the image, causing serious deviations in the surgeon's judgment of tissue depth and the distance between the instrument tip and key anatomical structures (such as blood vessels and nerves), greatly increasing surgical risks.

[0003] Therefore, how to correct the image distortion of endoscopes is a problem that urgently needs to be solved. Summary of the Invention

[0004] The main objective of this application is to provide an image distortion correction method, apparatus, endoscopic device, and storage medium, aiming to solve the technical problem of field of view loss during endoscopic image distortion correction, and to achieve field of view preservation while eliminating geometric distortion.

[0005] To achieve the above objectives, this application provides an image distortion correction method, the image distortion correction method comprising:

[0006] Acquire the camera intrinsic parameter matrix, distortion coefficients, and raw distorted image of the endoscopic imaging device;

[0007] Identify the effective field of view region in the original distorted image, and generate a corresponding field of view mask based on the effective field of view region;

[0008] A virtual spherical space is established based on the camera's optical center, and a pixel inverse mapping relationship between the corrected image and the original distorted image is established based on the virtual spherical space.

[0009] Based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship, the source pixel coordinates of the corrected image relative to the original distorted image are obtained;

[0010] Pixel sampling is performed on the original distorted image based on the source pixel coordinates to obtain pixel sampling results, and a corrected image is generated based on the pixel sampling results and the field mask.

[0011] In one embodiment, the step of identifying the effective field of view region in the original distorted image and generating a corresponding field of view mask based on the effective field of view region includes:

[0012] The original distorted image is binarized, and the bright areas of the original distorted image are extracted.

[0013] Perform contour detection on the bright area, calculate the area enclosed by each contour, and obtain multiple contour-enclosed areas;

[0014] Compare the areas enclosed by the contours and take the contour with the largest area as the effective field of view contour;

[0015] The effective field of view contour is optimized to generate a corresponding field of view mask.

[0016] In one embodiment, the step of establishing a virtual spherical space based on the camera optical center and establishing a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space includes:

[0017] Establish a virtual spherical space with the camera's optical center as the origin;

[0018] Generate the target pixel coordinate matrix of the corrected image, and map the target pixel coordinate matrix to the virtual spherical space to obtain the target incident angle and azimuth angle corresponding to each pixel in the target pixel coordinate matrix;

[0019] The source incident angle is calculated by mapping the target incident angle according to a preset nonlinear compression function;

[0020] Based on the source incident angle and azimuth angle, a pixel-wise inverse mapping relationship is established between the corrected image and the original distorted image.

[0021] In one embodiment, the step of obtaining the source pixel coordinates of the corrected image relative to the original distorted image based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship includes:

[0022] The ideal coordinates are obtained by projecting the source incident angle and the azimuth angle onto the coordinates of the ideal normalized plane;

[0023] Based on the distortion coefficient and the ideal coordinates, radial distortion correction and tangential distortion correction are performed on the ideal coordinates to obtain distortion-normalized coordinates;

[0024] Based on the camera intrinsic parameter matrix, the distortion normalized coordinates are transformed to obtain the source pixel coordinates of the corrected image relative to the original distorted image.

[0025] In one embodiment, the step of performing pixel sampling on the original distorted image based on the source pixel coordinates to obtain a pixel sampling result, and generating a corrected image based on the pixel sampling result and the field mask, includes:

[0026] Traverse each target pixel in the corrected image;

[0027] Based on the pixel inverse mapping relationship, calculate the coordinates of the source pixel corresponding to the target pixel;

[0028] Determine whether the coordinates of the source pixel corresponding to the target pixel are located within the effective area defined by the field mask;

[0029] If the coordinates of the source pixel corresponding to the target pixel are located within the effective area defined by the field mask, then the pixel value of the source pixel coordinates is calculated by an interpolation algorithm.

[0030] If the coordinates of the source pixel corresponding to the target pixel are outside the effective area defined by the field mask, then the pixel value of the source pixel coordinates is set to the preset background color.

[0031] A corrected image is generated based on the pixel values ​​at the coordinates of the source pixel.

[0032] In one embodiment, the step of establishing a virtual spherical space based on the camera optical center and establishing a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space includes:

[0033] A virtual unit spherical space is used as the specific implementation of the virtual spherical space. The pixel coordinates of the corrected image are converted into normalized spatial coordinates and mapped to the virtual unit sphere. The specific calculation model is as follows:

[0034]

[0035] The source incident angle is calculated based on the nonlinear compression function, and the specific calculation model is as follows:

[0036]

[0037] Where u and v are the target coordinate matrices of the corrected image, c_x and c_y are the coordinates of the center point of the camera optical axis on the image plane, x and y are the coordinates on the normalized spatial imaging plane, r is the distance from the coordinate point on the normalized spatial imaging plane to the center of the camera optical axis, f_{\text{new} is the virtual target focal length; \theta_{dst} is the target incident angle, \phi is the azimuth angle, \theta_{src} is the source incident angle, C_0 is the preset linear coefficient, and C_1 is the preset compression coefficient.

[0038] In one embodiment, in the step of obtaining the source pixel coordinates of the corrected image relative to the original distorted image based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship:

[0039] The source incident angle is projected back onto the ideal normalized plane. The specific calculation model is as follows:

[0040]

[0041] Where x_{ideal} and y_{ideal} are ideal coordinates, r_{ideal} is the ideal radial distance, φ is the azimuth angle, and theta_{src} is the source incident angle;

[0042] Based on the distortion coefficients and the ideal coordinates, radial distortion correction and tangential distortion correction are performed on the ideal coordinates to obtain distortion-normalized coordinates. The specific calculation model is as follows:

[0043]

[0044] Where r^2 is the square of the radial radius, Radial is the radial distortion factor, k_1, k_2, k_3 are the radial distortion coefficients, p_1, p_2 are the tangential distortion coefficients, and x_{distorted} and y_{distorted} are the distortion-normalized coordinates;

[0045] The distortion-normalized coordinates are converted into source pixel coordinates. The specific calculation model is as follows:

[0046]

[0047] in, and f_x and f_y are the source pixel coordinates, f_x and f_y are the focal lengths of the camera along the x and y axes, respectively, in pixels, and c_x and c_y are the coordinates of the center point of the camera's optical axis on the image plane.

[0048] Furthermore, to achieve the above objectives, this application also provides an image distortion correction device, the image distortion correction device comprising:

[0049] The acquisition unit is used to acquire the camera intrinsic parameter matrix, distortion coefficients, and original distorted image of the endoscope imaging device;

[0050] A field-of-view mask generation unit is used to identify the effective field-of-view region in the original distorted image and generate a corresponding field-of-view mask based on the effective field-of-view region.

[0051] The mapping relationship determination unit is used to establish a virtual spherical space based on the camera optical center, and to establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space;

[0052] The source pixel coordinate calculation unit is used to obtain the source pixel coordinates of the corrected image relative to the original distorted image based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship.

[0053] The corrected image generation unit is used to perform pixel sampling on the original distorted image according to the coordinates of the source pixel points, obtain the pixel sampling result, and generate a corrected image according to the pixel sampling result and the field mask.

[0054] The field-of-view mask generation unit is further configured to perform binarization processing on the original distorted image and extract the bright areas of the original distorted image; perform contour detection on the bright areas, calculate the area enclosed by each contour to obtain multiple contour-enclosed areas; compare the contour-enclosed areas and take the contour with the largest area as the effective field-of-view contour; perform contour optimization on the effective field-of-view contour to generate the corresponding field-of-view mask.

[0055] The mapping relationship determination unit is further configured to establish a virtual spherical space with the camera optical center as the origin; generate the target pixel coordinate matrix of the corrected image, and map the target pixel coordinate matrix to the virtual spherical space to obtain the target incident angle and azimuth angle corresponding to each pixel in the target pixel coordinate matrix; map the target incident angle according to a preset nonlinear compression function to calculate the source incident angle; and establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the source incident angle and azimuth angle.

[0056] The source pixel coordinate calculation unit is further configured to project the source incident angle and the azimuth angle onto the coordinates of the ideal normalized plane to obtain ideal coordinates; perform radial distortion correction and tangential distortion correction on the ideal coordinates according to the distortion coefficient and the ideal coordinates to obtain distortion normalized coordinates; and transform the distortion normalized coordinates according to the camera intrinsic parameter matrix to obtain the source pixel coordinates of the corrected image relative to the original distorted image.

[0057] The corrected image generation unit is further configured to traverse each target pixel in the corrected image; calculate the source pixel coordinates corresponding to the target pixel according to the pixel inverse mapping relationship; determine whether the source pixel coordinates corresponding to the target pixel are located within the effective area defined by the field mask; if the source pixel coordinates corresponding to the target pixel are located within the effective area defined by the field mask, then the pixel value corresponding to the source pixel coordinates is acquired in the original distorted image by an interpolation algorithm; if the source pixel coordinates corresponding to the target pixel are located outside the effective area defined by the field mask, then the pixel value of the source pixel coordinates is set to a preset background color; and generate a corrected image based on the pixel value of the source pixel coordinates.

[0058] The mapping relationship determination unit is further configured to use a virtual unit spherical space as the specific implementation of the virtual spherical space, convert the pixel coordinates of the corrected image into normalized spatial coordinates and map them to the virtual unit sphere. The specific calculation model is as follows:

[0059]

[0060] The source incident angle is calculated based on the nonlinear compression function, and the specific calculation model is as follows:

[0061]

[0062] Where u and v are the target coordinate matrices of the corrected image, c_x and c_y are the coordinates of the center point of the camera optical axis on the image plane, x and y are the coordinates on the normalized spatial imaging plane, r is the distance from the coordinate point on the normalized spatial imaging plane to the center of the camera optical axis, f_{\text{new} is the virtual target focal length; \theta_{dst} is the target incident angle, \phi is the azimuth angle, \theta_{src} is the source incident angle, C_0 is the preset linear coefficient, and C_1 is the preset compression coefficient.

[0063] The source pixel coordinate calculation unit is also used to project the source incident angle back to the ideal normalized plane. The specific calculation model is as follows:

[0064]

[0065] Where x_{ideal} and y_{ideal} are ideal coordinates, r_{ideal} is the ideal radial distance, φ is the azimuth angle, and theta_{src} is the source incident angle;

[0066] Based on the distortion coefficients and the ideal coordinates, radial distortion correction and tangential distortion correction are performed on the ideal coordinates to obtain distortion-normalized coordinates. The specific calculation model is as follows:

[0067]

[0068] Where r^2 is the square of the radial radius, Radial is the radial distortion factor, k_1, k_2, k_3 are the radial distortion coefficients, p_1, p_2 are the tangential distortion coefficients, and x_{distorted} and y_{distorted} are the distortion-normalized coordinates;

[0069] The distortion-normalized coordinates are converted into source pixel coordinates. The specific calculation model is as follows:

[0070]

[0071] in, and f_x and f_y are the source pixel coordinates, f_x and f_y are the focal lengths of the camera along the x and y axes, respectively, in pixels, and c_x and c_y are the coordinates of the center point of the camera's optical axis on the image plane.

[0072] In addition, to achieve the above objectives, this application also proposes an endoscope device, the device comprising: a memory, a processor, and a computer program stored in the memory and executable on the processor, the computer program being configured to implement the steps of the image distortion correction method described above.

[0073] In addition, to achieve the above objectives, this application also provides a storage medium, which is a computer-readable storage medium, on which a program for implementing an image distortion correction method is stored. The program for implementing the image distortion correction method is executed by a processor to implement the steps of the image distortion correction method as described above.

[0074] This application provides an image distortion correction method that effectively solves the problems of image distortion caused by lens optical characteristics in endoscopic imaging and field-of-view loss caused by traditional correction methods. By obtaining the camera intrinsic parameter matrix and distortion coefficients, it ensures that the correction process can be strictly performed according to the physical characteristics of the imaging system. By identifying the effective field-of-view region of the original distorted image and generating a field-of-view mask, it is possible to accurately distinguish between the actual imaging region and the invalid edge region, avoiding misprocessing of informationless regions during the correction process. Furthermore, based on a virtual spherical space and deriving the pixel inverse mapping relationship, it achieves accurate coordinate inverse calculation from the ideal corrected image space to the actual distorted image space, ensuring the accuracy of geometric transformation. Combining the distortion coefficients and the inverse mapping relationship, the precise source coordinates of each pixel in the corrected image in the original image are calculated, solving the problem of nonlinear pixel position offset caused by distortion. Finally, pixel sampling is performed on the original image based on the coordinates, and the final corrected image is generated by combining the field mask. This restores the natural geometry of the image, eliminates barrel or pincushion distortion, and ensures the edge integrity and visual consistency of the output image through masking. This significantly improves the reliability and usability of endoscopic images in clinical observation, measurement, and computer-aided diagnosis. Attached Figure Description

[0075] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application.

[0076] To more clearly illustrate the technical solutions in the embodiments of this application or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0077] Figure 1 This is a schematic diagram of the original distortion image of the endoscope in an embodiment of this application;

[0078] Figure 2 This is a schematic diagram of the corrected image generated by the standard geometric correction method in the embodiments of this application;

[0079] Figure 3 This is a schematic diagram of the corrected image generated by the image distortion correction method in the embodiments of this application;

[0080] Figure 4 This is a flowchart illustrating an embodiment of the image distortion correction method in this application.

[0081] Figure 5 This is a schematic diagram of the module structure for image distortion correction in an embodiment of this application;

[0082] Figure 6 This is a schematic diagram of the device structure of the hardware operating environment involved in the image distortion correction method in the embodiments of this application.

[0083] The purpose, features, and advantages of this application will be further explained in conjunction with the embodiments and with reference to the accompanying drawings. Detailed Implementation

[0084] It should be understood that the specific embodiments described herein are only used to explain the technical solutions of this application and are not intended to limit this application.

[0085] To better understand the technical solution of this application, a detailed description will be provided below in conjunction with the accompanying drawings and specific implementation methods.

[0086] In modern minimally invasive surgery, endoscopic distortion is not merely a simple aesthetic issue, but directly leads to a series of serious clinical problems. For example, distortion causes straight instruments (such as puncture needles and surgical scissors) to appear curved on the screen, making it difficult for surgeons to determine their true orientation. Simultaneously, the shape and size of lesions (such as polyps or tumors) can be misperceived due to their different positions on the screen, severely interfering with diagnosis and surgical planning. Furthermore, distortion disrupts perspective, causing significant deviations in the surgeon's judgment of tissue depth and the distance between instrument tips and critical anatomical structures (such as blood vessels and nerves), greatly increasing surgical risks. Currently, to preserve the wide-angle field of view of the endoscope to 100%, many endoscopic imaging systems choose not to perform any correction, referring to… Figure 1 This method directly presents the raw "fisheye" image to the doctor. While it preserves the field of view, it completely fails to address the aforementioned clinical problems. This can lead to misjudgments by the doctor during surgery regarding the shape, size, and depth of the imaging target. It also reduces the geometric accuracy of the endoscopic image. Furthermore, referring to... Figure 2 Existing technologies have employed standard geometric correction methods (such as pinhole-based distortion correction) to stretch barrel-shaped distorted images outwards to restore geometric linearity. This involves obtaining the camera's intrinsic parameter matrix and distortion coefficients through offline or dynamic calibration. Subsequently, these parameters are used to remap pixels in the real-time image, straightening the curved lines. This process inevitably results in the corrected effective image area being larger than the original circular field of view, with image edges expanding beyond the original boundaries. To display this on a screen, the expanded portion must be cropped, directly leading to the loss of the endoscope's most valuable edge field of view (FOV), creating a visual blind spot and contradicting the design intent of a wide-angle lens. Therefore, how to correct image distortion in endoscopes is a pressing issue that needs to be addressed.

[0087] This application effectively solves the image distortion problem caused by lens optical characteristics in endoscopic imaging. By obtaining the camera intrinsic parameter matrix and distortion coefficients, it ensures that the correction process can be strictly performed according to the physical characteristics of the imaging system. By identifying the effective field of view region of the original distorted image and generating a field of view mask, it is possible to accurately distinguish between the actual imaging region and the invalid edge region, avoiding misprocessing of informationless regions during the correction process. Furthermore, based on a virtual spherical space and deriving the pixel inverse mapping relationship, it achieves accurate coordinate inverse calculation from the ideal corrected image space to the actual distorted image space, ensuring the accuracy of geometric transformation. Combining the distortion coefficients and the inverse mapping relationship, the precise source coordinates of each pixel in the corrected image in the original image are calculated, solving the problem of nonlinear pixel position offset caused by distortion. Finally, pixel sampling is performed in the original image based on these coordinates, and the final corrected image is generated by combining the field of view mask, restoring the natural geometric shape of the image, eliminating barrel or pincushion distortion, and ensuring the edge integrity and visual consistency of the output image through mask processing, significantly improving the reliability and usability of endoscopic images in clinical observation, measurement, and computer-aided diagnosis.

[0088] This application provides an image distortion correction method. Please refer to the following embodiments. Figure 4 The image distortion correction method includes steps S10 to S50:

[0089] Step S10: Obtain the camera intrinsic parameter matrix, distortion coefficients, and original distorted image of the endoscopic imaging device;

[0090] It should be noted that the raw distorted image is an image directly acquired by the endoscopic imaging device without any geometric processing. The camera intrinsic parameter matrix K describes the ideal geometric relationship in which the camera projects points in three-dimensional space onto the two-dimensional image plane. It mainly contains information such as focal length (f_x, f_y) and principal point coordinates (c_x, c_x), for example:

[0091]

[0092] Where f_x and f_y are the camera's focal lengths (in pixels); c_x and c_y are the coordinates of the center point (principal point) of the camera's optical axis on the image plane.

[0093] The distortion factor D describes the degree of physical distortion caused by the lens manufacturing process:

[0094]

[0095] Where k_1, k_2, and k_3 are radial distortion coefficients (mainly causing barrel distortion). p_1 and p_2 are tangential distortion coefficients (mainly caused by non-parallel lens mounting).

[0096] The camera intrinsic parameter matrix and distortion coefficients can be directly read from the pre-stored configuration file of the endoscope model, or they can be calculated by acquiring calibration plate images captured by the endoscope using the Zhang Zhengyou calibration method or other calibration techniques.

[0097] Step S20: Identify the effective field of view region in the original distorted image, and generate a corresponding field of view mask based on the effective field of view region;

[0098] It should be noted that the effective field of view refers to the portion of the original distorted image that actually captures light signals by the endoscope lens. It typically appears as a circular or nearly circular bright area, contrasting with the surrounding black, informationless background. The field of view mask is a binary image of the same size as the original image, where pixels belonging to the effective field of view are marked as 1 (or 255, representing the foreground), and background areas are marked as 0 (representing the background). By generating the field of view mask, the actual field of view shape of the current frame can be dynamically adapted, ensuring that subsequent pixel mapping and sampling operations are only performed on the effective area. This avoids erroneous interpolation calculations for informationless backgrounds, providing an accurate contour template for the corrected image and improving processing efficiency and accuracy.

[0099] Specifically, the original distorted image can be converted to a grayscale image and Gaussian filtering can be applied for noise reduction. Then, binarization is performed using a fixed or adaptive threshold (such as the Otsu algorithm) to separate bright areas (foreground) from the dark background. Next, a contour-finding algorithm (such as OpenCV's findContours) is used to detect all contours in the binary image, and the area of ​​each contour is calculated. Finally, the contour with the largest area is selected as the effective field-of-view contour, and a binary mask image with an inner fill of 1 and an outer fill of 0 is generated based on this contour. For cases with rough boundaries, polygon approximation or convex hull calculation can be performed on the contours to smooth the edges.

[0100] Step S30: Establish a virtual spherical space based on the camera optical center, and establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space;

[0101] Specifically, the virtual spherical space defines the rules for mapping pixels in the target corrected image to three-dimensional spatial angles, and then back to the original spatial angles via a nonlinear function. The pixel inverse mapping relationship is a mathematical relationship that derives the physical incident angle in the original distorted image for each pixel (u, v) in the distortion-free corrected image (target image) through the virtual spherical space. By constructing the virtual spherical space and deriving the inverse mapping relationship, for any pixel in the corrected image, its position in the original distorted image can be traced back through this relationship, thus ensuring that the corrected image is continuous and complete.

[0102] Step S40: Based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship, obtain the source pixel coordinates of the corrected image relative to the original distorted image;

[0103] It should be noted that the source pixel coordinates refer to the corresponding pixel coordinates on the original distorted image calculated from a point (u, v) on the corrected image based on the inverse mapping relationship and the lens distortion model. , This coordinate is usually not an integer, indicating that the color value needs to be obtained through interpolation in the original distorted image. By combining the inverse mapping relationship (ideal model) with the distortion coefficients (true deviation) for calculation, the inverse process of the imaging process of a real lens is accurately simulated. For each point on the corrected image, a theoretically unique corresponding sampling point in the original distorted image can be found, thus ensuring that the correction operation can accurately counteract the geometric distortion introduced by the lens and restore the perspective and linear features of the image.

[0104] Step S50: Perform pixel sampling on the original distorted image according to the source pixel coordinates to obtain pixel sampling results, and generate a corrected image based on the pixel sampling results and the field mask.

[0105] It should be noted that, referring to Figure 3 This is a schematic diagram of the corrected image generated by the image distortion correction method in this embodiment of the application. Source pixel coordinates ( , The color value is usually not an integer and needs to be obtained by interpolation from the original distorted image. Common algorithms include bilinear interpolation, nearest neighbor interpolation, or bicubic interpolation. The pixel sampling result refers to the sampled color value. The field mask is used to determine whether the sampling point is valid and ultimately constrain the shape of the output image. By sampling pixels according to the precisely calculated source coordinates, the color information in the original distorted image is redistributed to the target position of the corrected image according to the correct geometric relationship, thus intuitively showing the image after the distortion has been eliminated. Combined with the field mask, it ensures that the field of view of the final generated corrected image is consistent with that of the original image. For areas marked as valid by the mask, their corrected content is preserved; for areas outside the mask (the black background of the original image), they are filled with the specified background color (such as black), realizing the transformation from the original distorted input to a geometrically correct and field-of-view complete corrected output.

[0106] This embodiment establishes a precise physical model foundation for correction by acquiring the camera's intrinsic distortion coefficients; it dynamically defines the effective range of image processing and output by identifying the effective field of view and generating a mask, ensuring the integrity of the field of view; it establishes a mathematical path from the ideal image space back to the original image space by constructing a virtual spherical space and an inverse mapping relationship; it accurately simulates the reverse process of lens imaging by calculating the source pixel coordinates by combining camera intrinsic parameters, distortion coefficients, and mapping relationships, thereby eliminating geometric distortion; finally, it generates an image by sampling pixels based on the source coordinates and combining them with the field of view mask, redistributing color information according to the correct geometric relationship and strictly constraining it within the original field of view boundary. This effectively corrects the geometric deformation caused by the lens while completely preserving the endoscope's original wide-angle field of view, overcoming the technical contradiction of the traditional method where "geometric accuracy" and "field of view integrity" are mutually exclusive.

[0107] In one feasible implementation, step S20 may include steps S21 to S24:

[0108] Step S21: Binarize the original distorted image and extract the bright areas of the original distorted image;

[0109] Step S22: Perform contour detection on the bright area, calculate the area enclosed by each contour, and obtain multiple contour-enclosed areas.

[0110] Step S23: Compare the areas enclosed by the contours and take the contour with the largest area as the effective field of view contour;

[0111] Step S24: Optimize the effective field of view contour to generate a corresponding field of view mask.

[0112] It should be noted that binarization transforms the grayscale value of each pixel in a grayscale or color image into either 0 (black) or 255 (white) based on a threshold, thus simplifying the image to black and white for easier subsequent analysis. When extracting bright areas, pixel regions with brightness above the threshold (i.e., circular light spots in endoscopic imaging) are separated and designated as the foreground (white), while dark background regions are designated as the background (black).

[0113] Specifically, binarization can employ a globally fixed threshold method, such as setting an empirical threshold (e.g., 30-50), setting pixels with grayscale values ​​greater than this threshold to 255 (white), and the rest to 0 (black); alternatively, a more adaptive Otsu's algorithm can be used to automatically calculate an optimal global threshold. Furthermore, to reduce noise interference, Gaussian blur or other filtering operations can be applied to the grayscale image before binarization.

[0114] It's important to note that contour detection extracts object boundary curves (contours) from a binary image, identifying the sequence of boundary points for all white connected regions in the image. The total number of pixels contained within a closed contour curve is the area enclosed by the contour. By performing contour detection on bright areas, all independent white regions in the image can be enumerated (potentially including the main field of view, as well as smaller regions caused by reflections and noise). The area of ​​each contour is calculated, and the contour with the largest area is selected to determine the effective field of view contour, representing the actual main imaging area of ​​the endoscope.

[0115] Specifically, contour detection can use functions such as findContours from the OpenCV library to find all contours in a binary image. For each contour in the list, the system calculates its area, for example, using the contourArea function.

[0116] It should be noted that contour optimization of the selected effective field of view contour is an operation that smooths its boundaries, fills small gaps, or makes it more regular, such as polygon approximation, convex hull calculation, or morphological closing operation. Field of view mask refers to creating a binary mask image by filling the area inside the contour with white (255) on a blank (completely black) image based on the optimized contour.

[0117] This embodiment provides an image distortion correction method. By binarizing the original image, it efficiently separates the bright effective field of view from the dark background. Contour detection and area calculation are performed on the bright areas, and the contour with the largest area is selected as the effective field of view. Utilizing the inherent characteristic that the endoscopic field of view is typically the largest connected region, adaptive field of view localization is achieved. Finally, contour optimization and mask generation ensure the smoothness and accuracy of the field of view boundary, and generate a precise binary template for constraining subsequent processing. This method can dynamically adapt to different imaging conditions and field of view shapes, providing a reliable spatial range definition for subsequent distortion correction.

[0118] In one feasible implementation, step S30 may include steps S31 to S34:

[0119] Step S31: Establish a virtual spherical space with the camera's optical center as the origin;

[0120] Step S32: Generate the target pixel coordinate matrix of the corrected image, and map the target pixel coordinate matrix to the virtual spherical space to obtain the target incident angle and azimuth angle corresponding to each pixel in the target pixel coordinate matrix;

[0121] Step S33: Map the target incident angle according to a preset nonlinear compression function to calculate the source incident angle;

[0122] Step S34: Based on the source incident angle and azimuth angle, establish a pixel inverse mapping relationship between the corrected image and the original distorted image.

[0123] It should be noted that the virtual spherical space can be used to describe the mathematical model of an ideal (distortion-free) imaging process. By constructing a normalized space based on the camera's optical center, a standardized and measurable relationship is established between the pixel positions on the corrected image and the light directions in three-dimensional space. This allows subsequent calculations to focus on geometric relationships and distortion correction itself, without repeatedly considering specific pixel sizes and image center offsets, simplifying mathematical derivation and improving the model's versatility.

[0124] It should be noted that the "virtual spherical space" mentioned in this embodiment refers to a three-dimensional spherical projection model constructed with the camera's optical center as the center. In the specific mathematical calculation process, in order to simplify the complexity of coordinate transformation and achieve normalization, this embodiment preferably uses a "virtual spherical space" with a radius R=1 as the specific implementation of the model.

[0125] It should be understood that virtual spherical space is a special case of virtual spherical space. Those skilled in the art will understand that, without departing from the technical concept of this application, equivalent calculations can also be performed using a spherical space with a non-unit radius (i.e., R ≠ 1) (e.g., multiplying all coordinates by a scaling factor K). This transformation essentially still falls within the scope of the projection model based on spherical space as defined in this application. Therefore, for the sake of simplicity, the following detailed derivation uses "virtual spherical space" as an example, but this calculation process is equally applicable to virtual spherical space in a generalized sense.

[0126] It should be noted that the target coordinate matrix contains the set of (u, v) coordinates of all pixels in the final corrected image, usually obtained by generating a mesh. The target incident angle is the angle between the ray of light originating from the camera's optical center and pointing to the ideal scene point represented by a pixel in the corrected image, and the camera's optical axis (Z-axis), denoted as θ_dst. The azimuth angle is the angle between the projection of this ray onto the XY plane perpendicular to the optical axis and the positive X-axis, denoted as φ. Mapping the target coordinates to the virtual spherical space, i.e., using the inverse transformation of the intrinsic parameter matrix, transforms the pixel coordinates (u, v) to the normalized space, and then calculates their corresponding angles. By generating the target coordinate matrix and mapping it to the virtual spherical space, a corresponding "ideal line of sight" direction (denoted by θ_dst and φ) is calculated for each pixel in the corrected image, describing the spatial orientation from which the scene point should be "seen" by the pixel in the corrected image under ideal distortion-free conditions.

[0127] In one feasible implementation, a virtual unit spherical space is used as the specific implementation of the virtual spherical space. The pixel coordinates of the corrected image are converted into normalized spatial coordinates and mapped to the virtual unit sphere. The specific calculation model is as follows:

[0128]

[0129] The source incident angle is calculated based on the nonlinear compression function. The specific calculation model is as follows:

[0130]

[0131] Where u and v are the target coordinate matrices of the corrected image, c_x and c_y are the coordinates of the center point (principal point) of the camera optical axis on the image plane, x and y are the coordinates on the normalized spatial imaging plane, r is the distance from the coordinate point on the normalized spatial imaging plane to the center of the camera optical axis, f_{\text{new} is the virtual target focal length; \theta_{dst} is the target incident angle, \phi is the azimuth angle, \theta_{src} (θ_src) is the source incident angle, C_0 is the preset linear coefficient, and C_1 is the preset compression coefficient.

[0132] It should be noted that each discrete pixel (u, v) on the corrected image is ultimately mapped to a pair of angles (\theta_{src}, \phi) representing its true physical ray direction through a series of deterministic mathematical transformations, including normalization, radial distance calculation, angle calculation, and nonlinear compression. The virtual target focal length f_{\text{new} is an adjustable parameter used to control the scaling ratio of the output corrected image, usually set to a value close to the original camera focal length f_x to maintain a consistent field of view. C_0 and C_1 are calibrated coefficients that determine the specific form of the nonlinear compression function g. C_0 is the linear coefficient, usually set to 1.0. At small angles (image center), the cubic term is close to 0, and the formula is approximately \theta_{src} = \theta_{dst}, which follows the pinhole imaging principle to ensure no distortion in the central region. C_1 is the compression coefficient, usually set to a positive number (e.g., 0.1\sim0.5). At large angles (image edges), the cubic term increases significantly, causing a smaller target display angle \theta_{dst} to correspond to a larger real physical angle \theta_{src}, thus "compressing" the vast edge field of view into a limited display area.

[0133] In this embodiment, a normalized virtual spherical space is established to provide a standardized geometric reference system for the entire mapping process. By converting the target pixel coordinates into the target incident angle and azimuth angle, the problem is transformed from the two-dimensional pixel domain to the three-dimensional spherical orientation domain, which is more in line with the optical nature. Through a preset nonlinear compression function, linear mapping is maintained in the central region to ensure geometric authenticity, while controllable nonlinear compression is introduced in the edge region. This ensures that after standard distortion inversion, the final corrected image content can be completely constrained within the original circular field of view boundary, thus achieving the correction of geometric distortion of the endoscopic image and the preservation of the entire field of view.

[0134] In one feasible implementation, step S40 may include steps S41 to S43:

[0135] Step S41: Project the source incident angle and the azimuth angle onto the coordinates of the ideal normalized plane to obtain the ideal coordinates;

[0136] Step S42: Based on the distortion coefficient and the ideal coordinates, perform radial distortion correction and tangential distortion correction on the ideal coordinates to obtain distortion-normalized coordinates;

[0137] Step S43: Based on the camera intrinsic parameter matrix, the distortion normalized coordinates are transformed to obtain the source pixel coordinates of the corrected image relative to the original distorted image.

[0138] It should be noted that the ideal normalized plane refers to the imaging plane located at focal length f=1 in an ideal pinhole camera model without any distortion. Ideal coordinates refer to the coordinates (x_{ideal}, y_{ideal}) of the intersection point of a ray with a given direction (defined by the source incident angle \theta_{src} and azimuth angle \phi) and this ideal normalized plane. Projection calculations are based on the geometric transformation from spherical coordinates to planar coordinates.

[0139] Understandably, the purpose of this step is to convert the angular information (\theta_{src}, \phi) representing the real physical direction of the ray obtained from the nonlinear compression function into its position on the ideal imaging plane. This is the first step in returning from the angular domain to the planar coordinate domain. Through this projection, the effect is to obtain the "correct" position on the sensor that the ray should have fallen on if the camera were a perfect, distortion-free pinhole model. This position is the reference point for subsequently introducing real lens distortion effects and is the starting coordinate of the entire "distortion inversion" process.

[0140] It should be noted that, according to the distortion coefficients D=[k_1, k_2, p_1, p_2, k_3], radial distortion correction applies a displacement correction along the radial direction (pointing towards the image center) to the ideal coordinate point based on the radial distortion coefficients (k_1, k_2, k_3), simulating the distortion caused by the lens causing the radial magnification to change with the distance from the center. Tangential distortion correction applies a displacement correction perpendicular to the radial direction (tangential) to the ideal coordinate point based on the tangential distortion coefficients (p_1, p_2), simulating the distortion caused by the lens not being parallel to the sensor. The distortion normalized coordinates (x_{distorted}, y_{distorted}) are the coordinates after applying these two distortion corrections, simulating the "distortion" effect of a real lens on ideal light. Using the camera intrinsic parameter matrix K, the distortion coordinates (x_{distorted}, y_{distorted}) located on the normalized plane (f=1) are mapped to the actual pixel coordinate system of the specific camera, finally obtaining the source pixel coordinates ( , This allows us to pinpoint exactly where to obtain color information in the original distorted image in order to fill the target pixels (u, v) in the corrected image.

[0141] In one feasible implementation, the source incident angle is projected back to the ideal normalized plane, and the specific calculation model is as follows:

[0142]

[0143] Where x_{ideal} and y_{ideal} are ideal coordinates, r_{ideal} is the ideal radial distance, φ is the azimuth angle, and theta_{src} is the source incident angle;

[0144] Based on the distortion coefficients and the ideal coordinates, radial distortion correction and tangential distortion correction are performed on the ideal coordinates to obtain distortion-normalized coordinates. The specific calculation model is as follows:

[0145]

[0146] Where r^2 is the square of the radial radius, Radial is the radial distortion factor, k_1, k_2, k_3 are the radial distortion coefficients, p_1, p_2 are the tangential distortion coefficients, and x_{distorted} and y_{distorted} are the distortion-normalized coordinates;

[0147] The distortion-normalized coordinates are converted into source pixel coordinates. The specific calculation model is as follows:

[0148]

[0149] Where u' and v' are the coordinates of the source pixel, f_x and f_y are the focal lengths of the camera in the x and y directions, respectively, in pixels, and c_x and c_y are the coordinates of the center point of the camera's optical axis on the image plane.

[0150] Understandably, a real-world ray with a specific angle (\theta_{src}, \phi) falls on a point (x_{ideal}, y_{ideal}) in the normalized plane; this point is "distorted" by the distortion characteristics of the real lens (described by k_1, k_2, k_3, p_1, p_2), shifting to (x_{distorted}, y_{distorted}); finally, this "distorted" point, after being magnified and shifted by the intrinsic parameters (focal length and principal point) of the real camera, ultimately falls on the original image sensor's (\theta_{src}, y_{ideal}). , At the pixel position. By calculating the coordinates of the source pixel ( , The location can be used to find the color source in the original distorted image corresponding to a point on the corrected image.

[0151] In this embodiment, a distortion-free reference position is obtained by projecting spherical angular coordinates representing the direction of real light rays back onto the ideal imaging plane. Subsequently, based on the precisely calibrated distortion coefficients, radial and tangential distortion corrections are applied to this reference position to inversely simulate the deformation effect of a real lens, thus obtaining the normalized coordinates of the actual landing point of the light ray under a real lens. Finally, the normalized coordinates are transformed to the pixel coordinate system of the original image using the camera intrinsic parameter matrix, resulting in the accurate source pixel sampling position. This realizes the reverse process of the complete imaging link from real-world light rays to sensor pixels.

[0152] In one possible implementation, step S50 may include steps S51 to S56:

[0153] Step S51: Traverse each target pixel in the corrected image;

[0154] Step S52: Calculate the coordinates of the source pixel corresponding to the target pixel based on the pixel inverse mapping relationship;

[0155] Step S53: Determine whether the coordinates of the source pixel corresponding to the target pixel are located within the effective area defined by the field mask;

[0156] Step S54: If the coordinates of the source pixel corresponding to the target pixel are located within the effective area defined by the field mask, then the pixel value of the source pixel coordinates is acquired in the original distorted image by an interpolation algorithm.

[0157] Step S55: If the coordinates of the source pixel corresponding to the target pixel are outside the effective area defined by the field mask, then the pixel value of the source pixel coordinates is set to a preset background color.

[0158] Step S56: Generate a corrected image based on the pixel values ​​of the source pixel coordinates.

[0159] It should be noted that for each position in the corrected image, a corresponding source coordinate has been calculated through the aforementioned steps. The loop iterates through the "target coordinate matrix" representing all pixel positions in the corrected image. In each iteration, a pair of target coordinates (u, v) and its pre-calculated corresponding source pixel coordinates are retrieved. , ), determine the coordinates of the source pixel ( , The field mask defines the region in the original image containing actual image content (the effective region) and the region with a pure black background. By determining whether the source coordinates fall within the effective region, it decides whether the output pixel (u, v) to be processed should sample the true color from the original image or be directly filled with the background color, ensuring that the final corrected image does not contain invalid interpolated data from the black background of the original image.

[0160] It should be noted that, due to the source pixel coordinates ( , The value () is a floating-point number and may not fall exactly at the center of an integer pixel in the original image. Calculating its pixel value using an interpolation algorithm achieves sub-pixel precision color sampling, thus avoiding the jagged edges and discontinuities that can result from directly assigning the color of the nearest integer pixel. This makes the generated corrected image visually smooth and continuous, maintaining good image quality even after complex geometric transformations. Interpolation algorithms such as bilinear interpolation, nearest neighbor interpolation, or bicubic interpolation can be used, with bilinear interpolation being the preferred method.

[0161] It should be noted that if the coordinates of the source pixel corresponding to the original pixel are outside the effective area defined by the field of view mask, and the position corresponding to the source coordinates belongs to the background or an area without information in the original image, the background color is directly assigned. The preset background color is usually black (0, 0, 0), which is consistent with the background of the original image, but it can also be other specific colors used to identify invalid areas. This ensures that the background area of ​​the corrected image is clean and consistent, and maintains visual continuity with the original image, achieving the visual effect of "preserving the entire field of view" and preventing the stretched rectangle from being cropped or having an incomplete shape during image correction. Based on the pixel values ​​calculated or assigned to each target pixel (u, v) in the above steps, they are filled into a new, blank target image matrix according to their coordinate positions, thereby generating a complete corrected image.

[0162] In this embodiment, the final image synthesis process is driven by traversing all target pixels of the corrected image and using their pre-calculated source coordinates. A field-of-view mask is used to determine the validity of each source coordinate, distinguishing between areas requiring actual sampling and areas to be filled with background. For valid areas, an interpolation algorithm is used for high-quality sub-pixel color sampling, ensuring the smoothness and visual fidelity of the corrected image. For invalid areas, the background color is uniformly filled, forcibly constraining the shape boundaries of the output image. Finally, all pixel values ​​are integrated to generate a complete corrected image, resulting in a clinically usable image that eliminates distortion and provides a complete peripheral field of vision.

[0163] This application also provides an image distortion correction device, please refer to... Figure 5 The image distortion correction device includes:

[0164] Acquisition unit 10 is used to acquire the camera intrinsic parameter matrix, distortion coefficients and original distorted image of the endoscope imaging device;

[0165] The field mask generation unit 20 is used to identify the effective field region in the original distorted image and generate a corresponding field mask based on the effective field region.

[0166] The mapping relationship determination unit 30 is used to establish a virtual spherical space based on the camera optical center, and to establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space;

[0167] The source pixel coordinate calculation unit 40 is used to obtain the source pixel coordinates of the corrected image relative to the original distorted image based on the camera intrinsic parameter matrix, the distortion coefficients and the pixel inverse mapping relationship;

[0168] The corrected image generation unit 50 is used to perform pixel sampling on the original distorted image according to the coordinates of the source pixel points, obtain the pixel sampling result, and generate a corrected image according to the pixel sampling result and the field mask.

[0169] The field-of-view mask generation unit 20 is further configured to perform binarization processing on the original distorted image and extract the bright areas of the original distorted image; perform contour detection on the bright areas, calculate the area enclosed by each contour to obtain multiple contour-enclosed areas; compare the contour-enclosed areas and take the contour with the largest area as the effective field-of-view contour; perform contour optimization on the effective field-of-view contour to generate the corresponding field-of-view mask.

[0170] The mapping relationship determination unit 30 is further configured to establish a virtual spherical space with the camera optical center as the origin; generate the target pixel coordinate matrix of the corrected image, and map the target pixel coordinate matrix to the virtual spherical space to obtain the target incident angle and azimuth angle corresponding to each pixel in the target pixel coordinate matrix; map the target incident angle according to a preset nonlinear compression function to calculate the source incident angle; and establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the source incident angle and azimuth angle.

[0171] The source pixel coordinate calculation unit 40 is further configured to project the source incident angle and the azimuth angle onto the coordinates of the ideal normalized plane to obtain ideal coordinates; perform radial distortion correction and tangential distortion correction on the ideal coordinates according to the distortion coefficient and the ideal coordinates to obtain distortion normalized coordinates; and transform the distortion normalized coordinates according to the camera intrinsic parameter matrix to obtain the source pixel coordinates of the corrected image relative to the original distorted image.

[0172] The corrected image generation unit 50 is further configured to traverse each target pixel in the corrected image; calculate the source pixel coordinates corresponding to the target pixel according to the pixel inverse mapping relationship; determine whether the source pixel coordinates corresponding to the target pixel are located within the effective area defined by the field mask; if the source pixel coordinates corresponding to the target pixel are located within the effective area defined by the field mask, then the pixel value corresponding to the source pixel coordinates is acquired in the original distorted image by an interpolation algorithm; if the source pixel coordinates corresponding to the target pixel are located outside the effective area defined by the field mask, then the pixel value of the source pixel coordinates is set to a preset background color; and generate a corrected image based on the pixel value of the source pixel coordinates.

[0173] The mapping relationship determination unit 30 is further configured to use a virtual unit spherical space as the specific implementation of the virtual spherical space, convert the pixel coordinates of the corrected image into normalized spatial coordinates and map them to the virtual unit sphere. The specific calculation model is as follows:

[0174]

[0175] The source incident angle is calculated based on the nonlinear compression function, and the specific calculation model is as follows:

[0176]

[0177] Where u and v are the target coordinate matrices of the corrected image, c_x and c_y are the coordinates of the center point of the camera optical axis on the image plane, x and y are the coordinates on the normalized spatial imaging plane, r is the distance from the coordinate point on the normalized spatial imaging plane to the center of the camera optical axis, f_{\text{new} is the virtual target focal length; \theta_{dst} is the target incident angle, \phi is the azimuth angle, \theta_{src} is the source incident angle, C_0 is the preset linear coefficient, and C_1 is the preset compression coefficient.

[0178] The source pixel coordinate calculation unit 40 is also used to project the source incident angle back to the ideal normalized plane. The specific calculation model is as follows:

[0179]

[0180] Where x_{ideal} and y_{ideal} are ideal coordinates, r_{ideal} is the ideal radial distance, φ is the azimuth angle, and theta_{src} is the source incident angle;

[0181] Based on the distortion coefficients and the ideal coordinates, radial distortion correction and tangential distortion correction are performed on the ideal coordinates to obtain distortion-normalized coordinates. The specific calculation model is as follows:

[0182]

[0183] Where r^2 is the square of the radial radius, Radial is the radial distortion factor, k_1, k_2, k_3 are the radial distortion coefficients, p_1, p_2 are the tangential distortion coefficients, and x_{distorted} and y_{distorted} are the distortion-normalized coordinates;

[0184] The distortion-normalized coordinates are converted into source pixel coordinates. The specific calculation model is as follows:

[0185]

[0186] in, and f_x and f_y are the source pixel coordinates, f_x and f_y are the focal lengths of the camera along the x and y axes, respectively, in pixels, and c_x and c_y are the coordinates of the center point of the camera's optical axis on the image plane.

[0187] The image distortion correction device provided in this application, employing the image distortion correction method described in the above embodiments, can solve the technical problem of field of view loss during endoscopic image distortion correction. Compared with the prior art, the beneficial effects of the image distortion correction device provided in this application are the same as those of the image distortion correction method described in the above embodiments, and other technical features in the image distortion correction device are the same as those disclosed in the methods of the above embodiments, and will not be repeated here.

[0188] This application provides an endoscope device, which includes: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, which are executed by the at least one processor to enable the at least one processor to perform the image distortion correction method in Embodiment 2 above.

[0189] The following is for reference. Figure 6 The diagram illustrates a structural schematic of an endoscope device suitable for implementing embodiments of this application. The endoscope device in the embodiments of this application may include, but is not limited to, mobile terminals such as mobile phones, laptops, digital broadcast receivers, PDAs (Personal Digital Assistants), PADs (Portable Application Descriptions), PMPs (Portable Media Players), in-vehicle terminals (e.g., in-vehicle navigation terminals), and fixed terminals such as digital TVs and desktop computers. Figure 5 The endoscopic device shown is merely an example and should not be construed as limiting the functionality and scope of use of the embodiments of this application.

[0190] like Figure 6As shown, the endoscopic device may include a processing unit 1001 (e.g., a central processing unit, a graphics processing unit, etc.) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) 1002 or a program loaded from a storage device 1003 into a random access memory (RAM) 1004. The RAM 1004 also stores various programs and data required for the operation of the endoscopic device. The processing unit 1001, ROM 1002, and RAM 1004 are interconnected via a bus 1005. An input / output (I / O) interface 1006 is also connected to the bus. Typically, the following systems can be connected to the I / O interface 1006: input devices 1007 including, for example, a touchscreen, touchpad, keyboard, mouse, image sensor, microphone, accelerometer, gyroscope, etc.; output devices 1008 including, for example, a liquid crystal display (LCD), speaker, vibrator, etc.; storage devices 1003 including, for example, magnetic tape, hard disk, etc.; and communication devices 1009. Communication device 1009 allows the endoscopic apparatus to communicate wirelessly or wiredly with other devices to exchange data. While the figures show endoscopic apparatuses with various systems, it should be understood that implementation or possession of all the systems shown is not required. More or fewer systems may be implemented alternatively.

[0191] Specifically, according to the embodiments disclosed in this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments disclosed in this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program containing program code for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication device, or installed from storage device 1003, or installed from ROM 1002. When the computer program is executed by processing device 1001, it performs the functions defined in the methods of the embodiments disclosed in this application.

[0192] The endoscopic device provided in this application, employing the image distortion correction method described in the above embodiments, solves the technical problem of field of view loss during endoscopic image distortion correction. Compared with the prior art, the beneficial effects of the endoscopic device provided in this application are the same as those of the image distortion correction method described in the above embodiments, and other technical features of this endoscopic device are the same as those disclosed in the previous embodiment method, and will not be repeated here.

[0193] It should be understood that the various parts disclosed in this application can be implemented using hardware, software, firmware, or a combination thereof. In the description of the above embodiments, specific features, structures, materials, or characteristics can be combined in any suitable manner in one or more embodiments or examples.

[0194] The above description is merely a specific embodiment of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

[0195] This application provides a computer-readable storage medium having computer-readable program instructions (i.e., a computer program) stored thereon, the computer-readable program instructions being used to execute the image distortion correction method in the above embodiments.

[0196] The computer-readable storage medium provided in this application may be, for example, a USB flash drive, but is not limited to, electrical, magnetic, optical, electromagnetic, infrared, or semiconductor systems, devices, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM or flash memory), optical fiber, portable compact disk read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this embodiment, the computer-readable storage medium may be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, system, or device. The program code contained on the computer-readable storage medium may be transmitted using any suitable medium, including but not limited to: wires, optical cables, RF (Radio Frequency), etc., or any suitable combination thereof.

[0197] The aforementioned computer-readable storage medium may be included in the endoscope device or may exist independently without being assembled into the endoscope device.

[0198] The aforementioned computer-readable storage medium carries one or more programs that, when executed by an endoscopic device, cause the endoscopic device to: acquire the camera intrinsic parameter matrix, distortion coefficients, and original distorted image of the endoscopic imaging device; identify the effective field of view region in the original distorted image and generate a corresponding field of view mask based on the effective field of view region; establish a virtual spherical space based on the camera optical center and establish a pixel inverse mapping relationship between the corrected image and the original distorted image based on the virtual spherical space; obtain the source pixel coordinates of the corrected image relative to the original distorted image based on the camera intrinsic parameter matrix, the distortion coefficients, and the pixel inverse mapping relationship; perform pixel sampling in the original distorted image based on the source pixel coordinates to obtain pixel sampling results, and generate a corrected image based on the pixel sampling results and the field of view mask.

[0199] Computer program code for performing the operations of this application can be written in one or more programming languages ​​or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, and C++, and conventional procedural programming languages ​​such as the "C" language or similar programming languages. The program code can be executed entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer can be connected to the user's computer via any type of network—including a Local Area Network (LAN) or a Wide Area Network (WAN)—or can be connected to an external computer (e.g., via the Internet using an Internet service provider).

[0200] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0201] The modules described in the embodiments of this application can be implemented in software or hardware. The names of the modules do not necessarily limit the functionality of the unit itself.

[0202] The readable storage medium provided in this application is a computer-readable storage medium that stores computer-readable program instructions (i.e., a computer program) for executing the above-described image distortion correction method, thereby solving the technical problem of field-of-view loss during endoscopic image distortion correction. Compared with the prior art, the beneficial effects of the computer-readable storage medium provided in this application are the same as those of the image distortion correction method provided in the above embodiments, and will not be repeated here.

[0203] The above are merely preferred embodiments of this application and do not limit the patent scope of this application. Any equivalent structural or procedural transformations made using the content of this application's specification and drawings, or direct or indirect applications in other related technical fields, are similarly included within the patent scope of this application.

Claims

1. An image distortion correction method characterized by, The image distortion correction method comprises: obtaining a camera intrinsic matrix, a distortion coefficient and an original distortion image of an endoscope imaging device; identifying an effective field of view area in the original distortion image, and generating a corresponding field of view mask according to the effective field of view area; establishing a virtual spherical space according to a camera optical center, and establishing a pixel reverse mapping relationship between a corrected image and the original distortion image according to the virtual spherical space; obtaining source pixel point coordinates of the corrected image relative to the original distortion image according to the camera intrinsic matrix, the distortion coefficient and the pixel reverse mapping relationship; performing pixel sampling in the original distortion image according to the source pixel point coordinates to obtain a pixel sampling result, and generating a corrected image according to the pixel sampling result and the field of view mask.

2. The method of claim 1, wherein, The step of identifying the effective field of view area in the original distortion image and generating the corresponding field of view mask according to the effective field of view area comprises: performing binary processing on the original distortion image, and extracting a bright area of the original distortion image; performing contour detection on the bright area, calculating the area surrounded by each contour to obtain a plurality of contour surrounding areas; comparing the contour surrounding areas, and taking the contour with the largest area as an effective field of view contour; performing contour optimization on the effective field of view contour to generate a corresponding field of view mask.

3. The method of claim 1, wherein, The step of establishing a virtual spherical space according to a camera optical center, and establishing a pixel reverse mapping relationship between a corrected image and the original distortion image according to the virtual spherical space comprises: establishing a virtual spherical space with the camera optical center as the origin; generating a target pixel coordinate matrix of the corrected image, and mapping the target pixel coordinate matrix to the virtual spherical space to obtain a target incident angle and an azimuth angle corresponding to each pixel point in the target pixel coordinate matrix; mapping the target incident angle according to a preset nonlinear compression function to calculate a source incident angle; establishing a pixel reverse mapping relationship between a corrected image and the original distortion image according to the source incident angle and the azimuth angle.

4. The method of claim 3, wherein, The step of obtaining source pixel point coordinates of the corrected image relative to the original distortion image according to the camera intrinsic matrix, the distortion coefficient and the pixel reverse mapping relationship comprises: projecting the source incident angle and the azimuth angle to coordinates on an ideal normalized plane to obtain ideal coordinates; performing radial distortion correction and tangential distortion correction on the ideal coordinates according to the distortion coefficient and the ideal coordinates to obtain distortion normalized coordinates; performing conversion on the distortion normalized coordinates according to the camera intrinsic matrix to obtain the source pixel point coordinates of the corrected image relative to the original distortion image.

5. The method of claim 1, wherein, The step of performing pixel sampling in the original distortion image according to the source pixel point coordinates to obtain a pixel sampling result, and generating a corrected image according to the pixel sampling result and the field of view mask comprises: traversing each target pixel point in the corrected image; calculating source pixel point coordinates corresponding to the target pixel point according to the pixel reverse mapping relationship; determine whether the source pixel point coordinate corresponding to the target pixel point is located in the effective area defined by the field mask; if the source pixel point coordinate corresponding to the target pixel point is located in the effective area defined by the field mask, then the pixel value corresponding to the source pixel point coordinate in the original distorted image is collected through an interpolation algorithm; if the source pixel point coordinate corresponding to the target pixel point is located outside the effective area defined by the field mask, then the pixel value of the source pixel point coordinate is set to a preset background color; generate a corrected image according to the pixel value of the source pixel point coordinate.

6. The method of claim 3, wherein, In the step of establishing a virtual spherical space according to the camera optical center and establishing a pixel reverse mapping relationship between the corrected image and the original distorted image according to the virtual spherical space: A virtual unit spherical space is used as the specific implementation of the virtual spherical space, the pixel coordinates of the corrected image are converted into normalized space coordinates and mapped to the virtual unit spherical space, and the specific calculation model is: A source incident angle is calculated according to the nonlinear compression function, and the specific calculation model is: Wherein, u and v are target coordinate matrices of the corrected image, c_x and c_y are center point coordinates of the camera optical axis on the image plane, x and y are coordinates on the normalized space imaging plane, r is the distance from the coordinate point on the normalized space imaging plane to the center of the camera optical axis, f_{new} is a virtual target focal length; \theta_{dst} is a target incident angle, \phi is an azimuth angle, \theta_{src} is a source incident angle, C_0 is a preset linear coefficient, and C_1 is a preset compression coefficient.

7. The method of claim 6, wherein, In the step of obtaining the source pixel point coordinate of the corrected image relative to the original distorted image according to the camera intrinsic matrix, the distortion coefficient and the pixel reverse mapping relationship: The source incident angle is projected back to an ideal normalized plane, and the specific calculation model is: Wherein, x_{ideal} and y_{ideal} are ideal coordinates, r_{ideal} is an ideal radial distance, \phi is an azimuth angle, and \theta_{src} is a source incident angle; The ideal coordinates are subjected to radial distortion correction and tangential distortion correction according to the distortion coefficient and the ideal coordinates, to obtain distorted normalized coordinates, and the specific calculation model is: Wherein, r^2 is a radial radius square, Radial is a radial distortion factor, k_1, k_2 and k_3 are radial distortion coefficients, p_1 and p_2 are tangential distortion coefficients, and x_{distorted} and y_{distorted} are distorted normalized coordinates; The distorted normalized coordinates are converted into source pixel point coordinates, and the specific calculation model is: wherein, and are the coordinates of the source pixel point, f_x and f_y are the focal lengths of the camera in the x and y axis directions, respectively, and are in units of pixels, and c_x and c_y are the coordinates of the center point of the optical axis of the camera on the image plane.

8. An image distortion correction apparatus characterized by comprising: It includes: An acquisition unit is configured to acquire a camera intrinsic matrix, a distortion coefficient and an original distorted image of an endoscope imaging device; A field mask generation unit is configured to identify an effective field area in the original distorted image, and generate a corresponding field mask according to the effective field area; The mapping relationship determining unit is configured to establish a virtual spherical space according to a camera optical center, and establish a pixel reverse mapping relationship between a corrected image and the original distorted image according to the virtual spherical space; The source pixel point coordinate calculating unit is configured to obtain source pixel point coordinates of the corrected image relative to the original distorted image according to the camera intrinsic parameter matrix, the distortion coefficient and the pixel reverse mapping relationship; The corrected image generating unit is configured to perform pixel sampling in the original distorted image according to the source pixel point coordinates to obtain a pixel sampling result, and generate a corrected image according to the pixel sampling result and the field of view mask.

9. An endoscope apparatus characterized by comprising: The endoscope device comprises a memory, a processor and a computer program stored on the memory and executable on the processor, and the computer program is configured to implement the steps of the image distortion correction method according to any one of claims 1 to 7.

10. A storage medium, characterized by The storage medium is a computer readable storage medium, and the computer readable storage medium stores a computer program, and the computer program is executed by the processor to implement the steps of the image distortion correction method according to any one of claims 1 to 7.

Images