Image dedistortion method and system accompanying laparoscopic surgery

Abstract

The present invention relates to the field of image correction technology, and in particular to an image dedistortion method and system accompanying the laparoscopic surgery stage. The present invention first obtains an organ surface image obtained by a laparoscope; further performs edge detection on the organ surface image and screens out spot areas; further obtains the lens distance parameters of each spot area and the corresponding inclination parameters of the laparoscope; further corrects the initial distance of each pixel point to obtain the corrected distance of each pixel point; and obtains a dedistorted image based on the corrected distance of each pixel point. The present invention analyzes the inclination characteristics of the laparoscope based on the distribution characteristics and grayscale characteristics of the organ surface features in the image; and then corrects the pixels based on the different positional relationships between the pixels and the laparoscope lens to reduce the influence of tangential distortion, so that the image can more accurately represent the real organ surface.

Description

Technical Field

[0001] The present invention relates to the technical field of image correction, and in particular to an image dedistortion method and system accompanying a laparoscopic surgery stage. Background Art

[0002] Similar to electronic organoscopes, laparoscopes are medical devices equipped with a miniature camera. Laparoscopic surgery is performed using a laparoscope and related instruments. During laparoscopic surgery, the angle between the lens shooting direction and the surface of the photographed organ may not be a right angle due to the limited placement of the laparoscope lens, causing tangential distortion of the image, which is not conducive to image analysis.

[0003] The existing technology for correcting tangential distortion of images in organs mainly relies on the distance of characteristic textures on the surface of the organ to correct the pixels in the image along a fixed direction. It does not take into account the three-dimensionality of the inclination angle between the lens and the organ surface, and there are problems in correcting the pixels in the image. Summary of the invention

[0004] In order to solve the technical problem of inaccurate correction of tangential distortion of organ surface images obtained by laparoscope, the purpose of the present invention is to provide an image dedistortion method and system accompanying the laparoscopic surgery stage. The technical solution adopted is as follows:

[0005] A method for image dedistortion during laparoscopic surgery, the method comprising:

[0006] obtain images of organ surfaces obtained through laparoscopy;

[0007] Perform edge detection on the organ surface image to obtain segmented regions; select any of the segmented regions as a target segmented region; obtain the spot possibility of the target segmented region based on the area characteristics of the target segmented region, the distribution density characteristics of the target segmented region and adjacent segmented regions, and the grayscale difference characteristics of the edge pixels of the target segmented region and the adjacent pixels outside the target segmented region; and screen out the spot region based on the spot possibility;

[0008] According to the grayscale change characteristics of the neighboring pixel points of the edge pixel points of each spot area, the grayscale characteristics inside the spot area, and the area characteristics of the spot area, the lens distance parameter of each spot area is obtained; according to the change characteristics of the lens distance parameter, combined with the distance change characteristics between the spot areas, the inclination angle parameters of the current organ surface image between the laparoscope and the organ surface are obtained;

[0009] Taking the center point of the spot area where the lens distance parameter is the smallest as the origin, the initial distance from each pixel in the image to the origin is obtained; according to the inclination parameter and the angle between the line connecting the origin and each pixel and the inclination direction of the laparoscope lens, the initial distance of each pixel is corrected to obtain the corrected distance of each pixel;

[0010] A dedistorted image is acquired based on the corrected distance of each pixel.

[0011] Furthermore, the method for obtaining the spot possibility includes:

[0012] Taking the reciprocal of the number of pixels in the target segmentation area as a first possibility parameter;

[0013] Taking the reciprocal of the sum of the distances between the center point of the target segmented region and the center points of the adjacent segmented regions as the second possibility parameter;

[0014] Taking the line connecting the center point of the target segmentation area and each edge pixel point as the extension direction away from the center point, extending a preset first number of pixel points as the external neighboring pixel points of each edge pixel point; taking the sum of the gray value differences between all the external neighboring pixel points and the corresponding edge pixel point as the third possibility parameter;

[0015] The product of the first possibility parameter, the second possibility parameter and the third possibility parameter corresponding to the target segmentation region is used as the spot possibility of the target segmentation region.

[0016] Furthermore, the method for obtaining the lens distance parameter includes:

[0017] Select any of the spot areas as the target spot area; use the number of pixels within the minimum circumscribed circle of the target spot area as the area parameter of the target spot area; and take the inverse of the area parameter as the first distance parameter;

[0018] Taking the reciprocal of the grayscale value and value of all pixels in the target spot area as the second distance parameter;

[0019] Select any edge pixel point of the target spot area as the target edge pixel point; take a line connecting the center point of the target spot area and the target edge pixel point, and obtain a pixel point sequence consisting of the target edge pixel point and the adjacent pixels on both sides of the line; in the pixel point sequence corresponding to the target edge pixel point, take the sum of the gray value differences of all adjacent pixels as the blurriness of the target edge pixel point; take the sum of the blurriness of all edge pixels of the target spot area as the third distance parameter;

[0020] The product of the first distance parameter, the second distance parameter and the third distance parameter of the target spot area is used as the lens distance parameter of the target spot area.

[0021] Furthermore, the method for obtaining the inclination parameter includes:

[0022] Using the range of the lens distance parameter as the first tilt angle sub-parameter;

[0023] Connecting the center point of the spot area with the largest lens distance parameter with the center point of the spot area with the smallest lens distance parameter, the connecting line is called a target connecting line; analyzing the distance variation characteristics of the center points of adjacent spot areas on the target connecting line to obtain a second tilt angle sub-parameter;

[0024] The product of the first tilt sub-parameter and the second tilt sub-parameter is used as the tilt parameter.

[0025] Furthermore, the method for acquiring the second tilt sub-parameter includes:

[0026] On the target line, the distance between the center point of the spot area with the largest lens distance parameter and the center point of the adjacent spot area is obtained as the first center distance, and the distance between the center point of the spot area with the smallest lens distance parameter and the center point of the adjacent spot area is obtained as the second center distance; after mapping the difference between the first center distance and the second center distance through a preset negative correlation mapping function, the mapping value is used as the second inclination sub-parameter.

[0027] Furthermore, the direction in which the center point of the spot area with the smallest lens distance parameter points to the center point of the spot area with the largest lens distance parameter is used as the inclination direction of the laparoscope lens.

[0028] Furthermore, the method for obtaining the corrected distance includes:

[0029] Select any pixel point as the target pixel point; use the angle between the direction away from the origin and the direction of the inclination angle on the line connecting the origin and the target pixel point as the denominator of the ratio, use the product of the initial distance and the inclination angle parameter as the numerator of the ratio, and normalize the ratio as a correction coefficient;

[0030] The product of the sum of the correction coefficient and the constant 1 and the initial distance of the target pixel point is used as the corrected distance of the target pixel point.

[0031] Furthermore, the method for acquiring a dedistorted image based on the corrected distance of each pixel point includes:

[0032] In the direction of the line connecting the origin and each pixel point, the corresponding pixel point is moved, and the distance between the moved pixel point and the origin point is the correction distance. After all the pixel points are moved, a dedistorted image is obtained.

[0033] Furthermore, after the pixel points are moved, the method further includes: filling the blank positions with a cubic interpolation method.

[0034] The present invention also proposes an image dedistortion system accompanying the laparoscopic surgery stage, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, the processor implementing any one of the steps of the image dedistortion method accompanying the laparoscopic surgery stage when executing the computer program.

[0035] The present invention has the following beneficial effects:

[0036] The embodiment of the present invention first obtains an organ surface image obtained by laparoscope to provide an analysis basis; further performs edge detection on the organ surface image and screens out spot areas, so as to facilitate the subsequent analysis of the spatial relationship between the laparoscope lens and the organ surface through the distribution position and size relationship of the spots in the image; further obtains the lens distance parameter of each spot area, characterizes the distance characteristics between the spot area and the lens, and prepares for the subsequent analysis of the laparoscope lens tilt angle characteristics; further, based on the characteristics that the laparoscope and the organ surface have different changes in lens distance parameters and different distance changes between spot areas due to different tilt angles, the laparoscope and the organ surface of the current organ surface image are obtained to characterize the tilt angle, and provide a basis for the subsequent accurate correction of the organ surface image; further obtains the initial distance from each pixel point to the origin, determines the initial position of each pixel point, and facilitates the subsequent correction; further corrects the initial distance from multiple angles to obtain the corrected distance of each pixel point, and lays the foundation for the final image correction; finally, the dedistorted image is obtained based on the corrected distance of each pixel point. The embodiment of the present invention analyzes the inclination characteristics of the laparoscope according to the distribution characteristics and grayscale characteristics of the organ surface characteristics in the image; and then corrects the pixel points according to the different positional relationships between the pixel points and the laparoscope lens to reduce the influence of tangential distortion, so that the image can more accurately show the real organ surface. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] In order to more clearly illustrate the technical solutions and advantages in the embodiments of the present invention or the prior art, the drawings required for use in the embodiments or the prior art descriptions are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0038] Figure 1A flowchart of an image dedistortion method accompanying a laparoscopic surgery stage provided by an embodiment of the present invention;

[0039] Figure 2 An organ surface image obtained through a laparoscope is provided in one embodiment of the present invention;

[0040] Figure 3 A flow chart of a method for obtaining spot likelihood provided by an embodiment of the present invention;

[0041] Figure 4 The present invention is a flowchart of a method for obtaining lens distance parameters provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0042] In order to further explain the technical means and effects adopted by the present invention to achieve the predetermined invention purpose, the following is a detailed description of the image dedistortion method and system for the laparoscopic surgery stage proposed by the present invention in combination with the accompanying drawings and preferred embodiments, and its specific implementation, structure, features and effects are described as follows. In the following description, different "one embodiment" or "another embodiment" does not necessarily refer to the same embodiment. In addition, specific features, structures or characteristics in one or more embodiments may be combined in any suitable form.

[0043] Unless defined otherwise, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this invention belongs.

[0044] The following is a detailed description of a specific scheme of an image dedistortion method and system during a laparoscopic surgery provided by the present invention in conjunction with the accompanying drawings.

[0045] See also Figure 1 , which shows a flow chart of an image dedistortion method accompanying a laparoscopic surgery stage provided by an embodiment of the present invention, specifically comprising:

[0046] Step S1: Acquire an organ surface image obtained through a laparoscope.

[0047] Tangential distortion is mainly caused by the fact that the lens and the imaging plane are not completely parallel, or the angle between the lens shooting direction and the surface of the photographed organ is not a right angle. This distortion will cause perspective distortion of the image, especially at the edge of the image, which is not conducive to image analysis; common distortions also include radial distortion and thin prism distortion, which can be avoided by selecting an appropriate lens, paying attention to the way the lens is used, etc., and are usually eliminated through quality inspection before the scope leaves the factory. The embodiment of the present invention mainly corrects the tangential distortion when the laparoscope is used to photograph the surface of the organ.

[0048] In the embodiment of the present invention, firstly, an organ surface image obtained by laparoscope is obtained to provide a basis for analysis. Figure 2 , which shows an organ surface image obtained through a laparoscope provided by an embodiment of the present invention.

[0049] Step S2: Perform edge detection on the organ surface image to obtain the segmented area; select any segmented area as the target segmented area; obtain the spot possibility of the target segmented area according to the area characteristics of the target segmented area, the distribution density characteristics of the target segmented area and the adjacent segmented areas, and the grayscale difference characteristics of the edge pixels of the target segmented area and the adjacent pixels outside the target segmented area; and screen out the spot area based on the spot possibility.

[0050] Considering that the mucosal surface of a normal organ will show evenly distributed spots with low grayscale, and the size of each spot is the same, the spatial relationship between the laparoscope lens and the organ surface can be analyzed through the distribution position and size relationship of the spots in the image, and finally the image can be corrected; in addition to flat areas, the organ surface may also have invaginated, wrinkled or lesion areas. In order to facilitate image correction based on spot areas, the edge detection of the organ surface image is first performed to obtain the segmented area, and any segmented area is selected as the target segmented area to avoid repeated analysis; considering that the spot area is smaller than other areas, the distribution is more concentrated, and the grayscale value is smaller, so according to the area characteristics of the target segmented area, the distribution density characteristics of the target segmented area and the adjacent segmented areas, combined with the grayscale difference characteristics of the edge pixels of the target segmented area and the adjacent pixels outside the target segmented area, the spot possibility of the target segmented area is obtained; the spot area is screened out based on the spot possibility.

[0051] It should be noted that edge detection of organ surface images is a technical means well known to those skilled in the art. In one embodiment of the present invention, a Canny edge detection algorithm is used to obtain a number of edges, and the edge lines are regarded as segmentation lines, and the areas enclosed by the edges are used as segmentation areas. This is already a prior art and will not be described in detail.

[0052] Preferably, in one embodiment of the present invention, the method for obtaining the probability of a spot includes: Figure 3 , which shows a flow chart of a method for obtaining spot likelihood provided by an embodiment of the present invention, specifically comprising:

[0053] Step S211: taking the inverse of the number of pixels in the target segmentation area as the first possibility parameter.

[0054] Since the area of ​​the spot area is relatively small and contains fewer pixels, the area characteristics of the target segmentation area are represented by the number of pixels in the segmentation area. The reciprocal of the number of pixels in the target segmentation area is taken. The smaller the number, the larger the reciprocal, the more likely it is a spot area, and the larger the first possibility parameter.

[0055] Step S212: taking the reciprocal of the sum of the distances between the center point of the target segmented region and the center points of the adjacent segmented regions as the second possibility parameter.

[0056] Taking into account the relatively concentrated distribution of the spot area, the center point of the segmented area is used to represent the position of the segmented area. The distance between the center point of the target segmented area and the center point of the adjacent segmented area is analyzed to reflect the distribution density characteristics of the target segmented area and the adjacent segmented area. The closer the distance, the more concentrated the distribution of the target segmented area and the surrounding segmented areas, the greater the distribution density characteristics, and the more consistent with the distribution characteristics of the spot area. Therefore, the reciprocal of the sum of the distances between the center point of the target segmented area and the center point of the adjacent segmented area is taken as the second possibility parameter.

[0057] It should be noted that, in one embodiment of the present invention, when two segmented areas have common edge pixels or two segmented areas have adjacent edge pixels, the two segmented areas are considered to be adjacent; the center points of the areas mentioned in the embodiment of the present invention are all obtained through the centroid formula, and the distances mentioned are all Euclidean distances, which are obtained by establishing a coordinate system for the organ surface image, which are all technical means well known to those skilled in the art and will not be repeated here.

[0058] Step S213: Take the line connecting the center point of the target segmentation area and each edge pixel point as the extension direction away from the center point, and extend a preset first number of pixel points as the external neighboring pixel points of each edge pixel point; and take the sum of the gray value differences between all the external neighboring pixel points and the corresponding edge pixel points as the third possibility parameter.

[0059] Taking into account that the grayscale value of the spot area is relatively small, the grayscale difference between the edge pixels and the external neighboring pixels is compared, and the line connecting the center point of the target segmentation area and each edge pixel is extended away from the center point by a preset first number of pixels as the external neighboring pixels of each edge pixel; the larger the grayscale value difference between the external neighboring pixels and the corresponding edge pixels, the more similar the internal grayscale of the segmented area obtained by edge detection is, indicating that the lower the grayscale of the target segmentation area is compared with the outside of the area, the more it conforms to the characteristics of the spot area, and therefore the sum of the grayscale value differences between all external neighboring pixels and the corresponding edge pixels is taken as the third possibility parameter.

[0060] As an example, the first number is preset to be 5.

[0061] Step S214: taking the product of the first possibility parameter, the second possibility parameter and the third possibility parameter corresponding to the target segmentation region as the spot possibility of the target segmentation region.

[0062] The first possibility parameter measures the speckle possibility from the perspective of the number characteristics of pixels in the target segmentation area, the second possibility parameter measures the speckle possibility from the distribution characteristics of the segmentation area, and the third possibility parameter measures the speckle possibility from the perspective of the grayscale difference characteristics between edge pixels and adjacent pixels outside the target segmentation area. Finally, multiple analysis angles are integrated, and the product of the first possibility parameter, the second possibility parameter, and the third possibility parameter corresponding to the target segmentation area is taken as the speckle possibility of the target segmentation area.

[0063] Preferably, in one embodiment of the present invention, the segmented area whose speckle probability is greater than a first preset threshold is regarded as a speckle area. As an example, the first preset threshold is 0.7, the speckle probability is normalized using a sigmoid function, and the segmented area whose normalized speckle probability is greater than 0.7 is regarded as a speckle area.

[0064] Step S3: Obtain the lens distance parameter of each spot area according to the grayscale change characteristics of the neighboring pixels of the edge pixels of each spot area, the grayscale characteristics inside the spot area, and the area characteristics of the spot area; obtain the inclination parameters of the laparoscope and the organ surface of the current organ surface image according to the change characteristics of the lens distance parameters and the distance change characteristics between the spot areas.

[0065] During surgery, the laparoscope will change angles according to the surgeon's needs. Due to the limited mobility of the patient in the abdominal cavity, tangential distortion is prone to occur during image sampling. In order to accurately correct the tangential distortion of the laparoscope-derived images, it is necessary to accurately analyze the three-dimensional spatial distribution of the laparoscope and the patient's organ surface. Therefore, it is necessary to obtain the lens distance parameters of each spot area, and then evaluate the current inclination parameters of the laparoscope and the organ surface, so as to accurately correct the image in the end.

[0066] Taking into account that when the laparoscope is shooting, different spot areas are at different distances from the lens, and the areas presented are different; and due to the depth of field effect, the range that the laparoscope can clearly capture is limited. The farther the spot area is from the lens, the blurrier it is, the lower the boundary clarity of the spot area, the more obvious the grayscale change of the adjacent pixels of the edge pixels, and the grayscale will be lower due to insufficient light. Therefore, according to the grayscale change characteristics of the adjacent pixels of the edge pixels of each spot area, the grayscale characteristics inside the spot area, and the area characteristics of the spot area, the lens distance parameters of each spot area are obtained to characterize the distance characteristics between the spot area and the lens, so as to prepare for the subsequent analysis of the laparoscope lens inclination characteristics.

[0067] Preferably, in one embodiment of the present invention, the method for obtaining the lens distance parameter includes: Figure 4 , which shows a flow chart of a method for obtaining lens distance parameters provided by an embodiment of the present invention, specifically comprising:

[0068] Step S311: select any spot area as the target spot area; use the number of pixels in the minimum circumscribed circle of the target spot area as the area parameter of the target spot area; and take the inverse of the area parameter as the first distance parameter.

[0069] Considering that spots are usually circular or nearly circular, the number of pixels in the minimum circumscribed circle of the target spot area is used as the area parameter of the target spot area, and the inverse of the area parameter is used as the first distance parameter.

[0070] The area characteristics of the spot area are reflected by the minimum circumscribed circle. The smaller the minimum circumscribed circle is, the fewer pixels it contains, the smaller the area is, and the farther the spot area is from the lens, which is the first distance parameter.

[0071] It should be noted that the method for obtaining the minimum circumscribed circle is already an existing technology and will not be described in detail.

[0072] Step S312: taking the inverse of the grayscale value and value of all pixels in the target spot area as the second distance parameter.

[0073] When the spot area is farther away from the lens, due to insufficient light, the spot area is in a darker area and the grayscale value inside is lower, so the reciprocal of the grayscale value and value of all pixels in the target spot area is taken as the second distance parameter.

[0074] The grayscale characteristics inside the spot area are represented by the grayscale value and value of all pixels in the spot area. The smaller the grayscale value and value of all pixels in the spot area is, the smaller the overall grayscale value inside the spot area is. The larger the second distance parameter after the inverse is, the farther the spot area is from the lens.

[0075] Step S313: select any edge pixel point of the target spot area as the target edge pixel point; use the line connecting the center point of the target spot area and the target edge pixel point to obtain a pixel point sequence consisting of the target edge pixel point and the adjacent pixels on both sides of the line; in the pixel point sequence corresponding to the target edge pixel point, use the sum of the gray value differences of all adjacent pixels as the blurriness of the target edge pixel point; use the sum of the blurriness of all edge pixels of the target spot area as the third distance parameter.

[0076] Taking into account that the spots on the surface of the organ farther away will be more blurred in the image, the grayscale consistency within different areas will decrease, and the grayscale of the pixels in the edge area of ​​the spot area will gradually increase from the inside to the outside. In order to facilitate the analysis of this grayscale change feature, the edge pixels of any target spot area are selected as the target edge pixels; the line connecting the center point of the target spot area and the target edge pixel is extended by a preset second number of pixels in the direction away from the center point, and by a preset third number of pixels in the direction close to the center point. According to the distance between the pixel point and the center point of the target spot area, the extended pixels and the target edge pixels are sorted to form a pixel sequence.

[0077] As an example, analogous to step S213, on the line connecting the center point of the target spot area and the target edge pixel point, with the target edge pixel point as the starting point, a preset second number of pixel points are extended in the direction away from the center point, and a preset third number of pixel points are extended in the direction close to the center point, and the extended pixel points and the target edge pixel points are sorted according to the distance between the pixel point and the center point of the target spot area to form a pixel point sequence, the preset second number is 5, and the preset third number is 5; the pixel point sequence includes 11 pixel points.

[0078] In the pixel sequence corresponding to the target edge pixel, the sum of the grayscale value differences of all adjacent pixels is used as the blurriness of the target edge pixel; the sum of the blurriness of all edge pixels in the target spot area is used as the third distance parameter. By analyzing the grayscale changes of adjacent pixels in the pixel sequence, the grayscale change characteristics of the spot area are reflected, thereby obtaining a more accurate lens distance parameter.

[0079] The calculation formula of the third distance parameter includes: ;in, The third distance parameter representing the target spot area; Represents the number of edge pixels in the target spot area; Indicates The number of pixels in the pixel sequence of edge pixels; The target spot area In the pixel sequence of edge pixels, the The gray value of each pixel; The target spot area In the pixel sequence of edge pixels, the The gray value of each pixel; The target spot area The blurriness of edge pixels.

[0080] Step S314: taking the product of the first distance parameter, the second distance parameter and the third distance parameter of the target spot area as the lens distance parameter of the target spot area.

[0081] After obtaining the first distance parameter, the second distance parameter and the third distance parameter, multiple analysis angles can be integrated to measure the distance characteristics between the target spot area and the lens, and the product of the first distance parameter, the second distance parameter and the third distance parameter of the target spot area is used as the lens distance parameter of the target spot area.

[0082] The lens distance parameter of the spot area closest to the laparoscope lens is the smallest, and the lens distance parameter of the spot area farthest from the laparoscope lens is the largest. The farther away from the lens, the closer the distance between the spot areas. The different inclination parameters between the laparoscope and the organ surface cause different changes in the lens distance parameters and the distance changes between the spot areas. Therefore, according to the changing characteristics of the lens distance parameters and the distance change characteristics between the spot areas, the inclination parameters of the laparoscope and the organ surface of the current organ surface image are obtained, providing a basis for the subsequent accurate correction of the organ surface image.

[0083] Preferably, in one embodiment of the present invention, considering that the larger the range of the lens distance parameter is, the larger the variation range of the lens distance parameter is, and the larger the inclination angle of the lens is, the range of the lens distance parameter is used as the first inclination angle sub-parameter, and the variation characteristics of the lens distance parameter are reflected by the range.

[0084] Considering that the lens distance parameter changes the most in the direction of lens tilt, it is easier to show the distance change characteristics between spot areas, so the center point of the spot area with the largest lens distance parameter is connected with the center point of the spot area with the smallest lens distance parameter, and the connecting line is called the target connecting line; the distance change characteristics of the center points of adjacent spot areas are analyzed on the target connecting line to obtain the second tilt angle sub-parameter;

[0085] The larger the first inclination sub-parameter is, the greater the change in the lens distance parameter is, and the greater the inclination angle between the laparoscope and the organ surface in the current organ surface image is. The larger the second inclination sub-parameter is, the greater the change in the distance between the spot areas is, and the greater the inclination angle is. Therefore, the product of the first inclination sub-parameter and the second inclination sub-parameter is taken as the inclination parameter.

[0086] Preferably, in one embodiment of the present invention, considering that the distance between the spot areas closest to the lens is the smallest, and the distance between the spot areas farthest from the lens is the largest, when the lens inclination angle is larger, the difference between the minimum distance and the maximum distance is larger, and using this characteristic, on the target line, the distance between the center point of the spot area with the largest lens distance parameter and the center point of the adjacent spot area is obtained as the first center distance, and the distance between the center point of the spot area with the smallest lens distance parameter and the center point of the adjacent spot area is obtained as the second center distance; after mapping the difference between the first center distance and the second center distance through a preset negative correlation mapping function, the mapping value is used as the second inclination angle sub-parameter;

[0087] As an example, the preset negative correlation mapping function is Function, the difference between the first center distance and the second center distance is calculated by After function mapping, the mapped value is used as the second inclination sub-parameter; It represents an exponential function with the natural constant e as the base, and x represents the independent variable.

[0088] The smaller the difference between the first center distance and the second center distance, the smaller the first center distance and the larger the second center distance, which reflects that the greater the difference between the minimum distance and the maximum distance, the greater the lens inclination parameter. After the function is negatively correlated and mapped, it serves as the second inclination sub-parameter to provide a basis for obtaining the inclination parameter.

[0089] It should be noted that when there are pixels in the spot area that belong to the target line, the spot area is marked as a spot area to be analyzed; when two spot areas to be analyzed have common edge pixels or two spot areas to be analyzed have adjacent edge pixels, the two spot areas to be analyzed are considered to be adjacent; when calculating the first center distance, it is considered that there may be multiple spot areas to be analyzed that are adjacent to the spot area with the largest lens distance parameter. At this time, the average value of the distances between the center point of the spot area with the largest lens distance parameter and the center points of all adjacent spot areas to be analyzed is calculated as the first center distance, and the second center distance is obtained in the same way.

[0090] Step S4: Taking the center point of the spot area with the smallest lens distance parameter as the origin, obtain the initial distance from each pixel in the image to the origin; according to the inclination parameter and the angle between the line connecting the origin and each pixel and the inclination direction of the laparoscope lens, correct the initial distance of each pixel to obtain the corrected distance of each pixel.

[0091] The de-distortion operation mainly uses a series of technical means to correct the image distortion caused by factors such as lens design, installation errors or inaccurate internal parameters of the camera, so as to make the image closer to the real scene. The embodiment of the present invention mainly corrects the tangential distortion in the image, mainly by adjusting the position distance of the pixel points in the image according to the lens distance of the laparoscope and the inclination angle of the laparoscope lens, so that the adjusted displayed image is closest to the real situation. Considering that the degree of tangential distortion is related to the distribution position of the pixel points, the closer the pixel points are to the lens, the smaller the degree of distortion. Therefore, the center point of the spot area with the smallest lens distance parameter is taken as the origin, and the initial distance from each pixel point in the image to the origin is obtained, and the initial position of each pixel point is determined, which is convenient for subsequent correction.

[0092] Considering that the imaging effect of the spot area closest to the lens is the best, no distortion correction is required. The farther the pixel is from the lens, the more correction is needed. At the same time, the angle between the line connecting the origin and each pixel and the inclination angle of the laparoscope lens also reflects the distance characteristics between the pixel and the lens, and then reflects the degree of distortion of the pixel. Therefore, according to the inclination parameter and the angle between the line connecting the origin and each pixel and the inclination direction of the laparoscope lens, the initial distance of each pixel is corrected to obtain the corrected distance of each pixel.

[0093] Preferably, in one embodiment of the present invention, considering that the change of the lens distance parameter is the largest in the direction of lens tilt, the center point of the spot area with the smallest lens distance parameter is pointed in the direction of the center point of the spot area with the largest lens distance parameter as the tilt direction of the laparoscope lens;

[0094] Considering that the farther the pixel is from the lens, the greater the degree of scaling, the greater the degree of distortion, and the greater the initial distance; at the same time, the farther the pixel is from the lens, the smaller the angle between the line connecting the origin and the pixel and the inclination direction, or in other words, when the angle between the line connecting the origin and the pixel and the inclination direction is small, the probability of the pixel being distributed farther from the lens is higher; also considering that the larger the inclination parameter, the greater the degree of scaling of the pixel, the greater the degree of distortion, and the more it is necessary to increase the initial distance for correction, based on this, any pixel is selected as the target pixel;

[0095] The angle between the direction away from the origin and the direction of the inclination angle on the line connecting the origin and the target pixel is used as the denominator of the ratio, the product of the initial distance and the inclination angle parameter is used as the numerator of the ratio, and the ratio is normalized and used as the correction coefficient;

[0096] The product of the sum of the correction coefficient and the constant 1 and the initial distance of the target pixel is taken as the corrected distance of the target pixel.

[0097] It should be noted that the pixel at the origin may not be involved in the correction, or the correction distance of the pixel at the origin may be considered to be 0; the direction away from the origin on the line connecting the origin and the target pixel may also be referred to as the direction from the origin to the target pixel.

[0098] Step S5: Obtain a dedistorted image based on the corrected distance of each pixel.

[0099] After obtaining the corrected distance of each pixel, the image can be corrected to remove the distortion effect and obtain a dedistorted image.

[0100] Preferably, in one embodiment of the present invention, considering that the tangential distortion will only affect or scale the position of the pixel point, the corresponding pixel point is moved in the direction of the line connecting the origin and each pixel point, and the distance between the moved pixel point and the origin point is to the corresponding correction distance, and the dedistorted image is obtained after all the pixel points are moved. As an example, the origin coordinates are (0, 0), the initial coordinates of a pixel point are (0, 5), and the correction distance is 10, then the coordinates of this pixel point after moving are (0, 10).

[0101] Preferably, in one embodiment of the present invention, considering that after the pixels are moved and corrected, there may be blank positions without pixels, in order to ensure the continuity of the image texture, after the pixels are moved, the method further includes: filling the blank positions with a cubic interpolation method. The cubic interpolation method is a technical means well known to those skilled in the art and will not be described in detail.

[0102] It should be noted that after obtaining the dedistorted image, the original organ surface image obtained by the laparoscope and the corrected dedistorted image are also transmitted to the computer for processing and displayed on the monitor to facilitate observation and analysis by relevant personnel.

[0103] It should be noted that the embodiment of the present invention does not involve a specific diagnosis process or method, but only performs distortion correction on the image.

[0104] An embodiment of the present invention also provides an image dedistortion system accompanying the laparoscopic surgery stage, the system comprising a memory, a processor and a computer program, wherein the memory is used to store the corresponding computer program, the processor is used to run the corresponding computer program, and when the computer program runs in the processor, it can implement an image dedistortion method accompanying the laparoscopic surgery stage described in steps S1-S5.

[0105] In summary, in order to solve the technical problem of inaccurate correction of tangential distortion of organ surface images acquired by laparoscope, the embodiment of the present invention first acquires the organ surface image acquired by laparoscope; further performs edge detection on the organ surface image and screens out spot areas; further acquires the lens distance parameters of each spot area and the corresponding inclination parameters of the laparoscope; further corrects the initial distance of each pixel point to obtain the corrected distance of each pixel point; and acquires the dedistorted image based on the corrected distance of each pixel point. The embodiment of the present invention analyzes the inclination characteristics of the laparoscope according to the distribution characteristics and grayscale characteristics of the organ surface features in the image; and then corrects the pixels according to the different positional relationships between the pixels and the laparoscope lens to reduce the influence of tangential distortion, so that the image can more accurately represent the real organ surface.

[0106] It should be noted that the sequence of the above embodiments of the present invention is only for description and does not represent the advantages and disadvantages of the embodiments. The processes depicted in the accompanying drawings do not necessarily require the specific order or continuous order shown to achieve the desired results. In some embodiments, multitasking and parallel processing are also possible or may be advantageous.

[0107] The various embodiments in this specification are described in a progressive manner, and the same or similar parts between the various embodiments can be referenced to each other, and each embodiment focuses on the differences from other embodiments.

Claims

1. A method for image dedistortion during laparoscopic surgery, characterized in that: The method comprises: obtain images of organ surfaces obtained through laparoscopy; Perform edge detection on the organ surface image to obtain segmented regions; select any of the segmented regions as a target segmented region; obtain the spot possibility of the target segmented region based on the area characteristics of the target segmented region, the distribution density characteristics of the target segmented region and adjacent segmented regions, and the grayscale difference characteristics of the edge pixels of the target segmented region and the adjacent pixels outside the target segmented region; and screen out the spot region based on the spot possibility; According to the grayscale change characteristics of the neighboring pixel points of the edge pixel points of each spot area, the grayscale characteristics inside the spot area, and the area characteristics of the spot area, the lens distance parameter of each spot area is obtained; according to the change characteristics of the lens distance parameter, combined with the distance change characteristics between the spot areas, the inclination angle parameters of the current organ surface image between the laparoscope and the organ surface are obtained; Taking the center point of the spot area with the smallest lens distance parameter as the origin, obtaining the initial distance from each pixel point in the organ surface image to the origin; correcting the initial distance of each pixel point according to the inclination parameter and the angle between the line connecting the origin and each pixel point and the inclination direction of the laparoscope lens to obtain the corrected distance of each pixel point; Acquire a dedistorted image based on the corrected distance of each pixel; The method for obtaining the spot probability includes: Taking the reciprocal of the number of pixels in the target segmentation area as a first possibility parameter; Taking the reciprocal of the sum of the distances between the center point of the target segmented region and the center points of the adjacent segmented regions as the second possibility parameter; Taking the line connecting the center point of the target segmentation area and each edge pixel point as the extension direction away from the center point, extending a preset first number of pixel points as the external neighboring pixel points of each edge pixel point; taking the sum of the gray value differences between all the external neighboring pixel points and the corresponding edge pixel point as the third possibility parameter; The product of the first possibility parameter, the second possibility parameter and the third possibility parameter corresponding to the target segmentation area is used as the spot possibility of the target segmentation area; The method for obtaining the lens distance parameter includes: Select any of the spot areas as the target spot area; use the number of pixels within the minimum circumscribed circle of the target spot area as the area parameter of the target spot area; and take the inverse of the area parameter as the first distance parameter; Taking the reciprocal of the grayscale value and value of all pixels in the target spot area as the second distance parameter; Select any edge pixel point of the target spot area as the target edge pixel point; take a line connecting the center point of the target spot area and the target edge pixel point, and obtain a pixel point sequence consisting of the target edge pixel point and the adjacent pixels on both sides of the line; in the pixel point sequence corresponding to the target edge pixel point, take the sum of the gray value differences of all adjacent pixels as the blurriness of the target edge pixel point; take the sum of the blurriness of all edge pixels of the target spot area as the third distance parameter; The product of the first distance parameter, the second distance parameter and the third distance parameter of the target spot area is used as the lens distance parameter of the target spot area; The method for obtaining the inclination parameter includes: Using the range of the lens distance parameter as the first tilt angle sub-parameter; Connecting the center point of the spot area with the largest lens distance parameter with the center point of the spot area with the smallest lens distance parameter, the connecting line is called a target connecting line; analyzing the distance variation characteristics of the center points of adjacent spot areas on the target connecting line to obtain a second tilt angle sub-parameter; Taking the product of the first tilt angle sub-parameter and the second tilt angle sub-parameter as the tilt angle parameter; The method for obtaining the second inclination sub-parameter includes: On the target line, obtaining the distance between the center point of the spot area with the largest lens distance parameter and the center point of the adjacent spot area as the first center distance, obtaining the distance between the center point of the spot area with the smallest lens distance parameter and the center point of the adjacent spot area as the second center distance; mapping the difference between the first center distance and the second center distance through a preset negative correlation mapping function, and using the mapping value as the second inclination sub-parameter; Pointing the center point of the spot area with the smallest lens distance parameter to the direction of the center point of the spot area with the largest lens distance parameter is used as the inclination direction of the laparoscope lens; The method for obtaining the corrected distance includes: Select any pixel point as the target pixel point; use the angle between the direction away from the origin and the direction of the inclination angle on the line connecting the origin and the target pixel point as the denominator of the ratio, use the product of the initial distance and the inclination angle parameter as the numerator of the ratio, and normalize the ratio as a correction coefficient; The product of the sum of the correction coefficient and the constant 1 and the initial distance of the target pixel point is used as the corrected distance of the target pixel point.

2. The method for image dedistortion during laparoscopic surgery according to claim 1, characterized in that: The method for acquiring a dedistorted image based on the corrected distance of each pixel point comprises: In the direction of the line connecting the origin and each pixel point, the corresponding pixel point is moved, and the distance between the moved pixel point and the origin point is the correction distance. After all the pixel points are moved, a dedistorted image is obtained.

3. The method for image dedistortion during laparoscopic surgery according to claim 2, characterized in that: After moving the pixel points, the method further includes: filling the blank positions with cubic interpolation.

4. An image dedistortion system accompanying a laparoscopic surgery stage, the system comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the steps of the image dedistortion method accompanying the laparoscopic surgery stage as described in any one of claims 1 to 3 are implemented.

Images