An endoscope measurement method and device based on image recognition

Abstract

The application relates to an endoscope measuring method and device based on image recognition. The endoscope measuring method based on image recognition comprises the following steps: S1. obtaining an object-image ratio. S2. obtaining a lesion image, and comparing the pixel value of the lesion image with a preset standard lesion pixel value. S3. repeating step S2 until the pixel value of the lesion image reaches the standard lesion pixel value. S4. performing image recognition processing on the lesion image to obtain a lesion center circle image, and establishing a cross line scale in the lesion center circle image. S5. dividing the lesion on the lesion center circle image into different two-dimensional grids, and calculating the area of a single two-dimensional grid. S6. calculating a virtual lesion area of the lesion through the area of the single two-dimensional grid. S7. calculating an actual lesion area through the object-image ratio. After processing the lesion image, the length, width or size of the lesion can be directly read, and the lesion area can be directly obtained.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of endoscopic measurement, in particular to an endoscopic measurement method based on image recognition and an endoscopic measurement device based on image recognition. BACKGROUND

[0002] Esophageal varices refers to the abnormal enlargement of esophageal veins, which is generally caused by blocked venous return. When the blood vessels in the esophagus rupture and bleed, symptoms such as vomiting blood, blackish and oily stool or blood stool, rapid heartbeat, cold and clammy skin, dizziness, etc. may occur. For a long time, when reading the size of the lesion image taken by an endoscope, the common methods are visual inspection, forceps method or instrument ruler, etc. The visual inspection method relies on the experience of medical staff, otherwise the data read is not accurate. The forceps method or instrument ruler is to read the data with the help of instruments, which is relatively cumbersome to operate. Although the common methods can read the length or width or diameter of the lesion, the area of the lesion cannot be immediately known, and it is also more troublesome to calculate by reading the numerical value. SUMMARY

[0003] Therefore, it is necessary to provide an endoscopic measurement method based on image recognition and an endoscopic measurement device based on image recognition to solve the problem that the area of the lesion in the lesion image cannot be conveniently obtained.

[0004] To achieve the above-mentioned purpose, the present application adopts the following technical solutions:

[0005] An endoscopic measurement method based on image recognition, the measurement method comprising the following steps:

[0006] S1. Adjust the endoscope so that the lateral distance between the lesion and the main optical axis of the endoscope is zero, fix the focal length of the endoscope, obtain the distance between the endoscope and the lesion, and calculate the object-image ratio;

[0007] S2. Take a picture of the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value;

[0008] S3. When the pixel value of the lesion image is less than the standard lesion pixel value, return to step S2 until the pixel value of the lesion image reaches the standard lesion pixel value;

[0009] S4. Perform image recognition processing on the lesion image to obtain a lesion center circle image, and establish a crosshair scale in the lesion center circle image;

[0010] S5. Perform edge processing on the lesion in the lesion center circle image to obtain an edge region, divide the edge region into different two-dimensional grids, refer to the crosshair scale to obtain a preliminary size of a single two-dimensional grid, obtain an error parameter, and add the error parameter and the preliminary size to calculate a virtual size;

[0011] S6. Calculate the area of a single two-dimensional grid by virtual size, count the number of two-dimensional grids, and multiply the number of two-dimensional grids by the area of a single two-dimensional grid to obtain the virtual lesion area of the lesion;

[0012] S7. Calculate the actual lesion area by the virtual lesion area through the object-image ratio, and the calculation formula is h / H = f / d, wherein h is the virtual lesion area, H is the actual lesion area, f is the focal length when the lesion image is shot, and d is the distance between the endoscope and the lesion.

[0013] Further, the calculation method of the error parameter comprises the following steps:

[0014] S100. Decompose the lesion center circle image into multiple paths, preset a pixel standard value, and remove the paths with a pixel value less than the pixel standard value;

[0015] S200. Approximate each remaining path as a polygon, and perform curve optimization on the polygon to obtain an optimal polygon;

[0016] S300. Convert the optimal polygon into a smooth vector contour, and integrate multiple vector contours to obtain a lesion center circle vector graph;

[0017] S400. Enlarge the lesion center circle vector graph according to a rated scale to obtain a vector enlarged graph, perform edge drawing processing on the lesion in the vector enlarged graph to obtain an edge drawing area of the vector enlarged graph, divide the edge drawing area of the vector enlarged graph into different two-dimensional grids, and refer to the cross line scale to obtain the accurate size of a single two-dimensional grid;

[0018] S500. Calculate the size error by difference between the preliminary size and the accurate size;

[0019] S600. Repeat step S100 until the number of repetitions reaches a preset number, and stop repeating;

[0020] S700. Calculate the average value of multiple size errors to obtain the error parameter.

[0021] Further, the cross line scale comprises the following steps:

[0022] Obtain the center of the lesion center circle image, and establish a cross line with the center as the center point;

[0023] Set a scale on the cross line that is proportionally matched with the radius value of the lesion center circle image to obtain the cross line scale.

[0024] In one embodiment, the image recognition processing specifically comprises the following steps:

[0025] Scanning each pixel in the lesion image with a Gaussian filter template, replacing the value of the center pixel of the template with the weighted average gray value of the pixels in the neighborhood determined by the template to obtain a denoising lesion image;

[0026] Calculating the gradient amplitude and gradient direction of the denoising lesion image, performing non-maximum suppression and double threshold processing on the gradient amplitude according to the gradient direction to obtain edge pixel points, and connecting the edge pixel points to obtain an edge detection lesion image;

[0027] Performing Hough circle transformation on the edge detection lesion image to obtain a lesion center circle image.

[0028] In one embodiment, the image recognition processing specifically includes the following steps:

[0029] S11. Converting the lesion image from RGB mode to Lab mode, extracting the blue color from the lesion image in Lab mode, and performing binaryzation processing to obtain a binaryzation lesion image;

[0030] S12. Performing median filter denoising on the binaryzation lesion image to obtain a denoising lesion image;

[0031] S13. Detecting all connected regions in the denoising lesion image using a region detection algorithm, filtering out connected regions with smaller areas, marking the remaining connected regions, and connecting the marked connected regions in clockwise order to obtain a lesion center circle image.

[0032] The application also includes an endoscope measurement device based on image recognition, which includes:

[0033] An object-image ratio calculation module for adjusting the lateral distance between the lesion and the main optical axis of the endoscope to zero, fixing the focal length of the endoscope, obtaining the distance between the endoscope and the lesion, and calculating the object-image ratio;

[0034] A pixel value comparison module for capturing the lesion to obtain a lesion image, and comparing the pixel value of the lesion image with a preset standard lesion pixel value;

[0035] A lesion image confirmation module for returning to the pixel value comparison module when the pixel value of the lesion image is less than the standard lesion pixel value until the pixel value of the lesion image reaches the standard lesion pixel value;

[0036] A lesion center circle image calculation module for performing image recognition processing on the lesion image to obtain a lesion center circle image, and establishing a crosshair scale on the lesion center circle image;

[0037] a virtual size calculation module, configured to perform an outline processing on the lesion on the lesion center circle image to obtain an outline area, divide the outline area into different two-dimensional grids, refer to a cross line scale to obtain a preliminary size of a single two-dimensional grid, obtain an error parameter, and add the error parameter and the preliminary size to obtain a virtual size;

[0038] a virtual lesion area calculation module, configured to calculate an area of the single two-dimensional grid through the virtual size, count a number of the two-dimensional grids, and multiply the number of the two-dimensional grids by the area of the single two-dimensional grid to obtain a virtual lesion area of the lesion;

[0039] an actual lesion area calculation module, configured to calculate an actual lesion area of the lesion through the virtual area and an object-image ratio, and a calculation formula is h / H=f / d, where h is the virtual lesion area, H is the actual lesion area, f is a focal length when the lesion image is captured, and d is a distance between the endoscope and the lesion.

[0040] Further, the measuring device further comprises an error parameter calculation module, configured to:

[0041] divide the lesion center circle image into a plurality of paths, preset a pixel standard value, and remove a path with a pixel less than the pixel standard value;

[0042] approximate each of the remaining paths as a polygon, perform curve optimization on the polygon to obtain an optimal polygon;

[0043] convert the optimal polygon into a smooth vector contour, and integrate a plurality of vector contours to obtain a lesion center circle vector graph;

[0044] enlarge the lesion center circle vector graph according to a rated ratio to obtain a vector enlarged graph, perform an outline processing on the lesion on the vector enlarged graph to obtain an outline area of the vector enlarged graph, divide the outline area of the vector enlarged graph into different two-dimensional grids, and refer to the cross line scale to obtain an accurate size of a single two-dimensional grid;

[0045] perform a difference calculation on the preliminary size and the accurate size to obtain a size error;

[0046] repeat the above operation until a preset number of repetitions is reached, and stop the repetition;

[0047] perform an average value calculation on a plurality of size errors to obtain the error parameter.

[0048] Further, the measuring device further comprises a cross line scale acquisition module, configured to:

[0049] acquire a center of the lesion center circle image, and establish a cross line with the center as a center point;

[0050] A scale is set on the cross line, which is proportionally matched with the radius value of the lesion center circle image, to obtain a cross line scale.

[0051] In one of the embodiments, the measuring device further comprises an image recognition processing module, which is configured to:

[0052] Each pixel in the lesion image is scanned by using a Gaussian filter template, and a weighted average gray value of the pixels in the neighborhood determined by the template is used to replace the value of the center pixel point of the template, to obtain a denoising lesion image.

[0053] The gradient amplitude and the gradient direction of the denoising lesion image are calculated, the non-maximum suppression and the double threshold processing are performed on the gradient amplitude according to the gradient direction, to obtain edge pixel points, and the edge pixel points are connected to obtain an edge detection lesion image.

[0054] The Hough circle transformation is performed on the edge detection lesion image to obtain a lesion center circle image.

[0055] In one of the embodiments, the measuring device further comprises an image recognition processing module, which is configured to:

[0056] The lesion image is converted from the RGB mode to the Lab mode, the blue color is extracted from the lesion image in the Lab mode, and a binaryzation processing is performed to obtain a binaryzation lesion image.

[0057] The binaryzation lesion image is subjected to a median filter denoising to obtain a denoising lesion image.

[0058] All connected regions in the denoising lesion image are detected by using a region detection algorithm, the connected regions with small areas are filtered out, the remaining connected regions are labeled, and the labeled connected regions are connected in a clockwise order to obtain a lesion center circle image.

[0059] The technical scheme provided by the present application has the following beneficial effects:

[0060] 1. The present application directly reads the size of the lesion on the image and obtains the actual area of the lesion according to the object image ratio by performing image recognition processing on the lesion image and then establishing a cross line scale, without the need for calculation after visual inspection or the use of instruments for reading, thereby simplifying the complexity of obtaining the size of the lesion and conveniently and effectively obtaining the area of the lesion.

[0061] 2. The present application fixes two variables by adjusting the position of the endoscope, thereby facilitating the calculation of the object image ratio and the convenient obtaining of the actual area of the lesion.

[0062] 3. The present application further improves the accuracy of the obtained area of the lesion by obtaining an error parameter, reduces the existence of errors, and improves the precision. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 A flow chart of an endoscope measurement method based on image recognition in the present application;

[0064] Figure 2 A schematic diagram of superimposing a virtual crosshair ruler on a lesion image based on Figure 1 DETAILED DESCRIPTION

[0065] The technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by a person of ordinary skill in the art without creative labor fall within the protection scope of the present application.

[0066] The present embodiment aims at the problem that the lesion area in a lesion image cannot be conveniently obtained, and provides an endoscope measurement method based on image recognition. The lesion image is subjected to image recognition processing, and then a crosshair ruler is established to directly obtain the lesion size on the image and obtain the actual lesion area according to the object-image ratio, thereby reducing the influence of barrel distortion of the image.

[0067] As shown in Figure 1 , the specific steps of the measurement method are as follows:

[0068] S1. Adjust the endoscope so that the lateral distance of the lesion from the main optical axis of the endoscope is zero, fix the focal length of the endoscope, obtain the distance between the endoscope and the lesion, and calculate the object-image ratio. According to geometric optics, the object-image ratio on both sides of the lens is determined by the focal length, the position of the real object deviating from the main optical axis, and the object distance. If a fixed endoscope (the focal length is constant) is used and the lesion is kept centered (the lateral distance from the main optical axis is zero) when shooting, the object distance is the only variable affecting the object-image ratio. To know the actual size of the lesion, only the distance from the lens end to the lesion (object distance) and the lesion size in the image need to be measured, and the object-image ratio can be calculated to achieve the actual size of the lesion. For the acquisition of the object distance, a sensor can also be installed on the endoscope to directly obtain the distance, or the end cap of the endoscope can be directly attached to the lesion to obtain a fixed object distance. The appropriate acquisition method can be selected according to the actual situation.

[0069] S2. Shoot the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value.

[0070] S3. When the pixel value of the lesion image is less than the standard lesion pixel value, return to step S2 until the pixel value of the lesion image reaches the standard lesion pixel value.

[0071] S4. Subject the lesion image to image recognition processing to obtain a lesion center circle image, and establish a crosshair ruler on the lesion center circle image, as shown in​Figure 2 The center of the lesion center circle image is obtained, a cross line is established with the center as the center point, a scale matching the radius value of the lesion center circle image is set on the cross line, and a cross line ruler is obtained. There are two ways for image recognition processing. The first way is to scan each pixel in the lesion image with a Gaussian filter template, replace the value of the template center pixel point with the weighted average gray value of the pixels in the neighborhood determined by the template to obtain a denoising lesion image. The gradient amplitude and gradient direction of the denoising lesion image are calculated, the gradient amplitude is suppressed according to the gradient direction, and a double threshold value is processed to obtain an edge pixel point. The edge pixel points are connected to obtain an edge detection lesion image. The Hough circle transformation is performed on the edge detection lesion image to obtain the lesion center circle image. The second way is to convert the lesion image from RGB mode to Lab mode, extract the blue color from the lesion image in Lab mode, and perform binaryzation processing to obtain a binaryzation lesion image. The binaryzation lesion image is subjected to median filter denoising to obtain a denoising lesion image. All connected regions in the denoising lesion image are detected by a region detection algorithm, and the connected regions with small areas are filtered out. The remaining connected regions are labeled and connected in a clockwise order to obtain a lesion center circle image. According to the two different ways of image processing, the lesion center circle image is ultimately obtained.

[0072] S5. The lesion on the lesion center circle image is subjected to edge processing to obtain an edge region. The edge region is divided into different two-dimensional grids. The preliminary size of a single two-dimensional grid is obtained by referring to the cross line ruler. An error parameter is obtained. The virtual size is calculated by adding the error parameter and the preliminary size. The specific acquisition of the error parameter is that the lesion center circle image is decomposed into multiple paths. A pixel standard value is preset. The paths with pixel values less than the pixel standard value are removed. Each remaining path is approximated as a polygon. The polygon is subjected to curve optimization to obtain an optimal polygon. The optimal polygon is converted into a smooth vector contour. The multiple vector contours are integrated to obtain a lesion center circle vector graph. The lesion center circle vector graph is enlarged by a rated proportion to obtain a vector enlarged graph. The lesion on the vector enlarged graph is subjected to edge processing to obtain an edge region of the vector enlarged graph. The edge region of the vector enlarged graph is divided into different two-dimensional grids. The accurate size of a single two-dimensional grid is obtained by referring to the cross line ruler. The size error is calculated by subtracting the preliminary size from the accurate size. The above operation is repeated until the number of repetitions reaches a preset number, and the repetition is stopped. The size errors are averaged to obtain the error parameter.

[0073] S6. The area of a single two-dimensional grid is calculated by the virtual size. The number of two-dimensional grids is counted. The virtual lesion area of the lesion is calculated by multiplying the number of two-dimensional grids by the area of a single two-dimensional grid.

[0074] S7. The actual lesion area is calculated by the virtual lesion area through the object-image ratio, and the calculation formula is h / H=f / d, wherein h is the virtual lesion area, H is the actual lesion area, f is the focal length when the lesion image is taken, and d is the distance between the endoscope and the lesion.

[0075] The so-called "object-image ratio" is a description of the ratio of the overall size of the object and image using the same unit of measurement. If the method of rounding off to zero is adopted, the reciprocal form is taken first, and the centimeter number of the image is converted into pixel value, i.e. a new expression form of the object-image ratio, "pixel / cm", is obtained. Such a parameter is the basic unit for measuring the size of the lesion.

[0076] The actual size of the lesion can be obtained by obtaining the object-image ratio and the size of the lesion imaging. If the area of the lesion is to be known, the area of the lesion imaging can be calculated, the lesion image is placed in a two-dimensional grid, the two-dimensional grid with the lesion image is counted, the length of the single two-dimensional grid is obtained according to the crosshair scale, the area of the single two-dimensional grid is calculated, and then the number is multiplied to obtain the imaging area of the lesion. The actual area of the lesion can be obtained through the object-image ratio. For the number and area of the two-dimensional grid, the number is as large as possible, and the area is as small as possible, so that the actual error is small.

[0077] For the acquisition of the error parameter, the detailed steps are as follows: the lesion center circle image is binarized, so that the image has only black and white colors, the bitmap is placed in a coordinate system, and each pixel corner has an integer coordinate. Further, it is assumed that the background is white and the foreground is black. According to the convention, the area outside the bitmap boundary is assumed to be filled with white. A bitmap is decomposed into a path, which starts from a pair of adjacent pixels with different colors, such as the leftmost black pixel in a certain row. The two selected pixels meet on an edge, and the direction of the edge is changed so that the black pixel is on the left side of the edge and the white pixel is on the right side. The path is continued to expand so that the new edges all have a black pixel on the left side and a white pixel on the right side. The direction relative to the path is along the edge between the pixels. Each time a corner is encountered, the path is straight or turns left or right until it returns to the starting point, i.e. a closed graph point is defined. The color of all pixels of the closed graph is reversed to remove it from the graph, and a new bitmap is defined. The graph is recursively continued until there is no black pixel left, and a set of closed paths is obtained.

[0078] The de-noising point can be found by removing all paths with less than a preset number of pixels inside the path, first finding all straight paths, then approximating all non-value paths into straight paths, forming a polygon from the closed path, using a variable of a standard graph theory algorithm to find the optimal ring in the directed graph, i.e. the optimal polygon, adjusting the position of the polygon vertex to make it as consistent as possible with the source bitmap, mainly calculating the corner and curve according to the length of the adjacent line segment and the included angle between them, using a standard graph theory algorithm to decompose the given curve segment to an acceptable approximation using the shortest path, optimizing the number of segments to achieve the purpose of curve optimization, and converting the optimal polygon into a smooth vector contour through a linear transformation.

[0079] In actual application, the error parameters of each measurement can be stored to establish a database, when the data is stored to a rated amount, the stored error parameters are subjected to big data analysis to obtain an error parameter reference table, when the lesion image is shot, the error parameters can be directly obtained from the error parameter reference table according to the pixel value of the image, without the need of evaluating the error parameters again, so as to obtain the required virtual lesion size or virtual lesion area, and the steps are simplified.

[0080] On the basis of the foregoing image recognition-based endoscope measurement method, an image recognition-based endoscope measurement device is also provided, which realizes the foregoing image recognition-based endoscope measurement method in operation, and comprises an object-image ratio calculation module, a pixel value comparison module, a lesion image confirmation module, a lesion center circle image calculation module, a virtual size calculation module, a virtual lesion area calculation module, and an actual lesion area calculation module.

[0081] Specifically, the object-image ratio calculation module is used to adjust the transverse distance between the lesion and the main optical axis of the endoscope to zero, fix the focal length of the endoscope, obtain the distance between the endoscope and the lesion, and calculate the object-image ratio.

[0082] The pixel value comparison module is used to shoot the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value.

[0083] The lesion image confirmation module is used to return to the pixel value comparison module when the pixel value of the lesion image is less than the standard lesion pixel value, until the pixel value of the lesion image reaches the standard lesion pixel value.

[0084] The lesion center circle image calculation module is used to perform image recognition processing on the lesion image to obtain a lesion center circle image, and establish a crosshair scale on the lesion center circle image; the crosshair scale acquisition module is further used to: acquire the center of the lesion center circle image, establish a crosshair with the center as the center point, and set a scale on the crosshair that is proportionally matched with the radius value of the lesion center circle image to obtain the crosshair scale.

[0085] The recognition device further comprises an image recognition processing module. The image recognition module comprises two modes, which are consistent with the foregoing. However, the first mode adopts a Gaussian filter, a Canny edge detection and a Hough circle detection. The Gaussian filter is a linear smoothing filter, which is suitable for eliminating Gaussian noise and is widely used in the noise reduction process of image processing. The Gaussian filter is a linear filter, which can effectively suppress noise and smooth an image. The working principle of the Gaussian filter is similar to that of the mean filter. Both of them take the mean value of pixels in the filter window as the output. The window template coefficients of the Gaussian filter are different from those of the mean filter. The template coefficients of the mean filter are all the same, that is, 1. However, the template coefficients of the Gaussian filter decrease with the increase of the distance from the center of the template. Therefore, compared with the mean filter, the Gaussian filter has a smaller degree of image blurring. In a popular way, the Gaussian filter is a process of weighted average of the entire image. The value of each pixel point is obtained by weighted average of the pixel value in the neighborhood. The Canny edge detection is used to extract the edge of an image. The working steps of the Canny edge detection are as follows. Firstly, the gradient amplitude and direction are calculated to estimate the edge strength and direction at each point. Then, the non-maximum suppression is performed on the gradient amplitude according to the gradient direction. In essence, the non-maximum suppression is a further refinement of the results of Sobel, Prewitt and other operators. Finally, the edge is processed and connected by using a double threshold. The Hough circle detection adopts a “Hough gradient method”. The detection idea of the Hough circle detection is to traverse and accumulate all non-zero points corresponding to the center of the circle. The idea is to consider that each non-zero pixel point on the image may be a point on a potential circle. Like the Hough line transformation, the Hough circle detection is also through voting to generate a cumulative coordinate plane and set a cumulative weight to locate the circle. Finally, the detection result is the coordinates and the radius of the circle in the image.

[0086] The second mode is a mode of HSV color binarization, median filter noise reduction and region detection. The HSV color binarization is to convert the color of the video image from the RGB mode to the Lab mode, then extract the blue color from the image, and binarize the picture. The median filter is used for noise reduction. The median filter is a nonlinear digital filter, which is often used to remove noise in images or other signals. The algorithm of the median filter is to sort the observed numbers in the filter window, and then select the median value as the output of the filter window. The region detection algorithm is used to find all the connected regions in the image. After filtering out the regions with small areas, the remaining regions are the marking regions on the pipe. After sorting the marking regions in a clockwise direction, the center line of the regions is found. The midpoint of the center line is the center of the circle. The radius is the distance between the intersection points of the center line and the boundary of the marking region.

[0087] The virtual size calculation module is configured to perform edge drawing processing on the lesion on the lesion center circle image to obtain an edge drawing region, divide the edge drawing region into different two-dimensional grids, refer to the cross line scale to obtain a preliminary size of a single two-dimensional grid, obtain an error parameter, and add the error parameter and the preliminary size to obtain a virtual size.

[0088] The virtual lesion area calculation module is configured to calculate an area of the single two-dimensional grid through the virtual size, count the number of the two-dimensional grids, and multiply the number of the two-dimensional grids by the area of the single two-dimensional grid to obtain a virtual lesion area of the lesion.

[0089] The actual lesion area calculation module is configured to calculate an actual lesion area through the virtual area and an object-image ratio, and a calculation formula is h / H=f / d, where h is the virtual lesion area, H is the actual lesion area, f is a focal length when the lesion image is captured, and d is a distance between the endoscope and the lesion.

[0090] In addition, a display module can be further configured to display the actually measured size or area in real time, so that the user can immediately know the measurement result.

[0091] The measurement device realizes the measurement method when in operation, performs image recognition processing on the lesion image, then establishes the cross line scale, directly obtains the size of the lesion on the image, and obtains the actual lesion area according to the object-image ratio, thereby reducing the influence of the barrel distortion of the image.

[0092] The technical features of the above embodiments can be combined in any manner. To make the description concise, not all possible combinations of the technical features in the above embodiments are described, but as long as the combinations of the technical features do not exist contradictions, they should be considered as the scope of the present disclosure.

[0093] The above embodiments only express several implementation manners of the present application, and the description is relatively specific and detailed, but it should not be understood as a limitation on the patent scope of the present application. It should be noted that, for ordinary skilled persons in the art, several modifications and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application should be subject to the appended claims.

Claims

1. An image recognition-based endoscopic measurement method for measuring the area of a lesion, characterized by, The measurement method comprises the following steps: S1. Adjust the endoscope to make the lesion and the transverse distance of the main optical axis of the endoscope zero, fix the focal length of the endoscope, obtain the distance between the endoscope and the lesion, and calculate the object-image ratio; S2. Take a picture of the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value; S3. When the pixel value of the lesion image is less than the standard lesion pixel value, return to step S2 until the pixel value of the lesion image reaches the standard lesion pixel value; S4. Perform image recognition processing on the lesion image to obtain a lesion center circle image, and establish a crosshair scale on the lesion center circle image; S5. Perform edge processing on the lesion on the lesion center circle image to obtain an edge region, divide the edge region into different two-dimensional grids, refer to the crosshair scale to obtain the preliminary size of a single two-dimensional grid, obtain an error parameter, and add the error parameter and the preliminary size to calculate a virtual size; S6. Calculate the area of a single two-dimensional grid through the virtual size, count the number of two-dimensional grids, and multiply the number of two-dimensional grids by the area of a single two-dimensional grid to calculate a virtual lesion area of the lesion; S7. Calculate the actual lesion area of the lesion through the virtual lesion area and the object-image ratio, and the calculation formula is h / H=f / d, wherein h is the virtual lesion area, H is the actual lesion area, f is the focal length when the lesion image is taken, and d is the distance between the endoscope and the lesion; The image recognition processing specifically comprises the following steps: Scan each pixel in the lesion image with a Gaussian filter template, replace the value of the center pixel point of the template with the weighted average gray value of the pixels in the neighborhood determined by the template to obtain a denoising lesion image; Calculate the gradient amplitude and gradient direction of the denoising lesion image, perform non-maximum suppression and double-threshold processing on the gradient amplitude according to the gradient direction to obtain edge pixel points, and connect the edge pixel points to obtain an edge detection lesion image; Perform Hough circle transformation on the edge detection lesion image to obtain the lesion center circle image.

2. The image recognition-based endoscopic measurement method according to claim 1, characterized by, The calculation method of the error parameter comprises the following steps: S100. Decompose the lesion center circle image into multiple paths, preset a pixel standard value, and remove the paths with a pixel value less than the pixel standard value; S200. Approximate each remaining path as a polygon, perform curve optimization on the polygon to obtain an optimal polygon; S300. Convert the optimal polygon into a smooth vector contour, and integrate multiple vector contours to obtain a lesion center circle vector graph; S400. Enlarge the lesion center circle vector graph according to a rated scale to obtain a vector enlarged graph, perform edge processing on the lesion on the vector enlarged graph to obtain an edge region of the vector enlarged graph, divide the edge region of the vector enlarged graph into different two-dimensional grids, and refer to the crosshair scale to obtain the accurate size of a single two-dimensional grid; S500. Calculate the difference between the preliminary size and the accurate size to obtain a size error. S600. Repeat step S100 until the number of repetitions reaches a preset number, and stop repeating. S700. Calculate the average of the plurality of size errors to obtain the error parameter.

3. The image recognition-based endoscopic measurement method according to claim 1, characterized by, The crosshair ruler includes the following steps: Obtain the center of the lesion center circle image, and establish a crosshair with the center as the center point; Set a scale on the crosshair that is proportionally matched with the radius value of the lesion center circle image to obtain a crosshair ruler.

4. An image recognition-based endoscopic measurement method for measuring an area of a lesion, characterized by, The measurement method includes the following steps: S1. Adjust the endoscope so that the lateral distance between the lesion and the main optical axis of the endoscope is zero, fix the focal length of the endoscope, obtain the distance between the endoscope and the lesion, and calculate the object-image ratio; S2. Capture the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value; S3. When the pixel value of the lesion image is less than the standard lesion pixel value, return to step S2 until the pixel value of the lesion image reaches the standard lesion pixel value; S4. Perform image recognition processing on the lesion image to obtain a lesion center circle image, and establish a crosshair ruler on the lesion center circle image; S5. Perform edge detection processing on the lesion on the lesion center circle image to obtain an edge region, divide the edge region into different two-dimensional grids, refer to the crosshair ruler to obtain a preliminary size of a single two-dimensional grid, obtain an error parameter, and add the error parameter and the preliminary size to calculate a virtual size; S6. Calculate the area of a single two-dimensional grid through the virtual size, count the number of two-dimensional grids, and multiply the number of two-dimensional grids by the area of a single two-dimensional grid to calculate a virtual lesion area of the lesion; S7. Calculate the actual lesion area of the lesion through the virtual lesion area and the object-image ratio, and the calculation formula is h / H = f / d, where h is the virtual lesion area, H is the actual lesion area, f is the focal length when the lesion image is captured, and d is the distance between the endoscope and the lesion; The image recognition processing specifically includes the following steps: S11. Convert the lesion image from an RGB mode to an Lab mode, extract blue from the lesion image in the Lab mode, and perform binaryzation processing to obtain a binaryzation lesion image; S12. Perform median filter noise reduction on the binaryzation lesion image to obtain a noise reduction lesion image; S13. Detect all connected regions in the noise reduction lesion image using a region detection algorithm, filter out connected regions with small areas, mark the remaining connected regions, connect the marked connected regions in a clockwise order, and obtain the lesion center circle image.

5. The image recognition-based endoscopic measurement method according to claim 4, characterized by, The error parameter calculation method includes the following steps: S100. Decompose the lesion center circle image into a plurality of paths, preset a pixel standard value, and remove paths with a pixel value less than the pixel standard value; S200. Approximate each remaining path as a polygon, perform curve optimization on the polygon to obtain an optimal polygon; S300. Convert the optimal polygon into a smooth vector contour, and integrate a plurality of vector contours to obtain a lesion center circle vector graph; S400. Enlarge the lesion center circle vector diagram by a nominal scale to obtain a vector enlarged diagram, perform an outline processing on the lesion on the vector enlarged diagram to obtain an outline area of the vector enlarged diagram, divide the outline area of the vector enlarged diagram into different two-dimensional grids, and refer to the cross line scale to obtain the accurate size of a single two-dimensional grid; S500. Calculate the difference between the preliminary size and the accurate size to obtain a size error; S600. Repeat step S100 until the number of repetitions reaches a preset number, and stop repeating; S700. Calculate the average of a plurality of size errors to obtain the error parameter.

6. The image recognition-based endoscopic measurement method according to claim 4, characterized by, The cross line scale comprises the following steps: Obtain the center of the lesion center circle image, and establish a cross line with the center as a center point; Set a scale on the cross line that is proportionally matched with the radius value of the lesion center circle image to obtain a cross line scale.

7. An image recognition-based endoscopic measuring device for measuring the area of a lesion, characterized by The measuring device comprises: An object-image scale calculation module for adjusting the transverse distance between the lesion and the main optical axis of the endoscope to zero, fixing the focal length of the endoscope, obtaining the distance between the endoscope and the lesion, and calculating the object-image scale; A pixel value comparison module for shooting the lesion to obtain a lesion image, comparing the pixel value of the lesion image with a preset standard lesion pixel value; A lesion image confirmation module for returning to the pixel value comparison module when the pixel value of the lesion image is less than the standard lesion pixel value until the pixel value of the lesion image reaches the standard lesion pixel value; A lesion center circle image calculation module for performing image recognition processing on the lesion image to obtain a lesion center circle image, and establishing a cross line scale on the lesion center circle image; A virtual size calculation module for performing an outline processing on the lesion on the lesion center circle image to obtain an outline area, dividing the outline area into different two-dimensional grids, referring to the cross line scale to obtain a preliminary size of a single two-dimensional grid, obtaining an error parameter, and adding the error parameter and the preliminary size to calculate a virtual size; A virtual lesion area calculation module for calculating the area of a single two-dimensional grid through the virtual size, counting the number of two-dimensional grids, and multiplying the number of two-dimensional grids by the area of a single two-dimensional grid to calculate a virtual lesion area of the lesion; An actual lesion area calculation module for calculating an actual lesion area from the virtual area through the object-image scale, and the calculation formula is h / H=f / d, wherein h is the virtual lesion area, H is the actual lesion area, f is the focal length when the lesion image is shot, and d is the distance between the endoscope and the lesion; The measuring device further comprises an image recognition processing module for: Scanning each pixel in the lesion image with a Gaussian filter template, replacing the value of the center pixel point of the template with the weighted average gray value of the pixels in the neighborhood determined by the template to obtain a denoising lesion image; The gradient amplitude and the gradient direction of the denoising lesion image are calculated, non-maximum suppression and double threshold processing are performed on the gradient amplitude according to the gradient direction, edge pixel points are obtained, and the edge detection lesion image is obtained by connecting the edge pixel points; The edge detection lesion image is subjected to Hough circle transformation to obtain the lesion center circle image.

8. The image recognition based endoscopic measurement device of claim 7, wherein, The measurement device further comprises an error parameter calculation module configured to: The lesion center circle image is decomposed into a plurality of paths, a preset pixel standard value is set, and paths with pixel values less than the pixel standard value are removed; Each of the remaining paths is approximated as a polygon, the polygon is subjected to curve optimization to obtain an optimal polygon; The optimal polygon is converted into a smooth vector contour, and a plurality of vector contours are integrated to obtain a lesion center circle vector graph; The lesion center circle vector graph is enlarged according to a predetermined scale to obtain a vector enlarged graph, the lesion on the vector enlarged graph is subjected to edge processing to obtain an edge region of the vector enlarged graph, and the edge region of the vector enlarged graph is divided into different two-dimensional grids, and the accurate size of a single two-dimensional grid is obtained by referring to the cross line scale; The preliminary size and the accurate size are subjected to difference calculation to obtain a size error; The above operation is repeated until the number of repetitions reaches a preset number, and the repetition is stopped; The size errors are subjected to average value calculation to obtain the error parameter.

9. The image recognition based endoscopic measurement device of claim 7, wherein, The measurement device further comprises a cross line scale acquisition module configured to: acquire the center of the lesion center circle image, and establish a cross line with the center as a center point; set a scale on the cross line that is proportionally matched with the radius value of the lesion center circle image to obtain a cross line scale.

10. An image recognition-based endoscopic measuring device for measuring the area of a lesion, characterized in that, The measurement device comprises: an object image scale calculation module configured to adjust the transverse distance between the lesion and the main optical axis of the endoscope to zero, fix the focal length of the endoscope, acquire the distance between the endoscope and the lesion, and calculate the object image scale; a pixel value comparison module configured to capture the lesion to obtain a lesion image, and compare the pixel value of the lesion image with a preset standard lesion pixel value; a lesion image confirmation module configured to return to the pixel value comparison module when the pixel value of the lesion image is less than the standard lesion pixel value until the pixel value of the lesion image reaches the standard lesion pixel value; a lesion center circle image calculation module configured to perform image recognition processing on the lesion image to obtain a lesion center circle image, and establish a cross line scale on the lesion center circle image; a virtual size calculation module configured to perform edge processing on the lesion on the lesion center circle image to obtain an edge region, divide the edge region into different two-dimensional grids, and obtain the preliminary size of a single two-dimensional grid by referring to the cross line scale, acquire an error parameter, and calculate a virtual size by adding the error parameter and the preliminary size; a virtual lesion area calculation module configured to calculate the area of a single two-dimensional grid through the virtual size, count the number of two-dimensional grids, and calculate the virtual lesion area of the lesion by multiplying the number of two-dimensional grids by the area of a single two-dimensional grid. An actual lesion area calculation module is configured to calculate an actual lesion area by the virtual area and the object-image ratio, and a calculation formula is h / H=f / d, wherein h is the virtual lesion area, H is the actual lesion area, f is a focal length when the lesion image is captured, and d is a distance between the endoscope and the lesion; The measurement device further includes an image recognition processing module configured to: convert the lesion image from an RGB mode to a Lab mode, extract blue from the lesion image in the Lab mode, and perform a binaryzation process to obtain a binaryzation lesion image; perform a median filter denoising process on the binaryzation lesion image to obtain a denoised lesion image; detect all connected regions in the denoised lesion image using a region detection algorithm, filter out connected regions with small areas, mark the remaining connected regions, and connect the marked connected regions in a clockwise order to obtain the lesion center circle image.

11. The image recognition based endoscopic measurement device of claim 10, wherein, The measurement device further includes an error parameter calculation module configured to: divide the lesion center circle image into multiple paths, preset a pixel standard value, and remove paths with a pixel value less than the pixel standard value; approximate each of the remaining paths as a polygon, perform a curve optimization process on the polygon to obtain an optimal polygon, and convert the optimal polygon into a smooth vector contour; integrate multiple vector contours to obtain a lesion center circle vector graph; enlarge the lesion center circle vector graph according to a rated scale to obtain a vector enlarged graph, perform an edge drawing process on the lesion in the vector enlarged graph to obtain an edge drawing area of the vector enlarged graph, divide the edge drawing area of the vector enlarged graph into different two-dimensional grids, and obtain accurate dimensions of a single two-dimensional grid by referring to the cross line scale; perform a difference calculation on the preliminary dimensions and the accurate dimensions to obtain a dimension error; repeat the above operations until a preset number of repetitions is reached, and stop repeating; perform an average value calculation on multiple dimension errors to obtain the error parameter.

12. The image recognition based endoscopic measurement device of claim 10, wherein, The measurement device further includes a cross line scale acquisition module configured to: acquire a center of the lesion center circle image, and establish a cross line with the center as a center point; set a scale on the cross line in a proportionally matched manner with a radius value of the lesion center circle image to obtain a cross line scale.

Images