Image Processing Method and Image Processing System

Abstract

The present disclosure provides an image processing method and an image processing system. The image processing method of the present disclosure includes: obtaining an image captured by an endoscope, where the image includes an optical fiber; determining a target pixel size of the optical fiber in the image; determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber; using the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image; adding a target line segment to the head of the optical fiber in the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and displaying the image after adding the target line segment, where the target line segment is used to measure the actual size of a calculus when the calculus contacted by the head of the optical fiber is also included in the image.

Description

Technical Field

[0001] The present disclosure relates to the technical fields such as medical image processing, and particularly relates to an image processing method and an image processing system. Background Art

[0002] With the continuous progress of medical technology, endoscopic technology has been widely used in minimally invasive surgery, especially in the field of treatment of urinary system stones. Through its high-resolution imaging ability, the endoscope enables doctors to directly observe the lesion site in the body and perform precise operations.

[0003] In endoscopic surgery, accurately measuring the actual size of the stone is crucial for selecting an appropriate treatment plan (such as lithotripsy or direct removal). However, due to factors such as the viewing angle, focal length, and distortion of endoscopic images, it is difficult to accurately judge the actual size of the stone from the endoscopic image solely by the naked eye. Summary of the Invention

[0004] The present disclosure provides an image processing method and an image processing system.

[0005] According to one aspect of the present disclosure, there is provided an image processing method, including:

[0006] Obtaining an image captured by an endoscope, wherein the image includes an optical fiber;

[0007] Determining a target pixel size of the optical fiber in the image;

[0008] Predicting a pixel length of an actual unit length in the image according to the target pixel size and the actual size of the optical fiber;

[0009] Adding a target line segment to the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image;

[0010] Displaying the image after adding the target line segment, wherein the target line segment is used to measure the actual size of the stone when the stone in contact with the head of the optical fiber is also included in the image.

[0011] According to the image processing method of at least one embodiment of the present disclosure, determining the target pixel size of the optical fiber in the image includes:

[0012] Performing contour extraction on the content in the image to obtain the contour information of the optical fiber; and

[0013] Determining the target pixel size of the optical fiber in the image according to the contour information of the optical fiber.

[0014] An image processing method according to at least one embodiment of the present disclosure extracts the contour of the content in the image to obtain the contour information of the optical fiber, including:

[0015] Extract the contour of the content in the image to obtain a plurality of contour information;

[0016] Filter the plurality of contour information according to the characteristic information of the optical fiber to obtain the contour information of the optical fiber.

[0017] According to the image processing method of at least one embodiment of the present disclosure, the characteristic information of the optical fiber includes a contour size range, a target hue, and a target contour position;

[0018] Filter the plurality of contour information according to the characteristic information of the optical fiber to obtain the contour information of the optical fiber, including:

[0019] Use the contour information among the plurality of contour information whose contour size is within the contour size range, the hue is the target hue, and the contour position is the target contour position as the contour information of the optical fiber.

[0020] According to the image processing method of at least one embodiment of the present disclosure, determining the target pixel size of the optical fiber in the image according to the contour information of the optical fiber includes:

[0021] Determine the coordinates of two target points of the optical fiber from the contour information of the optical fiber;

[0022] Based on the coordinates of the two target points, calculate the distance between the two target points;

[0023] Use the distance as the target pixel size of the optical fiber in the image.

[0024] According to the image processing method of at least one embodiment of the present disclosure, predicting the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber includes:

[0025] Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber;

[0026] Use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0027] According to the image processing method of at least one embodiment of the present disclosure, the target pixel size includes the head pixel length of the optical fiber, and the actual size of the optical fiber includes the head actual width of the optical fiber;

[0028] Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber, including:

[0029] Use the quotient of the actual width of the head and the pixel length of the head as the actual size represented by a single pixel in the image.

[0030] According to the image processing method of at least one embodiment of the present disclosure, the target pixel size includes the body pixel length of the optical fiber and the head pixel length of the optical fiber, and the actual size of the optical fiber includes the preset actual body length of the optical fiber;

[0031] Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber, including:

[0032] Based on the mapping relationship between the preset actual body length and the preset body pixel length, determine the actual body length corresponding to the body pixel length;

[0033] Divide the product of the head pixel length and the actual body length by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image.

[0034] According to the image processing method of at least one embodiment of the present disclosure, adding a target line segment to the head of the optical fiber in the image includes:

[0035] Taking the head of the optical fiber in the image as the starting point, draw the target line segment in four directions of 0°, 90°, 180° and 270° respectively.

[0036] According to the image processing method of at least one embodiment of the present disclosure, determine the target pixel size of the optical fiber in the image, including:

[0037] In the case of detecting a measurement requirement for the actual size of the stone, determine the target pixel size of the optical fiber in the image.

[0038] According to another aspect of the present disclosure, there is provided an image processing system, including:

[0039] Optical fiber;

[0040] An endoscope capable of taking images;

[0041] A display screen capable of displaying the image;

[0042] A controller, configured to determine a target pixel size of the optical fiber in the image when the optical fiber enters the shooting range of the endoscope; predict a pixel length of an actual unit length in the image according to the target pixel size and the actual size of the optical fiber; add a target line segment to the head of the optical fiber in the image; and control the display screen to display the image with the target line segment added, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used to measure the actual size of a calculus when the calculus contacted by the head of the optical fiber is further included in the image. Description of the Drawings

[0043] The drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, are used to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are included in this specification and form a part of this specification.

[0044] Figure 1 It is a schematic flowchart of an image processing method according to an embodiment of the present disclosure.

[0045] Figure 2 It is a schematic diagram of the process of determining the target pixel size of the optical fiber according to an embodiment of the present disclosure.

[0046] Figure 3 It is a schematic diagram of the process of determining the contour information of the optical fiber according to an embodiment of the present disclosure.

[0047] Figure 4 It is a schematic diagram of the process of determining the target pixel size of the optical fiber according to another embodiment of the present disclosure.

[0048] Figure 5 It is a schematic diagram of the process of determining the pixel length of the actual unit length according to an embodiment of the present disclosure.

[0049] Figure 6 It is a schematic diagram of the process of determining the actual size represented by a single pixel in the image according to an embodiment of the present disclosure.

[0050] Figure 7 It is an exemplary schematic diagram of the image with the target line segment added according to an embodiment of the present disclosure.

[0051] Figure 8 It is a schematic flowchart of an image processing method according to another embodiment of the present disclosure.

[0052] Figure 9 It is a schematic block diagram of the structure of an image processing system according to an embodiment of the present disclosure. Detailed Embodiments

[0053] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It can be understood that the specific examples described herein are only used to explain the relevant content, rather than limiting the present disclosure. Additionally, it should be noted that for ease of description, only the parts related to the present disclosure are shown in the drawings.

[0054] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The technical solutions of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0055] The actual size of the calculus judged solely by the naked eye from the endoscopic image often has a large error, which poses challenges to the success rate and safety of the surgery. In the related art, the measurement of the actual size of the calculus relies on preoperative imaging examinations (such as CT or X-ray). However, the preoperative images cannot reflect the real-time state of the calculus during the operation. Taking the endoscopic laser lithotripsy surgery as an example, after the calculus is affected by the laser energy, it may produce fragmented particles with significantly different sizes, and the particle size distribution has significant randomness. If the actual size of the fragmented particles cannot be determined in real time, it may lead to the residue of fragmented particles with a larger diameter, which may cause the recurrence of the disease in the future. Therefore, it is crucial to measure the actual size of the calculus in real time.

[0056] For this reason, the present disclosure proposes an image processing method.

[0057] The image processing method of the present disclosure can be used to measure the actual size of the calculus in real time through the endoscopic image during the endoscopic examination or surgery. In the present disclosure, the electronic device includes but is not limited to mobile phones, tablet computers, laptop computers, personal computers, etc.

[0058] For ease of description and to make the technical solutions of the specific embodiments of the present disclosure easier to understand, before describing the image processing method implemented in the present disclosure, the technical terms involved in the specific embodiments of the present disclosure are explained as follows:

[0059] Size refers to the length and size of an object.

[0060] Length is a measure of one-dimensional space and is the distance from point to point. Usually, when measuring the side length of a line segment in a two-dimensional space, the one with a larger numerical value of the length is called the length, and the one that is not larger than its value or is on the "side" is called the width. Width is also a kind of length measurement.

[0061] The target pixel size refers to the number of pixels occupied by the target content in the image in the target direction.

[0062] The pixel length refers to the number of pixels occupied by a certain part of the content in the image in the length direction of that part of the content.

[0063] Pixel width refers to the number of pixels occupied by a certain part of the content in the width direction of that part of the content in the image.

[0064] Actual size, actual length, and actual width respectively refer to the size, length, and width of an object in the real world (such as the world geodetic coordinate system).

[0065] Unit length is a reference standard set artificially. The unit length is the available reference standard, and it has no fixed value and varies according to the setting.

[0066] Figure 1 The overall flowchart of the image processing method M100 according to an embodiment of the present disclosure is shown. As Figure 1 The method shown includes steps S110 to S150. Among them, the method can be executed by an electronic device such as a mobile phone or a computer.

[0067] Specifically, Figure 1 The method shown includes:

[0068] S110. Obtain the image captured by the endoscope, where the image includes an optical fiber.

[0069] In one example, the optical fiber is a holmium laser optical fiber.

[0070] S120. Determine the target pixel size of the optical fiber in the image.

[0071] The target pixel size of the optical fiber in the image is the number of pixels occupied by the optical fiber in the target direction of the optical fiber in the image. The target direction can be set according to the actual situation. For example, it can be the length direction of the head of the optical fiber, the length direction of the body of the optical fiber, etc.

[0072] Exemplarily, the target pixel size of the optical fiber in the image can be determined by an image recognition algorithm or an image recognition model in related technologies.

[0073] S130. Predict the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber.

[0074] The actual size of the optical fiber is the size of the optical fiber in the real world (such as the world geodetic coordinate system). The actual size of the optical fiber can be calibrated in advance.

[0075] The actual unit length is the unit length in the real world (such as the world geodetic coordinate system). The actual unit length can be the stone safety size threshold. For example, if the actual size of the stone is less than or equal to the safety size threshold, the stone may not be crushed; the actual unit length can also be other artificially set values, such as 0.1 mm, 0.5 mm, 1 mm, 5 mm, 1 cm, etc., which are not limited herein.

[0076] Since the actual size of the optical fiber is a known quantity, by comparing the actual size of the optical fiber with the target pixel size of the optical fiber, the correlation between the pixel size in the image and the actual size in the real world can be known, thus facilitating the prediction of the pixel length of the actual unit length in the image.

[0077] It should be noted that the specific values mentioned in this disclosure are only for detailed illustration of the implementation of this disclosure as examples and should not be construed as a limitation of this disclosure. In other examples, embodiments or implementations, other values can be selected according to this disclosure, and no specific limitation is made here.

[0078] S140. Add a target line segment to the head of the optical fiber in the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image.

[0079] The pixel width of the target line segment can be a pre-calibrated default value, and this disclosure does not limit the pixel width of the target line segment.

[0080] Exemplarily, the number of target line segments can be one or multiple. Multiple target line segments can be distributed in the image along the same direction. For example, all target line segments are connected end to end along the same direction; multiple target line segments can also be distributed in the image along multiple directions. For example, some of the target line segments are connected end to end along the first direction while the remaining target line segments are connected end to end along the second direction, and no limitation is made here.

[0081] At least one target line segment can have the head of the optical fiber as the midpoint, or have the head of the optical fiber as the starting point, or have the head of the optical fiber as an intermediate point other than the midpoint and the endpoint, or have a pixel point within the preset pixel length range of the head of the optical fiber as the midpoint, or have a pixel point within the preset pixel length range of the head of the optical fiber as the starting point, or have a pixel point within the preset pixel length range of the head of the optical fiber as an intermediate point other than the midpoint and the endpoint, and no limitation is made here.

[0082] S150. Display the image after adding the target line segment, where the target line segment is used to measure the actual size of the calculus when the image also includes the calculus contacted by the head of the optical fiber.

[0083] Since the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, the target line segment can act as a scale, thus facilitating the measurement of the actual size of the calculus contacted by the head of the optical fiber through the target line segment. In this disclosure, "contacted" can be understood as the distance between at least part of their positions being less than or equal to the preset distance threshold. Since the target line segment is equivalent to a scale, in addition to the calculus, the target line segment can also be used to measure the actual size of other items contacted by the head of the optical fiber, and the specific measurement method can be the same as the calculus measurement method.

[0084] The orientation of the target line segment in the image can be fixed or dynamically adjusted, which is not limited herein. Exemplarily, a plurality of preset orientations are determined in advance. Then, during the process of displaying the image with the target line segment added, if no operation for adjusting the orientation of the target line segment is received, the target line segment is displayed in the default orientation among the plurality of preset orientations in the image; if an operation for adjusting the orientation of the target line segment is received (for example, the user rotates the orientation adjustment knob of the target line segment or selects a preset orientation), the orientation of the target line segment displayed in the image is switched to a preset orientation other than the current orientation among the plurality of preset orientations. In this way, the dynamic adjustment of the orientation of the target line segment is realized, which can be applied to the actual size measurement of stones with different shapes and positions, improving the accuracy of the actual size measurement of stones.

[0085] The image processing method according to the embodiments of the present disclosure obtains an image including an optical fiber captured by an endoscope, determines the target pixel size of the optical fiber in the image, predicts the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber, adds a target line segment to the head of the optical fiber in the image, and displays the image with the target line segment added. Thus, by using the optical fiber as a reference, the accurate conversion between the actual unit length and the pixel length in the image is realized. Moreover, the target line segment added to the head of the optical fiber in the image has the same pixel length as the actual unit length, enabling the user to intuitively measure the actual size of the stone contacted by the head of the optical fiber in the image with the target line segment added, greatly simplifying the operation process of the user, significantly improving the operation efficiency, and at the same time improving the measurement accuracy, providing more efficient and accurate technical support for medical work. In addition, since an optical fiber is required for laser energy delivery during the endoscopic laser lithotripsy surgery process, and the image processing method according to the embodiments of the present disclosure is based on the optical fiber, that is, in the present disclosure, no additional tool needs to be introduced during the surgery process, realizing the rapid, intuitive, and real-time measurement of the actual size of the stone by using the existing tools in the surgery process, reducing the operation complexity and the infection risk.

[0086] Regarding step S120, in some embodiments of the present disclosure, it may include steps S121 and S122 as Figure 2 shown.

[0087] S121. Extract the contour of the content in the image to obtain the contour information of the optical fiber.

[0088] In the image captured by the endoscope, there may be human tissues, stones, etc. in addition to the optical fiber. Through contour extraction, the contour of the optical fiber in the image can be automatically determined, providing data support for determining the target pixel size of the optical fiber.

[0089] Exemplarily, the content in the image can be subjected to contour extraction through edge detection algorithms, image segmentation algorithms, or contour extraction models in related technologies, so as to obtain the contour information of the optical fiber.

[0090] S122. Determine the target pixel size of the optical fiber in the image according to the contour information of the optical fiber.

[0091] The contour information of the optical fiber can characterize the boundary position of the optical fiber in the image, such as the coordinates of the pixel points representing the boundary of the optical fiber in the image. Therefore, after determining the contour information of the optical fiber, the target pixel size of the optical fiber in the image can be determined through the contour information.

[0092] The image processing method of the above embodiment performs contour extraction on the content in the image to obtain the contour information of the optical fiber, and determines the target pixel size of the optical fiber in the image according to the contour information of the optical fiber. Thus, the contour of the optical fiber in the image is automatically recognized, avoiding cumbersome manual annotation, and improving the efficiency and consistency of measuring the target pixel size of the optical fiber. At the same time, by using the extracted contour to determine the target pixel size, interference from other content in the image can be avoided, improving the accuracy of the determined target pixel size, and providing reliable data for predicting the pixel length of the actual unit length in the image subsequently.

[0093] Regarding step S121, in some embodiments of the present disclosure, it may include steps S1211 and S1212 as Figure 3 shown.

[0094] S1211. Perform contour extraction on the content in the image to obtain multiple contour information.

[0095] Exemplarily, the image can be first converted from the RGB color space to a grayscale image to reduce the computational complexity and highlight the key features of the image; then, Gaussian blur is applied to the converted grayscale image to smooth the image and reduce the influence of noise on subsequent processing steps; then, the pixel values in the area where the pixel values in the Gaussian-blurred grayscale image are greater than the set threshold are set to 255, and the pixel values in the area where the pixel values in the Gaussian-blurred grayscale image are less than or equal to the set threshold are set to 0, so as to convert the Gaussian-blurred grayscale image into a binary image; finally, edge detection is performed on the content in the binary image to obtain multiple contour information.

[0096] S1212. Filter the multiple contour information according to the characteristic information of the optical fiber to obtain the contour information of the optical fiber.

[0097] The characteristics of the multiple contour information are different. Therefore, based on the characteristic information of the optical fiber, filtering the multiple contour information can accurately determine the contour information of the optical fiber from the multiple contour information.

[0098] The image processing method of the above embodiment extracts contour information of the content in the image to obtain a plurality of contour information, and filters the plurality of contour information according to the characteristic information of the optical fiber to obtain the contour information of the optical fiber. Thus, the contour information of the optical fiber can be accurately separated from the complex image, effectively reducing the interference of irrelevant contours, improving the robustness and reliability of optical fiber contour recognition, and is particularly suitable for the light changes or background clutter that may exist in endoscopic images.

[0099] Regarding step S1212, in some embodiments of the present disclosure, the characteristic information of the optical fiber includes a contour size range, a target color tone, and a target contour position. Correspondingly, step S1212 may specifically be: taking the contour information among the plurality of contour information whose contour size is within the contour size range, the color tone is the target color tone, and the contour position is the target contour position as the contour information of the optical fiber.

[0100] The characteristic information of the optical fiber can be pre-configured according to the actual situation.

[0101] Exemplarily, the contour information outside the contour size range among the plurality of contour information can be filtered out first to obtain the contour information after the first filtering; then the contour information whose color tone is not the target color tone among the contour information after the first filtering can be filtered out to obtain the contour information after the second filtering; finally, the contour information whose contour position is not the target contour position among the contour information after the second filtering can be filtered out, and the remaining contour information at this time is the contour information of the optical fiber.

[0102] The image processing method of the above embodiment filters a plurality of contour information based on the contour size range, the target color tone, and the target contour position, so as to ensure that the obtained contour information after filtering highly coincides with the actual characteristics of the optical fiber, improving the accuracy of optical fiber contour information extraction, reducing the misjudgment rate, and providing reliable data for predicting the pixel length of the actual unit length in the image.

[0103] Regarding step S122, in some embodiments of the present disclosure, it may include steps S1221 to S1223 as Figure 4 shown.

[0104] S1221. Determine the coordinates of two target points of the optical fiber from the contour information of the optical fiber.

[0105] The target points can be pre-configured according to requirements. For example, the target pixel size may include the head pixel length of the optical fiber and the body pixel length of the optical fiber. The head pixel length of the optical fiber is the number of pixels occupied by the head of the optical fiber in the length direction of the head contour of the optical fiber in the image, and the body pixel length of the optical fiber is the number of pixels occupied by the body of the optical fiber in the length direction of the body contour of the optical fiber in the image. Furthermore, in the process of calculating the head pixel length of the optical fiber, the two end points of the head contour of the optical fiber can be used as the target points; in the process of calculating the body pixel length of the optical fiber, the two end points of the body contour of the optical fiber can be used as the target points.

[0106] S1222. Calculate the distance between the two target points based on the coordinates of the two target points.

[0107] S1223. Use the distance as the target pixel size of the optical fiber in the image.

[0108] The image processing method of the above embodiment determines the coordinates of two target points of the optical fiber from the contour information of the optical fiber, calculates the distance between the two target points based on the coordinates of the two target points, and uses the distance as the target pixel size of the optical fiber in the image. Thus, through the coordinate calculation method, the target pixel size of the optical fiber in the image can be determined quickly and accurately, laying a foundation for predicting the pixel length of the actual unit length in the image.

[0109] Regarding step S120, in other embodiments, the minimum rectangular bounding box enclosing the contour of the optical fiber can be calculated, and the pixel size of the minimum rectangular bounding box can be used as the target pixel size of the optical fiber in the image; or, the pixel points representing the contour of the optical fiber are traversed to determine the pixel points representing the contour of the optical fiber in the target direction, and the number of pixel points representing the contour of the optical fiber in the target direction can be used as the target pixel size of the optical fiber in the image, which is not limited herein.

[0110] Regarding step S130, in some embodiments of the present disclosure, it may include steps S131 and S132 as Figure 5 shown.

[0111] S131. Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber.

[0112] Since the actual size of the optical fiber is a known quantity, therefore, by comparing the actual size of the optical fiber and the target pixel size of the optical fiber, the correlation between the pixel size in the image and the actual size in the real world can be known, thus facilitating the prediction of the pixel length of the actual unit length in the image.

[0113] S132. Use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0114] Specifically, divide the actual unit length by the actual size represented by a single pixel, and use the obtained result as the pixel length of the actual unit length in the image.

[0115] For the image processing method of the above embodiment, determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber, and use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image. Thus, taking full advantage of the optical fiber as a reference object with a known actual size, an accurate mapping between the pixel length and the actual length is achieved, improving the accurate conversion between the actual unit length and the pixel length in the image, and providing a reliable basis for the generation of the target line segment.

[0116] Regarding step S131, as a possible implementation, the target pixel size includes the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the actual width of the head of the optical fiber; correspondingly, step S131 may be: using the quotient of the actual width of the head and the pixel length of the head as the actual size represented by a single pixel in the image.

[0117] The actual width of the head of the optical fiber can be understood as the diameter of the head of the optical fiber in the real world.

[0118] Exemplarily, when evaluating the actual size of a stone, since the head of the optical fiber contacts the stone, that is, the distance between the head of the optical fiber and the stone is less than or equal to a preset distance threshold, the factors such as the viewing angle, focal length, and distortion corresponding to the head of the optical fiber and the stone in the image are basically the same. Therefore, dividing the actual width of the head of the optical fiber by the pixel length of the head of the optical fiber, the obtained result can be regarded as the actual size represented by a single pixel of the stone in the image. In the present disclosure, the actual size represented by a single pixel in the image should be understood as the actual size represented by a single pixel within a certain range around the head of the optical fiber in the image, rather than the actual size represented by each pixel in the entire image.

[0119] Furthermore, considering that when the optical fiber does not contact the stone, there are differences in factors such as the viewing angle, focal length, and distortion corresponding to the head of the optical fiber and the stone in the image, which may lead to inaccurate measurement results. Therefore, without automatically taking the product of the actual size represented by a single pixel and the target pixel size of the stone as the actual size of the stone and prompting the actual size of the stone to the user when the actual size represented by a single pixel is obtained, instead, by adding a target line segment to the head of the optical fiber to assist the user in measuring the actual size of the stone by themselves, thus avoiding misleading the user and improving the accuracy of the measured actual size of the stone.

[0120] The image processing method of the above embodiment uses the quotient of the actual width of the fiber head and the pixel length of the head as the actual size represented by a single pixel. Since the fiber head usually has a clear boundary and a known actual width, this method can quickly obtain the correspondence between a single pixel and the actual size, simplify the calculation process, and improve the real-time performance.

[0121] Regarding step S131, as another possible embodiment, the target pixel size includes the pixel length of the body part of the fiber and the pixel length of the head of the fiber, and the actual size of the fiber includes the preset actual length of the body part of the fiber; correspondingly, step S131 may include steps S1311 and S1312 as shown in Figure 6 the following.

[0122] S1311. Based on the mapping relationship between the preset actual length of the body part and the preset pixel length of the body part, determine the actual length of the body part corresponding to the pixel length of the body part.

[0123] The preset actual length of the body part of the fiber can be understood as a partial length of the body part of the fiber in the real world. The mapping relationship between the preset actual length of the body part and the preset pixel length of the body part can be calibrated in advance, so as to facilitate quickly determining the actual length of the body part corresponding to the current pixel length of the body part by looking up the mapping relationship when the current pixel length of the body part of the fiber is determined.

[0124] S1312. Divide the product of the pixel length of the head and the actual length of the body part by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image.

[0125] After obtaining the product of the pixel length of the head, the actual length of the body part, and the camera focal length of the endoscope, the actual size represented by a single pixel in the image can be calculated based on the similar triangle algorithm. Specifically, the actual size represented by a single pixel in the image = pixel length of the head × actual length of the body part ÷ camera focal length of the endoscope.

[0126] The image processing method of the above embodiment determines the actual length of the body part corresponding to the pixel length of the body part based on the mapping relationship between the preset actual length of the body part and the preset pixel length of the body part, and divides the product of the pixel length of the head and the actual length of the body part by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image. Thus, it makes full use of the actual length of the body part of the fiber and the imaging characteristics of the endoscope, can accurately obtain the correspondence between a single pixel and the actual size, and can be applicable to the adaptability of images under different focal lengths or imaging conditions.

[0127] Regarding step S140, in some embodiments of the present disclosure, specifically: starting from the head of the fiber in the image, draw target line segments in four directions of 0°, 90°, 180°, and 270° respectively.

[0128] Exemplarily, the 0° direction can be pre-calibrated. For example, the direction horizontally to the right in the image can be calibrated as the 0° direction. In the case where the 0° direction is calibrated, the directions of other angles can be obtained by rotating counterclockwise or clockwise starting from the 0° direction.

[0129] In one example, the image after adding the target line segments is as Figure 7 shown. From left to right in the figure are the optical fiber, four target line segments (in a cross shape) facing four directions at the head of the optical fiber, and the calculus.

[0130] The image processing method of the above embodiment can form a cross-shaped virtual scale at the head of the optical fiber in the image by drawing target line segments in four directions starting from the head of the optical fiber, thereby facilitating the user to intuitively compare the relative sizes of the calculus and the target line segments from different directions, and further improving the accuracy, flexibility and visualization effect of the actual size measurement of the calculus.

[0131] Regarding step S120, in some embodiments of the present disclosure, specifically, it may be: when detecting a measurement requirement for the actual size of the calculus, determining the target pixel size of the optical fiber in the image.

[0132] Exemplarily, when detecting a measurement requirement for the actual size of the calculus, determining the target pixel size of the optical fiber in the image, and predicting the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber, and then adding target line segments to the head of the optical fiber in the image, and displaying the image after adding the target line segments; when not detecting a measurement requirement for the actual size of the calculus, directly displaying the image captured by the endoscope.

[0133] Exemplarily, a calculus measurement mode switching button can be provided. For example, a physical button is provided on the operation handle of the endoscope, or a virtual button is provided on the human-computer interaction interface. Then, when detecting that the calculus measurement mode switching button is triggered, it can be determined that a measurement requirement for the actual size of the calculus is detected.

[0134] The image processing method of the above embodiment determines the target pixel size of the optical fiber in the image only when detecting a measurement requirement for the actual size of the calculus, realizes the function of processing on demand, avoids unnecessary computational overhead, improves the resource utilization efficiency of the system, and is particularly suitable for dynamically responding to user requirements during the operation.

[0135] Please combine Figure 8 In one example, the image processing method may include the following steps S201 to step S210. The content related to steps S201 to S210 can refer to the description of the above embodiment. For the sake of brevity, it will not be repeated here.

[0136] In step S201, an image captured by an endoscope is obtained, where the image includes an optical fiber.

[0137] In step S202, it is determined whether a measurement requirement for the actual size of a stone is detected. If so, step S203 is entered; otherwise, step S210 is entered.

[0138] In step S203, in the case where a measurement requirement for the actual size of a stone is detected, the content in the image is subjected to contour extraction to obtain a plurality of contour information.

[0139] In step S204, according to the characteristic information of the optical fiber, the plurality of contour information is filtered to obtain the contour information of the optical fiber.

[0140] In step S205, according to the contour information of the optical fiber, the target pixel size of the optical fiber in the image is determined.

[0141] In step S206, according to the target pixel size and the actual size of the optical fiber, the actual size represented by a single pixel in the image is determined.

[0142] In step S207, the quotient of the actual unit length and the actual size represented by a single pixel is used as the pixel length of the actual unit length in the image.

[0143] In step S208, a target line segment is added to the head of the optical fiber in the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image.

[0144] In step S209, the image after adding the target line segment is displayed.

[0145] In step S210, in the case where a measurement requirement for the actual size of a stone is not detected, the image captured by the endoscope is directly displayed.

[0146] Based on any of the above embodiments, the present disclosure further provides an image processing system.

[0147] Figure 9 Structural schematic diagram of an image processing system according to an embodiment of the present disclosure.

[0148] As Figure 9 shown, the image processing system includes an optical fiber 110, an endoscope 120, a display screen 130, and a controller 140, where the controller 140 is respectively connected to the endoscope 120 and the display screen 130.

[0149] The optical fiber 110 can be placed in the working channel of the endoscope 120. The optical fiber 110 can be a holmium laser optical fiber.

[0150] The endoscope 120 is capable of capturing images.

[0151] The display screen 130 can display images.

[0152] The controller 140 is configured to: when the optical fiber 110 is within the shooting range of the endoscope 120, determine the target pixel size of the optical fiber 110 in the image; predict the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber 110; add a target line segment to the head of the optical fiber 110 in the image; and control the display screen 130 to display the image with the target line segment added, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used to measure the actual size of a calculus when the calculus contacted by the head of the optical fiber 110 is also included in the image.

[0153] The display screen 130 and the controller 140 may be arranged on the same electronic device or may be respectively arranged on different electronic devices, which is not limited herein.

[0154] In some embodiments of the present disclosure, the controller 140 is further configured to: extract the contour of the content in the image to obtain the contour information of the optical fiber 110; and determine the target pixel size of the optical fiber 110 in the image according to the contour information of the optical fiber 110.

[0155] In some embodiments of the present disclosure, the controller 140 is further configured to: extract the contours of the content in the image to obtain a plurality of contour information; and filter the plurality of contour information according to the feature information of the optical fiber 110 to obtain the contour information of the optical fiber 110.

[0156] In some embodiments of the present disclosure, the feature information of the optical fiber 110 includes a contour size range, a target hue, and a target contour position; correspondingly, the controller 140 is further configured to: use the contour information in the plurality of contour information whose contour size is within the contour size range, the hue is the target hue, and the contour position is the target contour position as the contour information of the optical fiber 110.

[0157] In some embodiments of the present disclosure, the controller 140 is further configured to: determine the coordinates of two target points of the optical fiber 110 from the contour information of the optical fiber 110; calculate the distance between the two target points based on the coordinates of the two target points; and use the distance as the target pixel size of the optical fiber 110 in the image.

[0158] In some embodiments of the present disclosure, the controller 140 is further configured to: determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber 110; and use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0159] In some embodiments of the present disclosure, the target pixel size includes the head pixel length of the optical fiber 110, and the actual size of the optical fiber 110 includes the head actual width of the optical fiber 110; correspondingly, the controller 140 is further configured to: use the quotient of the head actual width and the head pixel length as the actual size represented by a single pixel in the image.

[0160] In some embodiments of the present disclosure, the target pixel size includes the body pixel length of the optical fiber 110 and the head pixel length of the optical fiber 110, and the actual size of the optical fiber 110 includes the preset actual body length of the optical fiber 110; correspondingly, the controller 140 is further configured to: determine the actual body length corresponding to the body pixel length based on the mapping relationship between the preset actual body length and the preset body pixel length; and divide the product of the head pixel length and the actual body length by the camera focal length of the endoscope 120 to obtain the actual size represented by a single pixel in the image.

[0161] In some embodiments of the present disclosure, the controller 140 is further configured to: draw target line segments in four directions of 0°, 90°, 180°, and 270° respectively starting from the head of the optical fiber 110 in the image.

[0162] In some embodiments of the present disclosure, the controller 140 is further configured to: determine the target pixel size of the optical fiber 110 in the image when detecting a measurement requirement for the actual size of a calculus.

[0163] For the implementation processes of the functions and roles of the various components in the above system, please refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated here.

[0164] Those skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0165] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, and the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate for implementing the process Figure 1One or more processes and / or blocks Figure 1 means for the functions specified in one or more blocks

[0166] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing apparatus to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions in the process Figure 1 One or more processes and / or blocks Figure 1 in one or more blocks

[0167] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, such that a series of operational steps are performed on the computer or other programmable apparatus to produce a computer-implemented process, so that the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions in the process Figure 1 One or more processes and / or blocks Figure 1 in one or more blocks

[0168] In the description of this specification, the description with reference to the terms "one embodiment / way", "some embodiments / ways", "example", "specific example", or "some examples", etc. means that the specific features, structures, or characteristics described in connection with the embodiment / way or example are included in at least one embodiment / way or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / way or example. Moreover, the specific features, structures, or characteristics described can be combined in any one or more embodiments / ways or examples in a suitable manner. In addition, without contradiction, those skilled in the art can combine and combine the different embodiments / ways or examples described in this specification and the features of different embodiments / ways or examples

[0169] In the description of the present disclosure, it should be understood that the orientation or positional relationship indicated by terms such as "center", "longitudinal", "lateral", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. is based on the orientation or positional relationship shown in the drawings. It is only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation. Therefore, it should not be construed as a limitation to the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly specifying the quantity of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of such features. In the description of the present disclosure, the meaning of "a plurality of" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0170] In the description of the present disclosure, it should be noted that unless otherwise clearly specified and limited, the terms "mounted", "connected" and "coupled" shall be understood in a broad sense. For example, it may be a fixed connection, a detachable connection or an integral connection. It may be a mechanical connection or an electrical connection. It may be directly connected or indirectly connected through an intermediate medium, and it may be the internal communication of two elements or the interaction relationship between two elements. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0171] Those skilled in the art should understand that the above embodiments are only for clearly illustrating the present disclosure and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications can be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that: include: Acquire an image captured by an endoscope, wherein the image includes the optical fiber; determining a target pixel size of the optical fiber in the image; Determining an actual size represented by a single pixel in the image based on the target pixel size and the actual size of the optical fiber; The quotient of the actual unit length and the actual size represented by the single pixel is used as the pixel length of the actual unit length in the image, wherein the actual unit length is a reference standard actually set; adding a target line segment at the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and displaying the image after the target line segment is added, wherein the target line segment is used as a ruler to measure the actual size of the stone when the stone contacted by the head of the optical fiber is also included in the image; During the process of displaying the image after adding the target line segment, if no operation for adjusting the orientation of the target line segment is received, the target line segment is displayed in the image with a default orientation among multiple preset orientations; if an operation for adjusting the orientation of the target line segment is received, the orientation of the target line segment displayed in the image is switched to a preset orientation among the multiple preset orientations except the current orientation.

2. The image processing method according to claim 1, characterized in that: Determining a target pixel size of the optical fiber in the image includes: Extracting contours of the contents in the image to obtain contour information of the optical fiber; and A target pixel size of the optical fiber in the image is determined according to the contour information of the optical fiber.

3. The image processing method according to claim 2, characterized in that: Extracting the outline of the content in the image to obtain the outline information of the optical fiber includes: Extracting contours of the content in the image to obtain a plurality of contour information; and The plurality of profile information are filtered according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber.

4. The image processing method according to claim 3, characterized in that: The characteristic information of the optical fiber includes a profile size range, a target hue, and a target profile position; Filtering the plurality of profile information according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber includes: The profile information whose profile size is within the profile size range, whose color tone is the target color tone, and whose profile position is the target profile position among the plurality of profile information is used as the profile information of the optical fiber.

5. The image processing method according to claim 2, characterized in that: Determining a target pixel size of the optical fiber in the image according to the profile information of the optical fiber includes: Determining coordinates of two target points of the optical fiber from profile information of the optical fiber; Based on the coordinates of the two target points, calculating the distance between the two target points; and The distance is taken as the target pixel size of the optical fiber in the image.

6. The image processing method according to claim 1, characterized in that: The target pixel size includes the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the actual width of the head of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: The quotient of the actual width of the head and the pixel length of the head is taken as the actual size represented by a single pixel in the image.

7. The image processing method according to claim 1, characterized in that: The target pixel size includes the pixel length of the body of the optical fiber and the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the preset actual length of the body of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: Determine the actual body length corresponding to the body pixel length based on the mapping relationship between the preset actual body length and the preset body pixel length; as well as The actual size represented by a single pixel in the image is obtained by dividing the product of the pixel length of the head and the actual length of the body by the focal length of the camera of the endoscope.

8. The image processing method according to claim 1, characterized in that: Adding a target line segment at the head of the optical fiber in the image includes: Taking the head of the optical fiber in the image as the starting point, the target line segments are drawn in four directions of 0°, 90°, 180° and 270° respectively.

9. The image processing method according to claim 1, characterized in that: Determining a target pixel size of the optical fiber in the image includes: In case a need for measurement of the actual size of the stone is detected, a target pixel size of the optical fiber in the image is determined.

10. An image processing system, characterized in that: include: optical fiber; an endoscope, which is capable of taking images; a display screen capable of displaying the image; a controller, configured to determine a target pixel size of the optical fiber in the image when the optical fiber enters a shooting range of the endoscope; Determining an actual size represented by a single pixel in the image based on the target pixel size and the actual size of the optical fiber; The quotient of the actual unit length and the actual size represented by the single pixel is used as the pixel length of the actual unit length in the image, wherein the actual unit length is a reference standard actually set; adding a target line segment at the head of the optical fiber in the image; and controlling the display screen to display the image after the target line segment is added, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used as a ruler to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber; in the process of displaying the image after the target line segment is added, if an operation for adjusting the orientation of the target line segment is not received, displaying the target line segment in the image with a default orientation among a plurality of preset orientations; If an operation for adjusting the orientation of the target line segment is received, the orientation of the target line segment displayed in the image is switched to a preset orientation among the plurality of preset orientations except the current orientation.

Images