Image processing method and image processing system

Abstract

The invention provides an image processing method and an image processing system. The image processing method comprises the steps that an image shot by an endoscope is obtained, and the image comprises an optical fiber; determining a target pixel size of the optical fiber in the image; determining the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber; taking the quotient value of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image; a target line segment is added to the head of the optical fiber in the image, and the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and displaying the image after the target line segment is added, and the target line segment is used for measuring the actual size of the stone under the condition that the image further comprises the stone in contact with the head of the optical fiber.

Description

Technical Field

[0001] The present disclosure relates to technical fields such as medical image processing, and in particular to an image processing method and an image processing system. Background Art

[0002] With the continuous advancement of medical technology, endoscopic technology has been widely used in minimally invasive surgery, especially in the treatment of urinary stones. Endoscopes enable doctors to directly observe the lesions in the body and perform precise operations through their high-resolution imaging capabilities.

[0003] In endoscopic surgery, accurate measurement of the actual size of stones is crucial for choosing appropriate treatment options (such as lithotripsy or direct removal). However, since endoscopic images are limited by factors such as viewing angle, focal length, and distortion, it is difficult to accurately determine the actual size of stones from endoscopic images simply by relying on the naked eye. Summary of the invention

[0004] The present disclosure provides an image processing method and an image processing system.

[0005] According to one aspect of the present disclosure, there is provided an image processing method, comprising: Acquire an image captured by an endoscope, wherein the image includes the optical fiber; determining a target pixel size of the optical fiber in the image; Predicting the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber; Adding a target line segment at the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; The image after the target line segment is added is displayed, wherein the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber.

[0006] According to at least one embodiment of the image processing method of the present disclosure, determining a target pixel size of the optical fiber in the image includes: Extracting contours of the contents in the image to obtain contour information of the optical fiber; and A target pixel size of the optical fiber in the image is determined according to the contour information of the optical fiber.

[0007] According to at least one embodiment of the image processing method of the present disclosure, contour extraction is performed on the content in the image to obtain contour information of the optical fiber, including: Extracting contours of the contents in the image to obtain a plurality of contour information; The plurality of profile information are filtered according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber.

[0008] According to the image processing method of at least one embodiment of the present disclosure, the characteristic information of the optical fiber includes a profile size range, a target hue, and a target profile position; Filtering the plurality of profile information according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber includes: The profile information whose profile size is within the profile size range, whose color tone is the target color tone, and whose profile position is the target profile position among the plurality of profile information is used as the profile information of the optical fiber.

[0009] According to at least one embodiment of the image processing method of the present disclosure, determining a target pixel size of the optical fiber in the image according to the contour information of the optical fiber includes: Determining coordinates of two target points of the optical fiber from profile information of the optical fiber; Based on the coordinates of the two target points, calculating the distance between the two target points; The distance is taken as the target pixel size of the optical fiber in the image.

[0010] According to at least one embodiment of the image processing method of the present disclosure, predicting the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber includes: Determining an actual size represented by a single pixel in the image based on the target pixel size and the actual size of the optical fiber; The quotient of the actual unit length and the actual size represented by the single pixel is taken as the pixel length of the actual unit length in the image.

[0011] According to the image processing method of at least one embodiment of the present disclosure, the target pixel size includes the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the actual width of the head of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: The quotient of the actual width of the head and the pixel length of the head is taken as the actual size represented by a single pixel in the image.

[0012] According to the image processing method of at least one embodiment of the present disclosure, the target pixel size includes the pixel length of the body of the optical fiber and the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the preset actual length of the body of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: Determine the actual body length corresponding to the body pixel length based on the mapping relationship between the preset actual body length and the preset body pixel length; The actual size represented by a single pixel in the image is obtained by dividing the product of the pixel length of the head and the actual length of the body by the focal length of the camera of the endoscope.

[0013] According to the image processing method of at least one embodiment of the present disclosure, adding a target line segment at the head of the optical fiber in the image includes: Taking the head of the optical fiber in the image as the starting point, the target line segments are drawn in four directions of 0°, 90°, 180° and 270° respectively.

[0014] According to at least one embodiment of the image processing method of the present disclosure, determining a target pixel size of the optical fiber in the image includes: In case a need for measurement of the actual size of the stone is detected, a target pixel size of the optical fiber in the image is determined.

[0015] According to another aspect of the present disclosure, there is provided an image processing system, comprising: optical fiber; an endoscope, which is capable of taking images; a display screen capable of displaying the image; A controller is used to determine the target pixel size of the optical fiber in the image when the optical fiber enters the shooting range of the endoscope; predict the pixel length of the actual unit length in the image based on the target pixel size and the actual size of the optical fiber; add a target line segment at the head of the optical fiber in the image; and control the display screen to display the image after the target line segment is added, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] The accompanying drawings illustrate exemplary embodiments of the present disclosure and together with the description serve to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are incorporated in and constitute a part of this specification.

[0017] Figure 1 It is a flowchart of an image processing method according to an embodiment of the present invention.

[0018] Figure 2 FIG. 4 is a schematic diagram of a process for determining a target pixel size of an optical fiber according to an embodiment of the present disclosure.

[0019] Figure 3 It is a schematic diagram of a process for determining optical fiber profile information according to an embodiment of the present disclosure.

[0020] Figure 4 FIG. 4 is a schematic diagram of a process for determining a target pixel size of an optical fiber according to another embodiment of the present disclosure.

[0021] Figure 5 It is a schematic diagram of a process of determining the pixel length of an actual unit length according to an embodiment of the present disclosure.

[0022] Figure 6 It is a schematic diagram of a process of determining the actual size of a single pixel representation in an image according to one embodiment of the present disclosure.

[0023] Figure 7 FIG. 4 is a schematic diagram of an example of adding a target line segment to an image according to an embodiment of the present disclosure.

[0024] Figure 8 It is a flowchart of an image processing method according to another embodiment of the present invention.

[0025] Fig. 9 It is a schematic block diagram of the structure of an image processing system according to one embodiment of the present disclosure. DETAILED DESCRIPTION

[0026] The present disclosure is further described in detail below in conjunction with the accompanying drawings and examples. It is understood that the specific examples described herein are only used to explain the relevant content, rather than to limit the present disclosure. It should also be noted that, for ease of description, only the parts related to the present disclosure are shown in the accompanying drawings.

[0027] It should be noted that, in the absence of conflict, the embodiments and features of the embodiments in the present disclosure can be combined with each other. The technical solution of the present disclosure will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0028] The actual size of stones judged from endoscopic images by naked eyes often has large errors, which poses a challenge to the success rate and safety of the operation. In related technologies, the measurement of the actual size of stones depends on preoperative imaging examinations (such as CT or X-rays). However, preoperative images cannot reflect the real-time status of stones during surgery. Taking endoscopic laser lithotripsy as an example, after the stones are acted on by laser energy, they may produce broken particles with significant size differences, and their particle size distribution has significant randomness. If the actual size of the broken particles cannot be determined in real time, it may cause broken particles with larger diameters to remain, which may cause recurrence of the disease in the future. Therefore, it is crucial to measure the actual size of the stones in real time.

[0029] To this end, the present disclosure proposes an image processing method.

[0030] The image processing method disclosed herein can be used to measure the actual size of stones in real time through endoscopic images during endoscopic examination or surgery.In the present disclosure, electronic devices include but are not limited to mobile phones, tablet computers, laptop computers, personal computers, etc.

[0031] For the convenience of description and to make the technical solutions of the specific embodiments of the present disclosure easier to understand, before describing the image processing method implemented in the present disclosure, the technical terms involved in the specific embodiments of the present disclosure are explained as follows: Size refers to the length and size of something.

[0032] Length is a measure of one-dimensional space, which is the distance between points. Usually when measuring the length of a line segment in two-dimensional space, the length with a larger value is called length, and the length that is not larger than its value or is on the "side" is called width. Width is also a measure of length.

[0033] The target pixel size refers to the number of pixels occupied by the target content in the image in the target direction.

[0034] Pixel length refers to the number of pixels that a part of the image occupies in the length direction of the part.

[0035] Pixel width refers to the number of pixels that a part of the image occupies in the width direction of the content.

[0036] Actual size, actual length, and actual width refer respectively to the size, length, and width of things in the real world (such as the world geodetic coordinate system).

[0037] Unit length is an artificially set reference standard. Unit length is a standard for reference. It has no fixed value and changes according to the setting.

[0038] Figure 1FIG. 2 is a schematic diagram showing the overall flow of an image processing method M100 according to an embodiment of the present disclosure. Figure 1 The method shown includes steps S110 to S150. The method can be executed by electronic devices such as mobile phones and computers.

[0039] Specifically, Figure 1 The methods shown include: S110, acquiring an image captured by an endoscope, wherein the image includes the optical fiber.

[0040] In one example, the optical fiber is a holmium laser fiber.

[0041] S120: Determine the target pixel size of the optical fiber in the image.

[0042] The target pixel size of the optical fiber in the image, that is, the number of pixels that the optical fiber occupies in the target direction of the optical fiber in the image. The target direction can be set according to the actual situation, for example, it can be the length direction of the optical fiber head, the length direction of the optical fiber body, etc.

[0043] Exemplarily, the target pixel size of the optical fiber in the image may be determined by an image recognition algorithm or an image recognition model in the related art.

[0044] S130 , predicting the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber.

[0045] The actual size of the optical fiber is the size of the optical fiber in the real world (such as the world geodetic coordinate system). The actual size of the optical fiber can be calibrated in advance.

[0046] The actual unit length is the unit length in the real world (such as the world geodetic coordinate system). The actual unit length can be a stone safety size threshold. For example, if the actual size of the stone is less than or equal to the safety size threshold, the stone may not be crushed. The actual unit length can also be other manually set values, such as 0.1 mm, 0.5 mm, 1 mm, 5 mm, 1 cm, etc., which are not limited here.

[0047] Since the actual size of the optical fiber is a known quantity, by comparing the actual size of the optical fiber with the target pixel size of the optical fiber, the correlation between the pixel size in the image and the actual size in the real world can be known, thereby facilitating the prediction of the pixel length of the actual unit length in the image.

[0048] It should be noted that the specific numerical values ​​mentioned in the present disclosure are only used as examples to illustrate the implementation of the present disclosure in detail, and should not be understood as limiting the present disclosure. In other examples or implementations or embodiments, other numerical values ​​can be selected according to the present disclosure, and are not specifically limited here.

[0049] S140: Add a target line segment at the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image.

[0050] The pixel width of the target line segment may be a pre-calibrated default value, and the present disclosure does not limit the pixel width of the target line segment.

[0051] Exemplarily, the number of target line segments may be one or more. Multiple target line segments may be distributed in the image along the same direction, for example, all target line segments are connected end to end along the same direction; multiple target line segments may also be distributed in the image along multiple directions, for example, some of the target line segments are connected end to end along a first direction and the remaining target line segments are connected end to end along a second direction, which is not limited here.

[0052] At least one target line segment can have the optical fiber head as the midpoint, or the optical fiber head as the starting point, or the optical fiber head as the middle point excluding the midpoint and the end point, or a pixel point within the preset pixel length range of the optical fiber head as the midpoint, or a pixel point within the preset pixel length range of the optical fiber head as the starting point, or a pixel point within the preset pixel length range of the optical fiber head as the middle point excluding the midpoint and the end point, and there is no limitation here.

[0053] S150, displaying the image after adding the target line segment, wherein the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber.

[0054] Since the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, the target line segment can act as a ruler, so that the actual size of the stone in contact with the optical fiber head can be measured through the target line segment. In the present disclosure, "contact" can be understood as the distance between at least part of the position of the two being less than or equal to the preset distance threshold. Since the target line segment is equivalent to a ruler, in addition to stones, the target line segment can also be used to measure the actual size of other objects in contact with the optical fiber head, and the specific measurement method can be the same as the stone measurement method.

[0055] The orientation of the target line segment in the image can be fixed or dynamically adjusted, which is not limited here. For example, multiple preset orientations are predetermined, and then in the process of displaying the image after adding the target line segment, if no operation for adjusting the orientation of the target line segment is received, the target line segment is displayed in the image with a default orientation among multiple preset orientations; if an operation for adjusting the orientation of the target line segment is received (for example, the user turns the target line segment orientation adjustment knob or selects a preset orientation), the orientation of the target line segment displayed in the image is switched to a preset orientation among multiple preset orientations except the current orientation. In this way, the dynamic adjustment of the orientation of the target line segment is realized, which can be applicable to the actual size measurement of stones of different shapes and positions, and improves the accuracy of the actual size measurement of stones.

[0056] The image processing method of the embodiment of the present disclosure obtains an image including an optical fiber taken by an endoscope, determines the target pixel size of the optical fiber in the image, predicts the pixel length of the actual unit length in the image according to the target pixel size and the actual size of the optical fiber, adds a target line segment at the head of the optical fiber in the image, and displays the image after adding the target line segment. Thus, the optical fiber is used as a reference to achieve accurate conversion between the actual unit length and the pixel length in the image. In addition, the target line segment added to the head of the optical fiber in the image has the same pixel length as the actual unit length, so that the user can intuitively measure the actual size of the stone contacted by the head of the optical fiber in the image with the target line segment added, greatly simplifying the user's operation process, significantly improving the operation efficiency, and improving the measurement accuracy, providing more efficient and accurate technical support for medical work. In addition, since the optical fiber is required for laser energy delivery during endoscopic laser lithotripsy, and the image processing method of the embodiment of the present disclosure is implemented based on the optical fiber, that is, in the present disclosure, there is no need to introduce additional tools during the operation, and the existing tools during the operation are used to quickly, intuitively, and in real time to measure the actual size of the stone, reducing the complexity of the operation and the risk of infection.

[0057] Regarding step S120, in some embodiments of the present disclosure, it may include the following steps: Figure 2 Step S121 and step S122 are shown.

[0058] S121. Extract the contour of the content in the image to obtain contour information of the optical fiber.

[0059] In addition to optical fibers, images captured by endoscopes may also contain human tissues, stones, etc. Through contour extraction, the contour of the optical fiber in the image can be automatically determined, thereby providing data support for determining the target pixel size of the optical fiber.

[0060] Exemplarily, the contour information of the optical fiber can be obtained by extracting the contour of the content in the image through an edge detection algorithm, an image segmentation algorithm or a contour extraction model in the related technology.

[0061] S122. Determine a target pixel size of the optical fiber in the image according to the profile information of the optical fiber.

[0062] The contour information of the optical fiber can characterize the boundary position of the optical fiber in the image, such as the coordinates of the pixel points characterizing the boundary of the optical fiber in the image. Therefore, after determining the contour information of the optical fiber, the target pixel size of the optical fiber in the image can be determined through the contour information.

[0063] The image processing method of the above embodiment extracts the outline of the content in the image to obtain the outline information of the optical fiber, and determines the target pixel size of the optical fiber in the image based on the outline information of the optical fiber. Thus, the outline of the optical fiber in the image is automatically identified, and tedious manual labeling is avoided, thereby improving the efficiency and consistency of the measurement of the target pixel size of the optical fiber. At the same time, by extracting the outline to determine the target pixel size, it is possible to avoid interference from other content in the image, improve the accuracy of the determined target pixel size, and provide reliable data for the subsequent prediction of the pixel length of the actual unit length in the image.

[0064] Regarding step S121, in some embodiments of the present disclosure, it may include the following: Figure 3 Steps S1211 and S1212 are shown.

[0065] S1211. Extract contours of contents in the image to obtain multiple contour information.

[0066] Exemplarily, the image can be first converted from the RGB color space to a grayscale image to reduce computational complexity and highlight the key features of the image; Gaussian blur is then applied to the converted grayscale image to smooth the image and reduce the impact of noise on subsequent processing steps; then, the pixel values ​​of areas in the grayscale image after Gaussian blur whose pixel values ​​are greater than a set threshold are set to 255, and the pixel values ​​of areas in the grayscale image after Gaussian blur whose pixel values ​​are less than or equal to the set threshold are set to 0, thereby converting the grayscale image after Gaussian blur into a binary image; finally, edge detection is performed on the content in the binary image to obtain multiple contour information.

[0067] S1212. Filter multiple profile information according to the characteristic information of the optical fiber to obtain profile information of the optical fiber.

[0068] The features of the multiple profile information are different. Therefore, based on the feature information of the optical fiber, the multiple profile information are filtered, so that the profile information of the optical fiber can be accurately determined from the multiple profile information.

[0069] The image processing method of the above embodiment extracts the contour of the content in the image to obtain multiple contour information, and filters the multiple contour information according to the characteristic information of the optical fiber to obtain the contour information of the optical fiber. As a result, the contour information of the optical fiber can be accurately separated from the complex image, effectively reducing the interference of irrelevant contours, and improving the robustness and reliability of optical fiber contour recognition, which is particularly suitable for light changes or background clutter that may exist in endoscopic images.

[0070] Regarding step S1212, in some embodiments of the present disclosure, the characteristic information of the optical fiber includes a profile size range, a target hue, and a target profile position. Accordingly, step S1212 may specifically be: using the profile information in which the profile size is within the profile size range, the hue is the target hue, and the profile position is the target profile position as the profile information of the optical fiber.

[0071] The characteristic information of the optical fiber can be pre-configured according to actual conditions.

[0072] Exemplarily, the contour information whose contour size is outside the contour size range among multiple contour information can be first filtered out to obtain the contour information after the first filtering; then, the contour information whose color tone is not the target color tone among the contour information after the first filtering can be filtered out to obtain the contour information after the second filtering; finally, the contour information whose contour position is not the target contour position among the contour information after the second filtering can be filtered out, and the remaining contour information at this time is the contour information of the optical fiber.

[0073] The image processing method of the above-mentioned embodiment filters multiple contour information based on the contour size range, target color tone and target contour position, thereby ensuring that the contour information obtained after filtering is highly consistent with the actual characteristics of the optical fiber, improving the accuracy of optical fiber contour information extraction, reducing the misjudgment rate, and providing reliable data for the subsequent prediction of the pixel length of the actual unit length in the image.

[0074] Regarding step S122, in some embodiments of the present disclosure, it may include the following steps: Figure 4 Steps S1221 to S1223 are shown.

[0075] S1221. Determine the coordinates of two target points of the optical fiber from the profile information of the optical fiber.

[0076] The target point can be pre-configured according to the requirements. For example, the target pixel size may include the pixel length of the optical fiber head and the pixel length of the optical fiber body. The pixel length of the optical fiber head is the number of pixels occupied by the optical fiber head in the length direction of the optical fiber head contour in the image, and the pixel length of the optical fiber body is the number of pixels occupied by the optical fiber body in the length direction of the optical fiber body contour in the image. Furthermore, in the process of calculating the pixel length of the optical fiber head, the two end points of the optical fiber head contour can be used as the target points; in the process of calculating the pixel length of the optical fiber body, the two end points of the optical fiber body contour can be used as the target points.

[0077] S1222. Calculate the distance between the two target points based on the coordinates of the two target points.

[0078] S1223. Use the distance as the target pixel size of the optical fiber in the image.

[0079] The image processing method of the above embodiment determines the coordinates of the two target points of the optical fiber from the contour information of the optical fiber, calculates the distance between the two target points based on the coordinates of the two target points, and uses the distance as the target pixel size of the optical fiber in the image. Thus, by means of coordinate calculation, the target pixel size of the optical fiber in the image can be determined quickly and accurately, laying a foundation for predicting the pixel length of the actual unit length in the image.

[0080] Regarding step S120, in other embodiments, the minimum rectangular bounding box surrounding the outline of the optical fiber can be calculated, and the pixel size of the minimum rectangular bounding box can be used as the target pixel size of the optical fiber in the image; or, the pixel points representing the optical fiber outline can be traversed to determine the pixel points representing the optical fiber outline in the target direction, and the number of pixel points representing the optical fiber outline in the target direction can be used as the target pixel size of the optical fiber in the image, which is not limited here.

[0081] Regarding step S130, in some embodiments of the present disclosure, it may include the following steps: Figure 5 Step S131 and step S132 are shown.

[0082] S131. Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber.

[0083] Since the actual size of the optical fiber is a known quantity, by comparing the actual size of the optical fiber with the target pixel size of the optical fiber, the correlation between the pixel size in the image and the actual size in the real world can be known, thereby facilitating the prediction of the pixel length of the actual unit length in the image.

[0084] S132. Taking the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0085] Specifically, the actual unit length is divided by the actual size represented by a single pixel, and the obtained result is used as the pixel length of the actual unit length in the image.

[0086] The image processing method of the above embodiment determines the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber, and uses the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image. Thus, the advantage of the optical fiber as a reference object with a known actual size is fully utilized to achieve an accurate mapping between the pixel length and the actual length, improve the accurate conversion between the actual unit length and the pixel length in the image, and provide a reliable basis for the generation of the target line segment.

[0087] Regarding step S131, as a possible implementation, the target pixel size includes the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the actual width of the head of the optical fiber; accordingly, step S131 can be: taking the quotient of the actual width of the head and the pixel length of the head as the actual size represented by a single pixel in the image.

[0088] The actual width of the optical fiber head can be understood as the diameter of the optical fiber head in the real world.

[0089] Exemplarily, since the optical fiber head is in contact with the stone when evaluating the actual size of the stone, that is, the distance between the optical fiber head and the stone is less than or equal to the preset distance threshold, the factors such as the viewing angle, focal length and distortion corresponding to the optical fiber head and the stone in the image are basically the same, and then the actual width of the optical fiber head is divided by the pixel length of the optical fiber head, and the result obtained can be regarded as the actual size represented by a single pixel of the stone in the image. In the present disclosure, the actual size represented by a single pixel in the image should be understood as the actual size represented by a single pixel in a certain range around the optical fiber head in the image, rather than the actual size represented by each pixel in the entire image.

[0090] Furthermore, considering that when the optical fiber is not in contact with the stone, there are differences in factors such as viewing angle, focal length and distortion corresponding to the optical fiber head and the stone in the image, which leads to inaccurate measurement results. Therefore, when the actual size represented by a single pixel is obtained, the product of the actual size represented by a single pixel and the target pixel size of the stone is not automatically used as the actual size of the stone and the actual size of the stone is not prompted to the user. Instead, a target line segment is added to the optical fiber head to assist the user in measuring the actual size of the stone by themselves, thereby avoiding misleading the user and improving the accuracy of the measured actual size of the stone.

[0091] The image processing method of the above embodiment uses the quotient of the actual width of the optical fiber head and the pixel length of the head as the actual size represented by a single pixel. Since the optical fiber head usually has a clear boundary and a known actual width, this method can quickly derive the corresponding relationship between a single pixel and the actual size, thereby simplifying the calculation process and improving real-time performance.

[0092] Regarding step S131, as another possible implementation, the target pixel size includes the pixel length of the fiber body and the pixel length of the fiber head, and the actual size of the fiber includes the preset actual length of the fiber body; accordingly, step S131 may include the following: Figure 6 Steps S1311 and S1312 are shown.

[0093] S1311. Determine the actual body length corresponding to the body pixel length based on a mapping relationship between a preset actual body length and a preset body pixel length.

[0094] The preset actual length of the optical fiber body can be understood as the partial length of the optical fiber body in the real world. The mapping relationship between the preset actual length of the body and the preset pixel length of the body can be pre-calibrated, so that when the current pixel length of the optical fiber body is determined, the actual length of the body corresponding to the current pixel length of the body can be quickly determined by looking up the mapping relationship.

[0095] S1312: Divide the product of the pixel length of the head and the actual length of the body by the focal length of the endoscope camera to obtain the actual size represented by a single pixel in the image.

[0096] After obtaining the product of the head pixel length, the actual body length and the endoscope camera focal length, the actual size represented by a single pixel in the image can be calculated based on the similar triangle algorithm. Specifically, the actual size represented by a single pixel in the image = head pixel length × actual body length ÷ endoscope camera focal length.

[0097] The image processing method of the above embodiment determines the actual length of the body corresponding to the body pixel length based on the mapping relationship between the preset actual length of the body and the preset pixel length of the body, and divides the product of the head pixel length and the actual length of the body by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image. Thus, the actual length of the optical fiber body and the imaging characteristics of the endoscope are fully utilized, and the corresponding relationship between a single pixel and the actual size can be accurately obtained, which can be applicable to the adaptability of images under different focal lengths or imaging conditions.

[0098] Regarding step S140, in some embodiments of the present disclosure, specifically, it may be: starting from the head of the optical fiber in the image, drawing target line segments in four directions of 0°, 90°, 180° and 270° respectively.

[0099] Exemplarily, the 0° direction may be pre-calibrated. For example, the horizontal rightward direction in the image may be calibrated as the 0° direction. When the 0° direction is calibrated, directions of other angles may be obtained by rotating counterclockwise or clockwise from the 0° direction.

[0100] In one example, the image after adding the target line segment is as follows Figure 7 As shown, from left to right in the figure are the optical fiber, four target line segments (in a cross shape) facing four directions at the head of the optical fiber, and the stone.

[0101] The image processing method of the above embodiment can form a cross-shaped virtual ruler at the optical fiber head in the image by drawing target line segments in four directions with the optical fiber head as the starting point, thereby facilitating the user to intuitively compare the relative size of the stone and the target line segment from different directions, further improving the accuracy, flexibility and visualization effect of the actual size measurement of the stone.

[0102] Regarding step S120, in some embodiments of the present disclosure, specifically, it may be: when a measurement requirement for the actual size of the stone is detected, determining a target pixel size of the optical fiber in the image.

[0103] Exemplarily, when a need to measure the actual size of the stone is detected, the target pixel size of the optical fiber in the image is determined, and the pixel length of the actual unit length in the image is predicted based on the target pixel size and the actual size of the optical fiber, and then a target line segment is added to the head of the optical fiber in the image, and the image with the added target line segment is displayed; when no need to measure the actual size of the stone is detected, the image taken by the endoscope is directly displayed.

[0104] Exemplarily, a stone measurement mode switching button may be provided, such as a physical button provided on the operating handle of the endoscope, or a virtual button provided on the human-computer interaction interface, and then when it is detected that the stone measurement mode switching button is triggered, it can be determined that the measurement requirement for the actual size of the stone has been detected.

[0105] The image processing method of the above-mentioned embodiment determines the target pixel size of the optical fiber in the image only when the measurement demand for the actual size of the stone is detected, thereby realizing the function of on-demand processing, avoiding unnecessary computing overhead, improving the resource utilization efficiency of the system, and being particularly suitable for dynamically responding to user needs during surgery.

[0106] Please combine Figure 8 In one example, the image processing method may include the following steps S201 to S210. The contents related to steps S201 to S210 can refer to the description of the above embodiment, and for the sake of brevity, they will not be repeated here.

[0107] In step S201, an image captured by an endoscope is acquired, wherein the image includes the optical fiber.

[0108] In step S202, it is determined whether a measurement requirement for the actual size of the stone is detected. If so, the process proceeds to step S203; otherwise, the process proceeds to step S210.

[0109] In step S203, when the measurement requirement for the actual size of the stone is detected, contour extraction is performed on the content in the image to obtain a plurality of contour information.

[0110] In step S204, multiple profile information are filtered according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber.

[0111] In step S205, the target pixel size of the optical fiber in the image is determined according to the contour information of the optical fiber.

[0112] In step S206, the actual size represented by a single pixel in the image is determined according to the target pixel size and the actual size of the optical fiber.

[0113] In step S207, the quotient of the actual unit length and the actual size represented by a single pixel is taken as the pixel length of the actual unit length in the image.

[0114] In step S208, a target line segment is added to the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image.

[0115] In step S209, the image after the target line segment is added is displayed.

[0116] In step S210 , when no measurement requirement for the actual size of the stone is detected, the image captured by the endoscope is directly displayed.

[0117] Based on any of the above implementations, the present disclosure also provides an image processing system.

[0118] Fig. 9 A schematic block diagram of the structure of an image processing system according to an embodiment of the present disclosure.

[0119] like Fig. 9 As shown, the image processing system includes an optical fiber 110, an endoscope 120, a display screen 130 and a controller 140, wherein the controller 140 is connected to the endoscope 120 and the display screen 130 respectively.

[0120] The optical fiber 110 may be placed in a working channel of the endoscope 120. The optical fiber 110 may be a holmium laser fiber.

[0121] The endoscope 120 is capable of capturing images.

[0122] The display screen 130 is capable of displaying images.

[0123] The controller 140 is used to: determine the target pixel size of the optical fiber 110 in the image when the optical fiber 110 enters the shooting range of the endoscope 120; predict the pixel length of the actual unit length in the image based on the target pixel size and the actual size of the optical fiber 110; add a target line segment at the head of the optical fiber 110 in the image; and control the display screen 130 to display the image after the target line segment is added, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber 110.

[0124] The display screen 130 and the controller 140 may be arranged on the same electronic device, or may be arranged on different electronic devices respectively, which is not limited here.

[0125] In some embodiments of the present disclosure, the controller 140 is further configured to: extract contours of the content in the image to obtain contour information of the optical fiber 110; and determine a target pixel size of the optical fiber 110 in the image based on the contour information of the optical fiber 110.

[0126] In some embodiments of the present disclosure, the controller 140 is further used to: extract contours of the content in the image to obtain multiple contour information; and filter the multiple contour information according to the characteristic information of the optical fiber 110 to obtain contour information of the optical fiber 110.

[0127] In some embodiments of the present disclosure, the characteristic information of the optical fiber 110 includes a profile size range, a target hue, and a target profile position; accordingly, the controller 140 is also used to: use the profile information in which the profile size is within the profile size range, the hue is the target hue, and the profile position is the target profile position as the profile information of the optical fiber 110.

[0128] In some embodiments of the present disclosure, the controller 140 is also used to: determine the coordinates of two target points of the optical fiber 110 from the profile information of the optical fiber 110; calculate the distance between the two target points based on the coordinates of the two target points; and use the distance as the target pixel size of the optical fiber 110 in the image.

[0129] In some embodiments of the present disclosure, the controller 140 is also used to: determine the actual size represented by a single pixel in the image based on the target pixel size and the actual size of the optical fiber 110; and use the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image.

[0130] In some embodiments of the present disclosure, the target pixel size includes the head pixel length of the optical fiber 110, and the actual size of the optical fiber 110 includes the actual width of the head of the optical fiber 110; accordingly, the controller 140 is also used to: use the quotient of the actual width of the head and the head pixel length as the actual size represented by a single pixel in the image.

[0131] In some embodiments of the present disclosure, the target pixel size includes the body pixel length of the optical fiber 110 and the head pixel length of the optical fiber 110, and the actual size of the optical fiber 110 includes the preset body actual length of the optical fiber 110; accordingly, the controller 140 is also used to: determine the actual body length corresponding to the body pixel length based on the mapping relationship between the preset body actual length and the preset body pixel length; and divide the product of the head pixel length and the actual body length by the camera focal length of the endoscope 120 to obtain the actual size represented by a single pixel in the image.

[0132] In some embodiments of the present disclosure, the controller 140 is further configured to: take the head of the optical fiber 110 in the image as the starting point, and draw target line segments in four directions of 0°, 90°, 180° and 270° respectively.

[0133] In some embodiments of the present disclosure, the controller 140 is further configured to: determine a target pixel size of the optical fiber 110 in the image when a measurement requirement for the actual size of the stone is detected.

[0134] The implementation process of the functions and effects of each component in the above system is specifically described in the implementation process of the corresponding steps in the above method, which will not be repeated here.

[0135] Those skilled in the art will appreciate that the embodiments of the present disclosure may be provided as methods, systems, or computer program products. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0136] The present disclosure is described with reference to flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the present disclosure. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing device to produce a machine, so that the instructions executed by the processor of the computer or other programmable data processing device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0137] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce an article of manufacture comprising an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0138] These computer program instructions can also be loaded onto a computer or other programmable data processing device so that a series of operating steps are executed on the computer or other programmable device to produce a computer-implemented process, thereby providing instructions for implementing the process. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0139] In the description of this specification, the description with reference to the terms "one embodiment / method", "some embodiments / methods", "example", "specific example", or "some examples" etc. means that the specific features, structures, or characteristics described in conjunction with the embodiment / method or example are included in at least one embodiment / method or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / method or example. Moreover, the specific features, structures, or characteristics described may be combined in any one or more embodiments / methods or examples in a suitable manner. In addition, those skilled in the art may combine and combine different embodiments / methods or examples described in this specification and features of different embodiments / methods or examples, unless they are contradictory.

[0140] In the description of the present disclosure, it should be understood that the terms "center", "longitudinal", "lateral", "length", "width", "thickness", "up", "down", "front", "back", "left", "right", "vertical", "horizontal", "top", "bottom", "inside", "outside", "clockwise", "counterclockwise" and the like indicate positions or positional relationships based on the positions or positional relationships shown in the accompanying drawings, and are only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and therefore cannot be understood as a limitation on the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes, and cannot be understood as indicating or implying relative importance or implicitly indicating the number of technical features indicated. Therefore, the features defined as "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "multiple" is at least two, such as two, three, etc., unless otherwise clearly and specifically defined.

[0141] In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and limited, the terms "installed", "connected", and "connected" should be understood in a broad sense. For example, it can be a fixed connection, a detachable connection, or an integral connection. It can be a mechanical connection or an electrical connection. It can be directly connected or indirectly connected through an intermediate medium. It can be the internal connection of two elements or the interaction relationship between two elements. For ordinary technicians in this field, the specific meanings of the above terms in the present disclosure can be understood according to the specific circumstances.

[0142] Those skilled in the art should understand that the above embodiments are only for the purpose of clearly illustrating the present disclosure, and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications may be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. An image processing method, characterized in that: include: Acquire an image captured by an endoscope, wherein the image includes the optical fiber; determining a target pixel size of the optical fiber in the image; Determining an actual size represented by a single pixel in the image based on the target pixel size and the actual size of the optical fiber; Taking the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image; adding a target line segment at the head of the optical fiber in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and The image after the target line segment is added is displayed, wherein the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber.

2. The image processing method according to claim 1, characterized in that: Determining a target pixel size of the optical fiber in the image includes: Extracting contours of the contents in the image to obtain contour information of the optical fiber; and A target pixel size of the optical fiber in the image is determined according to the contour information of the optical fiber.

3. The image processing method according to claim 2, characterized in that: Extracting the outline of the content in the image to obtain the outline information of the optical fiber includes: Extracting contours of the content in the image to obtain a plurality of contour information; and The plurality of profile information are filtered according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber.

4. The image processing method according to claim 3, characterized in that: The characteristic information of the optical fiber includes a profile size range, a target hue, and a target profile position; Filtering the plurality of profile information according to the characteristic information of the optical fiber to obtain the profile information of the optical fiber includes: The profile information whose profile size is within the profile size range, whose color tone is the target color tone, and whose profile position is the target profile position among the plurality of profile information is used as the profile information of the optical fiber.

5. The image processing method according to claim 2, characterized in that: Determining a target pixel size of the optical fiber in the image according to the profile information of the optical fiber includes: Determining coordinates of two target points of the optical fiber from profile information of the optical fiber; Based on the coordinates of the two target points, calculating the distance between the two target points; and The distance is taken as the target pixel size of the optical fiber in the image.

6. The image processing method according to claim 1, characterized in that: The target pixel size includes the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the actual width of the head of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: The quotient of the actual width of the head and the pixel length of the head is taken as the actual size represented by a single pixel in the image.

7. The image processing method according to claim 1, characterized in that: The target pixel size includes the pixel length of the body of the optical fiber and the pixel length of the head of the optical fiber, and the actual size of the optical fiber includes the preset actual length of the body of the optical fiber; Determining an actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber includes: Determine the actual body length corresponding to the body pixel length based on the mapping relationship between the preset actual body length and the preset body pixel length; as well as The actual size represented by a single pixel in the image is obtained by dividing the product of the pixel length of the head and the actual length of the body by the focal length of the camera of the endoscope.

8. The image processing method according to claim 1, characterized in that: Adding a target line segment at the head of the optical fiber in the image includes: Taking the head of the optical fiber in the image as the starting point, the target line segments are drawn in four directions of 0°, 90°, 180° and 270° respectively.

9. The image processing method according to claim 1, characterized in that: Determining a target pixel size of the optical fiber in the image includes: In case a need for measurement of the actual size of the stone is detected, a target pixel size of the optical fiber in the image is determined.

10. An image processing system, characterized in that: include: optical fiber; an endoscope, which is capable of taking images; a display screen capable of displaying the image; a controller, configured to determine a target pixel size of the optical fiber in the image when the optical fiber enters a shooting range of the endoscope; Determine the actual size represented by a single pixel in the image according to the target pixel size and the actual size of the optical fiber; take the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image; Adding a target line segment at the head of the optical fiber in the image; and controlling the display screen to display the image after adding the target line segment, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, and the target line segment is used to measure the actual size of the stone when the image also includes the stone contacted by the head of the optical fiber.

Images