Image processing system and image processing method

Abstract

The invention provides an image processing system and an image processing method. The image processing system of the present disclosure comprises: an endoscope capable of capturing an image including a stone; a display screen capable of displaying an image; the transmitting end and the receiving end of the laser distance measuring device extend to the head of the endoscope through optical fibers, and the laser distance measuring device is used for outputting the distance between the head and the calculus; the controller is used for determining the target pixel size of the stone in the image; predicting the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope and the target pixel size; a target line segment is added to the image, and the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; the display screen is controlled to display the image with the target line segment added, and the target line segment is used for evaluating the actual size of the stone.

Description

Technical Field

[0001] The present disclosure relates to the technical fields such as medical image processing, and particularly relates to an image processing system and an image processing method. Background Art

[0002] With the continuous progress of medical technology, endoscopic technology has been widely used in minimally invasive surgery, especially in the field of treatment of urinary system stones. Through its high-resolution imaging ability, the endoscope enables doctors to directly observe the lesion site in the body and perform precise operations.

[0003] In endoscopic surgery, accurately measuring the actual size of the stone is crucial for selecting an appropriate treatment plan (such as lithotripsy or direct removal). However, due to factors such as the viewing angle, focal length, and distortion of endoscopic images, it is difficult to accurately judge the actual size of the stone from the endoscopic images solely by the naked eye. Summary of the Invention

[0004] The present disclosure provides an image processing system and an image processing method.

[0005] According to one aspect of the present disclosure, there is provided an image processing system, including: An endoscope capable of taking images, wherein the images include stones; A display screen capable of displaying the images; A laser ranging device, the transmitting end and the receiving end of the laser ranging device respectively extend to the head of the endoscope through optical fibers, and the laser ranging device is used to output the distance between the head and the stone; and A controller for determining the target pixel size of the stone in the image; predicting the pixel length of the actual unit length in the image according to the distance between the head and the stone, the camera focal length of the endoscope, and the target pixel size; adding a target line segment in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and controlling the display screen to display the image after adding the target line segment, wherein the target line segment is used to evaluate the actual size of the stone.

[0006] According to the image processing method of at least one embodiment of the present disclosure, the laser ranging device is further used for: Calculating the distance between the transmitting end and the stone; and Calculating the distance between the head and the stone according to the distance between the transmitting end and the stone and the pre-calibrated distance between the transmitting end and the head.

[0007] An image processing method according to at least one embodiment of the present disclosure for determining a target pixel size of the calculus in the image includes: Performing contour extraction on the content in the image to obtain contour information of the calculus; and Determining the target pixel size of the calculus in the image according to the contour information of the calculus.

[0008] An image processing method according to at least one embodiment of the present disclosure for performing contour extraction on the content in the image to obtain contour information of the calculus includes: Performing contour extraction on the content in the image to obtain a plurality of contour information; and Filtering the plurality of contour information according to the characteristic information of the calculus to obtain the contour information of the calculus.

[0009] According to an image processing method of at least one embodiment of the present disclosure, the characteristic information of the calculus includes a contour size range, a target hue, and a target contour position; Filtering the plurality of contour information according to the characteristic information of the calculus to obtain the contour information of the calculus includes: Taking the contour information among the plurality of contour information whose contour size is within the contour size range, the hue is the target hue, and the contour position is the target contour position as the contour information of the calculus.

[0010] An image processing method according to at least one embodiment of the present disclosure for determining the target pixel size of the calculus in the image according to the contour information of the calculus includes: Determining the coordinates of two target points of the calculus from the contour information of the calculus; Calculating the distance between the two target points based on the coordinates of the two target points; and Taking the distance as the target pixel size of the calculus in the image.

[0011] An image processing method according to at least one embodiment of the present disclosure for predicting the pixel length of an actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size includes: Determining the actual size represented by a single pixel in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size; Taking the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image.

[0012] An image processing method according to at least one embodiment of the present disclosure, determining the actual size represented by a single pixel in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size, includes: Dividing the product value of the distance between the head and the calculus by the target pixel size by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image.

[0013] An image processing method according to at least one embodiment of the present disclosure, adding a target line segment to the image, includes: Taking the center point of the calculus in the image as the starting point, and drawing the target line segments in four directions of 0°, 90°, 180°, and 270° respectively.

[0014] In the image processing method according to at least one embodiment of the present disclosure, the pixel length of the target line segment has a negative correlation with the distance between the head and the calculus.

[0015] According to another aspect of the present disclosure, there is provided an image processing method, including: Obtaining an image captured by an endoscope, where the image includes a calculus; Determining the distance between the head of the endoscope and the calculus; Determining the target pixel size of the calculus in the image; Predicting the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size; Adding a target line segment to the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and Controlling a display screen to display the image after adding the target line segment, where the target line segment is used to evaluate the actual size of the calculus. Description of the Drawings

[0016] The drawings illustrate exemplary embodiments of the present disclosure and, together with the description thereof, are used to explain the principles of the present disclosure. These drawings are included to provide a further understanding of the present disclosure and are included in this specification and form a part of this specification.

[0017] Figure 1 is a flowchart of an image processing method according to an embodiment of the present disclosure.

[0018] Figure 2 is a schematic diagram of the process of determining the target pixel size of a calculus according to an embodiment of the present disclosure.

[0019] Figure 3 It is a schematic diagram of the process of determining the stone contour information according to an embodiment of the present disclosure.

[0020] Figure 4 It is a schematic diagram of the process of determining the target pixel size of the stone according to another embodiment of the present disclosure.

[0021] Figure 5 It is a schematic diagram of the process of determining the pixel length of the actual unit length according to an embodiment of the present disclosure.

[0022] Figure 6 It is an example schematic diagram of the image with the target line segment added according to an embodiment of the present disclosure.

[0023] Figure 7 It is a schematic flowchart of the image processing method according to another embodiment of the present disclosure Figure 8 It is a schematic block diagram of the structure of the image processing system according to an embodiment of the present disclosure. Detailed Embodiments

[0024] The present disclosure will be further described in detail below with reference to the accompanying drawings and examples. It can be understood that the specific examples described herein are only for explaining the relevant content, rather than limiting the present disclosure. In addition, it should be noted that for the convenience of description, only the parts related to the present disclosure are shown in the drawings.

[0025] It should be noted that, without conflict, the embodiments in the present disclosure and the features in the embodiments can be combined with each other. The technical solutions of the present disclosure will be described in detail below with reference to the accompanying drawings and embodiments.

[0026] The actual size of the stone judged solely by the naked eye from the endoscopic image often has a large error, which poses challenges to the success rate and safety of the surgery. In the related art, the measurement of the actual size of the stone relies on preoperative imaging examinations (such as CT or X-ray). However, the preoperative image cannot reflect the real-time state of the stone during the operation. Taking the endoscopic laser lithotripsy surgery as an example, the stone may generate broken particles with significantly different sizes after being affected by the laser energy, and the particle size distribution has significant randomness. If the actual size of the broken particles cannot be determined in real time, it may lead to the retention of broken particles with a larger diameter, which may cause the recurrence of the disease in the future. Therefore, it is crucial to measure the actual size of the stone in real time.

[0027] For this reason, the present disclosure proposes an image processing method.

[0028] The image processing method of the present disclosure can be used to measure the actual size of the stone in real time through the endoscopic image during the endoscopic examination or operation. In the present disclosure, the electronic device includes but is not limited to mobile phones, tablet computers, laptop computers, personal computers, etc.

[0029] For the sake of convenience in description and to make the technical solutions of the specific embodiments of the present disclosure easier to understand, before describing the image processing method implemented in the present disclosure, the technical terms involved in the specific embodiments of the present disclosure are explained as follows: Size refers to the length and size of an object.

[0030] Length is a measure of one-dimensional space and is the distance from point to point. Usually, when measuring the side length of a line segment in two-dimensional space, the one with a larger numerical value of the length is called the length, and the one that is not larger than its value or is on the "side" is called the width. Width is also a kind of length measurement.

[0031] The target pixel size refers to the number of pixels occupied by the target content in the target direction in the image.

[0032] The pixel length refers to the number of pixels occupied by a certain part of the content in the image in the length direction of that part of the content.

[0033] The pixel width refers to the number of pixels occupied by a certain part of the content in the image in the width direction of that part of the content.

[0034] The actual size, actual length, and actual width respectively refer to the size, length, and width of an object in the real world (such as the world geodetic coordinate system).

[0035] The unit length is a reference standard set artificially. The unit length is the reference standard that can be used for reference. It has no fixed value and varies according to the setting.

[0036] Figure 1 Fig. shows the overall flowchart of the image processing method M100 according to an embodiment of the present disclosure. As Figure 1 shown, the method includes steps S110 to S160. Among them, the method can be executed by an electronic device such as a mobile phone or a computer.

[0037] Specifically, Figure 1 the method shown includes: S110. Obtain the image captured by the endoscope, where the image includes a calculus; S120. Obtain the distance between the head of the endoscope and the calculus; Exemplarily, the laser ranging device can output the distance between the head of the endoscope and the calculus. The electronic device can communicate with the laser ranging device to obtain the distance between the head of the endoscope and the calculus through the laser ranging device. The transmitting end and the receiving end of the laser ranging device can extend to the head of the endoscope through optical fibers respectively. Furthermore, the laser ranging device can calculate the distance between the transmitting end and the calculus through the flight time of the laser signal, and calculate the distance between the head and the calculus according to the distance between the transmitting end and the calculus and the pre-calibrated distance between the transmitting end and the head. Since an optical fiber is required for laser energy transmission during the endoscopic laser lithotripsy surgery, and the distance between the head of the endoscope and the calculus in the image processing method of the present disclosure embodiment can be realized based on the optical fiber, that is to say, in the present disclosure, there is no need to introduce additional tools during the surgery, and the actual size of the calculus can be measured quickly, intuitively and in real time by using the existing tools during the surgery, reducing the operation complexity and the infection risk.

[0038] S130. Determine the target pixel size of the calculus in the image; The target pixel size of the calculus in the image is the number of pixels occupied by the calculus in the image in the target direction of the calculus. The target direction can be set according to the actual situation. For example, it can be the length direction of the calculus, the width direction of the calculus, etc.

[0039] Since the distance between the head of the endoscope and the calculus obtained in step S120 is obtained, it can be considered that the factors such as the viewing angle, focal length, and distortion corresponding to the pixel points in the calculus area in the image are basically the same, while the factors such as the viewing angle, focal length, and distortion corresponding to the pixel points outside the calculus area in the image do not match this distance. Therefore, the target pixel size of the calculus in the image is selected for subsequent calculation, so as to improve the accuracy of the calculation result.

[0040] Exemplarily, the target pixel size of the calculus in the image can be determined by an image recognition algorithm or an image recognition model in related technologies.

[0041] S140. Predict the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size; The camera focal length of the endoscope can be pre-calibrated.

[0042] The actual unit length is the unit length in the real world (such as the world geodetic coordinate system). The actual unit length can be the safety size threshold of the calculus. For example, if the actual size of the calculus is less than or equal to the safety size threshold, the calculus may not be subjected to lithotripsy treatment. If the actual size of the calculus is greater than the safety size threshold, the calculus should be subjected to lithotripsy treatment. The actual unit length can also be other artificially set values, such as 0.1 mm, 0.5 mm, 1 mm, 5 mm, 1 cm, etc., which are not limited herein.

[0043] Since the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size are all known quantities, the correlation between the pixel size in the image and the actual size in the real world can be known through the imaging characteristics of the endoscope camera, thus facilitating the prediction of the pixel length of the actual unit length in the image.

[0044] It should be noted that the specific values mentioned in this disclosure are only for detailed illustration of the implementation of this disclosure as examples, and should not be construed as a limitation of this disclosure. In other examples, embodiments or implementations, other values can be selected according to this disclosure, and no specific limitation is made here.

[0045] S150. Add a target line segment to the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; The pixel width of the target line segment can be a pre-calibrated default value, and this disclosure does not limit the pixel width of the target line segment.

[0046] Exemplarily, the number of target line segments can be one or multiple. Multiple target line segments can be distributed in the image along the same direction. For example, all target line segments are connected end to end along the same direction; multiple target line segments can also be distributed in the image along multiple directions. For example, some of the target line segments are connected end to end along the first direction while the remaining target line segments are connected end to end along the second direction, and no limitation is made here.

[0047] At least one target line segment can have the center point of the calculus as the midpoint, or have the center point of the calculus as the starting point, or have the center point of the calculus as the intermediate point other than the midpoint and the endpoint, or have the pixel point within the preset pixel length range of the center point of the calculus as the midpoint, or have the pixel point within the preset pixel length range of the center point of the calculus as the starting point, or have the pixel point within the preset pixel length range of the center point of the calculus as the intermediate point other than the midpoint and the endpoint, and no limitation is made here.

[0048] Exemplarily, the pixel length of the target line segment is negatively correlated with the distance between the head and the calculus. In this way, during the process of the endoscope moving towards the calculus, the target line segment in the displayed image can correspondingly become longer, and during the process of the endoscope moving away from the calculus, the target line segment in the displayed image can correspondingly become shorter, thereby adjusting the length of the target line segment in the image in real time and facilitating the user to accurately measure the actual size of the calculus at any time.

[0049] S160. Control the display screen to display the image after adding the target line segment, where the target line segment is used to evaluate the actual size of the calculus.

[0050] Since the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image, the target line segment can serve as a scale, thus facilitating the measurement of the actual size of the calculus through the target line segment. Since the target line segment is equivalent to a scale, in addition to the calculus, the target line segment can also be used to measure the actual sizes of other objects or tissues, and the specific measurement method can be the same as that for measuring the calculus.

[0051] The orientation of the target line segment in the image can be fixed or dynamically adjusted, which is not limited herein. Exemplarily, a plurality of preset orientations are determined in advance. Then, during the process of displaying the image with the target line segment added, if no operation for adjusting the orientation of the target line segment is received, the target line segment is displayed in the default orientation among the plurality of preset orientations in the image; if an operation for adjusting the orientation of the target line segment is received (for example, the user rotates the orientation adjustment knob of the target line segment or selects a preset orientation), the orientation of the target line segment displayed in the image is switched to a preset orientation other than the current orientation among the plurality of preset orientations. In this way, the dynamic adjustment of the orientation of the target line segment is achieved, which can be applied to the measurement of the actual sizes of calculi with different shapes and positions, improving the accuracy of the measurement of the actual size of the calculus.

[0052] The image processing method according to the embodiments of the present disclosure acquires an image including a calculus captured by an endoscope, acquires the distance between the head of the endoscope and the calculus, determines the target pixel size of the calculus in the image, predicts the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size, adds a target line segment in the image, and controls the display screen to display the image with the target line segment added. Thus, by using the distance between the head of the endoscope and the calculus, the camera focal length of the endoscope, and the target pixel size as references, the accurate conversion between the actual unit length and the pixel length in the image is achieved. Moreover, the target line segment added in the image has the same pixel length as the actual unit length, enabling the user to intuitively measure the actual size of the calculus in the image with the target line segment added, greatly simplifying the operation process of the user, significantly improving the operation efficiency, and at the same time improving the measurement accuracy, providing more efficient and accurate technical support for medical work.

[0053] Regarding step S130, in some embodiments of the present disclosure, it may include steps S131 and S132 as Figure 2 shown.

[0054] S131. Extract the contour of the content in the image to obtain the contour information of the calculus.

[0055] In the image captured by the endoscope, in addition to stones, there may be human tissues and other contents. Through contour extraction, the contour of the stone in the image can be automatically determined, thus providing data support for determining the target pixel size of the stone.

[0056] Exemplarily, the content in the image can be subjected to contour extraction through edge detection algorithms, image segmentation algorithms or contour extraction models in related technologies, so as to obtain the contour information of the stone.

[0057] S132. Determine the target pixel size of the stone in the image according to the contour information of the stone.

[0058] The contour information of the stone can represent the boundary position of the stone in the image, such as the coordinates of the pixel points representing the boundary of the stone in the image. Therefore, after determining the contour information of the stone, the target pixel size of the stone in the image can be determined through the contour information.

[0059] The image processing method of the above embodiment performs contour extraction on the content in the image to obtain the contour information of the stone, and determines the target pixel size of the stone in the image according to the contour information of the stone. Thus, the contour of the stone in the image is automatically recognized, avoiding cumbersome manual annotation, and improving the efficiency and consistency of measuring the target pixel size of the stone. At the same time, by using the extracted contour to determine the target pixel size, interference from other contents in the image can be avoided, improving the accuracy of the determined target pixel size, and providing reliable data for predicting the pixel length of the actual unit length in the image.

[0060] Regarding step S131, in some embodiments of the present disclosure, it may include steps S1311 and S1312 as Figure 3 shown.

[0061] S1311. Perform contour extraction on the content in the image to obtain multiple contour information.

[0062] Exemplarily, the image can be first converted from the RGB color space to a grayscale image to reduce the computational complexity and highlight the key features of the image; then, Gaussian blur is applied to the converted grayscale image to smooth the image and reduce the influence of noise on subsequent processing steps; then, the pixel values in the area where the pixel values in the Gaussian-blurred grayscale image are greater than the set threshold are set to 255, and the pixel values in the area where the pixel values in the Gaussian-blurred grayscale image are less than or equal to the set threshold are set to 0, so as to convert the Gaussian-blurred grayscale image into a binary image; finally, edge detection is performed on the content in the binary image to obtain multiple contour information.

[0063] S1312. Filter the multiple contour information according to the characteristic information of the stone to obtain the contour information of the stone.

[0064] The features of multiple contour information are different. Therefore, based on the feature information of the stone, filtering the multiple contour information can accurately determine the contour information of the stone from the multiple contour information.

[0065] The image processing method of the above embodiment extracts the contours of the content in the image to obtain multiple contour information, and filters the multiple contour information according to the feature information of the stone to obtain the contour information of the stone. Thus, the contour information of the stone can be accurately separated from the complex image, effectively reducing the interference of irrelevant contours, and improving the robustness and reliability of stone contour recognition. It is especially applicable to the situation of possible light changes or cluttered backgrounds in endoscopic images.

[0066] Regarding step S1312, in some embodiments of the present disclosure, the feature information of the stone includes the contour size range, the target hue, and the target contour position; correspondingly, step S1312 may specifically be: taking the contour information in the multiple contour information whose contour size is within the contour size range, the hue is the target hue, and the contour position is the target contour position as the contour information of the stone.

[0067] The feature information of the stone can be pre-configured according to the actual situation.

[0068] Exemplarily, the contour information outside the contour size range in the multiple contour information can be filtered out first to obtain the contour information after the first filtering; then the contour information whose hue is not the target hue in the contour information after the first filtering can be filtered out to obtain the contour information after the second filtering; finally, the contour information whose contour position is not the target contour position in the contour information after the second filtering can be filtered out. At this time, the remaining contour information is the contour information of the stone.

[0069] The image processing method of the above embodiment filters the multiple contour information based on the contour size range, the target hue, and the target contour position, so as to ensure that the obtained contour information after filtering highly matches the actual characteristics of the stone, improve the accuracy of stone contour information extraction, reduce the misjudgment rate, and provide reliable data for predicting the pixel length of the actual unit length in the image.

[0070] Regarding step S132, in some embodiments of the present disclosure, it may include steps S1321 to S1323 as Figure 4 shown.

[0071] S1321. Determine the coordinates of two target points of the stone from the contour information of the stone.

[0072] The target points can be automatically determined by the electronic device based on pre-configured rules. For example, if the pre-configured rule is to use the intersection points of the horizontal line (or a straight line in other directions) passing through the center point of the stone in the image and the contour of the stone as the target points, then when the electronic device obtains an image including the stone captured by the endoscope, it can first identify the center point of the stone in the image, and then use the two intersection points of the horizontal line passing through this center point and the contour of the stone as the two target points.

[0073] S1322. Calculate the distance between the two target points based on the coordinates of the two target points.

[0074] S1323. Use the distance as the target pixel size of the stone in the image.

[0075] The image processing method of the above embodiment determines the coordinates of two target points of the stone from the contour information of the stone, calculates the distance between the two target points based on the coordinates of the two target points, and uses the distance as the target pixel size of the stone in the image. Thus, through the method of coordinate calculation, the target pixel size of the stone in the image can be determined quickly and accurately, laying a foundation for predicting the pixel length of the actual unit length in the image.

[0076] Regarding step S130, in other embodiments, the minimum rectangular bounding box enclosing the contour of the stone can be calculated, and the pixel size of the minimum rectangular bounding box can be used as the target pixel size of the stone in the image; or, traverse the pixel points representing the contour of the stone, determine the pixel points representing the contour of the stone in the target direction, and use the number of pixel points representing the contour of the stone in the target direction as the target pixel size of the stone in the image, which is not limited here.

[0077] Regarding step S140, in some embodiments of the present disclosure, it may include steps S141 and S142 as Figure 5 shown.

[0078] S141. Determine the actual size represented by a single pixel in the image according to the distance between the head and the stone, the camera focal length of the endoscope, and the target pixel size.

[0079] Since the camera focal length of the endoscope can be pre-calibrated, after obtaining the distance between the head and the stone and the target pixel size, the actual size represented by a single pixel in the image can be calculated based on the similar triangle algorithm.

[0080] Exemplarily, since the actual size represented by a single pixel in the image is determined based on the distance between the endoscopic head and the stone, and the distances between other contents in the image and the endoscopic head may be different from that of the stone, therefore, the "actual size represented by a single pixel in the image" can be understood as the actual size represented by a single pixel within a certain range around the stone and its surroundings in the image, rather than the actual size represented by each pixel in the entire image.

[0081] Furthermore, considering that the target pixel size of the stone may be determined by randomly selecting two target points from the stone contour based on a pre-configured fixed rule, and these two target points may not be the maximum length of the irregular stone, therefore, without obtaining the actual size represented by a single pixel, the product of the actual size represented by a single pixel and the target pixel size of the stone is not automatically used as the actual size of the stone and the actual size of the stone is not prompted to the user. Instead, by adding a target line segment in the image, it assists the user to measure the actual size of the stone by themselves, thus avoiding misleading the user and improving the accuracy of the measured actual size of the stone.

[0082] S142. Use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0083] Specifically, divide the actual unit length by the actual size represented by a single pixel, and use the obtained result as the pixel length of the actual unit length in the image.

[0084] The image processing method of the above embodiment determines the actual size represented by a single pixel in the image according to the distance between the head and the stone, the camera focal length of the endoscope, and the target pixel size, and uses the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image. Thus, it makes full use of the imaging characteristics of the endoscope, can accurately obtain the corresponding relationship between a single pixel and the actual size, and further accurately predict the pixel length of the actual unit length in the image.

[0085] Regarding step S141, in some embodiments of the present disclosure, specifically, it may be: divide the product of the distance between the head and the stone and the target pixel size by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image.

[0086] Specifically, the actual size represented by a single pixel in the image = the distance between the head and the stone × the target pixel size ÷ the camera focal length of the endoscope.

[0087] The image processing method of the above embodiment makes full use of the imaging characteristics of the endoscope, so as to accurately obtain the actual size represented by a single pixel.

[0088] Regarding step S150, in some embodiments of the present disclosure, specifically, it may be: taking the center point of the stone in the image as the starting point, and drawing target line segments in four directions of 0°, 90°, 180°, and 270° respectively.

[0089] Exemplarily, the 0° direction can be pre-calibrated. For example, the horizontal right direction in the image can be calibrated as the 0° direction. In the case where the 0° direction is calibrated, the directions of other angles can be obtained by rotating counterclockwise or clockwise from the 0° direction.

[0090] In one example, the image after adding the target line segments is as Figure 6 shown. The figure includes four target line segments (in a cross shape) facing four directions located at the center point of the stone and the stone.

[0091] The image processing method of the above embodiment can form a cross-shaped virtual scale at the center of the stone in the image by drawing target line segments from the center point of the stone in four directions, thereby facilitating the user to intuitively compare the relative sizes of the stone and the target line segments from different directions, and further improving the accuracy, flexibility, and visualization effect of the actual size measurement of the stone.

[0092] Please combine Figure 7 In one example, the image processing method may include the following steps S201 to S211. The content related to steps S201 to S211 can be referred to the description of the above embodiment. For the sake of brevity, it will not be repeated here.

[0093] In step S201, an image captured by the endoscope is obtained, where the image includes a stone.

[0094] In step S202, it is determined whether a measurement requirement for the actual size of the stone is detected. If so, step S203 is entered; otherwise, step S210 is entered.

[0095] In step S203, in the case where a measurement requirement for the actual size of the stone is detected, the distance between the endoscope head and the stone is obtained. Exemplarily, a stone measurement mode switching button can be provided. For example, a physical button is provided on the operation handle of the endoscope, or a virtual button is provided on the human-machine interaction interface. Then, in the case where it is detected that the stone measurement mode switching button is triggered, it can be determined that a measurement requirement for the actual size of the stone is detected. In this way, the distance between the endoscope head and the stone is obtained only when a measurement requirement for the actual size of the stone is detected, realizing the function of processing on demand, avoiding unnecessary computational overhead, improving the resource utilization efficiency of the system, and being particularly suitable for dynamically responding to user requirements during the operation.

[0096] In step S204, the content in the image is subjected to contour extraction to obtain the contour information of the calculus.

[0097] In step S205, according to the characteristic information of the calculus, multiple contour information is filtered to obtain the contour information of the calculus.

[0098] In step S206, according to the contour information of the calculus, the target pixel size of the calculus in the image is determined.

[0099] In step S207, according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size, the actual size represented by a single pixel in the image is determined.

[0100] In step S208, the quotient of the actual unit length and the actual size represented by a single pixel is used as the pixel length of the actual unit length in the image.

[0101] In step S209, a target line segment is added to the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image.

[0102] In step S210, the image after adding the target line segment is displayed.

[0103] In step S211, in the case where the measurement requirement for the actual size of the calculus is not detected, the image captured by the endoscope is directly displayed.

[0104] Based on any of the above embodiments, the present disclosure also provides an image processing system.

[0105] Figure 8 The structural schematic block diagram of the image processing system according to an embodiment of the present disclosure.

[0106] As Figure 8 shown, the image processing system includes an endoscope 110, a display screen 120, a laser ranging device 130, and a controller 140, wherein the controller 140 is respectively connected to the endoscope 110, the display screen 120, and the laser ranging device 130.

[0107] The endoscope 110 can capture an image, wherein the image includes a calculus.

[0108] The display screen 120 can display an image.

[0109] The transmitting end and the receiving end of the laser ranging device 130 respectively extend to the head of the endoscope 110 through optical fibers, and the laser ranging device 130 is used to output the distance between the head and the calculus.

[0110] The controller 140 is configured to determine the target pixel size of the calculus in the image; predict the pixel length of the actual unit length in the image based on the distance between the head and the calculus, the camera focal length of the endoscope 110, and the target pixel size; add a target line segment in the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and control the display screen 120 to display the image with the target line segment added, where the target line segment is used to evaluate the actual size of the calculus.

[0111] The display screen 120 and the controller 140 may be provided on the same electronic device or may be provided on different electronic devices respectively, which is not limited herein.

[0112] In some embodiments of the present disclosure, the laser ranging device 130 is further configured to: calculate the distance between the transmitting end and the calculus; and calculate the distance between the head and the calculus based on the distance between the transmitting end and the calculus and the pre-calibrated distance between the transmitting end and the head.

[0113] In some embodiments of the present disclosure, the controller 140 is further configured to: extract the contour of the content in the image to obtain the contour information of the calculus; and determine the target pixel size of the calculus in the image based on the contour information of the calculus.

[0114] In some embodiments of the present disclosure, the controller 140 is further configured to: extract the contour of the content in the image to obtain a plurality of contour information; and filter the plurality of contour information based on the feature information of the calculus to obtain the contour information of the calculus.

[0115] In some embodiments of the present disclosure, the feature information of the calculus includes the contour size range, the target hue, and the target contour position; correspondingly, the controller 140 is further configured to: use the contour information in the plurality of contour information whose contour size is within the contour size range, the hue is the target hue, and the contour position is the target contour position as the contour information of the calculus.

[0116] In some embodiments of the present disclosure, the controller 140 is further configured to: determine the coordinates of two target points of the calculus from the contour information of the calculus; calculate the distance between the two target points based on the coordinates of the two target points; and use the distance as the target pixel size of the calculus in the image.

[0117] In some embodiments of the present disclosure, the controller 140 is further configured to: determine the actual size represented by a single pixel in the image based on the distance between the head and the calculus, the camera focal length of the endoscope 110, and the target pixel size; and use the quotient of the actual unit length and the actual size represented by a single pixel as the pixel length of the actual unit length in the image.

[0118] In some embodiments of the present disclosure, the controller 140 is further configured to: divide the product of the distance between the head and the calculus by the camera focal length of the endoscope 110 to obtain the actual size represented by a single pixel in the image.

[0119] In some embodiments of the present disclosure, the controller 140 is further configured to: draw target line segments from the center point of the calculus in the image in four directions of 0°, 90°, 180°, and 270°, respectively.

[0120] In some embodiments of the present disclosure, the pixel length of the target line segment is negatively correlated with the distance between the head and the calculus.

[0121] For the implementation processes of the functions and roles of each component in the above system, please refer to the implementation processes of the corresponding steps in the above method for details, which will not be elaborated herein.

[0122] Those skilled in the art should understand that the embodiments of the present disclosure may be provided as a method, a system, or a computer program product. Therefore, the present disclosure may take the form of a complete hardware embodiment, a complete software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present disclosure may take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0123] The present disclosure is described with reference to the flowcharts and / or block diagrams of methods, apparatuses (systems), and computer program products according to the present disclosure. It should be understood that each flow and / or block in the flowcharts and / or block diagrams, as well as the combination of flows and / or blocks in the flowcharts and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, such that the instructions executed by the processor of the computer or other programmable data processing devices generate means for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0124] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory generate a manufactured article including instruction means for implementing the functions specified in Figure 1 one or more flows and / or blocks Figure 1 one or more blocks.

[0125] These computer program instructions can also be loaded onto a computer or other programmable data processing apparatus, so that a series of operation steps are performed on the computer or other programmable apparatus to generate a computer-implemented process, and thus the instructions executed on the computer or other programmable apparatus provide steps for implementing the functions specified in a process Figure 1 one process or a plurality of processes and / or blocks Figure 1 steps of a block or a plurality of blocks.

[0126] In the description of this specification, the descriptions with reference to the terms "one embodiment / way", "some embodiments / ways", "example", "specific example", or "some examples", etc. mean that the specific features, structures, or characteristics described in connection with the embodiment / way or example are included in at least one embodiment / way or example of the present disclosure. In this specification, the schematic representations of the above terms do not necessarily refer to the same embodiment / way or example. Moreover, the specific features, structures, or characteristics described can be combined in any one or more embodiments / ways or examples in a suitable manner. In addition, without conflict, those skilled in the art can combine and combine the different embodiments / ways or examples described in this specification and the features of different embodiments / ways or examples.

[0127] In the description of the present disclosure, it should be understood that the orientation or positional relationship indicated by the terms "center", "longitudinal", "transverse", "length", "width", "thickness", "upper", "lower", "front", "rear", "left", "right", "vertical", "horizontal", "top", "bottom", "inner", "outer", "clockwise", "counterclockwise", etc. is based on the orientation or positional relationship shown in the drawings, and is only for the convenience of describing the present disclosure and simplifying the description, rather than indicating or implying that the device or element referred to must have a specific orientation, be constructed and operated in a specific orientation, and thus cannot be understood as a limitation of the present disclosure. In addition, the terms "first" and "second" are only used for descriptive purposes and cannot be understood as indicating or implying relative importance or implicitly indicating the number of the indicated technical features. Thus, the features defined with "first" and "second" may explicitly or implicitly include at least one of the features. In the description of the present disclosure, the meaning of "a plurality" is at least two, such as two, three, etc., unless otherwise specifically defined.

[0128] In the description of the present disclosure, it should be noted that, unless otherwise clearly specified and defined, the terms "installed", "connected", and "coupled" should be understood in a broad sense. For example, it may be a fixed connection, a detachable connection, or an integral connection. It may be a mechanical connection or an electrical connection. It may be directly connected or indirectly connected through an intermediate medium, and it may be the communication inside two components or the interaction relationship between two components. For those of ordinary skill in the art, the specific meanings of the above terms in the present disclosure can be understood according to specific circumstances.

[0129] Those skilled in the art should understand that the above embodiments are merely for clearly illustrating the present disclosure and are not intended to limit the scope of the present disclosure. For those skilled in the art, other changes or modifications can be made based on the above disclosure, and these changes or modifications are still within the scope of the present disclosure.

Claims

1. An image processing system, characterized in that, Comprising: An endoscope capable of capturing an image, wherein the image includes a calculus. A display screen capable of displaying the image. A laser ranging device, the transmitting end and the receiving end of which respectively extend to the head of the endoscope through optical fibers, and the laser ranging device is used to output the distance between the head and the calculus; and A controller for determining the target pixel size of the calculus in the image; predicting the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope and the target pixel size; adding a target line segment in the image, wherein the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and controlling the display screen to display the image after adding the target line segment, wherein the target line segment is used to evaluate the actual size of the calculus.

2. The image processing system according to claim 1, wherein The laser ranging device is further used for: Calculating the distance between the transmitting end and the calculus; and Calculating the distance between the head and the calculus according to the distance between the transmitting end and the calculus and the pre-calibrated distance between the transmitting end and the head.

3. The image processing system according to claim 1, wherein Determining the target pixel size of the calculus in the image includes: Performing contour extraction on the content in the image to obtain the contour information of the calculus; and Determining the target pixel size of the calculus in the image according to the contour information of the calculus.

4. The image processing system according to claim 3, wherein, Performing contour extraction on the content in the image to obtain the contour information of the calculus includes: Performing contour extraction on the content in the image to obtain a plurality of contour information; and Filtering the plurality of contour information according to the characteristic information of the calculus to obtain the contour information of the calculus.

5. The image processing system according to claim 4, wherein The characteristic information of the calculus includes a contour size range, a target hue and a target contour position. Filtering the plurality of contour information according to the characteristic information of the calculus to obtain the contour information of the calculus includes: Taking the contour information in the plurality of contour information with a contour size within the contour size range, a hue of the target hue and a contour position of the target contour position as the contour information of the calculus.

6. The image processing system according to claim 3, wherein Determining the target pixel size of the calculus in the image according to the contour information of the calculus includes: Determining the coordinates of two target points of the calculus from the contour information of the calculus; Calculating the distance between the two target points based on the coordinates of the two target points; and Taking the distance as the target pixel size of the calculus in the image.

7. The image processing system according to claim 1, characterized in that Predicting the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope and the target pixel size includes: Determining the actual size represented by a single pixel in the image according to the distance between the head and the calculus, the camera focal length of the endoscope and the target pixel size; and Taking the quotient of the actual unit length and the actual size represented by the single pixel as the pixel length of the actual unit length in the image.

8. The image processing system according to claim 7, wherein Determine the actual size represented by a single pixel in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size, including: Divide the product of the distance between the head and the calculus and the target pixel size by the camera focal length of the endoscope to obtain the actual size represented by a single pixel in the image.

9. The image processing system according to claim 1, wherein Add a target line segment to the image, including: Draw the target line segment in four directions of 0°, 90°, 180°, and 270° respectively starting from the center point of the calculus in the image.

10. An image processing method, characterized in that, Including: Obtain an image captured by an endoscope, where the image includes a calculus; Obtain the distance between the head of the endoscope and the calculus; Determine the target pixel size of the calculus in the image; Predict the pixel length of the actual unit length in the image according to the distance between the head and the calculus, the camera focal length of the endoscope, and the target pixel size; Add a target line segment to the image, where the pixel length of the target line segment is the same as the pixel length of the actual unit length in the image; and Control the display screen to display the image after adding the target line segment, where the target line segment is used to evaluate the actual size of the calculus.

Images