Endoscope system focusing method and endoscope system

Abstract

This invention relates to the field of image processing technology and discloses a focusing method for an endoscope system and an endoscope system. The method includes: step S1, acquiring the current image captured by the endoscope at the current focal length; step S2, determining the candidate focusing region on the current image; step S3, determining the sharpness evaluation value of the candidate focusing region; step S4, determining whether the sharpness evaluation value meets the preset sharpness condition; if not, changing the focal length of the endoscope and returning to step S1; if yes, determining that the endoscope system has completed focusing. This invention effectively addresses the problem of incorrect focusing, not only achieving more precise focusing and ensuring the acquisition of high-quality images, but also improving focusing efficiency, reducing the risk of blurring, and effectively enhancing the safety and effectiveness of medical operations.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and more specifically to a focusing method for an endoscope system and an endoscope system. Background Technology

[0002] Endoscopic technology is an important diagnostic tool in the medical field, capable of capturing real-time images of internal tissues. In recent years, with the rapid development of image processing technology, the imaging quality of endoscopes has significantly improved, and autofocus has gradually become an important component of their intelligence, enabling doctors to clearly observe target areas.

[0003] However, during surgery, factors such as tissue movement, instrument obstruction, or camera shake cause drastic changes in the scene. If a fixed central area or simple contrast detection is still used to select the focus area, the autofocus system may focus on irrelevant tissue, reflective surgical instruments, or noise at the image edges, rather than the lesion or target tissue that the surgeon is actually focusing on. This misfocusing will result in blurring of critical areas, thus affecting the surgical outcome. Summary of the Invention

[0004] This invention provides a focusing method and an endoscope system to solve the problem of low clarity when focusing on a target area using an endoscope in the prior art.

[0005] In a first aspect, the present invention provides a focusing method for an endoscope system, the method comprising: Step S1: Acquire the current image captured by the endoscope at the current focal length; Step S2: Determine the focus candidate region on the current image; Step S3: Determine the sharpness evaluation value of the candidate focus area; Step S4: Determine whether the sharpness evaluation value meets the preset sharpness conditions; if not, change the endoscope focal length and return to step S1; if yes, confirm that the endoscope system has completed focusing.

[0006] The focusing method for an endoscopic system provided by this invention first determines the candidate focus areas on the current image, and then calculates the sharpness evaluation value of the candidate focus areas. When the sharpness evaluation value of the candidate focus area at the current focal length does not meet the sharpness requirements, the system actively adjusts the focal length and re-acquires the image. This ensures that the system focuses only on the most relevant areas rather than the entire image, effectively addressing the problem of incorrect focusing on the main subject during surgery, thereby improving the autofocus performance of the endoscope. Furthermore, by automatically adjusting the focal length based on the sharpness evaluation value of the candidate focus areas, the system not only achieves more precise focusing and ensures the acquisition of high-quality images, but also improves focusing efficiency and reduces the risk of blurring, thus enhancing the safety and effectiveness of medical procedures.

[0007] In one alternative implementation, determining the focus candidate region on the current image includes: The current image is input into a pre-built neural network detection model to determine the target object detection region; If there is only one target object detection region, the target object detection region is used as the focus candidate region; When there are two or more target object detection areas, the system responds to the target area selected by the user and uses the target area as a focus candidate area.

[0008] In one alternative implementation, if the neural network detection model fails to detect the target object, the following steps are used to determine the candidate focus region; Convert the current image to a grayscale image; Perform frequency domain transformation on the grayscale image to determine the amplitude spectrum and phase spectrum; After performing a logarithmic operation on the amplitude spectrum, a low-pass filter is applied to obtain a background frequency domain estimate. Determine the spectral residuals based on background frequency domain estimation; Based on the spectral residuals and phase spectrum, the saliency map is determined; Based on the saliency map, candidate focus regions are determined.

[0009] In one alternative implementation, determining the focus candidate region based on the saliency map includes: The current image is input into a pre-built neural network recognition model to determine the surgical instrument region in the current image; Extract multiple candidate regions of interest from the saliency map; Candidate regions of interest that contain surgical instruments are removed; Determine the focus candidate region from the eliminated candidate regions of interest.

[0010] In this embodiment, by combining a visual saliency algorithm and a surgical instrument detection and recognition model, the surgical instrument region in the image can be effectively identified, and overlapping parts can be eliminated, thereby ensuring that the selected focus candidate region is more accurate. In addition, this embodiment adopts a multimodal screening mechanism of "target object detection first, clean region of interest as a backup," which greatly avoids the risk of focusing on surgical instruments or irrelevant backgrounds, ensuring that the focus subject is always the clinical focus.

[0011] In one alternative implementation, determining the focus candidate region on the current image includes: Determine the first candidate focus area under the previous focal length; Using the first focus candidate region as a template, determine the second focus candidate region at the current focal length in the current image, and use the second focus candidate region as the focus candidate region on the current image; the size and content of the second focus candidate region are the same as those of the first focus candidate region.

[0012] In this implementation, the first focused area is used as a template for region matching in subsequent images. Region matching ensures that the comparison at different focal lengths is of the same content, providing a fair basis for sharpness calculation and thus improving the accuracy of image processing.

[0013] In one alternative implementation, determining the sharpness evaluation value of the focus candidate region includes: Based on the first focus candidate area, determine whether the second focus candidate area needs brightness correction; The second focus candidate region, whether it has undergone brightness correction or not, is filtered through different frequency channels to obtain the image response map of each frequency channel. Determine the initial sharpness value for the image response map of each frequency channel; Based on all initial sharpness values, a sharpness evaluation value for the second focus candidate region is determined.

[0014] In this embodiment, multiple bandpass filters are used to calculate sharpness separately, and then weighted and fused. This results in a more comprehensive and robust sharpness evaluation value. Furthermore, multi-band evaluation can comprehensively capture the texture details of the image. Compared with a single gradient function, it is more adaptable to different types of image content, the evaluation results are more in line with human subjective perception, and the noise resistance is also better.

[0015] In one optional implementation, determining whether a second focus candidate region needs brightness correction based on a first focus candidate region includes: Determine the first average brightness value of the first focus candidate region; Determine the second average brightness value of the second focus candidate region; Determine the brightness difference between the first average brightness value and the second average brightness value; Determine whether the brightness difference value exceeds the preset brightness difference threshold; If the brightness difference value exceeds the preset brightness difference threshold and the second average brightness value is less than the first average brightness value, it is determined that the second focus candidate area needs to be brightness corrected.

[0016] In one optional implementation, when the brightness difference value exceeds a preset brightness difference threshold and the first average brightness value is less than the second average brightness value, brightness correction is performed on the first focus candidate area.

[0017] In this embodiment, before calculating sharpness, the brightness consistency of areas at different focal lengths is adjusted. This can effectively eliminate the influence of exposure, that is, eliminate the interference of exposure changes on the sharpness evaluation function, thereby improving the accuracy of sharpness assessment.

[0018] In one optional implementation, determining whether the sharpness evaluation value meets a preset sharpness condition includes: Determine whether the sharpness evaluation value exceeds the first preset sharpness threshold; or, Determine whether the current round corresponding to the sharpness evaluation value has reached the preset round, and whether the sharpness evaluation value at the focal length of the current round is the maximum sharpness evaluation value at the preset round.

[0019] In this embodiment, a closed-loop feedback mechanism is formed by determining whether the sharpness evaluation value meets preset conditions. When the image at the current focal length does not meet the sharpness requirements, the system will actively adjust the focal length and re-acquire the image. This adaptive adjustment can significantly improve the accuracy and reliability of focusing, ensuring that the key areas of interest to the user remain sharp.

[0020] In a second aspect, the present invention provides an endoscope system comprising: Endoscope body; The processing module is communicatively connected to the endoscope body and is used to execute the focusing method of the endoscope system in any of the above embodiments.

[0021] The endoscope system provided by this invention effectively addresses the problem of incorrect focusing. It not only achieves more precise focusing and ensures the acquisition of high-quality images, but also improves focusing efficiency, reduces the risk of blurring, and effectively enhances the safety and effectiveness of medical procedures. Attached Figure Description

[0022] To more clearly illustrate the specific embodiments of the present invention or the technical solutions in the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0023] Figure 1 This is a schematic flowchart of a first method for focusing an endoscope system according to an embodiment of the present invention. Figure 2 This is a structural block diagram of an endoscope system according to an embodiment of the present invention. Detailed Implementation

[0024] To make the objectives, technical solutions, and advantages of the embodiments of the present invention clearer, the technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0025] It is understood that before using the technical solutions disclosed in the various embodiments of the present invention, users should be informed of the types, scope of use, and usage scenarios of the personal information involved in the present invention and their authorization should be obtained in accordance with relevant laws and regulations through appropriate means.

[0026] The terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of that feature. In the description of this invention, "a plurality of" means two or more, unless otherwise explicitly specified.

[0027] During the use of endoscopes, the existing autofocus technology generally suffers from two major pain points: Unintelligent target area selection: Traditional methods often use a fixed central region or simple contrast detection to select the focus area. This can easily cause the system to focus on irrelevant tissue, reflective surgical instruments, or noise at the edges of the image, rather than the lesion or target tissue that the doctor is actually concerned with, resulting in a focus error.

[0028] Unfairness in sharpness evaluation: At different focal lengths, the size and position of the same object in the image change due to variations in the field of view. Calculating sharpness directly on the entire image or a fixed region is essentially comparing image patches with different content, leading to an unfair sharpness comparison and making focusing algorithms prone to getting stuck in local optima or failing to find the target.

[0029] To overcome the above limitations, according to an embodiment of the present invention, an embodiment of a focusing method for an endoscope system is provided. It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. Furthermore, although a logical order is shown in the flowchart, in some cases, the steps shown or described may be executed in a different order than that shown here.

[0030] This embodiment provides a focusing method for an endoscope system, which can be used in endoscope systems, servers, terminals, mobile terminals, etc. Figure 1 This is a flowchart of an endoscope system focusing method according to an embodiment of the present invention, such as... Figure 1As shown, the process includes the following steps: Step S1: Acquire the current image captured by the endoscope at the current focal length.

[0031] In this embodiment, the endoscope refers to a device capable of acquiring images, which typically consists of a slender tubular structure, an optical lens, and a light source. Focal length is a crucial parameter of the optical lens, determining the image sharpness. In this embodiment, the current image acquired at the current focal length can be an image acquired by the optical lens at any focal length. Prioritized in this embodiment, the current image acquired is one containing the target object, which can be a biological tissue composed of cell populations, such as polyps or tumors.

[0032] Step S2: Determine the focus candidate region on the current image.

[0033] In this embodiment, a deep learning model can be used to determine the candidate focus region on the image, or the user can select the candidate focus region manually. The candidate focus region in this embodiment includes lesions or other important structures of interest to the user. The selected candidate region in this embodiment can be adjusted according to the image content, rather than just a fixed central area, which effectively reduces the probability of focusing on irrelevant tissues or reflective instruments.

[0034] Step S3: Determine the sharpness evaluation value of the candidate focus area.

[0035] In this embodiment, the sharpness evaluation value can be calculated by comprehensively considering different image features, such as edge strength and contrast. For example, a gradient operator is used to detect the edges of the focus candidate region. Sharp regions usually have stronger edges, so sharpness can be evaluated by calculating edge strength. Another example is calculating the contrast of the focus candidate region (such as standard deviation, variance, etc.); sharp regions usually have higher contrast. Yet another example is using the Laplacian transform to calculate the sharpness of the focus candidate region. By calculating the Laplacian transform results of the focus candidate region and taking the mean of their absolute values, a sharpness index can be obtained. Generally, a larger value indicates greater sharpness.

[0036] Step S4: Determine whether the sharpness evaluation value meets the preset sharpness conditions; if not, change the endoscope focal length and return to step S1; if yes, confirm that the endoscope system has completed focusing.

[0037] In this embodiment, determining whether the sharpness evaluation value meets the preset sharpness condition can be done by determining whether the sharpness evaluation value reaches the preset sharpness threshold; it can also be done by determining whether the zoom number corresponding to the sharpness evaluation value reaches the preset number; or it can be done by determining whether the sharpness evaluation value is the maximum value among the sharpness evaluation values ​​determined under all current focal lengths.

[0038] In addition, in this embodiment, a search algorithm such as the hill climbing method or the Fibonacci search can be adopted to drive the focus motor of the endoscope to change the focal length. Algorithms such as the hill climbing method or the Fibonacci search are optimized search algorithms that can quickly approach the optimal solution by combining local exploration with trend judgment. Therefore, the current sharpness evaluation value can be determined by setting the number of zoom times to check whether it meets the sharpness condition.

[0039] Taking the hill climbing method as an example: The system will continuously determine the moving direction and moving step size of the motor based on the current and historical F ,

[0044] ,

[0043] , values. If at the current position P1, the sharpness f1 is calculated. Command the motor to move to a new position P2 according to the current direction and step size, and calculate the initial sharpness f2 at P2. Compare f2 and f1. If f2 > f1: "uphill", the direction is correct. The system will remember P2 as a better position and maintain the current direction, preparing for the next move. If f2 < f1: "downhill", the direction is completely wrong. This is a key signal indicating that the peak is between P1 and P2. After multiple cycles, the highest point, i.e., the position with the highest sharpness value, will be found.

[0040] In this embodiment, it is determined whether the image collected at the current focal length meets the user's requirements for sharpness by the sharpness evaluation value of the determined focus candidate area. The following is an example of the implementation process: First, start the endoscope system for self-check, including checking the status of components such as the light source, camera, and sensor. And configure the basic parameters of endoscope operation, such as the initial focal length A1, image resolution, frame rate, etc.

[0041] When the endoscope initially collects an image, first collect the current image B1 at the preset initial focal length A1. At this time, this initial focal length is the current focal length. An image processing library (such as OpenCV) can be used to read and store the image data.

[0042] After obtaining the current image B1 collected at the initial focal length A1, determine the focus candidate area C1 in the current image B1 through a deep learning detection model, or other image processing algorithms, or in response to the user's independent selection, etc. And further determine the sharpness evaluation value D1 of the focus candidate area C1.

[0043] Judge whether the sharpness evaluation value D1 meets the preset sharpness threshold Q. If it meets, the endoscope system completes the focusing. If it does not meet, determine the next round of endoscope focal length A2 through a search algorithm and restart the image collection. ​After acquiring the current image B2 at the current focal length A2, the focus candidate region C2 in the current image B2 is determined again through a deep learning detection model, other image processing algorithms, or in response to user selection. Furthermore, the sharpness evaluation value D2 of the focus candidate region C2 is determined.

[0045] Determine if the sharpness evaluation value D2 meets the preset sharpness threshold Q. If it does, the endoscope system has completed focusing. If not, determine the next round of endoscope focal length A2 using a search algorithm and start acquiring images again.

[0046] As an alternative judgment method, it can be determined whether the current round corresponding to the sharpness evaluation value D2 has reached the preset round L, and whether the sharpness evaluation value at the current round's focal length is the maximum sharpness evaluation value at the preset round. If the current round corresponding to the sharpness evaluation value D2 has reached the preset round L, and the sharpness evaluation value at the current round's focal length is the maximum sharpness evaluation value at the preset round L, then the endoscope system has completed focusing. If the current round corresponding to the sharpness evaluation value D2 has not reached the preset round L, then the endoscope focal length A2 for the next round is determined through a search algorithm, and image acquisition starts again.

[0047] In this embodiment, candidate focus regions on the current image are first determined, and then a sharpness evaluation value for each candidate focus region is calculated. When the sharpness evaluation value of a candidate focus region at the current focal length does not meet the sharpness requirements, the system actively adjusts the focal length and re-acquires the image. This ensures that the system focuses only on the most relevant region rather than the entire image, effectively addressing the problem of incorrect focusing on the subject during surgery, thereby improving the autofocus performance of the endoscope. Furthermore, by automatically adjusting the focal length based on the sharpness evaluation value of the candidate focus region, the system not only achieves more precise focusing and ensures high-quality image acquisition, but also improves focusing efficiency and reduces the risk of blurring, thus enhancing the safety and effectiveness of medical procedures.

[0048] This embodiment provides a focusing method for an endoscope system, which can be used in endoscope systems, servers, terminals, mobile terminals, etc. The process includes the following steps: Step S101: Acquire the current image captured by the endoscope at the current focal length. For details, please refer to [link to relevant documentation]. Figure 1 Step S1 of the illustrated embodiment will not be described again here.

[0049] Step S102: Determine the focus candidate region on the current image.

[0050] In some optional implementations, step S102 above includes: Step a1: Input the current image into the pre-built neural network detection model to determine the target object detection region.

[0051] In this embodiment, the neural network detection model can be a pre-trained lightweight neural network model (such as YOLO or SSD). The neural network detection model is used to detect target objects in the image, such as polyps, specific mucous membranes, blood vessels, etc., and output their location bounding boxes, thereby determining the target object detection area.

[0052] Step a21: If there is only one target object detection area, the target object detection area is used as the focus candidate area.

[0053] Step a22: If there are two or more target object detection regions, respond to the target region selected by the user and use the target region as a focus candidate region. That is, if multiple targets are detected, the user can select the target object detection region of interest from multiple target object detection regions, or the user can re-select the image region, such as selecting the target located in the center of the image as the optimal target region.

[0054] In some alternative implementations, when the neural network detection model fails to detect the target object, the following steps are used to determine the candidate focus region; Step a31: Convert the current image to a grayscale image; Step a32: Perform frequency domain transformation on the grayscale image to determine the amplitude spectrum and phase spectrum; Step a33: After performing a logarithmic operation on the amplitude spectrum, perform low-pass filtering to obtain the background frequency domain estimate; Step a34: Determine the spectral residuals based on background frequency domain estimation; Step a35: Determine the saliency map based on the spectral residuals and phase spectrum; Step a36: Based on the saliency map, determine the candidate focus region.

[0055] In this embodiment, a neural network detection model is preferred to detect whether a target object exists in the current image, which can effectively improve the detection efficiency.

[0056] If the neural network detection model fails to detect the target object, further visual saliency algorithms, such as the frequency domain-based spectral residual method, can be used to generate a saliency map of the image and extract multiple candidate regions of interest. Specifically: First, the current image I1 is converted to grayscale image I using the standard grayscale conversion formula. G : I G = 0.299×R+0.587×G+0.114×B; Among them, R, G, and B are the red, green, and blue components of I1.

[0057] Secondly, for grayscale image I GPerform a two-dimensional discrete Fourier transform The frequency domain representation is obtained. : ; Among them, Fourier transform The formula is: , For a digital image of size M×N, the frequency domain representation is... In this context, u and v are discrete variables.

[0058] Furthermore, according to the frequency domain representation The amplitude spectrum can be obtained. Phase spectrum .

[0059] To compress the dynamic range and simplify the model, the logarithmic amplitude spectrum was calculated. In addition, an n×n mean filter is used. ,right Low-pass filtering is applied to obtain an estimate of the gently changing background: .

[0060] Furthermore, the spectral residuals are calculated: In a physical sense, this residual corresponds to a visually salient target.

[0061] Then, use the inverse Fourier transform. This involves transforming the frequency domain to the spatial domain, specifically using spectral residuals. As a new logarithmic amplitude spectrum, combined with the original phase spectrum We can obtain: ; Among them, the inverse Fourier transform The formula is: .

[0062] Furthermore, Gaussian filtering can be used to refine the saliency map. Smoothing is performed to suppress high-frequency jitter introduced by noise and other factors, while also making the saliency map more consistent with human visual perception, resulting in the processed saliency map: .

[0063] In the saliency map obtained through the above steps, the brighter areas are generally the most visually "prominent" areas, such as lesions and abnormal structures. Then, by setting a threshold (e.g., selecting the top N% of the brightest areas in the saliency map) or using a region segmentation algorithm, multiple "brightness-concentrated and visually prominent" areas can be selected from the saliency map; these are the candidate regions of interest for focusing.

[0064] In some alternative implementations, step a36 above, i.e., determining the focus candidate region based on the saliency map, includes: Step a361: Input the current image into the pre-built neural network recognition model to determine the surgical instrument region in the current image; Step a362: Extract multiple candidate regions of interest from the saliency map; Step a363: Remove candidate regions of interest that contain surgical instrument areas; Step a364: Determine the focus candidate region from the eliminated candidate regions of interest.

[0065] In this embodiment, the neural network detection model is still preferred to find the target tissue. Only when it is not found will the alternative plan based on visual saliency be activated and the instrument excluded to avoid focusing interference.

[0066] In this embodiment, after detecting target objects that were not detected by the neural network detection model (i.e., obtaining the target object detection region), while generating a saliency map of the image using a visual saliency algorithm, a surgical instrument detection and recognition model is also used, namely the neural network recognition model in this embodiment (maintaining consistency with the neural network detection model, only requiring loading a different pre-trained model during use to reduce engineering burden), to identify the surgical instrument region in the image. Further, in each candidate region of interest, the portion overlapping with the surgical instrument region is removed, and from the remaining regions of interest, the one with the largest area is selected as the focus candidate region.

[0067] In this embodiment, by combining a visual saliency algorithm and a surgical instrument detection and recognition model, the surgical instrument region in the image can be effectively identified, and overlapping parts can be eliminated, thereby ensuring that the selected focus candidate region is more accurate. In addition, this embodiment adopts a multimodal screening mechanism of "target object detection first, clean region of interest as a backup," which greatly avoids the risk of focusing on surgical instruments or irrelevant backgrounds, ensuring that the focus subject is always the clinical focus.

[0068] In some alternative implementations, determining the focus candidate region on the current image includes: Determine the first candidate focus area under the previous focal length; Using the first focus candidate region as a template, determine the second focus candidate region at the current focal length in the current image, and use the second focus candidate region as the focus candidate region on the current image; the size and content of the second focus candidate region are the same as those of the first focus candidate region.

[0069] In this embodiment, to ensure fair sharpness comparison, the current image needs to be preprocessed before determining the candidate focus region, ensuring consistency in the content of the evaluated regions across different focal lengths. After the endoscope has focused more than once (i.e., when the current image is not the initial image captured by the endoscope), subsequent candidate focus regions must be identical in size and content to those determined in the previous round. For example, using the candidate focus region S1 determined by the endoscope at the first focal length F1 as a template, a multi-scale template matching algorithm is used to find the candidate focus region S1 most similar to the template in the image captured at the second focal length F2. m Ensure S1 and S m Size and content alignment.

[0070] In this embodiment, the first focused area is used as a template for region matching in subsequent images. Region matching ensures that the comparison at different focal lengths is of the same content, providing a fair basis for sharpness calculation and thus improving the accuracy of image processing.

[0071] Step S103: Determine the sharpness evaluation value of the candidate focus area.

[0072] In some optional implementations, step S103 above, namely determining the sharpness evaluation value of the focus candidate region, includes: Step b1: Based on the first focus candidate region, determine whether the second focus candidate region needs brightness correction.

[0073] In some alternative implementations, step b1 above includes: Step c1: Determine the first average brightness value of the first focus candidate region; Step c2: Determine the second average brightness value of the second focus candidate region; Step c3: Determine the brightness difference between the first average brightness value and the second average brightness value; Step c4: Determine whether the brightness difference value exceeds the preset brightness difference threshold; Step c5: When the brightness difference value exceeds a preset brightness difference threshold and the second average brightness value is less than the first average brightness value, it is determined that the second focus candidate area needs brightness correction. When the brightness difference value exceeds the preset brightness difference threshold and the first average brightness value is less than the second average brightness value, brightness correction is performed on the first focus candidate area.

[0074] The following are examples: For example: the first average brightness value is The second average brightness value is Where M×N is the size of the grayscale image matrix, and (x,y) are the pixel coordinates.

[0075] The brightness difference value is: ;in, Take 1e-6 to prevent division by zero.

[0076] If Diff > α, proceed to the next step to perform brightness correction; otherwise, skip. Here, α is the preset brightness difference threshold.

[0077] like If the first focus candidate area S is darker, then brightness correction is performed to obtain... Second focus candidate region S m Unchanged. Similarly, if That is, the second focus candidate region S m If it's even darker, then perform brightness correction to achieve the desired result. The first focus candidate region S remains unchanged. The gain coefficient is: .

[0078] In this embodiment, before calculating sharpness, the brightness consistency of areas at different focal lengths is adjusted. This can effectively eliminate the influence of exposure, that is, eliminate the interference of exposure changes on the sharpness evaluation function, thereby improving the accuracy of sharpness assessment.

[0079] Step b2 involves filtering the second focus candidate region (with or without brightness correction) through different frequency channels to obtain the image response map for each frequency channel.

[0080] In this embodiment, a set of bandpass filters with different center frequencies (such as a Difference Gaussian (DoG) filter or a Gabor filter bank) can be used to filter different normalized regions respectively.

[0081] For example: using three Gaussian difference filters (DoG) to filter different low, mid, and high frequencies: ; Among them, k∈{low, mid, high}, This represents the grayscale area after brightness correction.

[0082] Step b3: Determine the initial sharpness value of the image response map for each frequency channel.

[0083] In this embodiment, for each filtered output image, its energy value (i.e., the sum of the squares of all pixel gray values) is calculated as the initial sharpness value under that frequency component.

[0084] For example: use the squared L2 norm to calculate the squared energy (initial sharpness value) of the corresponding response map Pk, i.e.: .

[0085] Step b4: Based on all initial sharpness values, determine the sharpness evaluation value of the second focus candidate region.

[0086] Specifically, the initial sharpness values ​​of each frequency channel can be weighted and fused to obtain the final sharpness evaluation value F. final Since the human eye is more sensitive to mid-to-high frequency details, the weights can be set based on this prior knowledge, giving higher weights to mid-to-high frequency components to achieve better results.

[0087] For example: the weighted blending sharpness score is: .

[0088] In this embodiment, multiple bandpass filters are used to calculate sharpness separately, and then weighted and fused (especially with mid-to-high frequency weights). This results in a more comprehensive and robust sharpness evaluation value. Furthermore, multi-band evaluation can comprehensively capture the texture details of the image. Compared with a single gradient function, it is more adaptable to different types of image content (such as regions with both smooth and textured areas), and the evaluation results are more in line with human subjective perception, with better noise resistance.

[0089] Step S104: Determine whether the sharpness evaluation value meets the preset sharpness conditions; if not, change the endoscope focal length and return to step S101; if yes, confirm that the endoscope system has completed focusing.

[0090] In some optional implementations, step S104 above includes: The system determines whether the sharpness evaluation value exceeds a first preset sharpness threshold; if it does, the endoscope system has completed focusing. If not, a search algorithm determines the endoscopic focal length for the next round, and image acquisition begins again.

[0091] In some optional implementations, step S104 above includes: The system determines whether the current image sharpness evaluation value corresponds to a preset number of image sharpness evaluations, and whether the current image sharpness evaluation value at the current focal length is the maximum value within the preset number of image sharpness evaluations. If the current image sharpness evaluation value corresponds to a preset number of image sharpness evaluations, and the current image sharpness evaluation value at the current focal length is the maximum value within the preset number of image sharpness evaluations, then the endoscope system has completed focusing. If the current image sharpness evaluation value corresponds to a preset number of image sharpness evaluations, the endoscope focal length for the next image sharpness evaluation is determined using a search algorithm, and image acquisition begins again.

[0092] In this embodiment, a closed-loop feedback mechanism is formed by determining whether the sharpness evaluation value meets preset conditions. When the image at the current focal length does not meet the sharpness requirements, the system will actively adjust the focal length and re-acquire the image. This adaptive adjustment can significantly improve the accuracy and reliability of focusing, ensuring that the key areas of interest to the user remain sharp.

[0093] This invention provides an endoscopic focusing method that combines a multimodal target region screening mechanism with an adaptive frequency sharpness evaluation function to ensure that the focus is always on the key target and to compare sharpness on a consistent benchmark, thereby achieving fast, accurate and stable autofocus.

[0094] Furthermore, this invention combines intelligent region selection with advanced clarity evaluation depth to form a closed-loop optimization system, fundamentally improving the focusing performance of endoscopes in complex and dynamic surgical environments.

[0095] This embodiment also provides an endoscope system for implementing the above embodiments and preferred embodiments; details already described will not be repeated. As used below, the term "module" can refer to a combination of software and / or hardware that performs a predetermined function. Although the modules described in the following embodiments are preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0096] This embodiment provides an endoscope system, such as Figure 2 As shown, it includes: Endoscope body; The processing module is communicatively connected to the endoscope body and is used to execute the focusing method of the endoscope system in any of the above embodiments.

[0097] The endoscopic system provided by this invention first determines the candidate focus areas on the current image, and then calculates the sharpness evaluation value of the candidate focus areas. When the sharpness evaluation value of the candidate focus area at the current focal length does not meet the sharpness requirements, the system actively adjusts the focal length and re-acquires the image. This ensures that the system focuses only on the most relevant areas rather than the entire image, effectively addressing the problem of incorrect focusing on the subject during surgery, thereby improving the autofocus performance of the endoscope. Furthermore, by automatically adjusting the focal length based on the sharpness evaluation value of the candidate focus areas, the system not only achieves more precise focusing and ensures the acquisition of high-quality images, but also improves focusing efficiency and reduces the risk of blurring, thus enhancing the safety and effectiveness of medical procedures.

[0098] In some alternative implementations, the processing module is specifically used to perform the following steps: Step S1: Acquire the current image captured by the endoscope at the current focal length; Step S2: Determine the focus candidate region on the current image; Step S3: Determine the sharpness evaluation value of the candidate focus area; Step S4: Determine whether the sharpness evaluation value meets the preset sharpness conditions; if not, change the endoscope focal length and return to step S1; if yes, confirm that the endoscope system has completed focusing.

[0099] In some alternative implementations, the processing module includes: The focus candidate region determination unit is used to input the current image into a pre-built neural network detection model to determine the target object detection region. If there is only one target object detection region, it is used as a focus candidate region. If there are two or more target object detection regions, it responds to the target region selected by the user and uses that region as a focus candidate region. When the neural network detection model does not detect the target object, the focus candidate region is determined through the following steps: converting the current image to a grayscale image; performing a frequency domain transformation on the grayscale image to determine the amplitude spectrum and phase spectrum; performing a logarithmic operation on the amplitude spectrum and then performing a low-pass filter to obtain a background frequency domain estimate; determining the spectral residual based on the background frequency domain estimate; determining the saliency map based on the spectral residual and the phase spectrum; and determining the focus candidate region based on the saliency map. It is also used to input the current image into a pre-built neural network recognition model to determine the surgical instrument region in the current image; extracting multiple candidate regions of interest from the saliency map; removing candidate regions of interest containing the surgical instrument region; and determining the focus candidate region from the removed candidate regions of interest.

[0100] In some optional implementations, the focus candidate region determination unit is further configured to determine a first focus candidate region at the previous focal length; using the first focus candidate region as a template, determine a second focus candidate region at the current focal length from the current image, and use the second focus candidate region as the focus candidate region on the current image; the second focus candidate region has the same size and content as the first focus candidate region.

[0101] In some alternative implementations, the processing module includes: The sharpness evaluation value determination unit is used to determine whether the second focus candidate region needs brightness correction based on the first focus candidate region; to filter the second focus candidate region, whether it has undergone brightness correction or not, through different frequency channels to obtain the image response map of each frequency channel; to determine the initial sharpness value of the image response map of each frequency channel; and to determine the sharpness evaluation value of the second focus candidate region based on all the initial sharpness values.

[0102] In some alternative implementations, the processing module includes: The correction unit is used to determine a first average brightness value of a first focus candidate region; determine a second average brightness value of a second focus candidate region; determine a brightness difference value between the first average brightness value and the second average brightness value; determine whether the brightness difference value exceeds a preset brightness difference threshold; and determine that the second focus candidate region needs brightness correction when the brightness difference value exceeds the preset brightness difference threshold and the second average brightness value is less than the first average brightness value. When the brightness difference value exceeds the preset brightness difference threshold and the first average brightness value is less than the second average brightness value, brightness correction is performed on the first focus candidate region.

[0103] In some alternative implementations, the processing module includes: The judgment unit is used to determine whether the sharpness evaluation value exceeds the first preset sharpness threshold; or, to determine whether the current round corresponding to the sharpness evaluation value has reached the preset round, and the sharpness evaluation value under the focal length of the current round is the maximum sharpness evaluation value under the preset round.

[0104] The endoscope system provided in this embodiment of the invention can execute the focusing method of the endoscope system provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects for executing the method. Further functional descriptions of the various modules and units are the same as in the corresponding embodiments described above, and will not be repeated here.

[0105] Although embodiments of the invention have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the invention, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A focusing method for an endoscope system, characterized in that, The method includes: Step S1: Acquire the current image captured by the endoscope at the current focal length; Step S2: Determine the focus candidate region on the current image; Step S3: Determine the sharpness evaluation value of the candidate focus area; Step S4: Determine whether the sharpness evaluation value meets the preset sharpness condition; if not, change the endoscope focal length and return to step S1; if yes, confirm that the endoscope system has completed focusing.

2. The method according to claim 1, characterized in that, Determining the focus candidate region on the current image includes: The current image is input into a pre-built neural network detection model to determine the target object detection region; If there is only one target object detection region, the target object detection region is used as the focus candidate region. When there are two or more target object detection areas, the target area is selected by the user as the focus candidate area in response to the target area selected by the user.

3. The method according to claim 2, characterized in that, If the neural network detection model fails to detect the target object, the focus candidate region is determined through the following steps; Convert the current image to a grayscale image; The grayscale image is subjected to frequency domain transformation to determine the amplitude spectrum and phase spectrum; After performing a logarithmic operation on the amplitude spectrum, a low-pass filter is applied to obtain a background frequency domain estimate. Based on the background frequency domain estimation, the spectral residual is determined; Based on the spectral residuals and the phase spectrum, a saliency map is determined; Based on the saliency map, the focus candidate region is determined.

4. The method according to claim 3, characterized in that, The process of determining the focus candidate region based on the saliency map includes: The current image is input into a pre-built neural network recognition model to determine the surgical instrument region in the current image; Extract multiple candidate regions of interest from the saliency map; The candidate region of interest containing the surgical instrument area is removed; The focus candidate region is determined from the eliminated candidate regions of interest.

5. The method according to claim 1, characterized in that, Determining the focus candidate region on the current image includes: Determine the first candidate focus area under the previous focal length; Using the first focus candidate region as a template, a second focus candidate region at the current focal length is determined from the current image, and the second focus candidate region is used as the focus candidate region on the current image; the second focus candidate region has the same size and content as the first focus candidate region.

6. The method according to claim 5, characterized in that, Determining the sharpness evaluation value of the focus candidate region includes: Based on the first focus candidate region, determine whether the second focus candidate region needs brightness correction; The second focus candidate region, whether it has undergone brightness correction or not, is filtered through different frequency channels to obtain the image response map of each frequency channel. Determine the initial sharpness value of the image response map for each frequency channel; Based on all the initial sharpness values, the sharpness evaluation value of the second focus candidate region is determined.

7. The method according to claim 6, characterized in that, The step of determining whether the second focus candidate region needs brightness correction based on the first focus candidate region includes: Determine the first average brightness value of the first focus candidate region; Determine the second average brightness value of the second focus candidate region; Determine the brightness difference between the first average brightness value and the second average brightness value; Determine whether the brightness difference value exceeds a preset brightness difference threshold; If the brightness difference value exceeds a preset brightness difference threshold and the second average brightness value is less than the first average brightness value, it is determined that the second focus candidate area needs brightness correction.

8. The method according to claim 7, characterized in that, When the brightness difference value exceeds a preset brightness difference threshold and the first average brightness value is less than the second average brightness value, brightness correction is performed on the first focus candidate area.

9. The method according to claim 1, characterized in that, The step of determining whether the sharpness evaluation value meets the preset sharpness conditions includes: Determine whether the sharpness evaluation value exceeds a first preset sharpness threshold; or, Determine whether the current round corresponding to the sharpness evaluation value has reached the preset round, and the sharpness evaluation value at the focal length of the current round is the maximum sharpness evaluation value at the preset round.

10. An endoscope system, characterized in that, The system includes: Endoscope body; The processing module is communicatively connected to the endoscope body and is used to execute the endoscope system focusing method according to any one of claims 1 to 9.

Images