Autofocus method for endoscope, electronic device, and readable storage medium

Abstract

The present invention provides an autofocus method for an endoscope, an electronic device, and a readable storage medium. The focus method comprises: within a focusing range of an endoscope, randomly selecting a focusing position as an initial focus position (S100); on the basis of the initial focus position, determining a first focus position and a second focus position within the focusing range, the initial focus position being located between the first focus position and the second focus position (S200); on the basis of the initial focus position, the first focus position, the second focus position, and a pre-constructed optical defocus gradient model, determining a potential focal position (S300); and on the basis of the potential focal position, determining a final focal position (S400). The present invention may solve the problems that when an existing mainstream autofocus algorithm is applied to an endoscope, the autofocus efficiency is low, the stability is poor, and it is easy to get stuck in a local extremum.

Description

Automatic focusing method of endoscope, electronic device and readable storage medium Technical Field

[0001] The present invention relates to the field of optical imaging technology, and in particular to an automatic focusing method of an endoscope, an electronic device and a readable storage medium. Background Art

[0002] Optical imaging has a wide range of applications in various fields. For example, cameras, endoscopes, microscopes, etc., all of these devices need to be focused to produce clear images. In the prior art, when focusing optical imaging equipment, traversal search method, Fibonacci search method or hill climbing search method are usually used to find a suitable focus position. Among them, the traversal search method is a search method that finds the extreme value of the focus evaluation function by traversing all focus positions. The traversal search method calculates the focus evaluation function value once every step in the entire motor movement range, and continuously compares until the largest focus evaluation function value is found. The traversal search method traverses the entire focus interval in equal steps, resulting in a relatively large amount of calculation and low real-time performance.

[0003] The Fibonacci search method is a one-dimensional single-peak search method based on the Fibonacci function. The Fibonacci search method has a simple implementation principle and is easy to program. However, this method requires the focus evaluation curve to have an ideal single peak. The actual focus evaluation curve often does not meet the ideal smooth curve. Therefore, this method is prone to fall into local peak points and lead to misjudgment.

[0004] Hill climbing is essentially a blind, heuristic search. If the step size is not set appropriately, it can take a long time to find the target value, increasing user wait time and reducing the user experience. Furthermore, hill climbing is a local optimization method, susceptible to interference from local extremes and unable to find the global optimal solution, leading to focus failure.

[0005] It should be noted that the information disclosed in the background technology section of the invention is only intended to deepen the understanding of the general background technology of the invention, and should not be regarded as an admission or any form of implication that the information constitutes prior art already known to those skilled in the art.

[0006] Summary of the Invention

[0007] The purpose of the present invention is to provide an autofocus method, electronic device and readable storage medium for an endoscope, which can solve the problems of low autofocus efficiency, poor stability and easy falling into local extreme values ​​when the existing mainstream autofocus algorithms are applied to endoscopes.

[0008] To achieve the above object, the present invention provides an automatic focusing method for an endoscope, comprising:

[0009] Randomly selecting a focusing position within the focusing range of the endoscope as an initial focusing position;

[0010] Determining a first focus position and a second focus position within the focusing range according to the initial focus position, wherein the initial focus position is located between the first focus position and the second focus position;

[0011] determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position, and a pre-built optical defocus gradient model;

[0012] A final in-focus position is determined according to the potential in-focus position.

[0013] Optionally, determining the first focus position and the second focus position within the focusing range according to the initial focus position includes:

[0014] Selecting a focusing position located on a first side of the initial focusing position within the focusing range as a first focusing position;

[0015] Calculating image clarity evaluation values ​​corresponding to the initial focus position and the first focus position, respectively, wherein the image clarity evaluation value corresponding to the initial corresponding position is calculated based on the target object image captured at the initial focus position using a preset image clarity evaluation algorithm, and the image clarity evaluation value corresponding to the first focus position is calculated based on the target object image captured at the first focus position using the image clarity evaluation algorithm;

[0016] According to the initial focus position and the image clarity evaluation value corresponding to the first focus position, a focus position located on a second side of the initial focus position is determined within the focus range as a second focus position.

[0017] Optionally, determining a focusing position located on a second side of the initial focusing position within the focusing range as the second focusing position according to the image clarity evaluation values ​​corresponding to the initial focusing position and the first focusing position includes:

[0018] If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, searching within a focus position interval located on a second side of the initial focus position within the focus range until the image clarity evaluation value corresponding to the currently searched focus position is greater than the image clarity evaluation value corresponding to the initial focus position, or the current number of searches reaches a first preset number, then using the currently searched focus position as the second focus position;

[0019] If the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, a search is performed within the focus position interval located on the second side of the initial focus position within the focus range until the image clarity evaluation value corresponding to the currently searched focus position is less than the image clarity evaluation value corresponding to the initial focus position, or the current search number reaches a second preset number, and the currently searched focus position is used as the second focus position.

[0020] Optionally, the searching within a focus position interval located on a second side of the initial focus position within the focus range includes:

[0021] The search is performed within a focus position interval located on a second side of the initial focus position within the focus range according to a preset moving step length.

[0022] Optionally, the calculation formula of the preset moving step length is as follows: s=[(L-x0) / 3]

[0023] Wherein, s is the preset moving step, L is the total step corresponding to the focusing range, x0 is the step corresponding to the initial focusing position, and [] indicates rounding.

[0024] Optionally, determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position, and a pre-built optical defocus gradient model includes:

[0025] If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than or equal to the image clarity evaluation value corresponding to the initial focus position; or if the image clarity evaluation values ​​corresponding to the initial focus position are both less than or equal to the image clarity evaluation value corresponding to the first focus position and the image clarity evaluation value corresponding to the second focus position, substituting the initial focus position, the first focus position, and the second focus position and their respective corresponding image clarity evaluation values ​​into the optical defocus gradient model, and solving the optical defocus gradient model to determine a potential in-focus position;

[0026] If the image clarity evaluation value corresponding to the first focus position, the image clarity evaluation value corresponding to the initial focus position, and the image clarity evaluation value corresponding to the second focus position satisfy a monotonically increasing relationship; or if the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than the image clarity evaluation value corresponding to the initial focus position, then a focus position is determined within the focusing range as the third focus position, and the one of the initial focus position, the first focus position, and the second focus position that is farthest from the third focus position is eliminated, and the remaining two and the third focus position and their respective corresponding image clarity evaluation values ​​are respectively substituted into the optical defocus gradient model, and the optical defocus gradient model is solved to determine a potential in-focus position.

[0027] Optionally, if the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is greater than the image clarity evaluation value corresponding to the initial focus position, the calculation formula for the third focus position is as follows: x3 = x2 + [(x2 - x0) * 8 / L]

[0028] Wherein, x3 is the step length corresponding to the third focus position, x2 is the step length corresponding to the second focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus adjustment range, and [] indicates rounding;

[0029] If the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than the image clarity evaluation value corresponding to the initial focus position, then the calculation formula for the third focus position is as follows: x3 = x1 + [(x1 - x0) * 8 / L]

[0030] Among them, x3 is the step length corresponding to the third focus position, x1 is the step length corresponding to the first focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus range, and [] indicates rounding.

[0031] Optionally, the image clarity evaluation algorithm includes:

[0032] Divide the image to be evaluated into N×N image blocks, where N is a positive integer;

[0033] For each image block, the variance factor, appropriate exposure factor and contrast factor corresponding to the image block are calculated respectively;

[0034] Calculating the weighted variance factor, weighted appropriate exposure factor, and weighted contrast factor corresponding to the image to be evaluated respectively according to the weight values, variance factors, appropriate exposure factors, and contrast factors corresponding to all image blocks;

[0035] An image clarity evaluation value of the image to be evaluated is calculated according to the weighted variance factor, the weighted appropriate exposure factor, and the weighted contrast factor corresponding to the image to be evaluated.

[0036] Optionally, before dividing the image to be evaluated into N×N image blocks, the image clarity evaluation algorithm further includes:

[0037] Normalizing the pixel value of each pixel point of the image to be evaluated;

[0038] The step of calculating the variance factor, the appropriate exposure factor, and the contrast factor corresponding to each image block includes:

[0039] For each image block, the variance factor corresponding to the image block is calculated based on the normalized grayscale value corresponding to each pixel in the image block; the contrast factor corresponding to the image block is calculated based on the normalized grayscale value corresponding to each pixel in the image block and the normalized grayscale values ​​corresponding to its adjacent pixels; and the appropriate exposure factor corresponding to the image block is calculated based on the normalized red channel value, green channel value, and blue channel value corresponding to each pixel in the image block.

[0040] Optionally, calculating the image clarity evaluation value of the image to be evaluated based on the weighted variance factor, the weighted appropriate exposure factor, and the weighted contrast factor corresponding to the image to be evaluated includes:

[0041] The image clarity evaluation value of the image to be evaluated is calculated using the following formula:

[0042] Among them, V is the weighted variance factor, W is the weighted moderate exposure factor, and C is the weighted contrast factor. is the weight parameter of the weighted variance factor, is the weight parameter of the weighted moderate exposure factor, is the weight parameter of the weighted contrast factor.

[0043] Optionally, the calculation formula of the optical defocus gradient model is as follows:

[0044] Or as follows:

[0045] Among them, f(x) is the image clarity evaluation value corresponding to the focus position with a step size of x, and x f is the step size corresponding to the potential in-focus position, and a and b are constants.

[0046] Optionally, determining a final in-focus position according to the potential in-focus position includes:

[0047] taking a focus position within the focus adjustment range that is located on a first side of the potential in-focus position and is one step away from the potential in-focus position as a first candidate in-focus position;

[0048] determining whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position;

[0049] If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position, selecting a focusing position within the focusing range that is located on a second side of the potential in-focus position and one step away from the potential in-focus position as a second candidate in-focus position; and determining whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position;

[0050] If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position, the potential in-focus position is used as the final in-focus position; if the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the second candidate in-focus position, the second candidate in-focus position is used as the final in-focus position;

[0051] If the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the first candidate in-focus position, selecting a focus position within the focusing range that is located on the same side of the potential in-focus position and one step away from the first candidate in-focus position as a third candidate in-focus position, and determining whether the image clarity evaluation value corresponding to the first candidate in-focus position is greater than the image clarity evaluation value corresponding to the third candidate in-focus position;

[0052] If so, the first candidate in-focus position is used as the final in-focus position; if not, the third candidate in-focus position is used as the final in-focus position.

[0053] To achieve the above-mentioned object, the present invention further provides an electronic device, comprising a processor and a memory, wherein the memory stores a computer program, and when the computer program is executed by the processor, the auto-focusing method of the endoscope described above is implemented.

[0054] To achieve the above-mentioned object, the present invention further provides a readable storage medium, wherein the readable storage medium stores a computer program, and when the computer program is executed by a processor, the auto-focusing method of the endoscope described above is implemented.

[0055] Compared with the prior art, the auto-focusing method for an endoscope, the electronic device, and the readable storage medium provided by the present invention have the following beneficial effects:

[0056] The autofocus method for an endoscope provided by the present invention includes randomly selecting a focusing position within the focusing range of the endoscope as the initial focusing position; determining a first focusing position and a second focusing position within the focusing range based on the initial focusing position, wherein the initial focusing position is located between the first focusing position and the second focusing position; determining a potential quasi-focus position based on the initial focusing position, the first focusing position and the second focusing position and a pre-constructed optical defocus gradient model; and determining a final quasi-focus position based on the potential quasi-focus position. Therefore, compared with other mainstream methods that only blindly use clarity evaluation function feedback for core focus control, the present invention starts from the root cause of defocus and uses an optical defocus gradient model with optical imaging defocus theory as theoretical support, which effectively avoids the problems of poor algorithm robustness, slow parameter adjustment, and poor focusing accuracy caused by relying solely on experience or evaluation functions to find similar curve models; at the same time, the present invention avoids the problem of easily falling into local extreme values ​​by estimating the quasi-focus position based on the global optical defocus gradient model. In addition, the present invention can effectively solve the problem of repeated changes in focusing direction due to transition iterations in the traditional hill climbing method by introducing boundary conditions such as the first focusing position and the second focusing position.

[0057] Since the electronic device and readable storage medium provided by the present invention belong to the same inventive concept as the automatic focusing method of the endoscope provided by the present invention, the electronic device and readable storage medium provided by the present invention have all the beneficial effects of the automatic focusing method of the endoscope provided by the present invention. For details, please refer to the relevant content above regarding the beneficial effects of the automatic focusing method of the endoscope provided by the present invention. Therefore, the beneficial effects of the electronic device and readable storage medium provided by the present invention will not be described one by one here. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] FIG1 is a schematic flow chart of an auto-focusing method for an endoscope provided in one embodiment of the present invention;

[0059] FIG2 is a specific flow chart of an auto-focusing method for an endoscope provided in one embodiment of the present invention;

[0060] Figure 3a is a schematic diagram of an optical system imaging model;

[0061] FIG3 b is a schematic diagram of the circle of confusion on the image plane;

[0062] FIG4 is a flow chart of an image clarity evaluation algorithm provided by one embodiment of the present invention;

[0063] FIG5 is a block diagram of an electronic device according to an embodiment of the present invention. DETAILED DESCRIPTION

[0064] The following is a further detailed description of the autofocus method, electronic device and readable storage medium for an endoscope proposed in the present invention in conjunction with the accompanying drawings and specific embodiments. According to the following description, the advantages and features of the present invention will become clearer. It should be noted that the drawings are in a very simplified form and are not in precise proportions, which are only used to conveniently and clearly assist in explaining the purpose of the embodiments of the present invention. In order to make the purposes, features and advantages of the present invention more obvious and easy to understand, please refer to the drawings. It should be noted that the structures, proportions, sizes, etc. illustrated in the drawings of this specification are only used to match the contents disclosed in the specification for people familiar with this technology to understand and read, and are not used to limit the conditions for the implementation of the present invention. Any modification of the structure, change in the proportional relationship or adjustment of the size should still fall within the scope of the technical content disclosed in the present invention, provided that the effects and purposes that can be achieved are the same or similar to those that can be produced by the present invention.

[0065] It should be noted that, in this document, relational terms such as first and second, etc., are used only to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply the existence of any such actual relationship or order between these entities or operations. Moreover, the terms "comprises," "comprising," or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article, or device comprising a series of elements includes not only those elements, but also other elements not explicitly listed, or elements inherent to such process, method, article, or device. In the absence of further limitations, an element defined by the phrase "comprising a ..." does not exclude the presence of other identical elements in the process, method, article, or device comprising the element.

[0066] In addition, in the description of this specification, the description with reference to the terms "one embodiment", "some embodiments", "example", "specific example", or "some examples" means that the specific features, structures, materials or characteristics described in conjunction with the embodiment or example are included in at least one embodiment or example of the present invention. In this specification, the schematic representation of the above terms does not necessarily refer to the same embodiment or example. Moreover, the specific features, structures, materials or characteristics described can be combined in any one or more embodiments or examples in a suitable manner. In addition, those skilled in the art can combine and combine different embodiments or examples described in this specification and the features of different embodiments or examples without contradiction.

[0067] The core idea of ​​the present invention is to provide an autofocus method, electronic device and readable storage medium for an endoscope, which can solve the problems of low autofocus efficiency, poor stability and easy falling into local extreme values ​​when the existing mainstream autofocus algorithms are applied to endoscopes.

[0068] It should be noted that the autofocus method of the endoscope provided by the present invention can be applied to the electronic device provided by the present invention, wherein the electronic device can be a personal computer, a mobile terminal, etc., and the mobile terminal can be a mobile phone, a tablet computer, and other hardware devices with various operating systems.

[0069] To achieve the above idea, the present invention provides an endoscope autofocus method. Please refer to Figure 1, which is a flow chart of the endoscope autofocus method provided by one embodiment of the present invention. As shown in Figure 1, the endoscope autofocus method provided by the present invention includes the following steps:

[0070] Step S100: randomly selecting a focusing position within the focusing range of the endoscope as an initial focusing position.

[0071] Step S200: determining a first focus position and a second focus position within the focus adjustment range according to the initial focus position, wherein the initial focus position is located between the first focus position and the second focus position.

[0072] Step S300: determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position, and a pre-constructed optical defocus gradient model.

[0073] Step S400: determining a final in-focus position according to the potential in-focus position.

[0074] Therefore, compared with other mainstream methods that simply use clarity evaluation function feedback for core focus control, the present invention starts from the fundamental cause of defocus and uses an optical defocus gradient model with optical imaging defocus theory as its theoretical support, effectively avoiding the problems of poor algorithm robustness, slow parameter adjustment, and poor focusing accuracy caused by relying solely on experience or evaluation functions to find similar curve models; at the same time, the present invention estimates the quasi-focus position based on the global optical defocus gradient model, which can avoid the problem of easily falling into local extreme values. In addition, by introducing boundary conditions such as the first focus position and the second focus position, the present invention can effectively solve the problem of repeated changes in focusing direction due to transition iterations in the traditional hill climbing method.

[0075] It should be noted that, as can be understood by those skilled in the art, the initial focus position can be any focus position within the focus range of the endoscope. Preferably, the initial focus position is close to the middle area of ​​the focus range (i.e., the area close to the middle value of the focus range). Since the in-focus position generally does not appear in the two end areas of the focus range (i.e., the area close to the minimum critical value of the focus range and the area close to the maximum critical value of the focus range), by randomly selecting a focus position in the middle area of ​​the focus range as the initial focus position, the problem of easily falling into a local extreme value during the focusing process can be further avoided.

[0076] In some exemplary embodiments, determining the first focus position and the second focus position within the focus adjustment range according to the initial focus position includes:

[0077] Selecting a focusing position located on one side of the initial focusing position within the focusing range as a first focusing position;

[0078] Calculating image clarity evaluation values ​​corresponding to the initial focus position and the first focus position, respectively, wherein the image clarity evaluation value corresponding to the initial corresponding position is calculated based on the target object image captured at the initial focus position using a preset image clarity evaluation algorithm, and the image clarity evaluation value corresponding to the first focus position is calculated based on the target object image captured at the first focus position using the image clarity evaluation algorithm;

[0079] According to the image clarity evaluation values ​​corresponding to the initial focus position and the first focus position, a focus position located on the other side of the initial focus position is determined within the focus range as a second focus position.

[0080] Specifically, in some embodiments, any focusing position within the focusing range can be selected on the left side of the initial focusing position x0 (the step size corresponding to the initial focusing position is x0) as the first focusing position x1 (the step size corresponding to the first focusing position is x1). At this time, the second focusing position x2 (the step size corresponding to the second focusing position is x2) is located on the right side of the initial focusing position x0; of course, in some other embodiments, any focusing position within the focusing range can be selected on the right side of the initial focusing position x0 as the first focusing position x1. At this time, the second focusing position x2 (the step size corresponding to the second focusing position is x2) is located on the left side of the initial focusing position x0. It should be noted that, as can be understood by those skilled in the art, when the first focus position x1 is located on the left side of the initial focus position x0 and the second focus position x2 is located on the right side of the initial focus position x0, x1<x0, x2>x0; when the first focus position x1 is located on the right side of the initial focus position x0 and the second focus position x2 is located on the left side of the initial focus position x0, x1>x0, x2<x0.

[0081] Furthermore, when the initial focus position x0 is closer to the minimum critical value of the focus adjustment range, the first focus position x1 is preferably selected to the left of the initial focus position x0; when the initial focus position x0 is closer to the maximum critical value of the focus adjustment range, the first focus position x1 is preferably selected to the right of the initial focus position x0. This configuration can further avoid the problem of being easily trapped in local extreme values.

[0082] It should be noted that, as those skilled in the art can understand, before using a preset image clarity evaluation algorithm to calculate the image clarity evaluation value corresponding to the initial focus position x0, it is necessary to first control the driving device (such as a stepping motor) used to drive the objective lens of the endoscope to move to drive the objective lens of the endoscope to move to the initial focus position x0, and then image the target object to obtain the corresponding target object image, and then use the preset image clarity evaluation algorithm to calculate the image clarity evaluation value f(x0) of the target object image captured at the initial focus position x0 (that is, obtain the image clarity evaluation value f(x0) corresponding to the initial focus position x0). Similarly, before using the preset image clarity evaluation algorithm to calculate the image clarity evaluation value corresponding to the first focus position x1, it is necessary to first control the driving device (e.g., a stepping motor) to drive the objective lens of the endoscope to move to the first focus position x1, then image the target object to obtain the corresponding target object image, and then use the preset image clarity evaluation algorithm to calculate the image clarity evaluation value f(x1) of the target object image captured at the first focus position x1 (that is, obtain the image clarity evaluation value f(x1) corresponding to the first focus position x1). Specifically, the specific content of the preset image clarity evaluation algorithm can be referred to the relevant description below and will not be introduced here.

[0083] In some exemplary embodiments, determining, according to the image clarity evaluation values ​​corresponding to the initial focus position and the first focus position, a focus position located on the other side of the initial focus position within the focus adjustment range as the second focus position includes:

[0084] If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, searching within the focus position interval located on the other side of the initial focus position within the focus range until the image clarity evaluation value corresponding to the currently searched focus position is greater than the image clarity evaluation value corresponding to the initial focus position, or the current search number reaches a first preset number, and then using the currently searched focus position as the second focus position;

[0085] If the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, the search is also performed within the focus position interval located on the other side of the initial focus position within the focusing range, but until the image clarity evaluation value corresponding to the currently searched focus position is less than the image clarity evaluation value corresponding to the initial focus position, or the current search number reaches a second preset number, the currently searched focus position is used as the second focus position.

[0086] Specifically, please refer to FIG2 , which is a specific flow chart of the automatic focusing method of the endoscope provided by one embodiment of the present invention. As shown in FIG2 , taking the first focus position x1 as an example, if the image clarity evaluation value f(x0) corresponding to the initial focus position x0 is greater than the image clarity evaluation value f(x1) corresponding to the first focus position x1, a focus position x2' is selected on the right side of the initial focus position x0, and the driving device (such as a stepping motor) is controlled to drive the objective lens of the endoscope to move to the focus position x2', and then the target object is imaged to obtain the corresponding target object image, and then the preset image clarity evaluation algorithm is used to calculate the image clarity evaluation value f(x2') of the target object image captured at the focus position x2' (that is, the image clarity evaluation value f(x2') corresponding to the focus position x2' is obtained). If f(x2')>f(x0) is satisfied, the focus position x2' is used as the final second focus position x2'; if it is not satisfied, f(x2')>f(x0), then further determine whether the current search times (i.e., the number of focus positions selected on the right side of the initial focus position x0) reaches the first preset number (e.g., 2 times). If so, the focus position x2' is also used as the final second focus position x2; if the current search times (i.e., the number of focus positions selected on the right side of the initial focus position x0) does not reach the first preset number (e.g., 2 times), then continue to select a new focus position x2' on the right side of the initial focus position x0 (the new focus position x2' is located on the right side of the previous focus position x2'), and repeat the above steps until the image clarity evaluation value f(x2') corresponding to the currently searched focus position x2' is greater than the image clarity evaluation value f(x0) corresponding to the initial focus position x0, or the current search times reaches the first preset number, so as to complete the search for the second focus position x2. It should be noted that, as those skilled in the art can understand, the first focus position x1 can also be located to the right of the initial focus position x0. The present invention is not limited to this. Those skilled in the art can reasonably infer the specific focusing process when the first focus position x1 is located to the left of the initial focus position x0 based on the specific focusing process described in this article when the first focus position x1 is located to the left of the initial focus position x0, so the specific process of how to perform subsequent automatic focusing when the first focus position x1 is located to the right of the initial focus position x0 will not be described in detail.

[0087] Further, as shown in FIG2 , if the image clarity evaluation value f(x0) corresponding to the initial focus position x0 is less than or equal to the image clarity evaluation value f(x1) corresponding to the first focus position x1, a focus position x2′ is also selected on the right side of the initial focus position x0, and the driving device (such as a stepping motor) is controlled to drive the objective lens of the endoscope to move to the focus position x2′, and then the target object is imaged to obtain the corresponding target object image, and then the preset image clarity evaluation algorithm is used to calculate the image clarity evaluation value f(x2′) of the target object image captured at the focus position x2′ (that is, the image clarity evaluation value f(x2′) corresponding to the focus position x2′ is obtained). If f(x2′)<f(x0) is satisfied, the focus position x2′ is used as the final second focus position x2; if f(x2′)<f( x0), it is further determined whether the current search times (i.e., the number of focus positions selected on the right side of the initial focus position x0) reaches the second preset number (e.g., 2 times). If so, the focus position x2' is also used as the final second focus position x2; if the current search times (i.e., the number of focus positions selected on the right side of the initial focus position x0) does not reach the second preset number (e.g., 2 times), a new focus position x2' is continuously selected on the right side of the initial focus position x0 (the new focus position x2' is located on the right side of the previous focus position x2'), and the above steps are repeated until the image clarity evaluation value f(x2') corresponding to the currently searched focus position x2' is less than the image clarity evaluation value f(x0) corresponding to the initial focus position x0, or the current search times reaches the second preset number, so as to complete the search for the second focus position x2.

[0088] It should be noted that, as those skilled in the art can understand, the first preset number and the second preset number can be the same or different, and the present invention is not limited to this. The first preset number and the second preset number can both be 2 times or other values.

[0089] In some exemplary embodiments, searching within a focus position interval located on the other side of the initial focus position within the focus range includes:

[0090] The search is performed in a focus position interval located on the other side of the initial focus position within the focus range according to a preset moving step length.

[0091] Therefore, by searching for a second focus position within the focusing range in the same focusing direction (for example, focusing toward the right side of the initial focus position) according to a preset moving step, not only can the problem of repeated changes in the focusing direction due to transition iterations of the traditional hill climbing method be effectively solved, but the focusing efficiency can also be effectively improved.

[0092] It should be noted that, as can be understood by those skilled in the art, the first preset number and the second preset number are related to the preset moving step. The longer the preset moving step, the smaller the first preset number and the second preset number, and the shorter the preset moving step, the larger the first preset number and the second preset number.

[0093] In some exemplary embodiments, the calculation formula of the preset moving step length is as follows: s = [(L-x0) / 3] (1)

[0094] Wherein, s is the preset moving step, L is the total step corresponding to the focusing range, x0 is the step corresponding to the initial focusing position, and [] indicates rounding.

[0095] Therefore, by using the preset moving step calculated according to formula (1) to search for the second focus position x2, it can be ensured that the search for the second focus position x2 can be completed with a maximum of 2 searches, thereby further improving the efficiency of the automatic focusing method of the endoscope provided by the present invention.

[0096] In some exemplary embodiments, determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position, and a pre-built optical defocus gradient model includes:

[0097] If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than or equal to (i.e., not greater than) the image clarity evaluation value corresponding to the initial focus position; or if the image clarity evaluation value corresponding to the initial focus position is less than or equal to (i.e., not greater than) the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is greater than or equal to (i.e., not less than) the image clarity evaluation value corresponding to the initial focus position (i.e., the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position and the image clarity evaluation value corresponding to the second focus position), then the initial focus position, the first focus position, the second focus position, and their respective corresponding image clarity evaluation values ​​are respectively substituted into the optical defocus gradient model, and the optical defocus gradient model is solved to determine a potential in-focus position;

[0098] If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is greater than the image clarity evaluation value corresponding to the initial focus position (that is, the image clarity evaluation value corresponding to the first focus position, the image clarity evaluation value corresponding to the initial focus position, and the image clarity evaluation value corresponding to the second focus position satisfy a monotonically increasing relationship); or if the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position If the image clarity evaluation value corresponding to the second focusing position is less than the image clarity evaluation value corresponding to the initial focusing position, a focusing position is determined within the focusing range as the third focusing position, the one farthest from the third focusing position among the initial focusing position, the first focusing position and the second focusing position is eliminated, and the remaining two and the third focusing position and their respective corresponding image clarity evaluation values ​​are substituted into the optical defocus gradient model, and the optical defocus gradient model is solved to determine the potential in-focus position.

[0099] Please continue to refer to Figure 2. As shown in Figure 2, if f(x0)>f(x1) and f(x0)≥f(x2), or f(x0)≤f(x1) and f(x0))≤f(x2), it indicates that the potential focus position x f (The step size corresponding to the potential focus position is x f ) should be located between the initial focus position x0, the first focus position x1 and the second focus position x2. Thus, by substituting the initial focus position x0, the first focus position x1 and the second focus position x2 and f(x0), f(x1) and f(x2) into the optical defocus gradient model, and solving the optical defocus gradient model, the potential in-focus position x0 can be determined. f When f(x0)>f(x1) and f(x0)<f(x2), if the second focus position x2 is to the right of the initial focus position x0, the potential focus position x f Should be located to the right of the second focus position x2; if the second focus position x2 is located to the left of the initial focus position x0, then the potential focus position x f Should be located to the left of the second focus position x2. When f(x0)≤f(x1) and f(x0)>f(x2), if the first focus position x1 is located to the right of the initial focus position x0, then the potential focus position x f Should be located to the right of the first focus position x1; if the first focus position x1 is to the left of the initial focus position x0, then the potential focus position x fIt should be located to the left of the first focus position x1. Therefore, when f(x0)>f(x1) and f(x0)<f(x2) or f(x0)≤f(x1) and f(x0)>f(x2), by determining the third focus position x3 (the step size corresponding to the third focus position is x3), and eliminating the one farthest from the third focus position x3 among the initial focus position x0, the first focus position x1 and the second focus position x2, and substituting the remaining two and the third focus position x3 and their corresponding image clarity evaluation values ​​into the optical defocus gradient model, the optical defocus gradient model is solved to determine the potential in-focus position x3. f Specifically, when f(x0)>f(x1) and f(x0)<f(x2), if the second focus position x2 is located on the right side of the initial focus position x0, then the third focus position x3 is located on the right side of the second focus position x2. Since the first focus position x1 is farthest from the third focus position x3 in this case, the first focus position x1 can be eliminated. By substituting the initial focus position x0, the second focus position x2 and the third focus position x3 as well as f(x0), f(x2) and f(x3) into the optical defocus gradient model and solving them, the potential in-focus position x f ; If the second focus position x2 is located on the left side of the initial focus position x0, then the third focus position x3 is located on the left side of the second focus position x2. When f(x0)≤f(x1) and f(x0)>f(x2), if the first focus position x1 is located on the right side of the initial focus position x0, then the third focus position x3 is located on the right side of the first focus position x1; if the first focus position x1 is located on the left side of the initial focus position x0, then the third focus position x3 is located on the left side of the first focus position x1. Since the second focus position x2 is farthest from the third focus position x3 in this case, the second focus position x2 can be eliminated. By substituting the initial focus position x0, the first focus position x1 and the third focus position x3 as well as f(x0), f(x1) and f(x3) into the optical defocus gradient model and solving them, the potential quasi-focus position x2 can be determined. f It should be noted that, as can be understood by those skilled in the art, the objective lens of the endoscope is driven to move to the third focusing position x3 by controlling the driving device (e.g., a stepping motor), and then imaging the target object to obtain a corresponding target object image, and then a preset image clarity evaluation algorithm is used to calculate the image clarity evaluation value f(x3) of the target object image captured at the third focusing position x3 (that is, the image clarity evaluation value f(x3) corresponding to the third focusing position x3 is obtained).

[0100] In some exemplary embodiments, if the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is greater than the image clarity evaluation value corresponding to the initial focus position, then the calculation formula for the third focus position is as follows: x3=x2+[(x2-x0)*8 / L] (2)

[0101] Among them, x3 is the step length corresponding to the third focus position, x2 is the step length corresponding to the second focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus range, and [] indicates rounding.

[0102] Specifically, the 8 in formula (2) is determined based on the total step length that the driving device (e.g., stepper motor) can support (e.g., 1024 steps) and the reasonable moving step length (e.g., 128) of the driving device (e.g., stepper motor) during actual operation (1024 / 128=8, which is somewhat similar to dividing the total step length of the stepper motor into 8 gears). Therefore, when f(x0)>f(x1) and f(x0)<f(x2), by determining the third focus position x3 according to the above formula (2), the potential focus position x f It is located between the initial focus position x0, the second focus position x2 and the third focus position x3, thereby ensuring that the potential focus position x can be accurately calculated based on the initial focus position x0, the second focus position x2, the third focus position x3 and f(x0), f(x2), f(x3). f .

[0103] In some exemplary embodiments, if the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than the image clarity evaluation value corresponding to the initial focus position, then the calculation formula for the third focus position is as follows: x3 = x1 + [(x1 - x0) * 8 / L] (3)

[0104] Among them, x3 is the step length corresponding to the third focus position, x1 is the step length corresponding to the first focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus range, and [] indicates rounding.

[0105] Similarly, the 8 in formula (3) is also determined based on the total step length that the driving device (e.g., stepper motor) can support (e.g., 1024 steps) and the reasonable movement step length (e.g., 128) of the driving device (e.g., stepper motor) during actual operation (1024 / 128=8, which is somewhat similar to dividing the total step length of the driving device (e.g., stepper motor) into 8 gears). Therefore, when f(x0)≤f(x1) and f(x0)>f(x2), by determining the third focus position x3 according to the above formula (3), the potential focus position x f It is located between the initial focus position x0, the first focus position x1 and the third focus position x3, thereby ensuring that the potential focus position x can be accurately calculated based on the initial focus position x0, the first focus position x1, the third focus position x3 and f(x0), f(x1), and f(x3). f .

[0106] In some exemplary embodiments, the calculation formula of the optical defocus gradient model is as follows:

[0107] Among them, f(x) is the image clarity evaluation value corresponding to the focus position with a step size of x, and x f is the step size corresponding to the potential in-focus position, a and b are constants respectively, and b is used to represent the parameters of the endoscope and to avoid the denominator in the above formula (4) being zero.

[0108] According to the above formula (4), the focus position corresponding to the maximum value of the function curve f(x) is the potential focus position, which can be obtained by curve fitting. From the above formula (4), we know that there are 3 unknowns (x f , a and b), so only three sets of data are needed to solve the potential quasi-focus position x f , constant a, and constant b. In practical applications, the image clarity evaluation values ​​at three focus positions are needed to fit the curve and obtain the potential in-focus position.

[0109] In other exemplary embodiments, the calculation formula of the optical defocus gradient model may also be as follows:

[0110] It should be noted that, as those skilled in the art will appreciate, the optical defocus gradient model may also be expressed as follows:

[0111] The derivation process of the optical defocus gradient model in the present invention is described below. Please refer to Figures 3a and 3b, wherein Figure 3a is a schematic diagram of the optical system imaging model; Figure 3b is a schematic diagram of the confusion circle of the image plane. As shown in Figure 3a, p represents the target object point in the object space, D is the lens diameter, d is the diameter of the confusion circle, l is the object distance, l' is the ideal image distance, p' is the ideal image point, △ is the defocus amount, and p" is the center of the confusion circle. The confusion circle of the image plane is shown in Figure 3(b), where R represents the radius. For the confusion circle, the energy of each pixel point is 1 / πR 2 , since the confusion circle is a circle with relatively uniform energy, the brightness value of each point in the circle is the same, for example, 255, but there is a brightness difference between the boundary of the confusion circle and the background. Therefore, compared with a point A on the circumference of the confusion circle, when calculating its clarity evaluation value, the brightness values ​​of the pixels in the confusion circle cancel each other out, and only the pixels on the circumference are not affected. Therefore, it can be considered that the clarity evaluation value is only related to the pixels on the circumference of the confusion circle. When the image clarity evaluation algorithm is used to calculate the clarity evaluation value of the confusion circle, the pixels within the uniform circumference do not need to participate in the calculation. It is only necessary to calculate the circumference length of the confusion circle and the energy of each pixel. A small arc line segment on the circumference can be expressed as Rdθ. When θ is small enough, it can be approximated as a line segment, and its horizontal and vertical components can be expressed as Rdθsinθ and Rdθcosθ, respectively. Therefore, the clarity evaluation value f(x) of the confusion circle on the image plane can be obtained by the line integral on the circumference:

[0112] In the formula, C is a constant (i.e. 4 / π). From the above formula (8), we can conclude that the clarity evaluation value f(x) of the circle of confusion is inversely proportional to the radius R. Since the radius R of the circle of confusion is inversely proportional to the defocus amount |x–x f | is proportional to |, so for any focus position x, the defocus amount |x–x f The relationship between | and the sharpness evaluation value f(x) satisfies any one of the calculation formulas (4) to (7) of the above-mentioned optical defocus gradient model.

[0113] Since the optical defocus gradient model in the present invention is derived based on the clarity evaluation value function f(x) of the diffusion circle on the image plane, it can be seen that the optical defocus gradient model in the present invention is based on the fundamental cause of defocus, and uses the optical system imaging theory as a theoretical support, so the focusing accuracy is high. In addition, the three focus positions used to solve the optical defocus gradient model in the present invention are determined by global point selection, which can avoid the occurrence of local extreme values ​​and thus avoid the secondary peak effect. In addition, when determining the second focus position, the present invention can reduce the number of searches for the second focus position by setting a larger moving step size and a search end condition, thereby not only avoiding excessive iterations, but also improving the focusing efficiency and having strong real-time performance. In summary, the present invention can effectively avoid the problems of low focusing accuracy (the present invention), poor real-time performance, secondary peak effect, excessive iteration, etc. caused by relying solely on experience or evaluation functions to find similar curve models.

[0114] In some exemplary embodiments, determining a final in-focus position based on the potential in-focus position includes:

[0115] A focusing position within the focusing range that is located on one side of the potential in-focus position and is one step away from the potential in-focus position is selected as a first candidate in-focus position;

[0116] determining whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position;

[0117] If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position, selecting a focus position within the focusing range that is located on the other side of the potential in-focus position and one step away from the potential in-focus position as a second candidate in-focus position; and determining whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position;

[0118] If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position, the potential in-focus position is used as the final in-focus position; if the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the second candidate in-focus position, the second candidate in-focus position is used as the final in-focus position;

[0119] If the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the first candidate in-focus position, selecting a focus position within the focusing range that is located on the same side of the potential in-focus position and one step away from the first candidate in-focus position as a third candidate in-focus position, and determining whether the image clarity evaluation value corresponding to the first candidate in-focus position is greater than the image clarity evaluation value corresponding to the third candidate in-focus position;

[0120] If so, the first candidate in-focus position is used as the final in-focus position; if not, the third candidate in-focus position is used as the final in-focus position.

[0121] Therefore, the present invention determines the potential focus position, then uses the image clarity evaluation function and combines the hill climbing search method to find the potential focus position x. f Perform two-step fine search near the final focus position x f ', thereby further improving the accuracy of the final quasi-focus position.

[0122] Specifically, please continue to refer to FIG2. As shown in FIG2, after calculating the potential focus position x f Then, the control driving device (such as a stepping motor) drives the objective lens of the endoscope to move to the potential quasi-focus position x f Then the target object is imaged to obtain the corresponding target image, and the preset image clarity evaluation algorithm is used to calculate the potential focus position x f The image clarity evaluation value f(x f )(that is, obtaining the potential focal position x f The corresponding image clarity evaluation value f(x f )). Then use x f As the starting point, the random control driving device (such as a stepping motor) drives the objective lens of the endoscope to move one step in a certain direction to reach the first candidate quasi-focus position x f+1 , and image the target to obtain the corresponding target image, and then use the preset image clarity evaluation algorithm to calculate the first candidate focus position x f+1 The image clarity evaluation value f(x f+1 )(that is, obtaining the first candidate in-focus position x f+1 The corresponding image clarity evaluation value f(x f+1 )). If f(x f )>f(x f+1 ), then x f As the starting point, the control driving device (such as a stepping motor) drives the objective lens of the endoscope to move one step in the opposite direction to reach the second candidate focal position x f-1, and image the target to obtain the corresponding target image, and then use the preset image clarity evaluation algorithm to calculate the second candidate focus position x f-1 The image clarity evaluation value f(x f-1 )(that is, obtaining the second candidate in-focus position x f-1 The corresponding image clarity evaluation value f(x f-1 )). If f(x f )>f(x f-1 ), then the potential focus position x f As the final in-focus position x f ';If f(x f )≤f(x f-1 ), then the second candidate in-focus position x f-1 As the final in-focus position x f '. If f(x f )≤f(x f+1 ), then take the first candidate focal position x f+1 As the starting point, the control driving device (such as a stepping motor) drives the objective lens of the endoscope to continue moving one step in the same direction to reach the third candidate focal position x f+2 , and image the target to obtain the corresponding target image, and then use the preset image clarity evaluation algorithm to calculate the third candidate focus position x f+2 The image clarity evaluation value f(x f+2 )(that is, obtaining the third candidate in-focus position x f+2 The corresponding image clarity evaluation value f(x f+2 )). If f(x f+1 )>f(x f+2 ), then the first candidate in-focus position x f+1 As the final in-focus position x f ';If f(x f+1 )≤f(x f+2 ), then the third candidate in-focus position x f+2 As the final in-focus position x f '.

[0123] In summary, the present application selects the initial focus position x0, the first focus position x1, and the second focus position x2 (the second focus position can be searched out at least once, and when s = [(L-x0) / 3], the second focus position can be searched out at most twice), and then the potential in-focus position x0 can be calculated based on the initial focus position x1, the first focus position x2, and the optical defocus gradient model. f , that is, at least 3 focusing times (when s = [(L-x0) / 3], at most 4 focusing times) can determine the potential quasi-focus position xf ; then at the potential focal position x f Perform two fine focusing operations near the target to determine the final focus position x f It can be seen that the present invention only needs 5 to 8 focusing adjustments to complete focusing, and has the advantages of small calculation amount and high real-time performance.

[0124] Please continue to refer to Figure 4, which is a flow chart of an image clarity evaluation algorithm provided by one embodiment of the present invention. As shown in Figure 4, in some exemplary embodiments, the image clarity evaluation algorithm includes:

[0125] Divide the image to be evaluated into N×N image blocks, where N is a positive integer;

[0126] For each image block, the variance factor, appropriate exposure factor and contrast factor corresponding to the image block are calculated respectively;

[0127] Calculating the weighted variance factor, weighted appropriate exposure factor, and weighted contrast factor corresponding to the image to be evaluated respectively according to the weight values, variance factors, appropriate exposure factors, and contrast factors corresponding to all image blocks;

[0128] An image clarity evaluation value of the image to be evaluated is calculated according to the weighted variance factor, the weighted appropriate exposure factor, and the weighted contrast factor corresponding to the image to be evaluated.

[0129] Since the edge distortion of the endoscopic image is serious, the central area has little distortion and the image is clear, the doctor pays more attention to the central area of ​​the image. Therefore, by first dividing the image to be evaluated (i.e., the target object image described above) into N×N image blocks, and then calculating the variance factor, appropriate exposure factor and contrast factor corresponding to each image block respectively; then, according to the weight values ​​and variance factors corresponding to all the image blocks, the weighted variance factor corresponding to the image to be evaluated is calculated; according to the weight values ​​and appropriate exposure factors corresponding to all the image blocks, the weighted appropriate exposure factor corresponding to the image to be evaluated is calculated; according to the weight values ​​and contrast factors corresponding to all the image blocks, the weighted contrast factor corresponding to the image to be evaluated is calculated; finally, according to the weighted variance factor, weighted appropriate exposure factor and weighted contrast factor corresponding to the image to be evaluated, the image clarity evaluation value of the image to be evaluated is calculated, so that the image clarity evaluation algorithm of the present invention can more accurately calculate the clarity of the image.

[0130] Specifically, image variance is a method of statistically measuring the degree of dispersion of image grayscale. The larger the variance, the more dispersed the grayscale levels in the image, and the better the image quality. The moderate exposure factor is introduced based on the visual characteristics of the human eye and the spatial frequency characteristics of the image. The moderate exposure area can present richer texture details and color information. For a blurred image, the exposure area is large and the texture details are missing. On the contrary, for a clearly focused image, the exposure area is small and the texture details are rich. Image contrast reflects the clarity of the image and the depth of the texture grooves. For a clearly focused image, the deeper the texture grooves, the greater the contrast and the clearer the visual effect; on the contrary, the smaller the contrast, the shallower the grooves and the blurrier the effect. Therefore, the image clarity evaluation algorithm in the present invention calculates the image clarity evaluation value based on the three measurement factors of variance factor, comfortable exposure factor and contrast factor, which can more accurately reflect the clarity of the image.

[0131] Please continue to refer to FIG. 4 . As shown in FIG. 4 , in some exemplary embodiments, before dividing the image to be evaluated into N×N image blocks, the image clarity evaluation algorithm further includes:

[0132] Normalization is performed on the pixel value (including grayscale value, red channel value, green channel value and blue channel value) of each pixel point of the image to be evaluated.

[0133] Therefore, by first normalizing the pixel values ​​of each pixel point of the image to be evaluated, and then dividing the image and calculating the variance factor, the appropriate exposure factor and the contrast factor, the amount of calculation can be effectively reduced, and the efficiency of the autofocus method provided by the present invention can be further improved.

[0134] Specifically, normalization can be performed according to the following formula: nor (x,y)=f img (x,y) / 255 (9)

[0135] Among them, f img (x,y) is the pixel value before normalization of the pixel with coordinates (x,y), f nor (x,y) is the normalized pixel value of the pixel with coordinates (x,y).

[0136] It should be noted that, as those skilled in the art will appreciate, since the images generally collected by an endoscope are RGB images, for each pixel, the red channel value (i.e., R value), green channel value (i.e., G value), and blue channel value (i.e., B value) of the pixel can be normalized according to the above formula (9), and then the grayscale value corresponding to the pixel can be calculated based on the normalized red channel value (i.e., R value), green channel value (i.e., G value), and blue channel value (i.e., B value). Of course, the image to be evaluated in RGB format can also be converted into a grayscale image first, and then the grayscale value can be normalized according to formula (9).

[0137] In some exemplary embodiments, calculating the variance factor, the appropriate exposure factor, and the contrast factor corresponding to each image block includes:

[0138] For each image block, the variance factor corresponding to the image block is calculated according to the normalized grayscale value corresponding to each pixel in the image block.

[0139] Specifically, the variance factor corresponding to the image block can be calculated according to the following formula:

[0140] Among them, f avg is the average value of the grayscale value of the image block, f var is the variance factor corresponding to the image block, m is the width of the image block, and n is the height of the image block.

[0141] Furthermore, after calculating the variance factor f corresponding to each image block, var After that, the weighted variance factor V corresponding to the image to be evaluated can be calculated according to the following formula:

[0142] in, is the variance factor corresponding to the i-th image block, w i is the weight value corresponding to the i-th image block.

[0143] Furthermore, the weight of each image block (coordinates are (x, y)) satisfies the Gaussian distribution and can be calculated according to the following formula:

[0144] Assuming that the image is divided into 3×3 image blocks and σ=1, the weight matrix w is:

[0145] It should be noted that, as those skilled in the art will appreciate, in some other implementations, other weighting methods besides the Gaussian weighting method may be used to calculate the weight value corresponding to each image block.

[0146] In some exemplary embodiments, for each image block, calculating the comfort exposure factor corresponding to the image block includes:

[0147] For each image block, an appropriate exposure factor corresponding to the image block is calculated according to the normalized red channel value, green channel value, and blue channel value corresponding to each pixel in the image block.

[0148] Specifically, the appropriate exposure factor corresponding to the image block can be calculated according to the following formula:

[0149] in, is the normalized red channel value of the pixel with coordinates (x, y), is the normalized green channel value of the pixel with coordinates (x, y), is the normalized blue channel value of the pixel with coordinates (x, y), μ is the empirical value 0.6, σ is the empirical value 0.2, and W exp is the appropriate exposure factor corresponding to the image block, m is the width of the image block, and n is the height of the image block.

[0150] Furthermore, after calculating the appropriate exposure factor W corresponding to each image block, exp After that, the weighted appropriate exposure factor corresponding to the image to be evaluated can be calculated according to the following formula:

[0151] in, is the appropriate exposure factor corresponding to the i-th image block, w i is the weight value corresponding to the i-th image block.

[0152] In some exemplary embodiments, for each image block, respectively calculating a contrast factor corresponding to the image block includes:

[0153] For each image block, the contrast factor corresponding to the image block is calculated according to the normalized grayscale value corresponding to each pixel in the image block and the normalized grayscale values ​​corresponding to the adjacent pixels.

[0154] Specifically, the contrast factor corresponding to the image block can be calculated according to the following formula:

[0155] Among them, C lapis the contrast factor corresponding to the image block, f nor (x,y) is the normalized grayscale value of the pixel with coordinates (x,y), f nor (x+1,y) is the normalized grayscale value of the pixel with coordinates (x+1,y), f nor (x-1,y) is the normalized grayscale value of the pixel with coordinates (x-1,y), f nor (x,y+1) is the normalized grayscale value of the pixel with coordinates (x,y+1), f nor (x, y-1) is the normalized grayscale value of the pixel with coordinates (x, y-1), m is the width of the image block, and n is the height of the image block.

[0156] Furthermore, after calculating the contrast factor corresponding to each image block, the weighted contrast factor C corresponding to the image to be evaluated can be calculated according to the following formula:

[0157] in, is the contrast factor corresponding to the i-th image block, w i is the weight value corresponding to the i-th image block.

[0158] In some exemplary embodiments, calculating the image clarity evaluation value of the image to be evaluated based on the weighted variance factor, the weighted appropriate exposure factor, and the weighted contrast factor corresponding to the image to be evaluated includes:

[0159] The image clarity evaluation value of the image to be evaluated is calculated using the following formula:

[0160] Among them, V is the weighted variance factor, W is the weighted moderate exposure factor, and C is the weighted contrast factor. is the weight parameter of the weighted variance factor, is the weight parameter of the weighted moderate exposure factor, is the weight parameter of the weighted contrast factor.

[0161] According to experience, When 1 is selected, the weighted variance factor is more accurate in judging image clarity. Select 3 or 4. The value is 2. Generally, the range of these three weight parameters is [0-5].

[0162] Based on the same inventive concept, the present invention also provides an electronic device. Please refer to Figure 5, which is a schematic diagram of the block structure of an electronic device provided by one embodiment of the present invention. As shown in Figure 5, the electronic device includes a processor 101 and a memory 103. The memory 103 stores a computer program. When the computer program is executed by the processor 101, the automatic focusing method of the endoscope described above is implemented. Since the electronic device provided by the present invention and the automatic focusing method of the endoscope provided by the present invention belong to the same inventive concept, the electronic device provided by the present invention has all the beneficial effects of the automatic focusing method of the endoscope provided by the present invention. For details, please refer to the relevant content above regarding the beneficial effects of the automatic focusing method of the endoscope provided by the present invention. Therefore, the beneficial effects of the electronic device provided by the present invention will not be repeated here.

[0163] As shown in FIG5 , the electronic device further includes a communication interface 102 and a communication bus 104, wherein the processor 101, the communication interface 102, and the memory 103 communicate with each other via the communication bus 104. The communication bus 104 may be a Peripheral Component Interconnect (PCI) bus or an Extended Industry Standard Architecture (EISA) bus, etc. The communication bus 104 may be divided into an address bus, a data bus, a control bus, etc. For ease of illustration, only one thick line is used in the figure, but this does not mean that there is only one bus or one type of bus. The communication interface 102 is used for communication between the electronic device and other devices.

[0164] The processor 101 referred to in the present invention may be a central processing unit (CPU), other general-purpose processors, digital signal processors (DSP), application-specific integrated circuits (ASIC), field-programmable gate arrays (FPGA), or other programmable logic devices, discrete gate or transistor logic devices, discrete hardware components, etc. A general-purpose processor may be a microprocessor or any conventional processor. The processor 101 is the control center of the electronic device, connecting various parts of the entire electronic device using various interfaces and lines.

[0165] The memory 103 may be used to store the computer program. The processor 101 implements various functions of the electronic device by running or executing the computer program stored in the memory 103 and calling the data stored in the memory 103.

[0166] The memory 103 may include non-volatile and / or volatile memory. Non-volatile memory may include read-only memory (ROM), programmable ROM (PROM), electrically programmable ROM (EPROM), electrically erasable programmable ROM (EEPROM), or flash memory. Volatile memory may include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM is available in various forms, such as static RAM (SRAM), dynamic RAM (DRAM), synchronous DRAM (SDRAM), double data rate SDRAM (DDR SDRAM), enhanced SDRAM (ESDRAM), synchronous link DRAM (SLDRAM), RAMbus direct RAM (RDRAM), direct RAMbus dynamic RAM (DRDRAM), and RAMbus dynamic RAM (RDRAM).

[0167] The present invention also provides a readable storage medium having a computer program stored therein. When the computer program is executed by a processor, the computer program can implement the auto-focusing method for an endoscope described above. Since the readable storage medium provided by the present invention and the auto-focusing method for an endoscope provided by the present invention belong to the same inventive concept, the readable storage medium provided by the present invention has all the beneficial effects of the auto-focusing method for an endoscope provided by the present invention. For details, please refer to the relevant content above regarding the beneficial effects of the auto-focusing method for an endoscope provided by the present invention. Therefore, the beneficial effects of the readable storage medium provided by the present invention will not be repeated here.

[0168] The readable storage medium of the embodiment of the present invention can adopt any combination of one or more computer-readable media. The readable medium can be a computer-readable signal medium or a computer-readable storage medium. The computer-readable storage medium can be, for example, but not limited to, a system, device or component of electricity, magnetism, light, electromagnetic, infrared or semiconductor, or any combination thereof. More specific examples (non-exhaustive list) of computer-readable storage media include: an electrical connection with one or more wires, a portable computer hard disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this article, a computer-readable storage medium can be any tangible medium containing or storing a program, which can be used by an instruction execution system, device or device or used in combination with it.

[0169] A computer-readable signal medium may include a data signal propagated in baseband or as part of a carrier wave, which carries computer-readable program code. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. A computer-readable signal medium may also be any computer-readable medium other than a computer-readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium may be transmitted using any suitable medium, including but not limited to wireless, wire, optical cable, RF, etc., or any suitable combination thereof.

[0170] Computer program code for performing the operations of the present invention may be written in one or more programming languages, or a combination thereof, including object-oriented programming languages ​​such as Java, Smalltalk, C++, and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a stand-alone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving a remote computer, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computer (e.g., through the Internet using an Internet service provider).

[0171] In summary, compared with the prior art, the auto-focusing method for an endoscope, the electronic device, and the readable storage medium provided by the present invention have the following beneficial effects:

[0172] The present invention randomly selects a focusing position within the focusing range of the endoscope as the initial focusing position; determines a first focusing position and a second focusing position within the focusing range based on the initial focusing position, wherein the initial focusing position is located between the first focusing position and the second focusing position; determines a potential quasi-focus position based on the initial focusing position, the first focusing position, the second focusing position, and a pre-constructed optical defocus gradient model; and determines a final quasi-focus position based on the potential quasi-focus position. Therefore, compared with other mainstream methods that simply use clarity evaluation function feedback for core focus control, the present invention starts from the root cause of defocus and uses an optical defocus gradient model with optical imaging defocus theory as theoretical support, which effectively avoids the problems of poor algorithm robustness, slow parameter adjustment, and poor focusing accuracy caused by relying solely on experience or evaluation functions to find similar curve models; at the same time, the present invention avoids the problem of easily falling into local extreme values ​​by estimating the quasi-focus position based on a global optical defocus gradient model. Furthermore, by introducing boundary conditions such as the first and second focus positions, the present invention can effectively solve the problem of repeated changes in focus direction during transition iterations in traditional hill climbing methods. The present invention can achieve fast and accurate autofocus of the endoscope with just 5 to 8 focus passes.

[0173] It should be noted that the devices and methods disclosed in the embodiments of this document may also be implemented in other ways. The device embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of the devices, methods, and computer program products according to the various embodiments of this document. In this regard, each box in the flowchart or block diagram may represent a module, program, or portion of code, wherein the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function, and the module, program segment, or portion of code contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the boxes may also occur in an order different from that marked in the accompanying drawings. For example, two consecutive boxes may actually be executed substantially in parallel, or they may sometimes be executed in the opposite order, depending on the functions involved. It should also be noted that each box in the block diagram and / or flowchart, as well as combinations of boxes in the block diagram and / or flowchart, may be implemented using a dedicated hardware-based system for performing the specified functions or actions, or may be implemented using a combination of dedicated hardware and computer instructions. In addition, the functional modules in the various embodiments of this document may be integrated together to form an independent part, or each module may exist independently, or two or more modules may be integrated to form an independent part.

[0174] It should also be noted that the above description is merely a description of preferred embodiments of the present invention and does not limit the scope of the present invention. Any changes or modifications made by a person skilled in the art based on the above disclosure are within the scope of protection of the present invention. Obviously, various changes and modifications can be made to the present invention by a person skilled in the art without departing from the spirit and scope of the present invention. Thus, if such modifications and variations fall within the scope of the present invention and its equivalents, the present invention is intended to include such modifications and variations.

Claims

1. An automatic focusing method for an endoscope, characterized in that: The automatic focusing method comprises: Randomly selecting a focusing position within the focusing range of the endoscope as an initial focusing position; Determining a first focus position and a second focus position within the focus adjustment range according to the initial focus position, wherein the initial focus position is located between the first focus position and the second focus position; Determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position, and a pre-constructed optical defocus gradient model; A final in-focus position is determined according to the potential in-focus position.

2. The automatic focusing method of an endoscope according to claim 1, characterized in that: Determining a first focus position and a second focus position within the focus adjustment range according to the initial focus position includes: Selecting a focusing position located on a first side of the initial focusing position within the focusing range as a first focusing position; Calculating the image clarity evaluation values ​​corresponding to the initial focus position and the first focus position respectively, wherein the image clarity evaluation value corresponding to the initial corresponding position is calculated based on the target object image collected at the initial focus position using a preset image clarity evaluation algorithm, and the image clarity evaluation value corresponding to the first focus position is calculated based on the target object image collected at the first focus position using the image clarity evaluation algorithm; According to the initial focus position and the image definition evaluation value corresponding to the first focus position, a focus position located at a second side of the initial focus position is determined within the focus range as a second focus position.

3. The automatic focusing method of an endoscope according to claim 2, characterized in that: The determining, according to the image clarity evaluation value corresponding to the initial focus position and the first focus position, a focus position located on a second side of the initial focus position within the focus range as a second focus position, comprises: If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, searching is performed in a focus position interval located on a second side of the initial focus position within the focus range until the image clarity evaluation value corresponding to the currently searched focus position is greater than the image clarity evaluation value corresponding to the initial focus position, or the current search times reaches a first preset times, and the currently searched focus position is used as the second focus position; If the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, then a search is performed within the focus position interval located on the second side of the initial focus position within the focus range until the image clarity evaluation value corresponding to the currently searched focus position is less than the image clarity evaluation value corresponding to the initial focus position, or the current search number reaches a second preset number, then the currently searched focus position is used as the second focus position.

4. The automatic focusing method of an endoscope according to claim 3, characterized in that: The searching within a focus position interval located on a second side of the initial focus position within the focus range comprises: The search is performed in a focus position interval located on a second side of the initial focus position within the focus range according to a preset moving step length.

5. The automatic focusing method of an endoscope according to claim 4, characterized in that: The calculation formula of the preset moving step length is as follows: s = [(L-x0) / 3] Wherein, s is the preset moving step length, L is the total step length corresponding to the focusing range, x0 is the step length corresponding to the initial focusing position, and [] indicates rounding.

6. The automatic focusing method of an endoscope according to claim 3, characterized in that: The determining a potential in-focus position according to the initial focus position, the first focus position, the second focus position and a pre-constructed optical defocus gradient model comprises: If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than or equal to the image clarity evaluation value corresponding to the initial focus position; or if the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position and the image clarity evaluation value corresponding to the second focus position, the initial focus position, the first focus position and the second focus position and their respective corresponding image clarity evaluation values ​​are respectively substituted into the optical defocus gradient model, and the optical defocus gradient model is solved to determine a potential in-focus position; If the image clarity evaluation value corresponding to the first focus position, the image clarity evaluation value corresponding to the initial focus position and the image clarity evaluation value corresponding to the second focus position satisfy a monotonically increasing relationship; or if the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than the image clarity evaluation value corresponding to the initial focus position, then a focus position is determined within the focusing range as the third focus position, the one of the initial focus position, the first focus position and the second focus position that is farthest from the third focus position is eliminated, and the remaining two and the third focus position and their respective corresponding image clarity evaluation values ​​are respectively substituted into the optical defocus gradient model, and the optical defocus gradient model is solved to determine a potential in-focus position.

7. The automatic focusing method of an endoscope according to claim 6, characterized in that: If the image clarity evaluation value corresponding to the initial focus position is greater than the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is greater than the image clarity evaluation value corresponding to the initial focus position, the calculation formula for the third focus position is as follows: x3=x2+[(x2-x0)*8 / L] Wherein, x3 is the step length corresponding to the third focus position, x2 is the step length corresponding to the second focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus range, and [] indicates rounding; If the image clarity evaluation value corresponding to the initial focus position is less than or equal to the image clarity evaluation value corresponding to the first focus position, and the image clarity evaluation value corresponding to the second focus position is less than the image clarity evaluation value corresponding to the initial focus position, the calculation formula for the third focus position is as follows: x3=x1+[(x1-x0)*8 / L] Among them, x3 is the step length corresponding to the third focus position, x1 is the step length corresponding to the first focus position, x0 is the step length corresponding to the initial focus position, L is the total step length corresponding to the focus range, and [] indicates rounding.

8. The automatic focusing method of an endoscope according to claim 2, characterized in that: The image clarity evaluation algorithm comprises: Divide the image to be evaluated into N×N image blocks, where N is a positive integer; For each image block, the variance factor, the appropriate exposure factor and the contrast factor corresponding to the image block are calculated respectively; According to the weight values ​​and variance factors, appropriate exposure factors and contrast factors corresponding to all the image blocks, respectively, a weighted variance factor, a weighted appropriate exposure factor and a weighted contrast factor corresponding to the image to be evaluated are calculated; An image clarity evaluation value of the image to be evaluated is calculated according to the weighted variance factor, the weighted appropriate exposure factor and the weighted contrast factor corresponding to the image to be evaluated.

9. The automatic focusing method of an endoscope according to claim 8, characterized in that: Before dividing the image to be evaluated into N×N image blocks, the image clarity evaluation algorithm further includes: Normalizing the pixel value of each pixel point of the image to be evaluated; The step of calculating the variance factor, the appropriate exposure factor and the contrast factor corresponding to each image block includes: For each image block, the variance factor corresponding to the image block is calculated according to the normalized grayscale value corresponding to each pixel in the image block; the contrast factor corresponding to the image block is calculated according to the normalized grayscale value corresponding to each pixel in the image block and the normalized grayscale values ​​corresponding to the adjacent pixels; the appropriate exposure factor corresponding to the image block is calculated according to the normalized red channel value, green channel value and blue channel value corresponding to each pixel in the image block.

10. The automatic focusing method of an endoscope according to claim 8, characterized in that: The step of calculating the image clarity evaluation value of the image to be evaluated according to the weighted variance factor, the weighted appropriate exposure factor and the weighted contrast factor corresponding to the image to be evaluated comprises: The image clarity evaluation value of the image to be evaluated is calculated using the following formula: Among them, V is the weighted variance factor, W is the weighted moderate exposure factor, and C is the weighted contrast factor. is the weight parameter of the weighted variance factor, is the weight parameter of the weighted moderate exposure factor, is the weight parameter of the weighted contrast factor.

11. The automatic focusing method of an endoscope according to claim 1, characterized in that: The calculation formula of the optical defocus gradient model is as follows: Or as follows: Among them, f(x) is the image clarity evaluation value corresponding to the focus position with a step size of x, and x f is the step size corresponding to the potential in-focus position, and a and b are constants respectively.

12. The endoscope auto-focusing method according to claim 1, characterized in that: The determining of a final in-focus position according to the potential in-focus position comprises: Taking a focus position within the focus range that is located on a first side of the potential in-focus position and is one step away from the potential in-focus position as a first candidate in-focus position; Determining whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position; If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the first candidate in-focus position, a focusing position within the focusing range that is located on the second side of the potential in-focus position and is one step away from the potential in-focus position is used as a second candidate in-focus position; and it is determined whether the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position; If the image clarity evaluation value corresponding to the potential in-focus position is greater than the image clarity evaluation value corresponding to the second candidate in-focus position, the potential in-focus position is used as the final in-focus position; if the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the second candidate in-focus position, the second candidate in-focus position is used as the final in-focus position; If the image clarity evaluation value corresponding to the potential in-focus position is less than or equal to the image clarity evaluation value corresponding to the first candidate in-focus position, a focusing position within the focusing range that is located on the same side of the potential in-focus position and is one step away from the first candidate in-focus position is used as a third candidate in-focus position, and it is determined whether the image clarity evaluation value corresponding to the first candidate in-focus position is greater than the image clarity evaluation value corresponding to the third candidate in-focus position; If yes, the first candidate in-focus position is used as the final in-focus position; if no, the third candidate in-focus position is used as the final in-focus position.

13. An electronic device, characterized in that: The invention comprises a processor and a memory, wherein a computer program is stored in the memory, and when the computer program is executed by the processor, the auto-focusing method of the endoscope according to any one of claims 1 to 12 is implemented.

14. A readable storage medium, characterized in that: The readable storage medium stores a computer program, and when the computer program is executed by a processor, the auto-focusing method for an endoscope according to any one of claims 1 to 12 is implemented.

Images