Endoscope equipment and automatic focusing method

Abstract

The embodiment of the invention provides an endoscope device and an automatic focusing method, the endoscope device can obtain a to-be-processed image and perform blurring operation on the to-be-processed image to obtain a blurred image, the blurring operation can weaken high-frequency details in the image, and the high-frequency details in the to-be-processed image and the blurred image can be focused on the to-be-processed image. The gray values of the corresponding pixel points are subtracted to obtain a difference image, the difference image can be used for representing the contour of the target part, and similar parts in the to-be-processed image and the blurred image are weakened, so that the interference of a complex and changeable operation environment on the automatic focusing operation can be avoided, and the accuracy of the automatic focusing operation is improved. The focusing window is determined according to the centroid of the difference image, and the target focus is determined in the focusing window, so that the endoscope equipment can perform focusing again based on the target focus, and compared with focusing with the image center as the focus, focusing based on the target focus is more excellent in focusing effect on the target part; and the success rate and the safety of the operation are improved.

Description

Technical Field

[0001] This application relates to the field of medical technology, and in particular to an endoscope device and an automatic focusing method. Background Technology

[0002] With the rapid advancement of medical technology, endoscopic equipment plays a crucial role in surgical procedures. The introduction of autofocus technology into endoscopic equipment has increased the speed of surgical procedures using endoscopic devices.

[0003] Current autofocus technology often uses a fixed image center as the focus point for focusing. However, this method is difficult to guarantee focusing accuracy in complex and changing surgical environments, which affects the success rate and safety of the surgery. Summary of the Invention

[0004] To address the problems in the prior art, this application provides an endoscopic device and an autofocus method that can avoid interference from the complex and ever-changing surgical environment on the autofocus of the endoscopic device.

[0005] In a first aspect, embodiments of this application provide an endoscope device, the endoscope device including a camera, a processor, and a display:

[0006] The camera is configured to acquire an image to be processed that includes the target area;

[0007] The processor is configured to perform blurring processing on the image to be processed to obtain a blurred image;

[0008] A difference image is determined between the image to be processed and the blurred image; the difference image is obtained by subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image; the difference image is used to characterize the contour of the target region.

[0009] The focus window is determined based on the centroid of the difference image;

[0010] Determine the target focus in the focus window, and control the camera to refocus based on the target focus;

[0011] The display is configured to show the image captured after focusing.

[0012] In one possible implementation, the processor is configured to:

[0013] Determine the explicit focus and potential focus in the focusing window respectively;

[0014] The target focus is determined based on the first image sharpness of the manifest focus and the second image sharpness of the potential focus.

[0015] In one possible implementation, the processor is configured to:

[0016] Based on the grayscale values ​​at at least three locations in the focusing window and the dimension-reduced horizontal filter mask, determine the horizontal gradient corresponding to the at least three locations;

[0017] Based on the gray values ​​at the at least three locations and the dimension-reduced vertical filter mask, determine the vertical gradient corresponding to the at least three locations;

[0018] The first image sharpness at each of the at least three locations is determined based on the sum of the squares of the horizontal gradient and the squares of the vertical gradient at each of the at least three locations.

[0019] An image sharpness curve is fitted based on the first image sharpness at the at least three locations; the image sharpness curve includes the first image sharpness at multiple locations within the focus window;

[0020] The location with the highest image sharpness among the plurality of locations is designated as the dominant focus. In one possible implementation, the at least three locations are an initial location, a first location, and a second location, respectively; the initial location is located at an intermediate position between the first location and the second location; the processor is configured to:

[0021] If the first image sharpness corresponding to the first position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the second position is less than the first image sharpness corresponding to the initial position, then the position with the greatest first image sharpness between the initial position and the first position is determined as the dominant focus.

[0022] If the first image sharpness corresponding to the second position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the first position is less than the first image sharpness corresponding to the initial position, then the position with the greatest first image sharpness between the initial position and the second position is determined as the dominant focus.

[0023] In one possible implementation, the processor is configured to: obtain an optical defocus model curve corresponding to the image in the focusing window based on the correspondence between the second image sharpness and multiple positions in the optical defocus model; the optical defocus model curve includes the second image sharpness corresponding to multiple positions in the focusing window;

[0024] The location with the highest image clarity among the plurality of locations is selected as the potential focal point.

[0025] In one possible implementation, the processor is configured to:

[0026] If the first image sharpness of the manifest focus is greater than the second image sharpness of the potential focus, then the manifest focus is taken as the target focus;

[0027] If the second image sharpness of the potential focus is greater than the first image sharpness of the manifest focus, then the potential focus is taken as the target focus.

[0028] In one possible implementation, the processor is configured to:

[0029] The centroid of the difference image is used as the center point of the focus window, the width is set as the width of the focus window, and the length is set as the length of the focus window.

[0030] In one possible implementation, the processor is configured to:

[0031] Adjust the curvature of the lens in the camera to focus the light reflected by the camera onto the target.

[0032] Secondly, embodiments of this application provide an autofocus method, including:

[0033] Obtain an image to be processed containing the target area, and perform blur processing on the image to be processed to obtain a blurred image;

[0034] A difference image is determined between the image to be processed and the blurred image; the difference image is obtained by subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image; the difference image is used to characterize the contour of the target region.

[0035] The focus window is determined based on the centroid of the difference image;

[0036] Determine the target focus in the focus window, and focus the image to be processed according to the focus window and the target focus;

[0037] Displays the image taken after focusing.

[0038] Thirdly, embodiments of this application provide a computer-readable storage medium storing a computer program, which, when executed by a processor, implements the aforementioned autofocus method.

[0039] This application provides an endoscopic device and an autofocus method. The endoscopic device can acquire an image to be processed and perform a blur operation on the image to obtain a blurred image. The blur operation can weaken high-frequency details in the image. The gray values ​​of corresponding pixels in the image to be processed and the blurred image are subtracted to obtain a difference image. The difference image can be used to characterize the contour of the target area and weaken similar parts in the image to be processed and the blurred image, thus avoiding interference from the complex and ever-changing surgical environment on the autofocus operation. The focus window is determined based on the centroid of the difference image, and the target focus is determined within the focus window, so that the endoscopic device can refocus based on the target focus. Compared with focusing based on the image center, focusing based on the target focus has a better focusing effect on the target area, increasing the success rate and safety of the surgery. Attached Figure Description

[0040] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0041] Figure 1 This is a schematic diagram of the structure of an endoscope device provided in an embodiment of this application;

[0042] Figure 2 This is a schematic diagram of the structure of a camera host provided in an embodiment of this application;

[0043] Figure 3 A flowchart of an autofocus method provided in an embodiment of this application;

[0044] Figure 4 A schematic diagram of an image to be processed provided in an embodiment of this application;

[0045] Figure 5 A schematic diagram of a blurred image corresponding to an image to be processed, provided in an embodiment of this application;

[0046] Figure 6 A schematic diagram of an evaluation curve for determining the focus window using a different method provided in an embodiment of this application;

[0047] Figure 7 This is a schematic diagram of an image region with a pixel intensity of 3*3 provided in an embodiment of this application;

[0048] Figure 8 A schematic diagram of an optical system imaging model provided in an embodiment of this application;

[0049] Figure 9 A schematic diagram of a dispersion circle provided for an embodiment of this application;

[0050] Figure 10 A schematic diagram of an optical defocus model curve and a focus evaluation curve provided for embodiments of this application;

[0051] Figure 11 A schematic diagram illustrating the determination of three positions, provided for an embodiment of this application;

[0052] Figure 12 This is another schematic diagram illustrating the determination of three positions according to an embodiment of this application;

[0053] Figure 13 A detailed flowchart of an autofocus method provided in this application embodiment;

[0054] Figure 14 This is a structural block diagram of an autofocus device provided in an embodiment of this application. Detailed Implementation

[0055] To make the objectives, technical solutions, and advantages of the embodiments of this application clearer, the technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. The described embodiments are only some, not all, of the embodiments of this application. All other embodiments obtained by those skilled in the art based on the embodiments of this application without creative effort are within the scope of protection of this application.

[0056] Furthermore, in the description of the embodiments of this application, unless otherwise stated, "and" means "or", for example, A / B can mean A or B; "and / or" in the text is merely a description of the relationship between related objects, indicating that there can be three relationships, for example, A and / or B can mean: A exists alone, A and B exist simultaneously, and B exists alone.

[0057] Specifically, in the description of this application, it should be understood that the terms "center," "longitudinal," "lateral," "length," "width," "thickness," "upper," "lower," "front," "rear," "left," "right," "vertical," "horizontal," "top," "bottom," "inner," "outer," "clockwise," and "counterclockwise," etc., indicate the orientation or positional relationship based on the orientation or positional relationship shown in the accompanying drawings. They are only for the convenience of describing this application and simplifying the description, and do not indicate or imply that the device or element referred to must have a specific orientation, or be constructed and operated in a specific orientation. Therefore, they should not be construed as limitations on this application. Furthermore, the terms "connection" and "linkage" used in this application, unless otherwise specified, include both direct and indirect connections (linkages).

[0058] Furthermore, the terms "first" and "second" are used for descriptive purposes only and should not be construed as indicating or implying relative importance or implicitly specifying the number of technical features indicated. Thus, a feature defined as "first" or "second" may explicitly or implicitly include one or more of the stated features. In the description of this application, "a plurality of" means two or more, unless otherwise explicitly specified.

[0059] Figure 1 An exemplary schematic diagram of an endoscope device according to an embodiment of this application is shown, wherein the endoscope device includes a cold light source, a camera assembly, and a display.

[0060] The camera module can be used to capture images of endoscopic examinations and surgeries to obtain visual data. Its key components may include one or more of the following: a camera, buttons, a camera unit, a CMOS (Complementary Metal-Oxide-Semiconductor) image sensor, a video cable, an optical adapter, an objective lens field of view, a lens rod, a communication port, and a video output interface. The camera unit can be connected to a monitor via the video output interface and to a cold light source unit via the communication port.

[0061] In this embodiment of the application, the objective lens field of view, lens rod, optical adapter, button, CMOS and camera in the key components of the camera assembly can also be collectively referred to as an endoscope camera.

[0062] The monitor can be used to display endoscopic images acquired and generated by the camera component.

[0063] Cold light sources can be used to provide illumination for endoscopes during endoscopic examinations and surgeries; their key components typically include: a cold light source main unit, a beam guide, a communication port, and a cold light source output interface.

[0064] It should be noted that the endoscopic equipment in the embodiments of this application specifically refers to the endoscopic equipment used in surgery, which may be one or more of a laparoscope, thoracoscope, arthroscope, rhinoscope, cystoscope, rectoscope, duodenoscope, mediastinoscope, and cardiac endoscope.

[0065] It should be understood that, Figure 1 The schematic diagram of the endoscopic device shown is merely an example, and the endoscopic device can have more than... Figure 1 The more or fewer components shown can be combined into two or more components, or they can have different component configurations. The various components shown in the figure can be implemented in hardware, software, or a combination of hardware and software, including one or more signal processing and / or application-specific integrated circuits.

[0066] based on Figure 1The schematic diagram of the endoscopic device shown in this application also includes a schematic diagram of the camera host. Figure 2 As shown, the camera host 200 may include components such as a memory 210, a processor 220, a communication interface 230, and a power supply 240.

[0067] The memory 210 can be used to store software programs and data. The processor 220 executes various functions of the camera host 200 and performs data processing by running the software programs or data stored in the memory 210. The memory 210 may include high-speed random access memory, and may also include non-volatile memory, such as at least one disk storage device, flash memory device, or other volatile solid-state storage device. The memory 210 stores an operating system that enables the camera host 200 to run. In this application, the memory 210 may store the operating system and various application programs, and may also store code that executes the methods described in the embodiments of this application.

[0068] The processor 220 is the control center of the camera host 200. It connects various parts of the camera host 200 via various interfaces and lines. By running or executing software programs stored in the memory 210 and calling data stored in the memory 210, it performs various functions and processes data for the camera host 200. In some embodiments, the processor 220 may include one or more processing units; the processor 220 may also integrate an application processor and a baseband processor. The application processor primarily handles the operating system, endoscope device interface, and applications, while the baseband processor primarily handles wireless communication. It is understood that the baseband processor may not be integrated into the processor 220. In this application, the processor 220 can run the operating system, applications, endoscope device interface display and touch response, and the autofocus method described in the embodiments of this application. The specific process of the processor 220 executing the autofocus method will be described in detail below.

[0069] It should be noted that, in the embodiments of this application, when the endoscopic device transmits data to other electronic devices, it can perform wired or wireless transmission with other devices. This application does not limit the transmission method, the number of devices, or the type of other devices. For example, other electronic devices can be endoscopes, personal computers, mobile phones, tablets, laptops, e-book readers, intelligent voice interaction devices, smart home devices, in-vehicle terminals, and other computer devices with certain computing capabilities that run instant messaging software and websites or social networking software and websites.

[0070] The communication interface 230 is used for information exchange with other electronic devices. Specifically, the communication interface 230 may include one or more of the following: network port 231, WiFi module 232, Bluetooth module 233, 4G module 234, and USB 235. Using different communication interfaces allows for the use of corresponding communication methods to exchange information with other electronic devices.

[0071] The power supply 240 is used to supply power to the various components in the camera host 200. The power supply 240 can be logically connected to the processor 220 through the power management system, thereby realizing the functions of managing charging, discharging and power consumption through the power management system.

[0072] In actual operation, endoscopic examinations or surgeries are mainly assisted by dynamic video. After the medical staff inserts the endoscope into the human body, the camera on the endoscope captures and displays images of the human body. As the camera moves, the area illuminated by the camera is displayed on the monitor connected to the endoscope. The environment around the lesion can be clearly seen, and the surgical instruments can be moved and operated in a three-dimensional and intuitive manner.

[0073] The foregoing provides many different implementation methods or examples for implementing different structures of this application. To simplify the content of the embodiments of this application, only the components and settings of specific examples are described above. Of course, these are merely examples and are not intended to limit this application.

[0074] Next, as follows Figure 3 As shown, Figure 3 A flowchart of an autofocus method provided in an embodiment of this application is shown. This method can be applied to... Figure 1 In the processor of the camera host of the endoscopic device shown, such as Figure 3 As shown, the method may include the following steps:

[0075] Step S301: Obtain the image to be processed containing the target area, and perform blur processing on the image to be processed to obtain a blurred image.

[0076] In one possible embodiment, an image containing the target area can be acquired using a camera in an endoscope. After acquiring the image, a Gaussian blur can be applied. Gaussian blur weakens high-frequency details and edges in the image, enhancing its continuity and smoothness, resulting in a smoother and more continuous image after processing. Specifically, the image can be Gaussian blurred using the following formula:

[0077]

[0078] Where (p,q) represents the coordinates of any point in the blurred image obtained after Gaussian blurring of the image to be processed, O'(p,q) represents the gray value of any point in the blurred image, and σ is the Gaussian criterion error. The larger σ is, the larger the Gaussian blur radius, and the higher the continuity and smoothness of the image. For example, Figure 4 This illustration shows a schematic diagram of an image to be processed according to an embodiment of this application. After Gaussian blurring, the image can be obtained as shown below. Figure 5 The image shown is blurry.

[0079] It should be noted that in order to weaken the high-frequency details and edges of the image to be processed, it is not necessary to use the Gaussian blur method. Other blurring methods, such as mean blur and Gaussian bilateral blur, can also be used to blur the image to be processed. This application does not limit the method.

[0080] Step S302: Determine the difference image between the image to be processed and the blurred image.

[0081] In one possible embodiment, after obtaining the blurred image, the corresponding difference image can be determined based on the image to be processed and the blurred image using the following formula:

[0082] C(p,q)=O(p,q)-O'(p,q)

[0083] Where O(p,q) represents the gray value of any point in the image to be processed, O'(p,q) represents the gray value of any point in the blurred image, and C(p,q) represents the gray value of any point in the difference image. Subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image yields the gray values ​​of the corresponding points in the difference image, thus obtaining the difference image. Typically, the obtained difference image often contains the contour of the target area; therefore, the difference image can be considered as representing the contour of the target area.

[0084] Step S303: Determine the focus window based on the centroid of the difference image.

[0085] In one possible embodiment, having obtained the difference image, the centroid of the difference image can be determined according to the following formula:

[0086]

[0087] Where, x c and y c , , and y are the x and y coordinates of the centroid of the difference image in the coordinate system, respectively. x and y are the x and y coordinates of any point of the centroid of the difference image in the coordinate system, respectively. M is the length of the difference image, N is the width of the difference image, and f(x,y) represents the gray value of the point at position (x,y).

[0088] In one possible embodiment, since the centroid of the difference image has been determined, the focus window can be determined based on the centroid of the difference image and the size of the focus window. The centroid of the difference image can be used as the center point of the focus window, the width can be set as the width of the focus window, and the length can be set as the length of the focus window. For example, if the size of the focus window is set to 10 units long and 8 units wide, and the centroid coordinates of the interpolated image are (20, 20), then the positions of the four vertices of the focus window can be determined as (15, 24), (15, 16), (25, 24), and (25, 16). It should be noted that different sizes of the focus window can be set according to the different resolutions and sizes of images acquired by different endoscope cameras; this application does not impose any limitations on this.

[0089] In one possible embodiment, current autofocus technologies often employ a fixed-position focus window for focusing. However, the autofocus method provided in this application determines the corresponding focus window by identifying the centroid of the difference image. This avoids interference from the complex and changing surgical environment on autofocus operations and is more conducive to automatic focusing of endoscopic equipment on target areas. Current autofocus technologies employ methods such as the central window method (fixing the focus window at a fixed position), the salient window method (selecting the most salient feature region in the image to be processed as the focus window), and the first-order moment window method (selecting the region with the most edge information in the image to be processed as the focus window).

[0090] For example, the images in the focus window determined by the center window method, the salient window method, the first-order moment window method, and the method of this application can be used to determine the value of the focus evaluation function. The horizontal axis of the function is the image sequence, which can be considered as different regions segmented in the focus window, with each region corresponding to a different image sequence. The vertical axis of the function is the focus value, which can reflect different levels of sharpness. Figure 6 This illustration shows a schematic diagram of an evaluation curve for determining the focus window using a different method provided in an embodiment of this application, such as... Figure 6As shown, the peak value of each function is 1. Therefore, the sharpness among the center window method, salient window method, first-order moment window method, and the method of this application needs to be confirmed by the minimum value of the evaluation function. The larger the difference between the maximum and minimum values ​​of the evaluation function, the sharper the image in the focus window. The minimum values ​​corresponding to the center window method, salient window method, first-order moment window method, and the method of this application are 0.47, 0.63, 0.19, and infinitely close to 0, respectively. Therefore, the method of this application determines the highest image sharpness in the focus window. In addition, the slopes of the center window method, the salient window method, the first-order moment window method, and the method of this application are 0.24, 0.31, 0.11, and 0.07, respectively. The lower the slope, the larger the step size can be used to traverse the image sequence in the smooth region, which effectively saves the autofocus time. Furthermore, the fluctuation of the evaluation function in the smooth region reflects its noise resistance performance. It can be seen that the curve of the method of this application is smoother and the noise resistance performance is stronger. Therefore, determining the focus window by the method of this application can achieve high-precision, high-efficiency, and high-noise-resistant focusing.

[0091] Step S304: Determine the target focus in the focus window and control the camera to refocus based on the target focus.

[0092] In one possible embodiment, before determining the focus point within the focus window, a dimensionality reduction focus evaluation algorithm can be used to evaluate the focus value of the image region within the focus window. This focus evaluation algorithm uses the Sobel operator to obtain the image gradient. The Sobel operator is the vector sum of a pair of orthogonal vectors, each of which is a directional derivative estimate multiplied by a unit vector specifying the derivative direction. This produces a simple center gradient estimate. Figure 7 A schematic diagram of an image region with a pixel intensity of 3*3 is shown, such as... Figure 8 As shown, in the detector using the Sobel operator, the partial derivatives of a 3x3 mask are used.

[0093] Horizontal gradient I x1 It can be determined by the following formula:

[0094] I x1 = (I3 + 2I6 + I9) - (I1 + 2I4 + I7)

[0095] Vertical gradient I y1 It can be determined by the following formula:

[0096] I y1 = (I1+2I2+I3)-(I7+2I8+I9)

[0097] By extracting the horizontal gradient I x1 and vertical gradient I y1The coefficients I1 to I9 can be used to obtain the filter masks in the horizontal and vertical directions, respectively, as shown below:

[0098]

[0099] However, as can be seen from the above formula, there is a problem of repeated diagonal calculations for the same center pixel, resulting in additional calculations. A new I can be set... x2 and I y2 With the I above x1 and I y1 The same formula, I x2 and I y2 The formula is as follows:

[0100] I x2 = (I9-I1)+(I3-I7)+2(I6-I4)

[0101] I y2 = -(I9-I1)+(I3-I7)+2(I2-I8)

[0102] It can be seen that I x2 and I y2 The formulas all contain (I9-I1) and (I3-I7), therefore, when determining I... x2 and I y2 During the process, only the values ​​of (I9-I1) and (I3-I7) need to be determined once, therefore, I x2 and I y2 The corresponding g x2 and g y2 They are respectively:

[0103]

[0104] After determining g x2 and g y2 Then, the focus value of the image can be determined using the following formula:

[0105]

[0106] Where FV is the focus value at any position in the image, used to characterize the image sharpness corresponding to different positions in the focus window, I x (x,y) is the horizontal gradient of the pixel located at point (x,y), I y (x,y) represents the vertical gradient of the pixel at point (x,y). x (x,y) and I y The formulas for determining (x, y) are as follows:

[0107] Ix (x,y)=I(x,y)*g x

[0108] I y (x,y)=I(x,y)*g y

[0109] Where I(x,y) is the grayscale value of the pixel located at point (x,y). If g is used in the two formulas above... x1 and g y1 Because of g x2 and g y2 Since there are additional diagonal vectors in the horizontal and vertical directions, g is used. x1 and g y1 Will be more than using g x2 and g y2 The calculation involves more dimensions, meaning more computational formulas need to be processed. However, the dimensionality reduction focus evaluation algorithm and the focus evaluation algorithm have almost the same accuracy. However, the focus evaluation algorithm has a computation time of 0.05 seconds per frame, while the dimensionality reduction focus evaluation algorithm proposed in this application has a computation time of 0.04 seconds per frame. Therefore, the method provided in this application embodiment can determine the focus value of an image faster without affecting performance.

[0110] In one possible embodiment, after determining the focus values ​​at different positions within the focus window using a dimensionality reduction focus evaluation algorithm, at least three positions are identified, and their focus values ​​are determined using the same algorithm. Based on the correspondence between these three positions and their focus values, a focus evaluation curve for the image to be processed is fitted. After obtaining the focus evaluation curve, the position corresponding to the peak of the curve can be selected as the dominant focus. The specific method for finding the peak will be explained below.

[0111] In one possible embodiment, the determined dominant focus may not be the most ideal focus due to non-ideal imaging scattering caused by optical aberrations and diffraction of the optical system. Therefore, after determining the dominant focus, a potential focus can be determined using an optical defocusing model.

[0112] The derivation process of the optical defocus model is explained below. Figure 8 The diagram shows an imaging model of an optical system provided in an embodiment of this application. p represents the target object point in the object space, D is the lens diameter, d is the diameter of the circle of confusion, l is the object distance, l' is the ideal image distance, p' is the ideal image point, Δ is the defocus amount, and p” is the center of the circle of confusion. Figure 9A schematic diagram of a blur circle provided in an embodiment of this application is shown, where R represents the radius. For the blur circle, the energy of each pixel is 1 / πR². Since the blur circle is a relatively uniform circle in energy, the brightness value of each point within the circle is the same, for example, 255. However, there is a brightness difference between the boundary of the blur circle and the background. Therefore, compared to a point A on the circumference of the blur circle, when calculating its focus value, the brightness values ​​of each pixel within the blur circle cancel each other out, and only the pixels on the circumference are unaffected. Therefore, it can be considered that the focus value is only related to the pixels on the circumference of the blur circle. When determining the focus value, the pixels within the uniform circumference do not need to participate in the calculation; only the circumference length of the blur circle and the energy of each pixel need to be calculated. A small arc segment on the circumference can be represented as Rdθ. When θ is sufficiently small, it can be approximated as a line segment, and its horizontal and vertical components can be represented as Rdθsinθ and Rdθcosθ, respectively. Therefore, the sharpness evaluation value f(x) of the blur circle on the image plane can be obtained by the line integral on the circumference:

[0113]

[0114] Where C is a constant, the above formula shows that the focusing value f(x) of the circle of confusion is inversely proportional to its radius R. Furthermore, since the radius R of the circle of confusion is related to the defocusing amount |xx... f | is directly proportional to |, therefore, for any focus position x, the defocus amount |xx f The relationship between | and the sharpness evaluation value f(x) satisfies:

[0115]

[0116] Where f(x) is the focal value corresponding to the focal point at position x, |xx f |, where a and b are constants, and parameter b is used to represent the parameters of the imaging device to avoid the denominator being zero.

[0117] In one possible embodiment, after determining the expression for the optical defocus model and the focus evaluation curve, Figure 10 This illustration shows a schematic diagram of an optical defocus model curve and a focus evaluation curve provided in an embodiment of this application, as shown below. Figure 10 As shown, there is an optical defocus model curve and a focus evaluation curve. Taking the focus evaluation curve as an example, it can be seen that the focus evaluation curve can be determined based on x0, x1, and x2 and the focus values ​​corresponding to these three points.

[0118] Taking the example of finding the peak of the focusing evaluation curve, if the initial position of the driving device (such as a stepper motor) of the endoscope is x0, a focusing position within the focusing range can be arbitrarily selected to the left of the initial focusing position x0 (the step corresponding to the initial focusing position is x0) as the first focusing position x1 (the step corresponding to the first focusing position is x1). At this time, the second focusing position x2 (the step corresponding to the second focusing position is x2) is located to the right of the initial focusing position x0. Since f(x0) > f(x1) and f(x0) < f(x2), it can be determined that the position of the dominant focus is between x0 and x2. Therefore, the stepper motor can be used to drive the endoscope from the position of x0 to x2, and the position with the maximum focusing value during the movement, that is, the position x3 of the dominant focus, can be determined. Or, the positions corresponding to x4 and x5 can be selected again between x0 and x2 as reference positions, and the stepper motor is used to drive the endoscope from the position of x4 to x5, and the position with the maximum focusing value during the movement, that is, the position x3 of the dominant focus, can be determined.

[0119] It should be noted that there are two special cases, namely f(x0) > f(x1) > f(x2) as shown in Figure 11 and f(x0) < f(x1) < f(x2) as shown in Figure 12 . For the case of f(x0) > f(x1) > f(x2), another fitting position can be calculated through the following formula:

[0120]

[0121] After driving the stepper motor to the position x3, the position farthest from x3 is discarded, and the above steps are repeated using the remaining three points to determine the dominant focus. For example, Figure 11 shows a schematic diagram for determining three positions. Since f(x0) > f(x1) > f(x2), and the determined x3 is the farthest from x2, x2 can be discarded.

[0122] For the case of f(x0) < f(x1) < f(x2), another fitting position can be calculated through the following formula:

[0123]

[0124] Similarly, after driving the stepper motor to the position x3, the position farthest from x3 is discarded, and the above steps are repeated using the remaining three points to determine the dominant focus. For example, Figure 12 shows a schematic diagram for determining three positions. Since f(x0) < f(x1) < f(x2), and the determined x3 is the farthest from x0, x0 can be discarded.

[0125] Similarly, the same method can be used to determine the potential focus on the optical defocus model curve, which will not be elaborated upon here. It should be noted that after determining the potential focus, two other points can be found to the left and right of the potential focus according to the set step size. The focus values ​​of the potential focus, the point to the left of the potential focus, and the point to the right of the potential focus are compared respectively, and the point with the largest focus value is selected as the potential focus.

[0126] In one possible embodiment, after determining the potential focus and the manifest focus, the focus values ​​of the potential focus and the manifest focus are determined respectively, and the point with the higher focus value is selected as the target focus, thereby focusing. With the target focus as the center, the image within the focus window is focused, that is, the curvature of the lens in the camera is adjusted to converge the light reflected by the camera to the target focus.

[0127] Step S305: Display the image captured after focusing.

[0128] In one possible embodiment, the camera host can focus the light reflected from the camera onto the target focal point by adjusting the curvature of the lens in the camera. After focusing the captured image, the focused image can be sent to a display via a video output interface, where it can be displayed.

[0129] In one specific embodiment, the flow of an autofocus method provided in this application can be as follows: Figure 13 As shown, it includes the following steps:

[0130] Step S1301: Obtain the image to be processed containing the target area, and perform blur processing on the image to be processed to obtain a blurred image.

[0131] In one possible embodiment, step S1301 can begin when the endoscope receives an autofocus request from the endoscope operator. For example, the endoscope operator presses the autofocus button on the endoscope.

[0132] Step S1302: Determine the difference image between the image to be processed and the blurred image.

[0133] Step S1303: Determine the focus window based on the centroid of the difference image.

[0134] Step S1304: Use a dimension reduction focusing evaluation algorithm to determine the focusing values ​​at at least three positions in the focusing window, and fit the focusing evaluation curve of the image to be processed.

[0135] Step S1305: The position corresponding to the maximum value of the focus in the focus evaluation curve is taken as the explicit focus.

[0136] Step S1306: Fit the optical defocus model curve of the image in the focus window using the optical defocus model, and take the position corresponding to the maximum value of the focus value in the optical defocus model curve as the potential focus.

[0137] Step S1307: Determine whether the focus value of the manifest focus is greater than the potential focus. If yes, proceed to step S1308; otherwise, proceed to step S1309.

[0138] Step S1308: Use the dominant focal point as the target focal point.

[0139] Step S1309: Designate the potential focus as the target focus.

[0140] In step S1310, the camera is controlled to refocus based on the target focus and the focused image is displayed.

[0141] It should be noted that the process of determining the dominant focus in steps S1304 to S1305 and the process of determining the potential focus in step S1306 can be interchanged. That is, step S1306 can be executed first, followed by steps S1304 to S1305.

[0142] Based on the same inventive concept, this application also provides an autofocus device, such as... Figure 14 As shown, the autofocus device 1400 may include:

[0143] The image blurring processing unit 1401 is used to acquire an image to be processed containing the target part, and to perform blurring processing on the image to be processed to obtain a blurred image;

[0144] The difference image determination unit 1402 is used to determine the difference image between the image to be processed and the blurred image; the difference image is obtained by subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image; the difference image is used to characterize the contour of the target part.

[0145] The focus window determination unit 1403 is used to determine the focus window based on the centroid of the difference image;

[0146] The image focusing display unit 1404 is used to determine the target focus in the focusing window, focus the image to be processed according to the focusing window and the focus, and display the focused image.

[0147] In one possible implementation, the image focusing display unit 1404 is specifically used to determine the explicit focus and the potential focus in the focusing window, respectively.

[0148] The target focus is determined based on the first image sharpness of the manifest focus and the second image sharpness of the potential focus.

[0149] In one possible implementation, the image focusing display unit 1404 is specifically used to determine the horizontal gradient corresponding to the at least three positions based on the gray values ​​at at least three positions in the focusing window and the dimension-reduced horizontal filter mask.

[0150] Based on the gray values ​​at the at least three locations and the dimension-reduced vertical filter mask, determine the vertical gradient corresponding to the at least three locations;

[0151] The first image sharpness at each of the at least three locations is determined based on the sum of the squares of the horizontal gradient and the squares of the vertical gradient at each of the at least three locations.

[0152] An image sharpness curve is fitted based on the first image sharpness at the at least three locations; the image sharpness curve includes the first image sharpness at multiple locations within the focus window;

[0153] The position with the highest image clarity among the plurality of positions is designated as the dominant focal point.

[0154] In one possible implementation, the image focusing display unit 1404 is specifically configured to, if the first image sharpness corresponding to the first position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the second position is less than the first image sharpness corresponding to the initial position, then between the initial position and the first position, determine the position with the greatest first image sharpness as the visible focus.

[0155] If the first image sharpness corresponding to the second position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the first position is less than the first image sharpness corresponding to the initial position, then the position with the greatest first image sharpness between the initial position and the second position is determined as the dominant focus.

[0156] In one possible implementation, the image focusing display unit 1404 is specifically used to fit an optical defocus model curve corresponding to the image in the focusing window based on the functional relationship between the second image sharpness and position in the optical defocus model; the optical defocus model curve includes the second image sharpness corresponding to multiple positions in the focusing window.

[0157] The location with the highest image clarity among the plurality of locations is selected as the potential focal point.

[0158] In one possible implementation, the image focusing display unit 1404 is specifically configured to, if the first image sharpness of the visible focus is greater than the second image sharpness of the potential focus, then use the visible focus as the target focus;

[0159] If the second image sharpness of the potential focus is greater than the first image sharpness of the manifest focus, then the potential focus is taken as the target focus.

[0160] In one possible implementation, the focus window determination unit 1403 is specifically configured to use the centroid of the difference image as the center point of the focus window, set the width as the width of the focus window, and set the length as the length of the focus window.

[0161] In one possible implementation, the image focusing display unit 1404 is specifically used to adjust the curvature of the lens in the camera to converge the light reflected by the camera onto the target focus.

[0162] Those skilled in the art will understand that embodiments of this application can be provided as methods, systems, or computer program products. Therefore, this application can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this application can take the form of a computer program product embodied on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0163] This application is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to this application. It should be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, generate instructions for implementing the flowchart illustrations. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0164] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0165] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0166] Obviously, those skilled in the art can make various modifications and variations to this application without departing from the spirit and scope of this application. Therefore, if such modifications and variations fall within the scope of the claims of this application and their equivalents, this application also intends to include such modifications and variations.

Claims

1. An endoscopic device, characterized in that, The endoscopic device includes a camera, a processor, and a display: The camera is configured to acquire an image to be processed that includes the target area; The processor is configured to perform blurring processing on the image to be processed to obtain a blurred image; Determine the difference image between the image to be processed and the blurred image; the difference image is obtained by subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image. The difference image is used to characterize the contour of the target region; The focus window is determined based on the centroid of the difference image; Determine the target focus in the focus window, and control the camera to refocus based on the target focus; The display is configured to show the image captured after focusing.

2. The endoscopic device according to claim 1, characterized in that, The processor is configured to: Determine the explicit focus and potential focus in the focusing window respectively; The target focus is determined based on the first image sharpness of the manifest focus and the second image sharpness of the potential focus.

3. The endoscopic device according to claim 2, characterized in that, The processor is configured to: Based on the grayscale values ​​at at least three locations in the focusing window and the dimension-reduced horizontal filter mask, determine the horizontal gradient corresponding to the at least three locations; Based on the gray values ​​at the at least three locations and the dimension-reduced vertical filter mask, determine the vertical gradient corresponding to the at least three locations; The first image sharpness at each of the at least three locations is determined based on the sum of the squares of the horizontal gradient and the squares of the vertical gradient at each of the at least three locations. An image sharpness curve is fitted based on the first image sharpness at the at least three locations; the image sharpness curve includes the first image sharpness at multiple locations within the focus window; The position with the highest image clarity among the plurality of positions is designated as the dominant focal point.

4. The endoscopic device according to claim 3, characterized in that, The at least three positions are an initial position, a first position, and a second position, respectively; the initial position is located at the midpoint between the first position and the second position; the processor is configured to: If the first image sharpness corresponding to the first position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the second position is less than the first image sharpness corresponding to the initial position, then the position with the greatest first image sharpness between the initial position and the first position is determined as the dominant focus. If the first image sharpness corresponding to the second position is greater than the first image sharpness corresponding to the initial position, and the first image sharpness corresponding to the first position is less than the first image sharpness corresponding to the initial position, then the position with the greatest first image sharpness between the initial position and the second position is determined as the dominant focus.

5. The endoscopic device according to claim 2, characterized in that, The processor is configured to: Based on the functional relationship between the second image sharpness and position in the optical defocus model, the optical defocus model curve corresponding to the image in the focusing window is fitted; the optical defocus model curve includes the second image sharpness corresponding to multiple positions in the focusing window; The location with the highest image clarity among the plurality of locations is selected as the potential focal point.

6. The endoscopic device according to claim 2, characterized in that, The processor is configured to: If the first image sharpness of the manifest focus is greater than the second image sharpness of the potential focus, then the manifest focus is taken as the target focus; If the second image sharpness of the potential focus is greater than the first image sharpness of the manifest focus, then the potential focus is taken as the target focus.

7. The endoscopic device according to claim 1, characterized in that, The processor is configured to: The centroid of the difference image is used as the center point of the focus window, the width is set as the width of the focus window, and the length is set as the length of the focus window.

8. The endoscopic device according to claim 1, characterized in that, The processor is configured to: Adjust the curvature of the lens in the camera to focus the light reflected by the camera onto the target.

9. An autofocus method, characterized in that, include: Obtain an image to be processed containing the target area, and perform blur processing on the image to be processed to obtain a blurred image; Determine the difference image between the image to be processed and the blurred image; the difference image is obtained by subtracting the gray values ​​of corresponding pixels in the image to be processed and the blurred image. The difference image is used to characterize the contour of the target region; The focus window is determined based on the centroid of the difference image; Determine the target focus in the focus window, and focus the image to be processed according to the focus window and the target focus; Displays the image taken after focusing.

10. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the method of claim 9.

Images