Active vision-based root canal orifice detection and positioning method, device, equipment and medium

Abstract

Embodiments of the present application disclose an active vision-based root canal orifice detection and positioning method, device, equipment and medium, belonging to the field of oral medicine and medical image technology, wherein the method comprises: taking an image quality evaluation score as a judgment index of active vision, adjusting the lens according to the image quality scores of the front and rear positions in the process before and after focusing until the imaging position with the highest image quality evaluation score is found, and completing automatic focusing; based on the pulp bottom image collected after automatic focusing, the sub-pixel coordinates of the root canal orifice edge are extracted by using the method of gray gradient interpolation refinement and high-low threshold screening; based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is regarded as a circular model, and the least square method is used to fit the center coordinate of each root canal orifice aperture. The present application solves the problems of complicated operation, low accuracy and high repeatability of doctors in the root canal preparation process, saves the time of root canal treatment, and provides a convenient and efficient auxiliary tool for oral medical treatment.

Description

Technical Field

[0001] This application relates to a method, device, equipment, and medium for detecting and locating root canal orifices based on active vision, belonging to the fields of oral medicine and medical imaging technology. Background Technology

[0002] Root canal surgery is a common method for treating pulpitis and periapical periodontitis, with root canal preparation being the main task. During root canal preparation, the detection and positioning of the root canal orifice by a root canal robot can provide visual guidance, reducing errors caused by subjective human factors and assisting the dentist in quickly and accurately aligning the root canal file with the orifice. The introduction of robotic technology can make oral surgery semi-automatic or fully automated, resulting in more precise and safer procedures. Robot-assisted root canal treatment will save both time and labor costs for medical staff. Against this backdrop, how to apply an endoscopic active vision system to assist dentists in quickly and accurately locating the root canal orifice is a pressing technical problem that needs to be solved. Summary of the Invention

[0003] To address the aforementioned technical problems, embodiments of this application provide a method, device, equipment, and medium for detecting and locating root canal orifices based on active vision. Utilizing image processing technology, and building upon a root canal preparation robot, this method provides a sub-pixel localization method for root canals based on active vision to detect the alignment scheme of root canal files before root canal preparation. This method can quickly and accurately identify the root canal orifice of the target tooth, thereby solving problems such as cumbersome operation, low accuracy, and high repeatability during root canal preparation, saving root canal treatment time, and providing a convenient and efficient auxiliary tool for oral healthcare.

[0004] Therefore, the purpose of this application is to overcome the limitations of traditional methods in order to assist doctors in quickly and accurately locating the root canal orifice.

[0005] Other features and advantages of this application will become apparent from the following detailed description, or may be learned in part from practice of this application.

[0006] According to one aspect of the embodiments of this application, a method for detecting and locating root canal orifices based on active vision is provided, the method comprising:

[0007] Using the image quality score as the criterion for active vision, the lens is adjusted based on the image quality scores of the positions before and after focusing until the imaging position with the highest image quality score is found, thus completing autofocus.

[0008] Based on the pulp floor image acquired after autofocus, the sub-pixel coordinates of the root canal orifice edge are extracted by thinning the gray-level gradient through interpolation and filtering by high and low thresholds.

[0009] Based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is regarded as a circular model, and the center coordinates of the orifice diameter of each root canal orifice are fitted using the least squares method.

[0010] Furthermore, using the image quality score as a criterion for active vision, the lens is adjusted based on the image quality scores at different positions before and after focusing, until the imaging position with the highest image quality score is found, thus completing autofocus. This includes:

[0011] Calculate the image quality score;

[0012] The size of the central rectangular window is set by using a variable window width center focusing method, combined with the change in the object distance between the lens and the upper surface of the tooth crown;

[0013] Based on an automated adjustment strategy that proceeds from coarse to fine, the object distance is brought close to the extreme point of the image quality evaluation curve, enabling the lens to be fed to a suitable depth within the opening to complete focusing.

[0014] Further, an image quality assessment score is calculated, including:

[0015] Based on the grayscale gradient information in the x, y, and 45° directions, the Roberts function is improved by adding horizontal and vertical filters [-1, 0, 1] for each pixel. T The convolution operation yields the following function expression:

[0016]

[0017] In the formula, F mul_rob The result represents the convolution, where x represents the pixel coordinates in the X direction of the image, y represents the pixel coordinates in the Y direction of the image, M represents the maximum pixel value in the X direction of the image, N represents the maximum pixel value in the Y direction of the image, f(x+2,y) represents the pixel value at point (x+2,y) in the image, f(x,y) represents the pixel value at point (x,y) in the image, f(x,y+2) represents the pixel value at point (x,y+2) in the image, f(x+1,y+1) represents the pixel value at point (x+1,y+1) in the image, f(x+1,y) represents the pixel value at point (x+1,y) in the image, and f(x,y+1) represents the pixel value at point (x,y+1) in the image.

[0018] Pixel size is calculated using the following formula:

[0019]

[0020] In the formula, ρ represents the pixel size, ε represents the parameter associated with the root canal positioning accuracy Δ, Δ·ε=1 and when ρ≤1 is considered sufficient pixel size, D1 represents the actual field width, and D2 represents the field pixel width;

[0021] Based on the pixel size, the image quality score is calculated using the following formula:

[0022]

[0023] Where IQV represents the image quality assessment score, α and β represent the first and second coefficients respectively, when ρ>1, the first coefficient α is 1 and the second coefficient β is 0; when ρ≤1, the first coefficient α is 0 and the second coefficient β is 1; f represents sharpness, which is calculated by multiplying the absolute values ​​of the image gradient differences, satisfying f>100, and the image quality assessment score satisfies e represents the natural constant.

[0024] Furthermore, a variable window width center-focusing method is adopted, and the size of the central rectangular window is set by combining the change in object distance between the lens and the upper surface of the tooth crown, including:

[0025] The relationship between the object distance of the endoscopic camera and the actual length of the field of view is determined by the following formula:

[0026]

[0027] In the formula, L represents the object distance of the endoscopic camera, θ represents the field of view angle, and h represents the actual length of the field of view;

[0028] The actual length of the rectangular open medullary region is determined using the following formula:

[0029]

[0030] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the open region, and P represents the number of horizontal pixels in the image resolution.

[0031] Based on the relationship between the object distance of the endoscopic camera and the actual length of the field of view, as well as the actual length of the rectangular open region, the size of the central rectangular window is determined by the following formula:

[0032]

[0033] In the formula, k represents the ratio coefficient of the central rectangular window, h0 represents the distance from the lens to the crown surface at the initial position, and Δ represents the positioning accuracy.

[0034] Furthermore, based on the coarse-to-fine automated adjustment strategy, the object distance is approximated to the extreme point of the image quality evaluation curve, and the lens is fed to an appropriate depth into the pulp chamber to complete focusing, including:

[0035] The coarse-to-fine automated adjustment strategy is divided into two stages, namely the first stage and the second stage;

[0036] In the first stage, a first step size is used for movement to find the range of the extreme point; in the second stage, a second step size is used for movement to gradually approximate the true position of the extreme point; wherein, the first step size is greater than the second step size;

[0037] At the beginning of the second stage, according to the set current adjustment direction, move twice according to the second step size, collect three frames of images and calculate the image quality evaluation scores of the three frames of images, where the image quality evaluation scores of the three frames of images are respectively the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image;

[0038] If the comparison of the image quality evaluation values of the three frames of images collected in the current adjustment direction is f1 < f2 and f2 > f3, continue to collect and adjust according to the current adjustment direction until f1 > f2 > f3 appears, then change the current adjustment direction and move twice with a step size lower than the second step size, re-collect three frames of images and calculate the image quality evaluation scores of the three frames of images. When f1 > f2 > f3 appears again, determine the position where the image quality evaluation score f1 of the first frame of image in the re-collected three frames of images is located as the most suitable imaging position.

[0039] Furthermore, based on the pulp bottom image collected after autofocus, the sub-pixel coordinates of the root canal orifice edge are extracted by means of gray gradient interpolation refinement and high and low threshold screening, including:

[0040] A preprocessing method of segmentation and denoising is used to eliminate visual interference in the root canal orifice image; when using the preprocessing method of segmentation and denoising, the global threshold is set to u0 = 255×10%;

[0041] The original gray image is binarized to obtain a first image; wherein, a white connected domain is formed in the root canal orifice part of the first image;

[0042] Based on the extraction conditions of the connected domain, dark point cavities formed by dental calculus defects in the image are eliminated, and the position coordinates and width and height sizes in the connected domain information are used to set a rectangular border to intercept the local image at the root canal orifice as the binarized image; wherein, the extraction condition of the connected domain is S min =(d / δ) 2 =2500, S mind represents the pixel area of ​​the connected region, d represents the diameter of the root canal opening, and δ represents the actual distance dimension represented by each pixel unit in the first image;

[0043] The edges in the binarized image are disconnected by median filtering and opening operations to obtain a preprocessed image; wherein, the edges are the jagged points at the edges of the root canal openings in the binarized image.

[0044] Based on the preprocessed image, a grayscale model is established for adjacent consecutive pixels and the grayscale gradient of all pixels is calculated. The grayscale gradient magnitude of consecutive pixels is interpolated using a quadratic interpolation function. The magnitude of grayscale gradient magnitudes with the same gradient direction is determined. Only the edge points with the maximum value are retained as sub-pixel edge points. An adaptive high and low threshold is set to filter sub-pixel edges and check the connection status.

[0045] The quadratic interpolation function and sub-pixel edge points are determined using the following method:

[0046] Let the quadratic interpolation function be:

[0047] L(x) = Ax 2 +Bx+C (7)

[0048] In the formula, L(x) represents the quadratic interpolation function, A, B, and C represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0049] Then the quadratic interpolation basis functions for the first pixel P(-1,G(p)), the second pixel R(0,G(r)), and the third pixel Q(1,G(q)) are:

[0050]

[0051] In the formula, l0(x), l1(x), and l2(x) represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0052] Based on equation (8), the quadratic interpolation function is determined as follows:

[0053] L(x)=l0(x)G(p)+l1(x)G(r)+l2(x)G(q) (9)

[0054] In the formula, G(p) represents the gray-level gradient magnitude of the first pixel P, G(r) represents the gray-level gradient magnitude of the second pixel R, and G(q) represents the gray-level gradient magnitude of the third pixel Q.

[0055] Solving equations (8) and (9) simultaneously, the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R is:

[0056]

[0057] In the formula, λ is the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R.

[0058] Furthermore, based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is treated as a circular model, and the center coordinates of each root canal orifice diameter are fitted using the least squares method, including:

[0059] According to the principle of least squares, by traversing N sub-pixel points, the sum of squared residuals can be obtained:

[0060]

[0061] In the formula, Q represents the sum of squared residuals, and ε i Represents each sub-pixel in the edge chain (x i y i The residuals of ε i =(x i -a) 2 +(y i -b) 2 -r 2 a and b represent the center parameters, x i and y i Let r represent the x and y coordinates of the i-th sub-pixel, and r represent the radius of the circle.

[0062] Based on the extremum condition, the following system of equations is obtained:

[0063]

[0064] In the formula,

[0065] Solving the system of equations (12) yields the coordinates of the center and the radius of the fitted root canal orifice:

[0066]

[0067] According to one aspect of the embodiments of this application, a root canal orifice detection and positioning device based on active vision is provided, comprising:

[0068] The autofocus module is configured to use the image quality evaluation score as the judgment indicator for active vision. It adjusts the lens according to the image quality scores of the positions before and after focusing until it finds the imaging position with the highest image quality evaluation score, thus completing the autofocus.

[0069] The subpixel coordinate extraction module is configured to extract the subpixel coordinates of the root canal orifice edge based on the pulp bottom image acquired after autofocus, using grayscale gradient interpolation refinement and high and low threshold filtering.

[0070] The coordinate fitting module is configured to fit the center coordinates of each root canal aperture based on the sub-pixel coordinates of the root canal orifice edge, treating the root canal orifice as a circular model, and using the least squares method.

[0071] According to one aspect of the embodiments of this application, an electronic device is provided, including: a controller; and a memory for storing one or more programs, which, when executed by the controller, cause the controller to implement the above-described active vision-based root canal detection and localization method.

[0072] According to one aspect of the embodiments of this application, a computer-readable storage medium is also provided, on which computer-readable instructions are stored, which, when executed by a computer processor, cause the computer to perform the above-described active vision-based root canal detection and localization method.

[0073] According to one aspect of the embodiments of this application, a computer program product or computer program is also provided, which includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the above-described active vision-based method for detecting and locating root canal orifices.

[0074] The technical solutions provided in the embodiments of this application have at least the following advantages:

[0075] This application introduces an image quality evaluation score during the autofocus process to find the optimal imaging position. Based on the image acquired after autofocus, it extracts the sub-pixel coordinates of the root canal orifice edge using grayscale gradient interpolation refinement and high / low threshold filtering. Finally, for the edge point chain of the root canal orifice, the root canal orifice is treated as a circular model, and the least squares method is used to fit the center coordinates of each root canal orifice diameter. This allows for rapid and accurate identification of the root canal orifice of the target tooth, thus solving the problems of cumbersome operation, low accuracy, and high repeatability in root canal preparation, saving root canal treatment time, and providing a convenient and efficient auxiliary tool for oral healthcare.

[0076] It should be understood that the above general description and the following detailed description are exemplary and explanatory only, and are not intended to limit this application. Attached Figure Description

[0077] The accompanying drawings, which are incorporated in and form part of this specification, illustrate embodiments consistent with this application and, together with the description, serve to explain the principles of this application. It is obvious that the drawings described below are merely some embodiments of this application, and those skilled in the art can obtain other drawings based on these drawings without any inventive effort. In the drawings:

[0078] Figure 1 This is an exemplary embodiment of the present application illustrating an overall flowchart of a method for detecting and locating root canal orifices based on active vision;

[0079] Figure 2 This is an exemplary embodiment of the present application illustrating the autofocus flowchart;

[0080] Figure 3 This is a schematic diagram of a center focus window shown in an exemplary embodiment of this application;

[0081] Figure 4 This is a schematic diagram of an imaging model illustrated in an exemplary embodiment of this application;

[0082] Figure 5 This is a flowchart illustrating the adjustment steps based on an automated adjustment strategy from coarse to fine, as shown in an exemplary embodiment of this application.

[0083] Figure 6 This is an exemplary embodiment of the present application illustrating the image effect after focusing;

[0084] Figure 7 This is an exemplary embodiment of the present application showing a binarized image and a pre-processed image, wherein (a) is a binarized image; and (b) is a pre-processed image.

[0085] Figure 8 This is a flowchart illustrating a subpixel detection process in an exemplary embodiment of this application;

[0086] Figure 9 This is a schematic diagram of a grayscale value model shown in an exemplary embodiment of this application, wherein the horizontal axis represents the pixel coordinates of three adjacent pixels, and the vertical axis represents the grayscale gradient magnitude of the corresponding pixel.

[0087] Figure 10 This is a detection result diagram shown in an exemplary embodiment of this application, wherein (a) actual edge information; (b) fitted contour;

[0088] Figure 11 This is a structural diagram of a root canal detection and positioning device based on active vision, as illustrated in an exemplary embodiment of this application. Detailed Implementation

[0089] Exemplary embodiments will now be described in detail, examples of which are illustrated in the accompanying drawings. When the following description relates to the drawings, unless otherwise indicated, the same numbers in different drawings denote the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with this application. Rather, they are merely examples of apparatuses and methods consistent with some aspects of this application as detailed in the appended claims.

[0090] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0091] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all content and operations / steps, nor do they necessarily have to be performed in the described order. For example, some operations / steps can be broken down, while others can be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0092] In this application, "multiple" refers to two or more. "And / or" describes the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A alone, A and B simultaneously, or B alone. The character " / " generally indicates that the preceding and following related objects have an "or" relationship.

[0093] Please see Figure 1 , Figure 1 This is an overall flowchart illustrating an active vision-based method for detecting and locating root canal orifices, as shown in an exemplary embodiment of this application. One aspect of this application provides an active vision-based method for detecting and locating root canal orifices, such as... Figure 1 As shown, the method includes steps S100 to S300. Details are as follows.

[0094] The S100 uses image quality evaluation score as the judgment indicator for active vision. It adjusts the lens according to the image quality score of the front and back positions during the focusing process until it finds the imaging position with the highest image quality evaluation score and completes autofocus.

[0095] In this embodiment, when an ideal reference image is unavailable, image quality can be evaluated using an objective mathematical evaluation model, ensuring the evaluation results are largely consistent with subjective human assessment. Addressing a series of issues in natural imaging, an image quality score is designed as an indicator for active vision, combining image sharpness and pixel size. The corresponding focusing window is adjusted at different object distances to maintain the focused area within the pulp chamber of the tooth, minimizing the introduction of background information. Comparing image quality scores across different imaging positions determines the next adjustment method for the robotic arm's end effector. Through a continuous, back-and-forth single-step search process, the imaging position with the highest image quality score is found as the optimal imaging position for the root canal.

[0096] In some embodiments, please refer to Figure 2 , Figure 2 This is an exemplary embodiment of the present application illustrating the autofocus flowchart. Step S100 specifically includes the following steps S110 to S130.

[0097] S110, calculate the image quality evaluation score.

[0098] In this embodiment, a comprehensive evaluation value combining sharpness (f) and pixel size (ρ) is used as the Image Quality Value (IQV) to measure the imaging effect of the open pulp chamber region. On the one hand, the sharpness of the image is relatively high within the depth of field of the camera, while the sharpness is relatively low outside the depth of field, similar to the trend of a Gaussian distribution. On the other hand, considering the detection accuracy of the root canal, while achieving clear imaging, a sufficient pixel ratio is also required within the field of view. During autofocus, as the object distance gets closer, the pixel ratio increases, and the pixel size decreases.

[0099] For example, the image quality assessment score is calculated using the following method:

[0100] Firstly, a spatial domain evaluation method is primarily used to measure sharpness. However, images of the open pulp chamber area not only contain the root canal orifice but may also include unevenly polished pulp tissue. The edge directions of these complex objects vary, so it's necessary to comprehensively consider the grayscale gradient information in the x, y, and 45° directions. Therefore, the Roberts function will be improved by adding horizontal and vertical filters [-1,0,1] for each pixel. T The convolution operation is expressed by the following function:

[0101]

[0102] In the formula, F mul_robThe result represents the convolution, where x represents the pixel coordinates in the X direction of the image, y represents the pixel coordinates in the Y direction of the image, M represents the maximum pixel value in the X direction of the image, N represents the maximum pixel value in the Y direction of the image, f(x+2,y) represents the pixel value at point (x+2,y) in the image, f(x,y) represents the pixel value at point (x,y) in the image, f(x,y+2) represents the pixel value at point (x,y+2) in the image, f(x+1,y+1) represents the pixel value at point (x+1,y+1) in the image, f(x+1,y) represents the pixel value at point (x+1,y) in the image, and f(x,y+1) represents the pixel value at point (x,y+1) in the image.

[0103] Secondly, the pixel size can be described as:

[0104]

[0105] In the formula, ρ represents the pixel size, and ε represents the parameter associated with the root canal positioning accuracy Δ. Δ·ε=1 and when ρ≤1 is considered to have sufficient pixel size.

[0106] The smaller the actual physical distance corresponding to each pixel unit in an image, the higher the accuracy of the pixels describing the edge of the root canal orifice. Therefore, during active focusing, when the pixel ratio is insufficient, it is necessary to suppress the impact of sharpness on image quality and continuously narrow the field of view; when the number of pixels within the root canal orifice is sufficient, the impact of pixel size on image quality is then suppressed, and the search continues for images with higher sharpness. In summary, the Image Quality Evaluation Value (IQV) is defined as:

[0107]

[0108] Where ρ>1, α and β represent the first and second coefficients, respectively, with α being 1 and β being 0; when ρ≤1, α is 0 and β is 1. When the object distance is close, the image quality evaluation score (IQV) is mainly affected by the sharpness f. In this case, IQV and f have the same monotonicity, exhibiting two types of monotonicity: either it increases first and then decreases as the object distance decreases, or it only decreases. Furthermore, within the working distance of the endoscopic camera, the sharpness f is calculated by multiplying the absolute values ​​of the image gradient differences, thus satisfying f>100. When the object distance is large, the image quality assessment score (IQV) is mainly affected by the pixel size. In this case, IQV has the opposite monotonicity to ρ, meaning it increases only as the object distance decreases, satisfying 0 < ρ < 1. In summary, the image quality evaluation score exhibits a strict overall trend of first increasing and then decreasing within the working object distance range of the camera.

[0109] The S120 uses a variable window width center focusing method and combines the change in object distance between the lens and the upper surface of the tooth crown to set the size of the central rectangular window.

[0110] In this embodiment, a center-focusing method with variable window width is adopted. The size of the central rectangular window is set by combining the change of the object distance between the lens and the upper surface of the tooth crown. This can keep the focus window exactly containing the image information of the main part of the pulp chamber during the focusing process, and avoid unnecessary calculations when processing the sampled data.

[0111] For example, the size of the central rectangular window is set in conjunction with the change in the object distance between the lens and the upper surface of the tooth crown. Figure 3 This is a schematic diagram illustrating the center focus window in an exemplary embodiment of this application. Figure 3 As shown, the distance from the lens to the crown surface is h0 at the initial position. The length of the initial rectangular window is set to l, the width to d, and p and q represent the pixel length and pixel width of the access cavity, respectively. Figure 4 This is a schematic diagram of an imaging model illustrated in an exemplary embodiment of this application. The imaging model is established by the field of view and resolution of the endoscopic camera, as shown below. Figure 4 As shown, the field of view (FOV) is determined by the length of the imaging plane, with a size of θ, and the image resolution is P×Q, where P and Q represent the number of horizontal and vertical pixels, respectively.

[0112] The relationship between the object distance of the endoscopic camera and the actual length of the field of view can be obtained from mathematical relationships:

[0113]

[0114] In the formula, L represents the object distance of the endoscopic camera, θ represents the field of view angle, and h represents the actual length of the field of view. Then, by applying the rules of similar triangles, the actual length of the rectangular medullary can be obtained:

[0115]

[0116] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the open region, and P represents the number of horizontal pixels in the image resolution.

[0117]

[0118] In the formula, k is the ratio coefficient of the central rectangular window, and Δ represents the positioning accuracy.

[0119] When the next end point is adjusted to a position with an object distance of h0-Δ, the pixel length of the window can be updated using the above relationship. If the value of k is greater than 1, it means that the open region has basically covered the entire image at the currently reached object distance position, so the focus window can be selected as the entire image, that is, let k = 1.

[0120] Similarly, the width d of the window also satisfies a similar linear relationship as above, so it will not be elaborated here.

[0121] S130, Based on the coarse-to-fine automatic adjustment strategy, make the object distance approach the extreme point of the image quality evaluation curve, and realize that the lens feeds to an appropriate depth into the pulp opening to complete focusing.

[0122] In this embodiment, a coarse-to-fine automatic adjustment strategy is adopted to realize that the endoscope lens feeds to an appropriate depth into the pulp opening to complete focusing, and make the object distance approach the extreme point of the image quality evaluation curve as much as possible.

[0123] Exemplarily, the first stage and the second stage that are successively executed by the coarse-to-fine automatic adjustment strategy. In the first stage, a larger step size is used for movement to quickly find the range of the extreme point. In this stage, the gray-level gradient change of the image is relatively large and less affected by local peaks. In the second stage, a smaller step size is used for movement to gradually approach the true position of the extreme point. In this stage, the influence of other factors such as noise is relatively large. The adjustment direction is determined by comparing the magnitudes of adjacent consecutive image quality evaluation values.

[0124] At the beginning of the second stage, if the comparison of the image quality evaluation values of the three frames of images collected is f1 < f2 and f2 > f3, it may be at a local maximum point or a global maximum point at this time. Therefore, it is necessary to continue to collect and adjust in this direction until f1 > f2 > f3 appears to prove that the global maximum point has been crossed. Then change the adjustment direction once again. When f1 > f2 > f3 appears again, it can be considered that the position where the image quality evaluation score is f1 is the most suitable imaging position. Among them, the image quality evaluation scores of the three frames of images are the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image respectively.

[0125] In an exemplary embodiment, please refer to Figure 5 , Figure 5 is the block diagram of the adjustment steps based on the coarse-to-fine automatic adjustment strategy shown in an exemplary embodiment of the present application. Step S130 can be completed by the following steps S131 to S137 in specific implementation.

[0126] S131, Set the first step size L1 = x1 (unit: mm).

[0127] S132, move the first step by a length L1, acquire two images before and after the move, and calculate the image quality evaluation scores f11 and f12. The two acquired images are the image acquired before the move and the image acquired after the move, respectively. f11 is the image quality evaluation score of the image acquired before the move, and f12 is the image quality evaluation score of the image acquired after the move.

[0128] S133, determine whether the condition is met: f11 < f12; if yes, return to step S132; if no, execute step S134.

[0129] S134, change direction, set the second step size L2 = x1 / 2, and initialize flag = 1. Here, flag is a boolean variable.

[0130] S135, move L2 twice, acquire three frames of images and calculate the image quality evaluation scores f1, f2 and f3 for each frame of images respectively;

[0131] S136, compare f1, f2, and f3; if f1, f2, f3, and flag satisfy the first condition: f1 > f2 > f3 and flag = 1, then execute step S137; if f1, f2, f3, and flag satisfy the second condition: f1 > f2 > f3 and flag = 2, then end the automatic adjustment and take the position of f1 as the most suitable imaging position; if f1, f2, f3, and flag do not satisfy the first and second conditions (i.e., other cases), then return to step S135.

[0132] S137, change direction, set the second step length L2 = x1 / 4, flag = 2, and return to step S135.

[0133] In this embodiment, the final image effect obtained after autofocus is as follows: Figure 6 As shown.

[0134] The S200 extracts the subpixel coordinates of the root canal orifice edge by using grayscale gradient interpolation refinement and high / low threshold filtering based on the image of the pulp base acquired after autofocus.

[0135] In some embodiments, step S200 can be implemented as follows: Locating the root canal orifice first requires preprocessing the root canal image to unify image information and improve feature contrast. Then, an edge detection algorithm is used to extract the contour of the root canal orifice from the image. The contour is further refined and filtered using interpolation and high / low thresholding to obtain a set of point chains that highly overlap with the actual edge. The enamel on the pulp surface is relatively smooth, and there are small areas of high-reflectivity in the captured root canal orifice image. These scattered areas will lose image features such as the color and shape of the tooth surface. There may also be a small amount of difficult-to-clean plaque residue on the tooth surface, which will appear as subtle blemishes in the image. Here, a segmentation and denoising preprocessing method is used to eliminate visual interference in the root canal orifice image. A grayscale model is established for adjacent consecutive pixels and the grayscale gradient of all pixels is calculated. Before performing non-maximum suppression, the grayscale gradient magnitude of consecutive pixels needs to be interpolated to refine the edges to the sub-pixel level. Then, the magnitude of grayscale gradient magnitude with the same gradient direction is judged, and only the edge points with the maximum value are retained. Finally, adaptive high and low thresholds are set to filter sub-pixel edges and check the connection status.

[0136] In an exemplary embodiment, step S200 is specifically implemented by steps S210 and S220.

[0137] S210, Image preprocessing.

[0138] A segmentation-based denoising preprocessing method is used to eliminate visual interference in the root canal orifice image. To avoid this problem, a global threshold of u0 = 255 × 10% is set. The 10% increase in grayscale value is to expand the range of the connected domain of the root canal orifice, making subsequent edge extraction more complete.

[0139] The binarized image obtained after binarizing the original grayscale image is as follows: Figure 7 As shown in (a), a white connected region will form at this point. The diameter d of the root canal orifice is less than 0.5 mm, and the actual distance δ represented by each pixel unit needs to be less than 0.01 mm. Therefore, the pixel area describing the connected region of the root canal orifice needs to satisfy a value greater than S. min =(d / δ) 2 =2500, used as the condition for extracting connected components to eliminate dark spots and cavities formed by plaque blemishes in the image. The position coordinates and dimensions from the connected component information are used to set a rectangular border to capture a local image of the root canal orifice. To prevent missing edge details around the root canal orifice, the rectangular border needs to be appropriately enlarged and adjusted. The x-coordinate (x11) and y-coordinate (y11) of the top-left corner pixel of the rectangular border are set as follows:

[0140] x11 = stats.at <int>(i,CC_STAT_X)–10

[0141] y11=stats.at <int>(i,CC_STAT_Y)–10

[0142] The `stats` structure contains the initial information of the bounding boxes of all connected component rectangles. `stats.at` indicates the operation to get the top-left corner pixel, `int` indicates the rounding operation, `i` indicates the current rectangle index, `CC_STAT_LEFT` indicates the X coordinate of the top-left corner pixel of the center rectangle, and `CC_STAT_TOP` indicates the Y coordinate of the top-left corner pixel of the center rectangle.

[0143] The width and height of the rectangular border are set as follows:

[0144] W11 = stats.at <int>(i,CC_STAT_WIDTH)+20

[0145] h11=stats.at <int>(i, CC_STAT_HEIGHT) + 20

[0146] Wherein, W11 represents the width of the rectangular frame, h11 represents the height of the rectangular frame; CC_STAT_WIDTH represents the rectangular frame width, CC_STAT_HEIGHT represents the rectangular frame height.

[0147] Further reduces the image field of view, the whole image only contains the root canal information of the teeth, the rubber barrier, crown and other redundant background in the image after preliminary focusing is completely excluded. However, the edge of the root canal after binarization is obviously uneven, which is caused by uneven illumination and uneven surface. Then the method of median filtering and opening operation is used to disconnect the connection of these small edge points, which smooths the edge under the condition of ensuring the overall shape unchanged. The root canal image obtained after preprocessing is shown in Figure 7 (b).

[0148] S220, root canal edge sub-pixel extraction.

[0149] In this embodiment, the improved Canny algorithm based on quadratic interpolation and adaptive high-low threshold is used to realize the sub-pixel edge detection of root canal, and the detection process is shown in Figure 8 First, the gray model of adjacent continuous pixels is established and the gray gradient of all pixels is calculated. The amplitude of the gray gradient of the continuous pixels is interpolated before non-maximum suppression, so as to refine the edge to the sub-pixel level. Then the size of the gray gradient amplitude with the same gradient direction is judged, and only the edge point with the maximum value is reserved. Finally, the adaptive high-low threshold is set to screen the sub-pixel edge and check the connection.

[0150] After the improved Canny algorithm completes the calculation of the amplitude and direction of each pixel point, it will use non-maximum suppression to refine the edge, and select the pixel point with the maximum gradient modulus as the edge point. The actual edge position of the root canal may be located in these pixel points. Here, the pixel gray value model is constructed as shown in Figure 9 The horizontal axis represents the pixel coordinates of the adjacent three pixel points, and the vertical axis represents the gray gradient amplitude of the corresponding pixel points.

[0151] Based on the pixel grayscale value model, three pixels are determined: the first pixel P(-1, G(p)), the second pixel R(0, G(r)), and the third pixel Q(1, G(q)). Here, G(p) represents the grayscale gradient magnitude of the first pixel P, G(r) represents the grayscale gradient magnitude of the second pixel R, and G(q) represents the grayscale gradient magnitude of the third pixel Q. Since G(r) > G(p) and G(r) > G(q), the Canny improved algorithm will consider point R as the edge point with the largest gradient magnitude. Between P and Q, there exists a point M with the largest actual gradient magnitude, which more accurately describes the edge location. Here, a quadratic interpolation is performed on the gradient magnitudes of the three adjacent points P, Q, and R, and the quadratic interpolation function is:

[0152] L(x) = Ax 2 +Bx+C (7)

[0153] In the formula, L(x) represents the quadratic interpolation function, A, B, and C represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0154] Then the quadratic interpolation basis functions for the first pixel P(-1,G(p)), the second pixel R(0,G(r)), and the third pixel Q(1,G(q)) are:

[0155]

[0156] In the formula, l0(x), l1(x), and l2(x) represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0157] The quadratic interpolation function L(x) can be expressed by the above basis functions as follows:

[0158] L(x)=l0(x)G(p)+l1(x)G(r)+l2(x)G(q) (9)

[0159] In the formula, G(p) represents the gray-level gradient magnitude of the first pixel P, G(r) represents the gray-level gradient magnitude of the second pixel R, and G(q) represents the gray-level gradient magnitude of the third pixel Q.

[0160] Solving equations (8) and (9) simultaneously, we can obtain the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R as λ = -B / 2A.

[0161]

[0162] In the formula, λ is the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R.

[0163] S300, based on the sub-pixel coordinates of the root canal orifice edge, treats the root canal orifice as a circular model and uses the least squares method to fit the center coordinates of the aperture of each root canal orifice.

[0164] In this embodiment, since the root canal orifice is generally a circular or elliptical shape, it is simplified as a circular hole. The least squares method is used to fit the obtained sub-pixel edge points to a circle, and the center coordinates of the root canal orifice can be calculated as the position reference information for root canal preparation.

[0165] In some embodiments, step S300 may be implemented as follows:

[0166] The coordinates of the center points within these point chains are calculated using the least squares method, combining geometric features. The parameters representing the circular contour are determined by minimizing the sum of squared residuals between the sub-pixel edge chains of the root canal orifice and the theoretical circular contour. The theoretical circumference can be represented by (xa). 2 +(yb) 2 =r 2 Let εi be the residual for each sub-pixel (xi, yi) in the edge chain. 2 +(yi-b) 2 -r 2 By traversing N sub-pixel points according to the least squares method, the sum of squared residuals can be obtained:

[0167]

[0168] In the formula, Q represents the sum of squared residuals, and ε i Represents each sub-pixel in the edge chain (x i y i The residuals of ε i =(x i -a) 2 +(y i -b) 2 -r 2 a and v represent the center parameters, x i and y i Let r represent the x and y coordinates of the i-th sub-pixel, and r represent the radius of the circle.

[0169] The following system of equations can be obtained from the extremum conditions:

[0170]

[0171] In the formula,

[0172] Solving the system of equations (12) will yield the coordinates of the center and the radius of the fitted root canal orifice:

[0173]

[0174] Please see Figure 10 , Figure 10 This is a schematic diagram illustrating the detection results of an exemplary embodiment of this application. Figure 10 Image (a) shows the actual location of the root canal orifice, clearly revealing the edge information of the root canal foramen. Figure 10 In (b), the fitted contour after subpixel edge extraction is superimposed. The high degree of overlap between the two contours indicates that the fitted circle is very close to the actual root canal opening. Using the center coordinates of the fitted circle to represent the actual opening position has high accuracy.

[0175] Please see Figure 11 , Figure 11 This is a structural diagram of a root canal orifice detection and positioning device based on active vision, illustrating an exemplary embodiment of this application. Another aspect of this application provides a root canal orifice detection and positioning device based on active vision, the device comprising:

[0176] The autofocus module 1101 is configured to use the image quality evaluation score as the judgment index of active vision, and adjust the lens according to the image quality scores of the front and back positions during the focusing process until the imaging position with the highest image quality evaluation score is found, thus completing the autofocus.

[0177] The subpixel coordinate extraction module 1102 is configured to extract the subpixel coordinates of the root canal orifice edge based on the pulp bottom image acquired after autofocus, using grayscale gradient interpolation refinement and high and low threshold filtering.

[0178] The coordinate fitting module 1103 is configured to fit the center coordinates of each root canal aperture based on the sub-pixel coordinates of the root canal orifice edge, treating the root canal orifice as a circular model, and using the least squares method.

[0179] In some embodiments, the autofocus module is further configured to:

[0180] Calculate the image quality score;

[0181] The size of the central rectangular window is set by using a variable window width center focusing method, combined with the change in the object distance between the lens and the upper surface of the tooth crown;

[0182] Based on an automated adjustment strategy that proceeds from coarse to fine, the object distance is brought close to the extreme point of the image quality evaluation curve, enabling the lens to be fed to a suitable depth within the opening to complete focusing.

[0183] In some embodiments, the autofocus module is further configured to:

[0184] Based on the grayscale gradient information in the x, y, and 45° directions, the Roberts function is improved by adding horizontal and vertical filters [-1, 0, 1] for each pixel. T The convolution operation yields the following function expression:

[0185]

[0186] In the formula, F mul_rob The result represents the convolution, where x represents the pixel coordinates in the X direction of the image, y represents the pixel coordinates in the Y direction of the image, M represents the maximum pixel value in the X direction of the image, N represents the maximum pixel value in the Y direction of the image, f(x+2,y) represents the pixel value at point (x+2,y) in the image, f(x,y) represents the pixel value at point (x,y) in the image, f(x,y+2) represents the pixel value at point (x,y+2) in the image, f(x+1,y+1) represents the pixel value at point (x+1,y+1) in the image, f(x+1,y) represents the pixel value at point (x+1,y) in the image, and f(x,y+1) represents the pixel value at point (x,y+1) in the image.

[0187] Pixel size is calculated using the following formula:

[0188]

[0189] In the formula, ρ represents the pixel size, ε represents the parameter associated with the root canal positioning accuracy Δ, Δ·ε=1 and when ρ≤1 is considered sufficient pixel size, D1 represents the actual field width, and D2 represents the field pixel width;

[0190] Based on the pixel size, the image quality score is calculated using the following formula:

[0191]

[0192] Where IQV represents the image quality assessment score, α and β represent the first and second coefficients respectively, when ρ>1, the first coefficient α is 1 and the second coefficient β is 0; when ρ≤1, the first coefficient α is 0 and the second coefficient β is 1; f represents sharpness, which is calculated by multiplying the absolute values ​​of the image gradient differences, satisfying f>100, and the image quality assessment score satisfies e represents the natural constant.

[0193] In some embodiments, the autofocus module is further configured to:

[0194] The relationship between the object distance of the endoscopic camera and the actual length of the field of view is determined by the following formula:

[0195]

[0196] In the formula, L represents the object distance of the endoscopic camera, θ represents the size of the field of view angle, and h represents the actual length of the field of view.

[0197] The actual length of the rectangular access opening area is determined by the following formula:

[0198]

[0199] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the access opening area, and P represents the number of horizontal pixels in the image resolution.

[0200] Based on the relationship between the object distance of the endoscopic camera and the actual length of the field of view, and the actual length of the rectangular access opening area, the size of the central rectangular window is determined by the following formula:

[0201]

[0202] In the formula, k is the ratio coefficient of the central rectangular window, h0 represents the distance from the lens to the crown surface at the initial position, and Δ represents the positioning accuracy.

[0203] In some embodiments, the autofocus module is further configured to:

[0204] The automated adjustment strategy from coarse to fine is divided into two stages, namely the first stage and the second stage;

[0205] In the first stage, a first step size is used for movement to find the range of the extreme point; in the second stage, a second step size is used for movement to gradually approach the position of the true extreme point; wherein, the first step size is greater than the second step size.

[0206] At the start of the second stage, according to the set current adjustment direction, move twice according to the second step size, collect three frames of images and calculate the image quality evaluation scores of the three frames of images, where the image quality evaluation scores of the three frames of images are the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image respectively;

[0207] If the comparison of the image quality evaluation values of the three frames of images collected in the current adjustment direction is f1 < f2 and f2 > f3, continue to perform acquisition adjustment in the current adjustment direction until f1 > f2 > f3 appears, then change the current adjustment direction and move twice with a step size lower than the second step size, re-collect three frames of images and calculate the image quality evaluation scores of the three frames of images. When f1 > f2 > f3 appears again, determine the position where the image quality evaluation score f1 of the first frame of image among the re-collected three frames of images is located as the most suitable imaging position.

[0208] In some embodiments, the sub-pixel coordinate extraction module is further configured to:

[0209] A segmentation-based denoising preprocessing method is used to eliminate visual interference in the root canal orifice image; when using the segmentation-based denoising preprocessing method, the global threshold is set to u0 = 255 × 10%;

[0210] The original grayscale image is binarized to obtain a first image; wherein, the root canal orifice portion of the first image forms a white connected region.

[0211] Based on the extraction criteria of connected components, dark spots and cavities formed by dental plaque blemishes in the image are eliminated. The position coordinates and dimensions of the connected components are used to set a rectangular border to extract a local image of the root canal orifice as a binarized image. The extraction criteria for the connected components are S... min =(d / δ) 2 =2500, S min d represents the pixel area of ​​the connected region, d represents the diameter of the root canal opening, and δ represents the actual distance dimension represented by each pixel unit in the first image;

[0212] The edges in the binarized image are disconnected by median filtering and opening operations to obtain a preprocessed image; wherein, the edges are the jagged points at the edges of the root canal openings in the binarized image.

[0213] Based on the preprocessed image, a grayscale model is established for adjacent consecutive pixels and the grayscale gradient of all pixels is calculated. The grayscale gradient magnitude of consecutive pixels is interpolated using a quadratic interpolation function. The magnitude of grayscale gradient magnitudes with the same gradient direction is determined. Only the edge points with the maximum value are retained as sub-pixel edge points. An adaptive high and low threshold is set to filter sub-pixel edges and check the connection status.

[0214] The quadratic interpolation function and sub-pixel edge points are determined using the following method:

[0215] Let the quadratic interpolation function be:

[0216] L(x) = Ax 2 +Bx+C (7)

[0217] In the formula, L(x) represents the quadratic interpolation function, A, B, and C represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0218] Then the quadratic interpolation basis functions for the first pixel P(-1,G(p)), the second pixel R(0,G(r)), and the third pixel Q(1,G(q)) are:

[0219]

[0220] In the formula, l0(x), l1(x), and l2(x) represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0221] Based on equation (8), the quadratic interpolation function is determined as follows:

[0222] L(x)=l0(x)G(p)+l1(x)G(r)+l2(x)G(q) (9)

[0223] In the formula, G(p) represents the gray-level gradient magnitude of the first pixel P, G(r) represents the gray-level gradient magnitude of the second pixel R, and G(q) represents the gray-level gradient magnitude of the third pixel Q.

[0224] Solving equations (8) and (9) simultaneously, the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R is:

[0225]

[0226] In the formula, λ is the distance between the sub-pixel edge point M at the maximum gradient magnitude and the coarse positioning edge point R.

[0227] In some embodiments, the coordinate fitting module is further configured to:

[0228] According to the principle of least squares, by traversing N sub-pixel points, the sum of squared residuals can be obtained:

[0229]

[0230] In the formula, Q represents the sum of squared residuals, and ε i Represents each sub-pixel in the edge chain (x i y i The residuals of ε i =(x i -a) 2 +(y i -b) 2 -r 2 a and b represent the center parameters, x i and y i Let r represent the x and y coordinates of the i-th sub-pixel, and r represent the radius of the circle.

[0231] Based on the extremum condition, the following system of equations is obtained:

[0232]

[0233] In the formula,

[0234] Solving the system of equations (12) yields the coordinates of the center and the radius of the fitted root canal orifice:

[0235]

[0236] It should be noted that the active vision-based root canal orifice detection and positioning device provided in the above embodiments and the active vision-based root canal orifice detection and positioning method provided in the aforementioned embodiments belong to the same concept. The specific way of performing the steps has been described in detail in the method embodiments, and will not be repeated here.

[0237] Another aspect of this application provides an electronic device, including: a controller; and a memory for storing one or more programs, which, when executed by the controller, perform the active vision-based root canal detection and localization method described in the various embodiments above.

[0238] Specifically, according to embodiments of this application, the processes described above with reference to the flowcharts can be implemented as computer software programs. For example, embodiments of this application include a computer program product comprising a computer program carried on a computer-readable medium, the computer program including a computer program for performing the methods shown in the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network via a communication component, and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU) 701, it performs various functions defined in the system of this application.

[0239] It should be noted that the computer-readable medium shown in the embodiments of this application can be a computer-readable signal medium or a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of a computer-readable storage medium may include, but are not limited to: an electrical connection having one or more wires, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage device, magnetic storage device, or any suitable combination thereof. In this application, a computer-readable storage medium can be any tangible medium containing or storing a program that can be used by or in conjunction with an instruction execution system, apparatus, or device. In this application, a computer-readable signal medium can include a data signal propagated in baseband or as part of a carrier wave, carrying a computer-readable computer program. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to wireless, wired, etc., or any suitable combination thereof.

[0240] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of this application. Each block in a flowchart or block diagram may represent a module, segment, or portion of code, which contains one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram or flowchart, and combinations of blocks in a block diagram or flowchart, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.

[0241] The modules / units described in the embodiments of this application can be implemented in software or hardware, and the described units can also be located in a processor. The names of these modules / units do not necessarily constitute a limitation on the module / unit itself.

[0242] Another aspect of this application provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the aforementioned active vision-based method for detecting and locating root canal orifices. This computer-readable storage medium may be included in the electronic device described in the above embodiments, or it may exist independently and not incorporated into the electronic device.

[0243] Another aspect of this application provides a computer program product or computer program that includes computer instructions stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium and executes the computer instructions, causing the computer device to perform the active vision-based root canal detection and localization method provided in the various embodiments described above.

[0244] According to one aspect of the embodiments of this application, a computer system is also provided, including a Central Processing Unit (CPU), which can perform various appropriate actions and processes based on a program stored in read-only memory (ROM) or a program loaded from storage into random access memory (RAM), such as performing the methods described above. Various programs and data required for system operation are also stored in the RAM. The CPU, ROM, and RAM are interconnected via a bus. Input / output (I / O) interfaces are also connected to the bus.

[0245] The following components are connected to the I / O interface: input sections including keyboards, mice, etc.; output sections including cathode ray tubes (CRTs), liquid crystal displays (LCDs), and speakers; storage sections including hard drives; and communication sections including network interface cards such as LAN (Local Area Network) cards and modems. The communication sections perform communication processing via networks such as the Internet. Drives are also connected to the I / O interface as needed. Removable media, such as disks, optical discs, magneto-optical discs, semiconductor memories, etc., are installed on the drive as needed so that computer programs read from them can be installed into the storage section as required.

[0246] The above content is merely a preferred exemplary embodiment of this application and is not intended to limit the implementation of this application. Those skilled in the art can easily make corresponding modifications or alterations based on the main concept and spirit of this application. Therefore, the scope of protection of this application should be determined by the scope of protection claimed in the claims.< / int> < / int> < / int> < / int>

Claims

1. A method for active vision based detection and localization of root canal orifice, characterized in that, The method comprises: The image quality evaluation score is taken as a judgment index of active vision, the lens is adjusted according to the image quality scores of front and rear positions in the process before and after focusing, until the imaging position with the highest image quality evaluation score is found, and automatic focusing is completed; Based on the sub-pixel coordinates of the root canal orifice edge extracted by the gray gradient interpolation thinning and high-low threshold screening method based on the pulp bottom image collected after automatic focusing, the root canal orifice is regarded as a circular model, and the least square method is used to fit the center coordinate of each root canal orifice aperture; The calculation formula of the image quality evaluation score is: The image quality evaluation score is taken as a judgment index of active vision, the lens is adjusted according to the image quality scores of front and rear positions in the process before and after focusing, until the imaging position with the highest image quality evaluation score is found, and automatic focusing is completed, comprising: Wherein, IQV represents the image quality evaluation score, and a and β represent the first coefficient and the second coefficient respectively, and ρ represents the pixel size, when ρ>1, the first coefficient a takes 1, and the second coefficient β takes 0; when ρ≤1, the first coefficient a takes 0, and the second coefficient β takes 1; f represents the definition, the definition f is calculated by the product of the absolute value of the image gradient difference, and f>100 is satisfied, and the image quality evaluation score satisfies e represents a natural constant.

2. The active vision based detection and localization of root canal orifice method according to claim 1, wherein, The image quality evaluation score is calculated; The size of the center rectangular window is set by combining the variable window width center focusing method with the object distance change of the lens and the upper surface of the tooth crown; Based on the coarse-to-fine automatic adjustment strategy, the object distance is approximated to the extreme point of the image quality evaluation curve, and the lens is fed to the appropriate depth in the opening pulp hole to complete focusing. The image quality evaluation score is calculated, comprising:

3. The active vision based root canal orifice detection and localization method of claim 2, wherein, The pixel size is calculated by the following formula: Based on the grayscale gradient information in the X, Y, and 45° directions, the Roberts function is improved by adding horizontal and vertical filters [-1, 0, 1] for each pixel. T The convolution operation yields the following function expression: In the formula, F mul_rob represents the convolution result, x represents the X direction pixel point coordinate of the image, y represents the Y direction pixel point coordinate of the image, M represents the maximum value of the X direction pixel point of the image, N represents the maximum value of the Y direction pixel point of the image, f(x+2, y) represents the pixel value of the (x+2, y) point in the image, f(x, y) represents the pixel value of the (x, y) point in the image, f(x, y+2) represents the pixel value of the (x, y+2) point in the image, f(x+1, y+1) represents the pixel value of the (x+1, y+1) point in the image, f(x+1, y) represents the pixel value of the (x+1, y) point in the image, and f(x, y+1) represents the pixel value of the (x, y+1) point in the image. In the formula, represents a parameter associated with the positioning accuracy of the root canal Δ, satisfies Δ·ε=1, and is regarded as sufficient pixel size when ρ≤1, D1 represents the actual field width, and D2 represents the field pixel width; Based on the pixel size, the image quality evaluation score is calculated. The size of the center rectangular window is set by combining the variable window width center focusing method with the object distance change of the lens and the upper surface of the tooth crown, comprising:

4. The active vision based root canal orifice detection and localization method of claim 2, wherein, The relationship between the endoscopic camera object distance size and the actual field length size is determined by the following formula: In the formula, L represents the endoscopic camera object distance size, θ represents the size of the field angle, and h represents the actual field length size; The actual length size of the rectangular opening pulp region is determined by the following formula: In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the opening pulp region, and P represents the horizontal pixel number in the resolution of the image; Based on the relationship between the endoscopic camera object distance size and the actual field length size and the actual length size of the rectangular opening pulp region, the size of the center rectangular window is determined by the following formula: In the formula, k represents the ratio coefficient of the center rectangular window, h0 represents the distance from the lens to the tooth crown surface at the initial position, and Δ represents the positioning accuracy. Based on the coarse-to-fine automatic adjustment strategy, the object distance is approximated to the extreme point of the image quality evaluation curve, and the lens is fed to the appropriate depth in the opening pulp hole to complete focusing, comprising:

5. The active vision based detection and localization of root canal orifice method according to claim 2, wherein, The coarse-to-fine automatic adjustment strategy is divided into two stages, namely the first stage and the second stage; In the first stage, a first step length is used to find the range of the extreme point, and in the second stage, a second step length is used to gradually approach the real extreme point position; wherein the first step length is greater than the second step length; In the first stage, a first step length is used to find the range of the extreme point, and in the second stage, a second step length is used to gradually approach the real extreme point position; wherein the first step length is greater than the second step length; At the beginning of the second stage, according to the current adjustment direction, three images are collected and the image quality evaluation scores of the three images are calculated by moving twice with a second step length, wherein the image quality evaluation scores of the three images are respectively the image quality evaluation score f1 of the first image, the image quality evaluation score f2 of the second image and the image quality evaluation score f3 of the third image; If the image quality evaluation values of the three images collected in the current adjustment direction are f1 < f2 and f2 > f3, the collection adjustment is continued in the current adjustment direction until f1 > f2 > f3 appears, the current adjustment direction is changed and three images are collected again by moving twice with a step length lower than the second step length, the image quality evaluation scores of the three images are calculated, and when f1 > f2 > f3 appears again, the position of the image quality evaluation score f1 of the first image in the three images collected again is determined as the most suitable imaging position.

6. The active vision based detection and localization of root canal orifice method according to claim 1, wherein, Based on the pulp bottom image collected after automatic focusing, sub-pixel coordinates of the root canal orifice edge are extracted by using a method of gray scale gradient interpolation refinement and high-low threshold screening, including: A preprocessing method of separate denoising is used to eliminate visual interference in the root canal orifice image; when the preprocessing method of separate denoising is used, a global threshold is set as u0=255×10%; A first image is obtained by performing binaryzation processing on the original gray scale image; wherein the root canal orifice part in the first image forms a white connected domain; The dark dot cavity formed in the image is eliminated by the extraction condition of the connected domain, and the position coordinates and width and height size in the connected domain information are used to set a rectangular frame to intercept a local image at the root canal orifice as a binary image; wherein the extraction condition of the connected domain is S min =(d / δ) 2 =2500, S min The pixel area of the connected domain is represented, d represents the diameter of the root canal orifice, and δ represents the actual distance size represented by each pixel unit in the first image. A preprocessing image is obtained by using a method of median filtering and open operation to disconnect the connection of edge points in the binaryzation image; wherein the edge points are jagged points at the root canal orifice edge in the binaryzation image; Based on the preprocessing image, a gray scale model is established for adjacent continuous pixel points and the gray scale gradients of all pixel points are calculated, a quadratic interpolation function is used to interpolate the gray scale gradient amplitudes of the continuous pixel points, the sizes of the gray scale gradient amplitudes with the same gradient direction are judged, only the edge points with the maximum value are reserved as sub-pixel edge points, adaptive high-low thresholds are set to screen the sub-pixel edges and check the connection conditions; The quadratic interpolation function and the sub-pixel edge points are determined by the following method: Let the quadratic interpolation function be: L(x) = Ax 2 + Bx + C (7) In the formula, L(x) represents the quadratic interpolation function, A, B and C represent the coefficients of the quadratic polynomial, and x represents the variable of the quadratic function; The quadratic interpolation base function about the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)) and the third pixel point Q(1, G(q)) is: In the formula, l0(x), l1(x) and l2(x) represent the interpolation base functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q); Based on formula (8), the quadratic interpolation function is determined as: L(x)=l0(x)G(p)+l1(x)G(r)+l2(x)G(q) (9) In the formula, G(p) represents the gray scale gradient amplitude of the first pixel point P, G(r) represents the gray scale gradient amplitude of the second pixel point R, and G(q) represents the gray scale gradient amplitude of the third pixel point Q. The distance between the sub-pixel edge point M at the maximum gradient amplitude and the rough positioning edge point R is obtained by solving the equations (8) and (9) simultaneously. In the formula, λ is the distance between the sub-pixel edge point M at the maximum gradient amplitude and the rough positioning edge point R.

7. The active vision based root canal orifice detection and localization method of claim 1, wherein, Based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is regarded as a circular model, and the least square method is used to fit the center coordinates of each root canal orifice aperture, including: According to the least square method principle, the residual sum of squares can be obtained by traversing N sub-pixel points: where Q represents a residual sum of squares, ε i represents a residual of each sub-pixel point (x i , y i ) in the edge chain, ε i = (x i -a) 2 +(y i -b) 2 -r 2 , a and b represent the center parameters, x i and y i represent the horizontal and vertical coordinates of the i-th sub-pixel point, and r represents the radius of the circle; According to the extreme value condition, the following equation group is obtained: In the formulae, Solving the equation group (12) obtains the center coordinates and radius size of the fitted root canal orifice:

8. An active vision based root canal orifice detection and positioning apparatus, characterized by, The device comprises: The automatic focusing module is configured to take the image quality evaluation score as the judgment index of active vision, adjust the lens according to the image quality scores of the front and rear positions in the process before and after focusing, until the imaging position with the highest image quality evaluation score is found, and automatic focusing is completed; The sub-pixel coordinate extraction module is configured to extract the sub-pixel coordinates of the root canal orifice edge by using the gray gradient interpolation thinning and high-low threshold screening method based on the pulp bottom image collected after automatic focusing; The coordinate fitting module is configured to regard the root canal orifice as a circular model based on the sub-pixel coordinates of the root canal orifice edge, and use the least square method to fit the center coordinates of each root canal orifice aperture. The calculation formula of the image quality evaluation score is: Wherein, IQV represents the image quality evaluation score, and a and β represent the first coefficient and the second coefficient respectively, and ρ represents the pixel size, when ρ>1, the first coefficient a takes 1, and the second coefficient β takes 0; when ρ≤1, the first coefficient a takes 0, and the second coefficient β takes 1; f represents the definition, the definition f is calculated by the product of the absolute value of the image gradient difference, and f>100 is satisfied, and the image quality evaluation score satisfies e represents a natural constant.

9. An electronic device, comprising: It comprises: A controller; A memory for storing one or more programs, when the one or more programs are executed by the controller, the controller realizes the root canal orifice detection and positioning method based on active vision in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that, A computer readable instruction is stored thereon, when the computer readable instruction is executed by the processor of the computer, the computer executes the root canal orifice detection and positioning method based on active vision in any one of claims 1 to 7.

Images