Root canal orifice detecting and positioning method, device and equipment based on active vision and medium

Abstract

The embodiment of the invention discloses a root canal orifice detecting and positioning method, device and equipment based on active vision and a medium, and belongs to the technical field of stomatology and medical image.The method comprises the steps that an image quality evaluation score serves as a judgment index of the active vision; and adjusting the lens according to the image quality scores of the front and back positions in the process before and after focusing until the imaging position with the highest image quality evaluation score is found, and completing automatic focusing. Based on a dental pulp bottom image collected after automatic focusing, extracting sub-pixel coordinates of the edge of a root canal orifice by adopting a gray gradient interpolation refining and high-low threshold screening mode; based on the sub-pixel coordinates of the edge of the root canal orifice, the root canal orifice is regarded as a circular model, and the aperture center coordinate of each root canal orifice is fitted by using a least square method; the problems of tedious doctor operation, low accuracy and high repeatability in the root canal preparation process are solved, the root canal treatment time is saved, and a convenient and efficient auxiliary tool is provided for oral medical treatment.

Description

Technical Field

[0001] The present application relates to a method, device, equipment and medium for detecting and positioning the root canal orifice based on active vision, belonging to the fields of oral medicine and medical image technology. Background Art

[0002] Root canal surgery is a common method for treating pulp diseases and periapical diseases, and root canal preparation is the main work of root canal surgery. During the process of root canal preparation by a root canal robot, the detection and positioning of the root canal orifice can provide visual guidance information, reduce the error caused by human subjective factors, and assist doctors to quickly and accurately operate the root canal file to align with the root canal orifice. The introduction of robot technology can make oral surgery semi-automatic or fully automatic, and the surgical operation is more precise and safer. The time cost and labor cost of robot-assisted medical staff for root canal treatment will be saved. Based on this background, how to apply the active vision system of the endoscope to assist doctors in quickly and accurately finding the position of the root canal orifice is a technical problem that urgently needs to be solved at present. Summary of the Invention

[0003] To solve the above technical problems, the embodiments of the present application respectively provide a method, device, equipment and medium for detecting and positioning the root canal orifice based on active vision. By using image processing technology, a method for sub-pixel positioning of the root canal orifice based on active vision is provided on the basis of a root canal preparation robot to detect the alignment scheme of the root canal file before root canal preparation, which can quickly and accurately identify the root canal orifice of the target tooth, thereby solving the problems of cumbersome operation, low accuracy and high repeatability of doctors during root canal preparation, saving the time of root canal treatment, and providing a convenient and efficient auxiliary tool for oral medicine.

[0004] Therefore, the purpose of the present application is to overcome the limitations of traditional methods to assist doctors in quickly and accurately finding the position of the root canal orifice.

[0005] Other features and advantages of the present application will become apparent through the following detailed description, or be learned in part through the practice of the present application.

[0006] According to one aspect of the embodiments of the present application, a method for detecting and positioning the root canal orifice based on active vision is provided, and the method includes:

[0007] Taking the image quality evaluation score as the judgment index of active vision, adjusting the lens according to the image quality scores of the front and rear positions during the process of focusing before and after until the imaging position with the highest image quality evaluation score is found to complete automatic focusing;

[0008] Based on the image of the bottom of the dental pulp collected after automatic focusing, the sub-pixel coordinates of the edge of the root canal orifice are extracted by means of gray-scale gradient interpolation refinement and high and low threshold screening;

[0009] Based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is regarded as a circular model, and the center coordinates of the aperture of each root canal orifice are fitted by the least square method.

[0010] Further, taking the image quality evaluation score as the judgment index of active vision, the lens is adjusted according to the image quality scores at the front and rear positions during the process before and after focusing until the imaging position with the highest image quality evaluation score is found to complete autofocus, including:

[0011] Calculating the image quality evaluation score;

[0012] Adopting the center focusing method with variable window width and setting the size of the central rectangular window in combination with the change of the object distance between the lens and the upper surface of the tooth crown;

[0013] Based on the automation adjustment strategy from coarse to fine, the object distance is approximated to the extreme point of the image quality evaluation curve, and the lens is advanced to an appropriate depth into the pulp chamber to complete focusing.

[0014] Further, calculating the image quality evaluation score includes:

[0015] Based on the gray gradient information in the x direction, y direction, and 45° direction, the Roberts function is improved by adding the convolution operation of each pixel point with the horizontal filter [-1, 0, 1] and the vertical filter [-1, 0, 1] T The obtained function expression is as follows:

[0016]

[0017] In the formula, F mul_rob represents the convolution result, x represents the pixel point coordinate in the X direction of the image, y represents the pixel point coordinate in the Y direction of the image, M represents the maximum value of the pixel points in the X direction of the image, N represents the maximum value of the pixel points in the Y direction of the image, f(x + 2, y) represents the pixel value at the point (x + 2, y) in the image, f(x, y) represents the pixel value at the point (x, y) in the image, f(x, y + 2) represents the pixel value at the point (x, y + 2) in the image, f(x + 1, y + 1) represents the pixel value at the point (x + 1, y + 1) in the image, f(x + 1, y) represents the pixel value at the point (x + 1, y) in the image, and f(x, y + 1) represents the pixel value at the point (x, y + 1) in the image;

[0018] The pixel size is calculated by the following formula:

[0019]

[0020] In the formula, ρ represents the pixel size, ε represents a parameter associated with the positioning accuracy Δ of the root canal, satisfying Δ·ε = 1 and when ρ ≤ 1, the pixel size is considered sufficient, D1 represents the actual field of view width, and D2 represents the pixel width of the field of view;

[0021] Based on the pixel size, the image quality evaluation score is calculated by the following formula:

[0022]

[0023] Where IQV represents the image quality evaluation score, α and β represent the first coefficient and the second coefficient respectively. When ρ > 1, the first coefficient α takes 1 and the second coefficient β takes 0; when ρ ≤ 1, the first coefficient α takes 0 and the second coefficient β takes 1; f represents the clarity, and the clarity f is calculated by the product of the absolute values of the image gradient differences, satisfying f > 100, and the image quality evaluation score satisfies e represents the natural constant.

[0024] Furthermore, the center focusing method with variable window width is adopted, and the size of the central rectangular window is set in combination with the change of the object distance between the lens and the upper surface of the dental crown, including:

[0025] The relationship between the object distance of the endoscope camera and the actual length of the field of view is determined by the following formula:

[0026]

[0027] In the formula, L represents the object distance of the endoscope camera, θ represents the size of the field of view angle, and h represents the actual length of the field of view;

[0028] The actual length of the rectangular pulp opening area is determined by the following formula:

[0029]

[0030] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the pulp opening area, and P represents the number of horizontal pixels in the resolution of the image.

[0031] Based on the relationship between the object distance of the endoscope camera and the actual length of the field of view and the actual length of the rectangular pulp opening area, the size of the central rectangular window is determined by the following formula:

[0032]

[0033] In the formula, k represents the ratio coefficient of the central rectangular window, h0 represents the distance from the lens to the dental crown surface at the initial position, and Δ represents the positioning accuracy.

[0034] Furthermore, based on the coarse-to-fine automatic adjustment strategy, the object distance is approximated to the extreme point of the image quality evaluation curve, and the lens is advanced to an appropriate depth into the pulp chamber to complete focusing, including:

[0035] The coarse-to-fine automatic adjustment strategy is divided into two stages, namely the first stage and the second stage;

[0036] In the first stage, a first step size is used for movement to find the range of the extreme point; in the second stage, a second step size is used for movement to gradually approach the position of the true extreme point; wherein, the first step size is greater than the second step size;

[0037] At the start of the second stage, according to the set current adjustment direction, move twice according to the second step size, collect three frames of images and calculate the image quality evaluation scores of the three frames of images, where the image quality evaluation scores of the three frames of images are respectively the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image;

[0038] If the comparison of the image quality evaluation values of the three frames of images collected in the current adjustment direction is f1 < f2 and f2 > f3, continue to collect and adjust according to the current adjustment direction until f1 > f2 > f3 appears, then change the current adjustment direction and move twice with a step size lower than the second step size, re-collect three frames of images and calculate the image quality evaluation scores of the three frames of images. When f1 > f2 > f3 appears again, determine the position where the image quality evaluation score f1 of the first frame of image among the re-collected three frames of images is located as the most suitable imaging position.

[0039] Furthermore, based on the pulp bottom image collected after autofocusing, the sub-pixel coordinates of the root canal orifice edge are extracted by means of gray gradient interpolation refinement and high and low threshold screening, including:

[0040] A preprocessing method of segmentation and denoising is used to eliminate the visual interference in the root canal orifice image; when using the preprocessing method of segmentation and denoising, the global threshold is set to u0 = 255×10%;

[0041] The original gray image is binarized to obtain a first image; wherein, a white connected domain is formed in the root canal orifice part of the first image;

[0042] Based on the extraction conditions of the connected domain, the dark spots and cavities formed by dental calculus defects in the image are eliminated, and the position coordinates, width and height size in the connected domain information are used to set a rectangular border to intercept the local image at the root canal orifice as the binarized image; wherein, the extraction condition of the connected domain is S min =(d / δ) 2 =2500, S minThe pixel area represents the connected domain, d represents the diameter of the root canal orifice, and δ represents the actual distance size represented by each pixel unit in the first image;

[0043] The connection of the edge points in the binary image is disconnected by using the methods of median filtering and opening operation to obtain a preprocessed image; wherein, the edge points are the uneven points at the edge of the root canal orifice in the binary image;

[0044] Based on the preprocessed image, a gray model is established for adjacent consecutive pixel points and the gray gradients of all pixel points are calculated. The quadratic interpolation function is used to interpolate the gray gradient amplitudes of the continuous pixel points, and the magnitudes of the gray gradient amplitudes with the same gradient direction are judged, and only the edge points with the maximum value are retained as sub-pixel edge points, and an adaptive high and low threshold is set to screen the sub-pixel edges and check the connection situation;

[0045] Among them, the quadratic interpolation function and the sub-pixel edge points are determined by the following method:

[0046] Let the quadratic interpolation function be:

[0047] L(x) = Ax 2 + Bx + C (7)

[0048] In the formula, L(x) represents the quadratic interpolation function, A, B, and C respectively represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0049] Then the quadratic interpolation basis functions for the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)), and the third pixel point Q(1, G(q)) are:

[0050]

[0051] In the formula, l0(x), l1(x), and l2(x) respectively represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0052] Based on Equation (8), the quadratic interpolation function is determined as:

[0053] L(x) = l0(x)G(p) + l1(x)G(r) + l2(x)G(q) (9)

[0054] In the formula, G(p) represents the gray gradient amplitude of the first pixel point P, G(r) represents the gray gradient amplitude of the second pixel point R, and G(q) represents the gray gradient amplitude of the third pixel point Q;

[0055] By combining Equation (8) and Equation (9), the distance between the sub-pixel edge point M at the maximum gradient amplitude and the roughly located edge point R is solved as:

[0056]

[0057] Wherein, λ is the distance between the sub-pixel edge point M at the maximum gradient amplitude and the roughly located edge point R.

[0058] Further, based on the sub-pixel coordinates of the root canal orifice edge, regarding the root canal orifice as a circular model, the center coordinates of the aperture of each root canal orifice are fitted by the least squares method, including:

[0059] According to the principle of the least squares method, traversing N sub-pixel points can obtain the sum of squared residuals:

[0060]

[0061] Wherein, Q represents the sum of squared residuals, and ε i represents the residual of each sub-pixel point (x i , y i ) in the edge chain, and ε i =(x i -a) 2 +(y i -b) 2 -r 2 , a and b represent the center parameters of the circle, x i and y i represent the abscissa and ordinate of the i-th sub-pixel point, and r represents the radius of the circle;

[0062] According to the extreme value conditions, the following system of equations is obtained:

[0063]

[0064] Wherein,

[0065] Solving the system of equations (12) to obtain the center coordinates and radius size of the fitted root canal orifice:

[0066]

[0067] According to one aspect of the embodiments of the present application, a root canal orifice detection and positioning device based on active vision is provided, including:

[0068] An autofocus module, configured to use the image quality evaluation score as the judgment index of active vision, and adjust the lens according to the image quality scores at the front and rear positions during the autofocus process until the imaging position with the highest image quality evaluation score is found to complete autofocus;

[0069] A sub-pixel coordinate extraction module, configured to extract the sub-pixel coordinates of the root canal orifice edge by means of gray-scale gradient interpolation refinement and high and low threshold screening based on the pulp bottom image collected after autofocus;

[0070] A coordinate fitting module, configured to regard the root canal orifice as a circular model based on the sub-pixel coordinates of the root canal orifice edge, and use the least squares method to fit the center coordinates of the aperture of each root canal orifice.

[0071] According to one aspect of the embodiments of the present application, an electronic device is provided, including: a controller; a memory for storing one or more programs, and when the one or more programs are executed by the controller, the controller implements the above-mentioned method for detecting and positioning the root canal orifice based on active vision.

[0072] According to one aspect of the embodiments of the present application, a computer-readable storage medium is further provided, on which computer-readable instructions are stored. When the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the above-mentioned method for detecting and positioning the root canal orifice based on active vision.

[0073] According to one aspect of the embodiments of the present application, a computer program product or a computer program is further provided. The computer program product or the computer program includes computer instructions, and the computer instructions are stored in a computer-readable storage medium. A processor of a computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the above-mentioned method for detecting and positioning the root canal orifice based on active vision.

[0074] In the technical solution provided by the embodiments of the present application, at least the following advantages are included:

[0075] In the automatic focusing process of the present application, an image quality evaluation score is introduced to find the best imaging position, and based on the image collected after automatic focusing, a method of interpolating and refining the gray gradient and screening with high and low thresholds is used to extract the sub-pixel coordinates of the root canal orifice edge. Finally, for the edge point chain of the root canal orifice, the root canal orifice is regarded as a circular model, and the least squares method is used to fit the center coordinates of the aperture of each root canal orifice. In this way, the root canal orifice of the target tooth can be quickly and accurately identified, thereby solving the problems of cumbersome operation, low accuracy, and high repeatability of doctors during root canal preparation, saving the time of root canal treatment, and providing a convenient and efficient auxiliary tool for oral medicine.

[0076] It should be understood that the above general description and the following detailed description are only exemplary and explanatory, and cannot limit the present application. Description of the Drawings

[0077] The accompanying drawings here are incorporated into the description and form a part of this description, showing embodiments consistent with this application, and are used together with the description to explain the principles of this application. Obviously, the accompanying drawings in the following description are only some embodiments of this application. For those of ordinary skill in the art, other accompanying drawings can be obtained based on these drawings without creative efforts. In the accompanying drawings:

[0078] Figure 1 is the overall flowchart of a method for detecting and positioning the root canal orifice based on active vision shown in an exemplary embodiment of this application;

[0079] Figure 2 is the autofocus flowchart shown in an exemplary embodiment of this application;

[0080] Figure 3 is the schematic diagram of the central focus window shown in an exemplary embodiment of this application;

[0081] Figure 4 is the schematic diagram of the imaging model shown in an exemplary embodiment of this application;

[0082] Figure 5 is the block diagram of the adjustment steps based on the coarse-to-fine automatic adjustment strategy shown in an exemplary embodiment of this application;

[0083] Figure 6 is the imaging effect diagram after focusing shown in an exemplary embodiment of this application;

[0084] Figure 7 is the binarized image and the effect diagram after preprocessing shown in an exemplary embodiment of this application, where (a) is the binarized image; (b) is the effect diagram after preprocessing;

[0085] Figure 8 is the sub-pixel detection flowchart shown in an exemplary embodiment of this application;

[0086] Figure 9 is the schematic diagram of the gray value model shown in an exemplary embodiment of this application, where the horizontal axis represents the pixel coordinates of three adjacent pixel points, and the vertical axis represents the gray gradient amplitude of the corresponding pixel points;

[0087] Figure 10 is the detection result diagram shown in an exemplary embodiment of this application, where (a) is the actual edge information; (b) is the fitted contour;

[0088] Figure 11 is the structural diagram of a device for detecting and positioning the root canal orifice based on active vision shown in an exemplary embodiment of this application. Detailed implementation manners

[0089] Here, exemplary embodiments will be described in detail, and examples thereof are shown in the accompanying drawings. When the following description refers to the accompanying drawings, unless otherwise indicated, the same numbers in different drawings represent the same or similar elements. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present application. On the contrary, they are merely examples of devices and methods consistent with some aspects of the present application as detailed in the appended claims.

[0090] The block diagrams shown in the accompanying drawings are merely functional entities and do not necessarily correspond to physically independent entities. That is, these functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0091] The flowcharts shown in the accompanying drawings are merely illustrative and do not necessarily include all contents and operations / steps, nor do they necessarily need to be executed in the described order. For example, some operations / steps can be decomposed, while some operations / steps can be combined or partially combined. Therefore, the actual execution order may change according to the actual situation.

[0092] In the present application, "a plurality of" means two or more. "And / or" describes the association relationship of associated objects and indicates that three relationships can exist. For example, A and / or B can represent: A exists alone, A and B exist simultaneously, and B exists alone. The character " / " generally represents an "or" relationship between the associated objects before and after.

[0093] Please refer to Figure 1 , Figure 1 which is the overall flowchart of a method for detecting and positioning the root canal orifice based on active vision shown in an exemplary embodiment of the present application. One aspect of the embodiments of the present application provides a method for detecting and positioning the root canal orifice based on active vision. As Figure 1 shown, the method includes steps S100 to S300. The details are introduced as follows.

[0094] S100, using the image quality evaluation score as the judgment index of active vision, adjusting the lens according to the image quality scores of the front and back positions during the process before and after focusing until the imaging position with the highest image quality evaluation score is found, and completing autofocus.

[0095] In this embodiment, in the case where an ideal reference image cannot be obtained, the quality evaluation of an image can be carried out by establishing an objective mathematical evaluation model to evaluate the quality of the image, so that the evaluation result is basically consistent with the subjective evaluation of the human eye. In view of a series of problems existing in the natural imaging state, an image quality evaluation score is designed in combination with image sharpness and pixel size as a judgment index for active vision. At different object distances, the corresponding focusing window is changed to keep the focusing area always within the pulp chamber orifice of the tooth, reducing the introduced background information. The image quality scores between different imaging positions are compared to determine the next adjustment method of the lens at the end of the robotic arm, and the imaging position with the highest image quality score is found as the best imaging position of the root canal through an iterative single-step search process.

[0096] In some embodiments, refer to Figure 2 , Figure 2 which is a flowchart of automatic focusing shown in an exemplary embodiment of the present application. Step S100 specifically includes the following steps S110 to S130.

[0097] S110, calculate the image quality evaluation score.

[0098] In this embodiment, a comprehensive evaluation value combining sharpness f and pixel size ρ is used as the image quality evaluation score IQV to measure the imaging effect of the pulp chamber area. On the one hand, the sharpness of the image within the depth of field of the camera is relatively high, while the sharpness outside the depth of field is relatively low, similar to the changing trend of the Gaussian distribution. On the other hand, considering the detection accuracy of the root canal, there needs to be a sufficient pixel proportion in the field of view while imaging clearly. During the process of autofocusing, as the object distance gets closer and closer, the pixel proportion will become larger and larger, and the pixel size will become smaller and smaller.

[0099] Exemplarily, the image quality evaluation score is calculated by the following method:

[0100] First, a spatial domain-based evaluation method is mainly used to measure sharpness. In fact, the image of the pulp chamber area not only contains the root canal orifice, but may also have unevenly polished pulp tissue. The edge directions of these complex objects are different, so it is necessary to comprehensively consider the gray gradient information in the x direction, y direction, and 45° direction. The Roberts function will be improved by adding the convolution operation of each pixel point with the horizontal filter [-1, 0, 1] and the vertical filter [-1, 0, 1] T . The function expression is as follows:

[0101]

[0102] In the formula, F mul_robrepresents the convolution result, x represents the pixel point coordinate in the X direction of the image, y represents the pixel point coordinate in the Y direction of the image, M represents the maximum value of the pixel points in the X direction of the image, N represents the maximum value of the pixel points in the Y direction of the image, f(x + 2, y) represents the pixel value at the point (x + 2, y) in the image, f(x, y) represents the pixel value at the point (x, y) in the image, f(x, y + 2) represents the pixel value at the point (x, y + 2) in the image, f(x + 1, y + 1) represents the pixel value at the point (x + 1, y + 1) in the image, f(x + 1, y) represents the pixel value at the point (x + 1, y) in the image, f(x, y + 1) represents the pixel value at the point (x, y + 1) in the image;

[0103] Secondly, the pixel size can be described as:

[0104]

[0105] In the formula, ρ represents the pixel size, ε represents a parameter associated with the positioning accuracy Δ of the root canal, satisfying Δ·ε = 1 and when ρ ≤ 1, it is regarded as having a sufficient pixel size.

[0106] The smaller the actual physical distance corresponding to each pixel unit in the image, the higher the accuracy of the pixel points describing the edge of the root canal orifice. Therefore, during the active focusing process, when the pixel ratio is insufficient, it is necessary to suppress the influence of sharpness on the image quality and continuously reduce the field of view; when the number of pixel points in the root canal orifice is sufficient, then suppress the influence of pixel size on the image quality and continue to find an image with higher sharpness. To sum up, the image quality evaluation score IQV is defined as:

[0107]

[0108] Among them, when ρ > 1, α and β represent the first coefficient and the second coefficient respectively. The first coefficient α takes 1 and the second coefficient β takes 0; when ρ ≤ 1, the first coefficient α takes 0 and the second coefficient β takes 1. When the object distance is relatively close, the image quality evaluation score IQV is mainly affected by sharpness f. At this time, IQV and f have the same monotonicity, and its monotonic characteristics are of two types: either increasing first and then decreasing as the object distance decreases, or only decreasing. And within the working distance of the endoscopic camera, the sharpness f is calculated by the product of the absolute values of the image gradient differences, so f > 100. When the object distance is relatively far, the image quality evaluation score IQV is mainly affected by the pixel size. At this time, IQV and ρ have opposite monotonicity, and its monotonic characteristic is only increasing as the object distance decreases, satisfying 0 < ρ < 1. To sum up the above analysis, the image quality evaluation score can meet the strict overall trend of increasing first and then decreasing within the working object distance range of the camera.

[0109] S120 adopts a central focusing method with variable window width and sets the size of the central rectangular window in combination with the change in the object distance between the lens and the upper surface of the dental crown.

[0110] In this embodiment, a central focusing method with variable window width is adopted, and the size of the central rectangular window is set in combination with the change in the object distance between the lens and the upper surface of the dental crown, so that the focusing window can always contain only the image information of the main part of the pulp opening during the focusing process as much as possible, avoiding unnecessary computational complexity when processing sampled data.

[0111] Exemplarily, the size of the central rectangular window is set in combination with the change in the object distance between the lens and the upper surface of the dental crown. Figure 3 It is a schematic diagram of the central focusing window shown in an exemplary embodiment of the present application. As Figure 3 shown, at the initial position, the distance from the lens to the surface of the dental crown is h0, the length of the initial rectangular window is set to l, the width is d, and p and q respectively represent the pixel length and pixel width of the pulp opening area. Figure 4 It is a schematic diagram of the imaging model shown in an exemplary embodiment of the present application. The imaging model established by the field of view angle and resolution of the endoscope camera is as Figure 4 shown. The field of view angle FOV is determined by the length of the imaging plane, with a size of θ, and the resolution of the image is P×Q, where P and Q respectively represent the number of horizontal pixels and vertical pixels.

[0112] From the mathematical relationship, the relationship between the object distance of the endoscope camera and the actual length of the field of view can be obtained:

[0113]

[0114] In the formula, L represents the object distance of the endoscope camera, θ represents the size of the field of view angle, and h represents the actual length of the field of view. Then, from the law of similar triangles, the actual length of the rectangular pulp opening area can be obtained:

[0115]

[0116] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the pulp opening area, and P represents the number of horizontal pixels in the resolution of the image.

[0117]

[0118] In the formula, k is the ratio coefficient of the central rectangular window, and Δ represents the positioning accuracy.

[0119] When the next end is adjusted to the position with an object distance of h0 - Δ, the pixel length of the window can be updated through the above relationship. If the value of k is greater than 1, it means that the pulp opening area has basically covered the entire image at the current object distance position, then the focusing window can be selected as the entire image, that is, let k = 1.

[0120] Similarly, the width d of the window also satisfies a similar linear relationship as above, so it will not be elaborated here.

[0121] S130. Based on the coarse-to-fine automatic adjustment strategy, make the object distance approach the extreme point of the image quality evaluation curve, and achieve the appropriate depth of the lens feeding into the pulp chamber to complete focusing.

[0122] In this embodiment, a coarse-to-fine automatic adjustment strategy is adopted to achieve the appropriate depth of the endoscope lens feeding into the pulp chamber to complete focusing, and make the object distance as close as possible to the extreme point of the image quality evaluation curve.

[0123] Exemplarily, the first stage and the second stage that are successively executed by the coarse-to-fine automatic adjustment strategy. In the first stage, a larger step size is used for movement to quickly find the range of the extreme point. In this stage, the gray level gradient change of the image is relatively large and the influence of local peaks is relatively small. In the second stage, a smaller step size is used for movement to gradually approach the true position of the extreme point. In this stage, the influence of other factors such as noise is relatively large. The adjustment direction is determined by comparing the magnitudes of adjacent consecutive image quality evaluation values.

[0124] At the beginning of the second stage, if the comparison of the image quality evaluation values of the three frames of images collected is f1 < f2 and f2 > f3, it may be at a local maximum point or a global maximum point at this time. Therefore, it is still necessary to continue to collect and adjust in this direction until f1 > f2 > f3 appears to prove that the global maximum point has been crossed. At this time, change the adjustment direction once again. When f1 > f2 > f3 appears again, it can be considered that the position where the image quality evaluation score is f1 is the most suitable imaging position. Among them, the image quality evaluation scores of the three frames of images are the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image respectively.

[0125] In an exemplary embodiment, please refer to Figure 5 , Figure 5 is the adjustment step block diagram based on the coarse-to-fine automatic adjustment strategy shown in an exemplary embodiment of the present application. Step S130 can be completed by the following steps S131 to S137 in specific implementation.

[0126] S131. Set the first step size L1 = x1 (unit: mm).

[0127] S132. Move the first step length L1, collect the images before and after the movement, and calculate the image quality evaluation scores f11 and f12. Among them, the images collected before and after are the images collected before the movement and the images collected after the movement respectively. f11 is the image quality evaluation score of the image collected before the movement, and f12 is the image quality evaluation score of the image collected after the movement.

[0128] S133. Determine whether the condition f11 < f12 is satisfied; if so, return to step S132; if not, execute step S134.

[0129] S134. Change the direction, set the second step length L2 = x1 / 2, and initialize flag = 1. Here, flag is a boolean variable.

[0130] S135. Move L2 twice, collect three frames of images, and calculate the image quality evaluation scores f1, f2, and f3 for each frame of the image respectively.

[0131] S136. Compare f1, f2, and f3; if f1, f2, f3, and flag satisfy the first condition: f1 > f2 > f3 and flag = 1, then execute step S137; if f1, f2, f3, and flag satisfy the second condition f1 > f2 > f3 and flag = 2, then end the automatic adjustment, and use the position where the image quality evaluation score is f1 as the most suitable imaging position; if f1, f2, f3, and flag do not satisfy the first condition and the second condition (i.e., other situations), then return to step S135.

[0132] S137. Change the direction, set the second step length L2 = x1 / 4, flag = 2, and return to step S135.

[0133] In this embodiment, the imaging effect finally obtained through autofocus is as Figure 6 shown.

[0134] S200. Based on the pulp bottom image collected after autofocus, use the method of interpolating and refining the gray gradient and screening with high and low thresholds to extract the sub-pixel coordinates of the root canal orifice edge.

[0135] In some embodiments, step S200 can be implemented in the following manner: To locate the position of the root canal orifice, it is first necessary to preprocess the root canal image to simplify the image information and enhance the feature contrast. Then, an edge detection algorithm is used to extract the contour of the root canal orifice from the image, and the contour is further refined and screened by combining interpolation and high and low thresholds to obtain a set of point chains that highly coincide with the actual edge. The enamel on the surface of the dental pulp is relatively smooth, and there are small areas of high-gloss reflection in the captured root canal orifice images. These scattered areas will lose image features such as the color and shape of the tooth surface. There may also be a small amount of dental calculus marks on the tooth surface that are difficult to clean, which will appear as fine defect spots in the image. Here, a preprocessing method of separating and denoising is used to eliminate the visual interference in the root canal orifice image. A gray model is established for adjacent consecutive pixel points and the gray gradient of all pixel points is calculated. Before performing non-maximum suppression, interpolation is required for the gray gradient amplitude of consecutive pixel points to refine the edge to the sub-pixel level. Then, the magnitudes of the gray gradients with the same gradient direction are judged, and only the edge points with the maximum value are retained. Finally, adaptive high and low thresholds are set to screen the sub-pixel edge and check the connection situation.

[0136] In an exemplary embodiment, step S200 is specifically implemented by the following step S210 and step S220.

[0137] S210, Image preprocessing.

[0138] A preprocessing method of separating and denoising is used to eliminate the visual interference in the root canal orifice image. To avoid this problem, here the global threshold is set to u0 = 255×10%, and the 10% increase in gray value here is to expand the range of the root canal orifice connected domain to make the subsequent edge extraction more complete.

[0139] The binary image obtained after binarizing the original gray image is as Figure 7 shown in (a). At this time, a white connected domain will be formed in the root canal orifice part. The diameter d of the root canal orifice is within 0.5 mm and the actual distance size δ represented by each pixel unit needs to be less than 0.01 mm. Therefore, the pixel area describing the root canal orifice connected domain needs to satisfy being greater than S min =(d / δ) 2 =2500, and this is used as the extraction condition for the connected domain to eliminate the dark point holes formed by dental calculus defects in the image. The position coordinates, width, and height information in the connected domain are used to set a rectangular border to intercept the local image at the root canal orifice. To prevent missing the edge details around the root canal orifice, the rectangular border also needs to be appropriately enlarged and adjusted. The abscissa x11 and ordinate y11 of the upper left corner pixel of the rectangular border are respectively set as:

[0140] x11 = stats.at <int>(i, CC_STAT_X) – 10

[0141] y11 = stats.at <int>(i, CC_STAT_Y) – 10

[0142] Among them, the structure stats contains the initial information of all the connected component rectangle borders segmented; stats.at represents the operation of obtaining the top-left pixel, int represents the integer operation, i represents the current rectangle border number, CC_STAT_LEFT represents the X coordinate of the top-left pixel of the central rectangle border, and CC_STAT_TOP represents the Y coordinate of the top-left pixel of the central rectangle border.

[0143] The width and height of the rectangle border are set as:

[0144] W11 = stats.at <int>(i, CC_STAT_WIDTH)+20

[0145] h11 = stats.at <int>(i, CC_STAT_HEIGHT) + 20

[0146] Among them, W11 represents the width of the rectangular border, h11 represents the height of the rectangular border; CC_STAT_WIDTH represents the width of the rectangular box, and CC_STAT_HEIGHT represents the height of the rectangular box.

[0147] Furthermore, the image field of view is further reduced, and the entire image only contains the root canal information of the teeth. The redundant backgrounds such as the rubber dam and the dental crown that are likely to appear in the image after preliminary focusing are completely excluded. However, the edges of the root canal orifices after binarization are significantly uneven, which is caused by uneven illumination and the uneven surface. Then, the median filtering and opening operation methods are used to disconnect the connections of these small edge points, and the edges are smoothed while ensuring the overall shape remains unchanged. The root canal orifice image obtained after preprocessing is as Figure 7 shown in (b) in the figure.

[0148] S220, Sub-pixel extraction of the root canal orifice edge.

[0149] In this embodiment, an improved Canny algorithm based on quadratic interpolation and adaptive high and low thresholds is used to achieve sub-pixel edge detection of the root canal orifice. The detection process is as Figure 8 shown. First, a gray model needs to be established for adjacent consecutive pixel points and the gray gradients of all pixel points are calculated. Before non-maximum suppression, interpolation needs to be performed on the gray gradient amplitudes of consecutive pixel points, so as to refine the edge to the sub-pixel level. Then, the magnitudes of the gray gradients with the same gradient direction are judged, and only the edge points with the maximum value are retained. Finally, adaptive high and low thresholds are set to screen the sub-pixel edges and check the connection situation.

[0150] After the improved Canny algorithm completes the calculation of the amplitude and direction of each pixel point, it will use non-maximum suppression to refine the edge, and select the pixel point with the largest gradient modulus value as the edge point. However, the actual edge position of the root canal orifice may be located within these pixel points. Here, a pixel point gray value model is constructed as Figure 9 shown. The horizontal axis represents the pixel coordinates of three adjacent pixel points, and the vertical axis represents the gray gradient amplitude of the corresponding pixel point.

[0151] Based on the pixel grayscale value model, three pixel points are determined, namely the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)), and the third pixel point Q(1, G(q)), where G(p) represents the grayscale gradient amplitude of the first pixel point P, G(r) represents the grayscale gradient amplitude of the second pixel point R, and G(q) represents the grayscale gradient amplitude of the third pixel point Q. Since G(r) > G(p) and G(r) > G(q), the Canny improved algorithm will take point R as the edge point with the maximum gradient modulus, while there is an actual point M with the maximum gradient modulus between P and Q, and point M can more accurately describe the edge position. Here, quadratic interpolation is performed on the gradient moduli of the adjacent three points P, Q, and R, and the quadratic interpolation function is defined as:

[0152] L(x) = Ax 2 + Bx + C (7)

[0153] In the formula, L(x) represents the quadratic interpolation function, A, B, and C respectively represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0154] Then the quadratic interpolation basis functions for the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)), and the third pixel point Q(1, G(q)) are:

[0155]

[0156] In the formula, l0(x), l1(x), and l2(x) respectively represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0157] The quadratic interpolation function L(x) can be expressed by the above basis functions as:

[0158] L(x) = l0(x)G(p) + l1(x)G(r) + l2(x)G(q) (9)

[0159] In the formula, G(p) represents the grayscale gradient amplitude of the first pixel point P, G(r) represents the grayscale gradient amplitude of the second pixel point R, and G(q) represents the grayscale gradient amplitude of the third pixel point Q.

[0160] By simultaneously solving equations (8) and (9), the distance λ between the sub-pixel edge point M with the maximum gradient amplitude and the roughly located edge point R can be obtained as λ = -B / 2A, that is:

[0161]

[0162] In the formula, λ is the distance between the sub-pixel edge point M with the maximum gradient amplitude and the roughly located edge point R.

[0163] S300. Based on the sub-pixel coordinates of the root canal orifice edge, the root canal orifice is regarded as a circular model, and the center coordinates of the aperture of each root canal orifice are fitted by the least square method.

[0164] In this embodiment, since the root canal orifice generally has a characteristic shape similar to a circle or an ellipse, the root canal orifice is simplified into a circular hole for processing. The least square method is used to perform circular fitting on the obtained sub-pixel edge points, and then the center coordinates of the root canal orifice can be calculated as the position reference information for root canal preparation.

[0165] In some embodiments, step S300 can be implemented in the following manner:

[0166] Combined with geometric features, the center point coordinates within these point chains are calculated by the least square method. The parameters representing the circular contour are determined by minimizing the sum of the squared residuals between the sub-pixel edge chain of the root canal orifice and the theoretically circular contour. The theoretical circumference can be expressed as (x - a) 2 +(y - b) 2 =r 2 . Then, for each sub-pixel point (xi, yi) in the edge chain, the residual is: εi = (xi - a) 2 +(yi - b) 2 -r 2 . According to the principle of the least square method, traversing N sub-pixel points can obtain the sum of the squared residuals:

[0167]

[0168] In the formula, Q represents the sum of the squared residuals, and ε i represents the residual of each sub-pixel point (x i , y i ) in the edge chain. ε i =(x i -a) 2 +(y i -b) 2 -r 2 . a and v represent the center parameters of the circle, x i and y i represent the abscissa and ordinate of the i-th sub-pixel point, and r represents the radius of the circle;

[0169] From the extreme value conditions, the following system of equations can be obtained:

[0170]

[0171] In the formula,

[0172] Solving the system of equations (12) can obtain the center coordinates and radius size of the fitted root canal orifice:

[0173]

[0174] Please refer to Figure 10 , Figure 10 which is a schematic diagram of the detection result shown in an exemplary embodiment of the present application. Figure 10 In (a), it is an image of the actual root canal orifice position, and the edge information of the root canal orifice can be clearly seen. Figure 10 In (b), it is the fitted contour after superposing the sub-pixel edge extraction. The high coincidence degree of the two contours indicates that the fitted circle is already very close to the actual root canal orifice. Using the center coordinates of the fitted circle to represent the actual orifice position has high accuracy.

[0175] Please refer to Figure 11 , Figure 11 which is a structural diagram of a root canal orifice detection and positioning device based on active vision shown in an exemplary embodiment of the present application. On the other hand, an embodiment of the present application also provides a root canal orifice detection and positioning device based on active vision, and the device includes:

[0176] An auto-focus module 1101, configured to use the image quality evaluation score as the judgment index of active vision, and adjust the lens according to the image quality scores of the front and rear positions during the focusing process until the imaging position with the highest image quality evaluation score is found to complete auto-focus;

[0177] A sub-pixel coordinate extraction module 1102, configured to extract the sub-pixel coordinates of the root canal orifice edge based on the pulp bottom image collected after auto-focusing, by means of interpolating and refining the gray gradient and screening with high and low thresholds;

[0178] A coordinate fitting module 1103, configured to regard the root canal orifice as a circular model based on the sub-pixel coordinates of the root canal orifice edge, and use the least squares method to fit the center coordinates of the aperture of each root canal orifice.

[0179] In some embodiments, the auto-focus module is further configured to:

[0180] Calculate the image quality evaluation score;

[0181] Adopt the center focusing method with variable window width, and set the size of the central rectangular window in combination with the change of the object distance between the lens and the upper surface of the tooth crown;

[0182] Based on the automation adjustment strategy from coarse to fine, make the object distance approach the extreme point of the image quality evaluation curve, and realize that the lens feeds to the appropriate depth in the pulp chamber to complete focusing.

[0183] In some embodiments, the auto-focus module is further configured to:

[0184] Improve the Roberts function based on the gray gradient information in the x - direction, y - direction, and 45° direction, and add the convolution operations of each pixel point with the horizontal filter [-1, 0, 1] and the vertical filter [-1, 0, 1] T The obtained function expression is as follows:

[0185]

[0186] In the formula, F mul_rob represents the convolution result, x represents the pixel point coordinate in the X - direction of the image, y represents the pixel point coordinate in the Y - direction of the image, M represents the maximum value of the pixel points in the X - direction of the image, N represents the maximum value of the pixel points in the Y - direction of the image, f(x + 2, y) represents the pixel value at the point (x + 2, y) in the image, f(x, y) represents the pixel value at the point (x, y) in the image, f(x, y + 2) represents the pixel value at the point (x, y + 2) in the image, f(x + 1, y + 1) represents the pixel value at the point (x + 1, y + 1) in the image, f(x + 1, y) represents the pixel value at the point (x + 1, y) in the image, f(x, y + 1) represents the pixel value at the point (x, y + 1) in the image;

[0187] Calculate the pixel size through the following formula:

[0188]

[0189] In the formula, ρ represents the pixel size, ε represents a parameter associated with the positioning accuracy Δ of the root canal, satisfying Δ·ε = 1 and when ρ ≤ 1, it is considered that the pixel size is sufficient, D1 represents the actual field - of - view width, and D2 represents the field - of - view pixel width;

[0190] Based on the pixel size, calculate the image quality evaluation score through the following formula:

[0191]

[0192] Among them, IQV represents the image quality evaluation score, α and β represent the first coefficient and the second coefficient respectively. When ρ > 1, the first coefficient α takes 1 and the second coefficient β takes 0; when ρ ≤ 1, the first coefficient α takes 0 and the second coefficient β takes 1; f represents the clarity, and the clarity f is calculated by the product of the absolute values of the image gradient differences, satisfying f > 100, and the image quality evaluation score satisfies e represents the natural constant.

[0193] In some embodiments, the autofocus module is further configured to:

[0194] Determine the relationship between the object distance of the endoscopic camera and the actual length of the field of view through the following formula:

[0195]

[0196] In the formula, L represents the object distance of the endoscope camera, θ represents the size of the field of view angle, and h represents the actual length of the field of view;

[0197] The actual length of the rectangular pulp opening area is determined by the following formula:

[0198]

[0199] In the formula, l represents the length of the initial rectangular window, p represents the pixel length of the pulp opening area, and P represents the number of horizontal pixels in the image resolution.

[0200] Based on the relationship between the object distance of the endoscope camera and the actual length of the field of view, and the actual length of the rectangular pulp opening area, the size of the central rectangular window is determined by the following formula:

[0201]

[0202] In the formula, k is the ratio coefficient of the central rectangular window, h0 represents the distance from the lens to the crown surface at the initial position, and Δ represents the positioning accuracy.

[0203] In some embodiments, the autofocus module is further configured to:

[0204] The automatic adjustment strategy from coarse to fine is divided into two stages, namely the first stage and the second stage;

[0205] In the first stage, a first step size is used for movement to find the range of the extreme point; in the second stage, a second step size is used for movement to gradually approach the position of the true extreme point; wherein, the first step size is greater than the second step size;

[0206] At the beginning of the second stage, according to the set current adjustment direction, move twice according to the second step size, collect three frames of images and calculate the image quality evaluation scores of the three frames of images, where the image quality evaluation scores of the three frames of images are the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image;

[0207] If the comparison of the image quality evaluation values of the three frames of images collected in the current adjustment direction is f1 < f2 and f2 > f3, continue the acquisition adjustment in the current adjustment direction until f1 > f2 > f3 appears, then change the current adjustment direction and move twice with a step size lower than the second step size, re-collect three frames of images and calculate the image quality evaluation scores of the three frames of images. When f1 > f2 > f3 appears again, determine the position where the image quality evaluation score f1 of the first frame of image among the re-collected three frames of images is located as the most suitable imaging position.

[0208] In some embodiments, the sub-pixel coordinate extraction module is further configured to:

[0209] Adopt a preprocessing method of separating and denoising to eliminate visual interference in the root canal orifice image; when adopting the preprocessing method of separating and denoising, set the global threshold as u0 = 255×10%;

[0210] Perform binarization processing on the original grayscale image to obtain a first image; wherein, the root canal orifice part in the first image forms a white connected domain;

[0211] Based on the extraction conditions of the connected domain, eliminate the dark point holes formed by dental calculus defects in the image, and use the position coordinates, width, and height size in the connected domain information to set a rectangular border to intercept the local image at the root canal orifice as the binarized image; wherein, the extraction condition of the connected domain is S min =(d / δ) 2 =2500, S min represents the pixel area of the connected domain, d represents the diameter of the root canal orifice, and δ represents the actual distance size represented by each pixel unit in the first image;

[0212] Adopt the methods of median filtering and opening operation to disconnect the connection of the edge points in the binarized image to obtain a preprocessed image; wherein, the edge points are the uneven points at the edge of the root canal orifice in the binarized image;

[0213] Based on the preprocessed image, establish a grayscale model for adjacent consecutive pixel points and calculate the grayscale gradients of all pixel points, use a quadratic interpolation function to interpolate the grayscale gradient amplitudes of the consecutive pixel points, judge the magnitudes of the grayscale gradient amplitudes with the same gradient direction, and only retain the edge points with the maximum value as sub-pixel edge points, and set adaptive high and low thresholds to screen the sub-pixel edges and check the connection situation;

[0214] Among them, the quadratic interpolation function and the sub-pixel edge points are determined by the following method:

[0215] Let the quadratic interpolation function be:

[0216] L(x)=Ax 2 +Bx+C (7)

[0217] In the formula, L(x) represents the quadratic interpolation function, A, B, and C respectively represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable;

[0218] Then the quadratic interpolation basis function for the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)), and the third pixel point Q(1, G(q)) is:

[0219]

[0220] In the formula, l0(x), l1(x), and l2(x) respectively represent interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q);

[0221] Based on Equation (8), the quadratic interpolation function is determined as:

[0222] L(x) = l0(x)G(p) + l1(x)G(r) + l2(x)G(q) (9)

[0223] In the formula, G(p) represents the gray-scale gradient amplitude of the first pixel point P, G(r) represents the gray-scale gradient amplitude of the second pixel point R, and G(q) represents the gray-scale gradient amplitude of the third pixel point Q;

[0224] By simultaneously solving Equation (8) and Equation (9), the distance between the sub-pixel edge point M with the maximum gradient amplitude and the roughly located edge point R is obtained as:

[0225]

[0226] In the formula, λ is the distance between the sub-pixel edge point M with the maximum gradient amplitude and the roughly located edge point R.

[0227] In some embodiments, the coordinate fitting module is further configured to:

[0228] According to the least squares principle, traversing N sub-pixel points can obtain the sum of squared residuals:

[0229]

[0230] In the formula, Q represents the sum of squared residuals, and ε i represents the residual of each sub-pixel point (x i , y i ) in the edge chain, ε i = (x i - a) 2 + (y i - b) 2 - r 2 , a and b represent the center parameters, x i and y i represent the abscissa and ordinate of the i-th sub-pixel point, and r represents the radius of the circle;

[0231] According to the extreme value condition, the following system of equations is obtained:

[0232]

[0233] In the formula,

[0234] Solve the system of equations (12) to obtain the center coordinates and radius of the fitted root canal orifice:

[0235]

[0236] It should be noted that the root canal orifice detection and positioning device based on active vision provided in the above embodiments and the root canal orifice detection and positioning method based on active vision provided in the foregoing embodiments belong to the same concept. The specific manner of performing the steps has been described in detail in the method embodiments and will not be elaborated here.

[0237] On the other hand, an embodiment of the present application further provides an electronic device, including: a controller; a memory for storing one or more programs, which, when executed by the controller, are used to execute the root canal orifice detection and positioning method based on active vision in the above various embodiments.

[0238] In particular, according to the embodiments of the present application, the process described above with reference to the flowchart can be implemented as a computer software program. For example, an embodiment of the present application includes a computer program product, which includes a computer program carried on a computer-readable medium, and the computer program includes a computer program for executing the method shown in the flowchart. In such an embodiment, the computer program can be downloaded and installed from the network through the communication part and / or installed from a removable medium. When the computer program is executed by the central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0239] It should be noted that the computer-readable medium shown in the embodiments of the present application can be a computer-readable signal medium, a computer-readable storage medium, or any combination of the two. A computer-readable storage medium can be, for example, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination of the above. More specific examples of the computer-readable storage medium may include, but are not limited to: an electrical connection with one or more wires, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above. In the present application, the computer-readable storage medium can be any tangible medium that contains or stores a program, and this program can be used by or in combination with an instruction execution system, apparatus, or device. In the present application, a computer-readable signal medium can include a data signal propagated in a baseband or as part of a carrier wave, which carries a computer-readable computer program. Such a propagated data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. A computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, and this computer-readable medium can send, propagate, or transmit a program for use by or in combination with an instruction execution system, apparatus, or device. The computer program contained on the computer-readable medium can be transmitted using any appropriate medium, including but not limited to: wireless, wired, etc., or any suitable combination of the above.

[0240] The flowcharts and block diagrams in the accompanying drawings illustrate the possible architectures, functions, and operations of systems, methods, and computer program products according to various embodiments of the present application. Among them, each block in the flowchart or block diagram can represent a module, a program segment, or a part of code, and the above-mentioned module, program segment, or part of code contains one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in an order different from that marked in the accompanying drawings. For example, two consecutive blocks shown may actually be executed substantially in parallel, and they may sometimes be executed in the reverse order, depending on the functions involved. It should also be noted that each block in the block diagram or flowchart, and the combination of blocks in the block diagram or flowchart, can be implemented by a dedicated hardware-based system for performing the specified functions or operations, or can be implemented by a combination of dedicated hardware and computer instructions.

[0241] The modules / units involved in the embodiments of the present application can be implemented in software or in hardware, and the described units can also be provided in a processor. Among them, the names of these modules / units do not constitute a limitation to the modules / units themselves in some cases.

[0242] Another aspect of the present application further provides a computer-readable storage medium, on which a computer program is stored. When the computer program is executed by a processor, the method for detecting and positioning the root canal orifice based on active vision as described above is implemented. The computer-readable storage medium may be included in the electronic device described in the above embodiments, or may exist alone without being assembled into the electronic device.

[0243] Another aspect of the embodiments of the present application further provides a computer program product or a computer program, which includes computer instructions stored in a computer-readable storage medium. The processor of the computer device reads the computer instructions from the computer-readable storage medium, and the processor executes the computer instructions, so that the computer device executes the method for detecting and positioning the root canal orifice based on active vision provided in the above various embodiments.

[0244] According to one aspect of the embodiments of the present application, a computer system is further provided, including a central processing unit (CPU), which can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage part into a random access memory (RAM), such as executing the method in the above embodiments. In the RAM, various programs and data required for system operation are also stored. The CPU, ROM, and RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0245] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a cathode ray tube (CRT), a liquid crystal display (LCD), etc. and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as required. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is mounted on the drive as required so that a computer program read therefrom is installed in the storage section as required.

[0246] The above content is only a preferred exemplary embodiment of the present application and is not used to limit the implementation of the present application. Those of ordinary skill in the art can easily make corresponding adaptations or modifications according to the main concept and spirit of the present application. Therefore, the protection scope of the present application shall be subject to the protection scope required by the claims.< / int> < / int> < / int> < / int>

Claims

1. A root canal orifice detection and positioning method based on active vision, characterized in that: The method comprises: The image quality evaluation score is used as the judgment index of active vision. The lens is adjusted according to the image quality scores of the front and rear positions during the focusing process until the imaging position with the highest image quality evaluation score is found to complete the automatic focusing. Based on the pulp bottom image acquired after automatic focusing, the sub-pixel coordinates of the root canal orifice edge are extracted by grayscale gradient interpolation refinement and high and low threshold screening; Based on the sub-pixel coordinates of the edge of the root canal orifice, the root canal orifice is regarded as a circular model, and the least squares method is used to fit the center coordinates of each root canal orifice aperture.

2. The root canal orifice detection and positioning method based on active vision according to claim 1, characterized in that: The image quality evaluation score is used as the judgment index of active vision. The lens is adjusted according to the image quality scores of the front and rear positions during the focusing process until the imaging position with the highest image quality evaluation score is found to complete the autofocus, including: Calculate image quality assessment scores; The central focusing method with variable window width is adopted, and the size of the central rectangular window is set in combination with the change of the object distance between the lens and the upper surface of the crown; Based on the coarse-to-fine automated adjustment strategy, the object distance is brought close to the extreme point of the image quality evaluation curve, and the lens is fed to the appropriate depth in the medullary foramen to complete focusing.

3. The root canal orifice detection and positioning method based on active vision according to claim 2, characterized in that: Calculate image quality assessment scores, including: Based on the grayscale gradient information in the X direction, Y direction and 45° direction, the Roberts function is improved by adding a horizontal filter [-1, 0, 1] and a vertical filter [-1, 0, 1] to each pixel. T The convolution operation of , the obtained function expression is as follows: In the formula, F mul_rob represents the convolution result, x represents the pixel coordinates in the X direction of the image, y represents the pixel coordinates in the Y direction of the image, M represents the maximum value of the pixel in the X direction of the image, N represents the maximum value of the pixel in the Y direction of the image, f(x+2,y) represents the pixel value of the point (x+2,y) in the image, f(x,y) represents the pixel value of the point (x,y) in the image, f(x,y+2) represents the pixel value of the point (x,y+2) in the image, f(x+1,y+1) represents the pixel value of the point (x+1,y+1) in the image, f(x+1,y) represents the pixel value of the point (x+1,y) in the image, and f(x,y+1) represents the pixel value of the point (x,y+1) in the image; The pixel size is calculated using the following formula: Where ρ represents the pixel size, ε represents the parameter associated with the root canal positioning accuracy Δ, the pixel size is considered sufficient when Δ·ε=1 and ρ≤1, D1 represents the actual field of view width, and D2 represents the field of view pixel width; Based on the pixel size, the image quality evaluation score is calculated by the following formula: Where IQV represents the image quality evaluation score, α and β represent the first coefficient and the second coefficient respectively. When ρ>1, the first coefficient α is 1 and the second coefficient β is 0; when ρ≤1, the first coefficient α is 0 and the second coefficient β is 1; f represents the clarity, and the clarity f is calculated by the product of the absolute values ​​of the image gradient differences. If f>100, the image quality evaluation score satisfies e represents a natural constant.

4. The root canal orifice detection and positioning method based on active vision according to claim 2, characterized in that: The central focusing method with variable window width is adopted, and the size of the central rectangular window is set in combination with the change of the object distance between the lens and the upper surface of the crown, including: The relationship between the object distance of the endoscope camera and the actual length of the field of view is determined by the following formula: In the formula, L represents the object distance of the endoscope camera, θ represents the size of the field of view angle, and h represents the actual length of the field of view; The actual length of the rectangular opening area is determined by the following formula: Where l represents the length of the initial rectangular window, p represents the pixel length of the opening area, and P represents the number of horizontal pixels in the image resolution; Based on the relationship between the object distance of the endoscope camera and the actual length of the field of view and the actual length of the rectangular opening area, the size of the central rectangular window is determined by the following formula: Where k represents the ratio coefficient of the central rectangular window, h0 represents the distance from the lens to the crown surface at the initial position, and Δ represents the positioning accuracy.

5. The root canal orifice detection and positioning method based on active vision according to claim 2, characterized in that: Based on the coarse-to-fine automatic adjustment strategy, the object distance is made close to the extreme point of the image quality evaluation curve, and the lens is fed to the appropriate depth in the medullary foramen to complete the focusing, including: The coarse-to-fine automation adjustment strategy is divided into two stages, namely the first stage and the second stage; In the first stage, a first step length is used to move to find the range of the extreme point; in the second stage, a second step length is used to move to gradually approach the true extreme point position; wherein the first step length is greater than the second step length; At the start of the second stage, according to the set current adjustment direction, move twice according to the second step length, collect three frames of images, and calculate the image quality evaluation scores of the three frames of images. The image quality evaluation scores of the three frames of images are respectively the image quality evaluation score f1 of the first frame of image, the image quality evaluation score f2 of the second frame of image, and the image quality evaluation score f3 of the third frame of image. If the comparison of the image quality evaluation values of the three frames of images collected in the current adjustment direction is f1 < f2 and f2 > f3, continue to perform acquisition adjustment in the current adjustment direction until f1 > f2 > f3 appears. Then change the current adjustment direction and move twice with a step length lower than the second step length. Re-collect three frames of images and calculate the image quality evaluation scores of the three frames of images. When f1 > f2 > f3 appears again, determine that the position where the image quality evaluation score f1 of the first frame of image among the re-collected three frames of images is located is the most suitable imaging position.

6. The root canal orifice detection and positioning method based on active vision according to claim 1, characterized in that: Based on the pulp bottom image collected after autofocus, adopt the method of interpolating and refining the gray gradient and screening with high and low thresholds to extract the sub-pixel coordinates of the root canal orifice edge, including: Adopt a preprocessing method of separating and denoising to eliminate visual interference in the root canal orifice image. When adopting the preprocessing method of separating and denoising, set the global threshold as u0 = 255×10%. Perform binary processing on the original gray image to obtain the first image. Among them, a white connected domain is formed in the root canal orifice part of the first image. Based on the extraction condition of the connected domain, the dark spots and cavities formed by dental plaque defects in the image are eliminated, and the position coordinates and width and height in the connected domain information are used to set a rectangular frame to intercept the local image at the root canal orifice as a binary image; wherein the extraction condition of the connected domain is S min =(d / δ) 2 =2500, S min represents the pixel area of ​​the connected domain, d represents the diameter of the root canal orifice, and δ represents the actual distance size represented by each pixel unit in the first image; Adopt the method of median filtering and opening operation to disconnect the connection of the edge points in the binary image to obtain a preprocessed image. Among them, the edge points are the uneven points at the root canal orifice edge in the binary image. Based on the preprocessed image, establish a gray model for adjacent consecutive pixel points and calculate the gray gradients of all pixel points. Use a quadratic interpolation function to interpolate the gray gradient amplitudes of consecutive pixel points, judge the magnitudes of the gray gradient amplitudes with the same gradient direction, and only retain the edge points with the maximum value as sub-pixel edge points. Set adaptive high and low thresholds to screen the sub-pixel edge and check the connection situation. Among them, the quadratic interpolation function and sub-pixel edge points are determined by the following method: Let the quadratic interpolation function be: L(x)=Ax 2 +Bx+C (7) In the formula, L(x) represents the quadratic interpolation function, A, B, and C respectively represent the coefficients of the quadratic polynomial, and x represents the quadratic function variable. Then the quadratic interpolation basis functions for the first pixel point P(-1, G(p)), the second pixel point R(0, G(r)), and the third pixel point Q(1, G(q)) are: In the formula, l0(x), l1(x), and l2(x) respectively represent the interpolation basis functions, x0 represents G(p), x1 represents G(r), and x2 represents G(q); Based on formula (8), the quadratic interpolation function is determined as: L(x) = l0(x)G(p) + l1(x)G(r) + l2(x)G(q) (9) In the formula, G(p) represents the gray gradient amplitude of the first pixel point P, G(r) represents the gray gradient amplitude of the second pixel point R, and G(q) represents the gray gradient amplitude of the third pixel point Q. The solution of equations (8) and (9) is that the distance between the sub-pixel edge point M with the maximum gradient amplitude and the coarse positioning edge point R is: Where λ is the distance between the sub-pixel edge point M with the maximum gradient amplitude and the coarse positioning edge point R.

7. The root canal orifice detection and positioning method based on active vision according to claim 1, characterized in that: Based on the sub-pixel coordinates of the edge of the root canal orifice, the root canal orifice is regarded as a circular model, and the least square method is used to fit the center coordinates of each root canal orifice aperture, including: According to the principle of least squares method, the residual sum of squares can be obtained by traversing N sub-pixel points: In the formula, Q represents the residual sum of squares, ε i Represents each sub-pixel point (x i ,y i ), ε i =(x i -a) 2 +(y i -b) 2 -r 2 , a and b represent the center parameters, x i and i represents the horizontal and vertical coordinates of the ith sub-pixel point, and r represents the radius of the circle; According to the extreme value conditions, we get the following system of equations: In the formula, Solving equation group (12) yields the center coordinates and radius of the fitted root canal orifice:

8. A root canal orifice detection and positioning device based on active vision, characterized in that: The device comprises: The autofocus module is configured to use the image quality evaluation score as an active vision judgment indicator, and adjust the lens according to the image quality scores of the front and rear positions in the process of focusing, until the imaging position with the highest image quality evaluation score is found, and the autofocus is completed; The sub-pixel coordinate extraction module is configured to extract the sub-pixel coordinates of the edge of the root canal opening by interpolating and refining the grayscale gradient and screening the high and low threshold values ​​based on the pulp bottom image acquired after the automatic focus; The coordinate fitting module is configured to consider the root canal opening as a circular model based on the sub-pixel coordinates of the edge of the root canal opening, and use the least squares method to fit the center coordinates of each root canal opening aperture.

9. An electronic device, characterized in that: include: Controller; A memory for storing one or more programs, which, when executed by the controller, enables the controller to implement the root canal orifice detection and positioning method based on active vision as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: Computer-readable instructions are stored thereon, and when the computer-readable instructions are executed by a processor of a computer, the computer is caused to execute the root canal orifice detection and positioning method based on active vision according to any one of claims 1 to 7.

Images